Pollination device and method aiming at pollination problem of Amomum villosum

Abstract

The invention relates to a pollination device and method aiming at the pollination problem of Amomum villosum. The device comprises a chassis, the chassis is arranged above the back of the mechanical dog, and a mechanical arm, an alarm device and an air compressor are arranged above the chassis; the mechanical arm comprises a mechanical arm base and a plurality of middle mechanical arms sequentially connected to the rear portion of the mechanical arm base, the middle mechanical arms are connected through a plurality of hinge pieces, and a vertically-placed mounting platform is arranged at the tail end of the mechanical arm. A flexible clamp mechanism, an air blowing module, a vibration module, a turntable module and a depth camera are sequentially mounted on the mounting platform from bottom to top; wherein the blowing module and the vibration module are mounted on the turntable module; the air compressor is connected with the air blowing module through an airflow pipeline and used for providing an air source for the air blowing module, and the air source blows out from an air blowing needle arranged at the top of the air blowing module through the airflow pipeline. The method solves the problem that self-pollination of Amomum villosum flowers is difficult.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent pollination, and more specifically, to a pollination device and method for the pollination problem of Amomum villosum. Background Art

[0002] Due to the inconsistency of pollination methods and flower shapes of different plants, existing pollination robots can only perform specific pollination on certain types of plants and cannot adapt well to the flower shape and growth position of Amomum villosum. In existing technical solutions, artificial pollination requires manual crawling on wet ground and pollinating one flower at a time through pollination machines such as cotton swabs; honey pollination requires the cultivation of special bees that collect nectar for this plant, and the pollination of the plant is completed during the process of the bees collecting nectar; air blast pollination requires the use of a blower to collect pollen and transfer pollen through an air duct to complete the pollination process of the plant; electrostatic friction pollination generates static electricity through friction, absorbs pollen and transfers it To the stigma, the pollination process is completed; artificial pollination requires a lot of manpower and time, and the flowering period of Amomum villosum is short and the pollination process is long. This method is inefficient and may not be able to complete the pollination task within the effective time; for pollination by specific bees, the breeding process of specific bees is difficult and the cycle is long, and it cannot be put into large-scale agricultural use; secondly, for electrostatic friction pollination, due to the humid growth environment of Amomum villosum, it is difficult to transfer pollen from the pollen sac only through a blower, and humid air is easy to conduct electric charges. Even if it is frictionally charged, the charge is conducted into the air, and it is not easy to gather pollen.

[0003] Therefore, in view of the defects of the prior art, the present invention provides a pollination device and method for pollinating Amomum villosum that can bloom in low and humid areas, which fills some technical gaps in machine pollination and is of key significance to the development of pollination technology. Summary of the invention

[0004] The present invention aims to overcome at least one defect (shortcoming) of the above-mentioned prior art, and provides a pollination device and method for the pollination problem of Ampelopsis ternata, which is used to solve the problem that the existing pollination robots and intelligent pollination schemes are unable to pollinate the special flower type of Ampelopsis ternata, and realize intelligent pollination of Ampelopsis ternata with the help of a robot dog.

[0005] The technical solution adopted by the present invention is a pollination device for the pollination problem of Amomum villosum, the device comprising:

[0006] A chassis, the chassis is arranged above the back of the mechanical dog, and a mechanical arm, an alarm device and an air compressor are arranged above the chassis;

[0007] The robotic arm comprises a robotic arm base and a plurality of intermediate robotic arms sequentially connected to the rear of the robotic arm base, wherein the plurality of intermediate robotic arms are connected via a plurality of hinges, and a vertically placed mounting platform is provided at the end of the robotic arm, and a flexible clip mechanism, a blowing module, a vibration module, a turntable module and a depth camera are sequentially installed on the mounting platform from bottom to top; wherein the blowing module and the vibration module are installed on the turntable module;

[0008] The air compressor is connected to the blowing module through an air flow pipeline, and is used to provide an air source for the blowing module. The air source is blown out from a blowing needle arranged on the top of the blowing module through the air flow pipeline.

[0009] Due to the special structure of the floral organ of the Ampelopsis ovata, the position of its pollen sac is lower than the stigma, so it is difficult for pollen to fall onto the stigma and complete self-pollination. In order to solve this problem, in the present device, without destroying the structure of the flower, the depth camera is used to accurately identify the various structures of the Ampelopsis ovata, and the flexible clamp mechanism is used to clamp the style and move it upward for a distance so that the pollen sac is not buried by the petals. It also provides a possibility for the vibration module to approach the pollen sac, so that the vibration module can shake the pollen into the groove of the flower through the vibration between the pollen sac and the pollen sac, and then use the blowing needle to blow the pollen into the style stigma, thereby realizing self-pollination of the Ampelopsis ovata, solving the problem that the Ampelopsis ovata is difficult to self-pollinate, and can achieve accurate and rapid pollination. The preparation cycle is short and it can be put into use quickly, with a high pollination rate and little influence from the external environment.

[0010] On the other hand, the present application also proposes a pollination method for the pollination problem of Amomum villosum based on the above-mentioned pollination device, and the method comprises the following steps:

[0011] S1: Place the robot dog in the area where pollination is required, start the robot dog, measure its acceleration and angular velocity through its built-in inertial sensor, and calculate its position in combination with Kalman filtering;

[0012] S2: Use the model predictive control algorithm to control the robot dog’s posture;

[0013] S3: Use the D* algorithm to plan the path of the robot dog;

[0014] S4: When the robot dog moves to the vicinity of the target flower according to the planned path, the improved YOLOv8 is used to detect and identify the target flower of Amaryllis sinensis, and the target flower is positioned for the first time;

[0015] S5: Use the depth camera to perform a second positioning of the nearest target flower, identify the various structures of the target flower, and obtain the world coordinates of the various structures of the target flower;

[0016] S6: performing a pollination operation on the target flower according to the acquired world coordinates of each part of the target flower;

[0017] S7: After the pollination is completed, the pollination device is reset, and then steps S2-S6 are repeated to complete the pollination task of the next target flower.

[0018] In the present application, the data measured by the inertial sensor built into the mechanical dog is combined with the Kalman filter to calculate the position of the mechanical dog itself, and then the D* algorithm is used to plan the path of the mechanical dog so that the pollination component carried by the mechanical dog can accurately reach a certain range near the target flower, and then the target flower is identified and located twice by using a depth camera to accurately identify and locate the coordinates of the various parts of the flower organ of the Amomum vulgare, so that the pollination device can accurately perform self-pollination operations on the Amomum vulgare, ensuring the effective transfer of pollen and improving the success rate of pollination, thereby avoiding omissions or unevenness that may occur in traditional artificial pollination, reducing human intervention, and improving pollination efficiency. Through automated pollination, the dependence on natural pollinators (such as bees and other insects) is reduced, which helps to maintain crop growth and yield when the number of pollinators is insufficient or the ecological environment is disturbed, and is suitable for large-scale agricultural production.

[0019] Preferably, the step S1 includes: obtaining the position estimation of the robot dog by integrating the acceleration and angular velocity data according to the Kalman filter equation, and the formula is as follows:

[0020]

[0021] in, is the position vector, is the velocity vector, is the acceleration vector.

[0022] The above method can accurately calculate the exact position of the robot dog itself, thus laying the foundation for the subsequent establishment of a coordinate system to locate the target flower. The accurate coordinates of the target flower can be obtained, thereby achieving precise positioning.

[0023] Preferably, step S2 includes: establishing a position coordinate system by establishing a kinematic model to obtain a body coordinate system and a world coordinate system, wherein the body coordinate system is located at the center of mass of the robot dog, the positive direction of the x-axis is the forward direction of the robot dog, the positive direction of the z-axis is perpendicular to the plane of the robot dog body and is vertically upward, and the y-axis is determined by the right-hand rule; the world coordinate system is located in the vertical direction of the projection of the body coordinate system on the ground, and the distance to the body coordinate system is h.

[0024] By establishing a fuselage coordinate system and a world coordinate system according to the different relative positions of the robotic dog, when implementing attitude control of the robotic dog through a model predictive control algorithm, the movement route of the robotic dog is more accurate, and the position of the target flower reached is more accurate.

[0025] Preferably, in the step S3, it includes:

[0026] S31: Use the five groups of cameras and infrared sensing detectors carried by the robotic dog to sense the surrounding environment 360 degrees without dead angles, discretize the collected three-dimensional topographic map into a three-dimensional grid structure, set each small cube unit as a node, and determine the connection relationship between the nodes. Then set the starting position of the robotic dog as the starting node s and the expected destination as the target node g;

[0027] S32: According to the starting node and the target node, estimate the distance from the starting node to the target node through the start function;

[0028] S33: Take out the node n with the smallest evaluation function f(n) value from the priority queue OPEN;

[0029] S34: For each adjacent node m ∈ Neighbors(n) of the taken-out node n, calculate the movement cost c ∈ (n, m) from the node n to the adjacent node m, and then calculate the new path cost t according to the movement cost;

[0030] S35: Compare the new path cost t with the path cost g(m) of the adjacent node m. If t < g(m), then update g(m) = t, indicating that a better path to reach m is found; if m is not in the OPEN queue, then put m into the OPEN queue, and repeat steps S31 - S35 until the starting node is taken out or the OPEN queue is empty.

[0031] In this application, by using the cameras and infrared sensing detectors carried by the robotic dog to sense the surrounding environment, discretizing the collected three-dimensional topographic map into a three-dimensional grid structure, and then calculating and planning the optimal forward route of the robotic dog according to the relevant nodes. If the starting node is taken out, it means that the optimal path from the starting node to the target node has been found; if the OPEN queue is empty, it means that there is no feasible path. At this time, an alarm message can be provided through the alarm device, and the robotic dog will be re-placed and the path will be re-planned.

[0032] Preferably, in the step S4,

[0033] The improved YOLOv8 includes: optimizing YOLOv8 in the feature extraction part through improved depthwise separable convolution and attention mechanism, so as to improve the accuracy and speed of detecting Amomum villosum flowers;

[0034] The first positioning of the target flower comprises:

[0035] S41: The depth camera eliminates lens distortion on the image by using its internal reference to perform distortion correction to obtain a distortion-free image, and obtains the camera's internal reference data from a configuration file of a color frame or a depth frame;

[0036] S42: For each pixel point, convert its depth pixel value into an actual depth value Z;

[0037] S43: Obtaining the three-dimensional coordinates of the pixel point in the camera coordinate system according to the camera's intrinsic parameter data and the actual depth value;

[0038] S44: The rotation matrix and the translation vector of the robot dog are combined to form a transformation matrix, and then the three-dimensional coordinates of the pixel points to be obtained are converted into the world coordinate system using the transformation matrix, so as to achieve the first positioning of the target flower.

[0039] With the help of the path planning algorithm, the pollination device moves to the vicinity of the target flower, and with the help of the depth camera to collect information, the custom feature extraction module in the improved YOLOv8 is used to effectively train the model for the flower type of the Amaryllis japonica, and the feature extraction part of YOLOv8 is optimized through the improved depth separable convolution and attention mechanism, thereby improving the accuracy and speed of detecting the Amaryllis japonica. After detecting the Amaryllis japonica, the world coordinates of the target flower are determined based on the pixel points of the target flower image taken by the depth camera, thereby achieving the first positioning of the target flower. The pollination device reaches a more accurate position of the target flower based on the coordinate results of the first positioning, preparing for pollination.

[0040] Preferably, step S5 includes: using an FCN semantic segmentation algorithm to specifically identify the various structures of the target flower, and then using the method of first positioning the target flower to perform a second positioning on the various structures of the target flower, and obtaining the coordinates of each structure in the world coordinate system.

[0041] The target flower is identified through the FCN semantic segmentation algorithm, and the various structures of the floral organ, including petals, stigma, pollen sac, and style, are identified. Then, the above structures are located, so that the pollination device can accurately find the style and pollen sac.

[0042] Preferably, step S6 includes:

[0043] S61: The robot arm establishes a relative coordinate system for positioning according to the recorded relevant coordinates of the flower, and then calculates the relative coordinate system coordinates of the petals, stigma, pollen sac, and style of the target flower, and then suspends the flexible clamp mechanism, the vibration module, and the blowing module above the target flower;

[0044] S62: After receiving the clamping instruction, the flexible clamp mechanism calculates its force and opening and closing angle according to the relative coordinates of the style, and then drives the motor to clamp the style and move upward for a distance so that the pollen sac is not buried by the petals;

[0045] S63: The turntable module drives the vibration module and the blowing module to be located at appropriate positions by rotating, and the vibration between the vibration module and the pollen sac shakes the pollen to the groove of the flower; the blowing module generates gas through an air compressor and uses an air flow pipeline to transport the air flow, and blows the pollen into the top of the style through the blowing needle to complete a pollination;

[0046] S64: According to the number of petals, the vibration module, the blowing module and the blowing needle are placed at appropriate positions again by rotating the turntable module, thereby completing the pollination work of different petals near the same coordinate.

[0047] After completing the positioning of the target flowers, the robotic arm of the pollination device begins to perform pollination work, and through the cooperation of various modules of the pollination device, efficient and accurate self-pollination of the Amaryllis flower is achieved.

[0048] Further preferably, in step S63, the vibration module includes: using the pulse signal generated by the PWM module to affect its period and finally affect the vibration frequency f z , wherein the period calculation formula of the PWM module is:

[0049]

[0050] T clk is the clock period of the PWM module, f clk is the clock frequency of the PWM module; PERIOD is the period ratio register.

[0051] By calculating the cycle of the PWM module to control the vibration frequency of the vibration module, it can be ensured that the pollen in the pollen sac can be shaken off smoothly while the structure of the pollen sac will not be damaged, thereby ensuring the success rate of pollination.

[0052] Preferably, in step S7, resetting the pollination device includes resetting the robot arm through a total transformation matrix, and the total transformation matrix formula is:

[0053] T=A 1 A 2 …A i

[0054] Among them, A i is the homogeneous transformation matrix between adjacent intermediate manipulators obtained by the DH parameter method, and i is the number of manipulator joints.

[0055] After completing the pollination task for a target flower, the robotic arm is reset in time to ensure that the pollination task for the next target flower can be carried out in time, thereby improving the pollination efficiency.

[0056] Compared with the prior art, the present invention has the following beneficial effects:

[0057] Due to the special structure of the floral organs of the Ampelopsis ovata, the position of its pollen sac is lower than the stigma, so it is difficult for pollen to fall onto the stigma and complete self-pollination. In order to solve this problem, the present application proposes a pollination device and method for the pollination problem of Ampelopsis ovata. Firstly, a path planning algorithm is used to enable the pollination device to accurately reach a certain range near the target flower. Then, without destroying the structure of the flower, a depth camera is used to accurately identify the various structures of the Ampelopsis ovata, and the precise world coordinates of each structure are obtained. According to the coordinates, a flexible clamp mechanism is used to clamp the flower. The style moves upward for a distance so that the pollen sac is not buried by the petals, and it also provides the possibility for the vibration module to approach the pollen sac, so that the vibration module can shake the pollen into the groove of the flower through the vibration between the pollen sac, and then use the air blowing needle to blow the pollen into the stigma of the style, thereby realizing self-pollination of the Ampelopsis pilosa, solving the problem that Ampelopsis pilosa is difficult to self-pollinate, can realize accurate and rapid pollination, reduce human intervention, have a high pollination rate, are less affected by the external environment, have a short preparation period and can be put into use quickly, which is suitable for large-scale agricultural production. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 This is a schematic diagram of the structure of the pollination device provided by the present invention.

[0059] Figure 2 The present invention provides a schematic diagram of the structure of the Amomum villosum provided by the present invention.

[0060] Figure 3 The present invention provides a schematic diagram of the pollination method process.

[0061] Figure 4 A schematic diagram of the structure of another pollination device provided by the present invention.

[0062] Figure 5 This is a schematic diagram of the enlarged structure of another pollination device provided by the present invention.

[0063] Figure 6 A schematic diagram of another pollination method provided by the present invention.

[0064] Description of the drawings: chassis 1; robotic arm 2; middle robotic arm 21; hinge 22; mounting platform 23; alarm device 3; air compressor 4; air flow duct 5; flexible clamp mechanism 6; blowing module 7; blowing needle 71; vibration module 8; turntable module 9; depth camera 10; petals 11; stigma 12; pollen sac 13; style 14; end effector 15; flexible clamp 16; gripper 17; turntable 18; goose down 19; bearing 20; expansion dock 24; laser radar 25; rotating bearing 26. DETAILED DESCRIPTION

[0065] The drawings of the present invention are only for illustrative purposes and should not be construed as limiting the present invention. In order to better illustrate the following embodiments, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product; it is understandable to those skilled in the art that some well-known structures and their descriptions in the drawings may be omitted.

[0066] Example 1

[0067] like Figure 1 As shown, this embodiment provides a pollination device for the pollination problem of Amomum villosum, and the device comprises:

[0068] A chassis 1, wherein the chassis 1 is arranged above the back of the robot dog, and a robot arm 2, an alarm device 3 and an air compressor 4 are arranged above the chassis 1;

[0069] The robot arm 2 includes a robot arm base and a plurality of intermediate robot arms 21 sequentially connected to the rear of the robot arm base, wherein the plurality of intermediate robot arms 21 are connected via a plurality of hinges 22, and a vertically placed mounting platform 23 is provided at the end of the robot arm 2, and a flexible clip mechanism 6, a blowing module 7, a vibration module 8, a turntable module 9 and a depth camera 10 are sequentially mounted on the mounting platform 23 from bottom to top; wherein the blowing module 7 and the vibration module 8 are mounted on the turntable module 9;

[0070] The air compressor 4 is connected to the blowing module 7 through the air flow pipeline 5 to provide an air source for the blowing module 7 . The air source is blown out from the blowing needle 71 arranged on the top of the blowing module 7 through the air flow pipeline 5 .

[0071] like Figure 2As shown, due to the special structure of the flower organ of the Amaryllis odorata, the position of its pollen sac 13 is lower than the stigma 12, so it is difficult for pollen to fall on the stigma 12, and it is difficult to complete self-pollination. In order to solve this problem, in this device, without destroying the structure of the flower, the depth camera 10 is used to accurately identify the various structures of the Amaryllis odorata, and the flexible clamp mechanism 6 is used to clamp the style 14 and move it upward for a distance so that the pollen sac 13 is not buried by the petals. It also provides a possibility for the vibration module 8 to approach the pollen sac 13, so that the vibration module 8 can shake the pollen into the flower groove through the vibration effect between the pollen sac 13, and then use the blowing needle 71 to blow the pollen into the style stigma 12, so as to achieve self-pollination of the Amaryllis odorata, solve the problem that the Amaryllis odorata is difficult to self-pollinate, can achieve accurate and rapid pollination, and the preparation cycle is short and it can be quickly put into use, with a high pollination rate and little influence from the external environment.

[0072] Example 2

[0073] like Figure 3 As shown, this embodiment proposes a pollination method for the pollination problem of Amomum villosum based on the pollination device provided in Example 1, and the method includes the following steps:

[0074] Step S1: Place the robot dog in the area where pollination is required, start the robot dog, measure the acceleration and angular velocity of the robot dog through its built-in inertial sensor, and calculate its position in combination with Kalman filtering;

[0075] Preferably, the step S1 includes:

[0076] Assume the initial coordinates of the robot dog are (x 0 ,y 0 ,z 0 ), the robot dog measures its acceleration and angular velocity through the built-in inertial sensor IMU, and calculates its position in combination with the Kalman filter.

[0077] Assume the acceleration of the robot dog in three-dimensional space is Then measured by the accelerometer in the IMU, the output can be expressed as:

[0078] a i =gsin(θ i )+k i a i +b i

[0079] Among them, a i is the output value measured by the accelerometer, g is the acceleration due to gravity, θ i is the attitude angle (pitch, roll, yaw) of the robot dog, k i is the scale factor of the accelerometer, b iis the bias of the accelerometer.

[0080] Assume that the angular velocity of the robot dog in three-dimensional space is Then measured by the gyroscope in the IMU, the output can be expressed as:

[0081] ω i =ω j +b j ω j +b j

[0082] Among them, ω i is the output value measured by the gyroscope, ω j is the actual angular velocity, k j is the gyroscope scale factor, b j is the bias of the gyroscope.

[0083] Then the angular velocity is obtained

[0084] The Kalman filter equation combined with the data from the accelerometer and gyroscope can be used to estimate the position of the robot dog through integration. The formula is as follows:

[0085]

[0086]

[0087] in, is the position vector, is the velocity vector, is the acceleration vector.

[0088] The above method can accurately calculate the exact position of the robot dog itself, thus laying the foundation for the subsequent establishment of a coordinate system to locate the target flower. The accurate coordinates of the target flower can be obtained, thereby achieving precise positioning.

[0089] Step S2: Using a model predictive control algorithm to control the posture of the robot dog;

[0090] Preferably, in step S2, the robot dog realizes its movement through its built-in control system and electrical drive, and uses advanced control algorithms such as model predictive control (MPC) to realize posture control. Then, the position coordinate system is established by establishing a kinematic model to obtain the fuselage coordinate system and the world coordinate system, wherein the fuselage coordinate system {b} is located at the center of mass of the robot dog, the positive direction of the x-axis is the forward direction of the robot dog, the positive direction of the z-axis is perpendicular to the plane of the robot dog body and vertically upward, and the y-axis is determined by the right-hand rule; the world coordinate system {w} is located in the vertical direction of the fuselage coordinate projection on the ground, and the distance to the fuselage coordinate system {b} is h.

[0091] By establishing the body coordinate system and the world coordinate system according to the different relative positions of the robot dog, when the robot dog's posture is controlled through the model predictive control algorithm, the robot dog's movement route is more precise and the target flower position reached is more accurate.

[0092] Step S3: Use the D* algorithm to plan the path of the robot dog;

[0093] Preferably, in step S3, the robot dog uses its own five sets of cameras and infrared sensing detectors to achieve 360-degree perception of the surrounding environment without blind spots. The collected terrain information is combined with its own position coordinates to plan an optimal path to the area to be pollinated. Step S3 specifically includes the following steps:

[0094] Step S31: Using the robot dog's five cameras and infrared sensing detectors to sense the surrounding environment at 360 degrees without blind spots, the collected three-dimensional terrain map is discretized into a three-dimensional grid structure, each small cube unit is set as a node, and the connection relationship between nodes is determined, that is, the neighboring nodes Neighbors (n). Then the robot dog's starting position (s x ,s y ,s z ) is set as the starting node s, and the desired destination (i.e., the area to be pollinated) (g x ,g y ,g z ) is set as the target node g;

[0095] Step S32: According to the starting node s and the target node g, the distance from the starting node to the target node is estimated by starting the function; the heuristic function formula is the Manhattan distance formula, which is as follows:

[0096] h(n)=|s x -g x |+|s y -g y |+|s y -g y |

[0097] Step S33: Take out the node n with the smallest evaluation function f(n) value from the priority queue OPEN; the evaluation function f(n) = g(n) + h(n), where g(n) is the continuously updated actual path cost from the starting node s to the node n, and h(n) is the heuristic estimated cost from the node n to the target node g.

[0098] Step S34: For each adjacent node m ∈ Neighbors(n) of the retrieved node n, calculate the movement cost c ∈ (n, m) from node n to the adjacent node m. In this embodiment, simply set the movement cost between adjacent nodes without obstacles and with little terrain difference to 1, and for special terrains, a value greater than 1 can be set according to the actual situation.

[0099] Then calculate the new path cost t = g(n) + c(n, m) according to the movement cost;

[0100] Step S35: Compare the new path cost t with the path cost g(m) of the adjacent node m,

[0101] If t < g(m), it means that the path cost to reach m through node n is smaller, then update g(m) = t, indicating that a better path to reach m is found;

[0102] If m is not in the OPEN queue, then put m into the OPEN queue. Since its path cost has changed and needs to be re-evaluated, repeat steps S31 - S35 until the start node is retrieved or the OPEN queue is empty.

[0103] If the start node s is retrieved, it means that the optimal path from the start node s to the target node g has been found; if the OPEN queue is empty, it means that there is no feasible path. At this time, an alarm message can be provided through the alarm device 3, and the robot dog will be re - placed and the path will be re - planned.

[0104] In this application, by using the camera and infrared sensing detector built in the robot dog to sense the surrounding environment, and discretizing the collected three - dimensional topographic map into a three - dimensional grid structure, then according to the relevant nodes, the D* algorithm can be selected to calculate and plan the optimal forward route of the robot dog, so that the pollination device can accurately reach near the target flower and be ready to complete the pollination work.

[0105] Step S4: When the robot dog moves to near the target flower according to the planned path, use the improved YOLOv8 to detect and identify the target flower of Amomum villosum flower and perform the first positioning on the target flower;

[0106] Preferably, in the step S4, first use a depth camera to collect the images of Amomum villosum flower and the images of the structures of each part of its flower organs, then optimize YOLOv8 in the feature extraction part through the improved depth - separable convolution and attention mechanism, and then input the images collected by the depth camera into the improved YOLOv8 model for effective training to obtain a network model for the flower shape of Amomum villosum flower, so as to improve the accuracy and speed of detecting Amomum villosum flower;

[0107] Then perform the first positioning on the target flower, including the following steps:

[0108] Step S41: The depth camera eliminates the lens distortion on the image by using its internal parameters for distortion correction to obtain a distortion-free image, and obtains the camera's internal parameter data from the configuration file of the color frame or depth frame, including the focal length f x 、f y , optical center coordinates ppx, ppy, and image width and height, etc.;

[0109] Step S42: For each pixel point, convert its depth pixel value into an actual depth value Z, and the formula is as follows:

[0110] Z=depth pixel *depth scale

[0111] Among them, depth pixel is the depth pixel value of the pixel in the depth frame; depth scale is the depth scale factor, which is used to convert the depth pixel value to the actual physical distance;

[0112] Step S43: Obtain pixel point P according to the camera's intrinsic parameter data and the actual depth value s The three-dimensional coordinates (x s ,y s ,z s ), whose formula is as follows:

[0113]

[0114] z s =Z

[0115] Among them, (px,py) is the coordinate of the pixel point in the image coordinate system, ppx and ppy are the coordinates of the optical center, and f x 、f y is the focal length, Z is the depth value calculated by the above method;

[0116] Step S44: Combine the rotation matrix R and the translation vector T of the robot dog to form a transformation matrix M, where the rotation matrix R is a 3*3 matrix and the translation vector T = [T X ,T Y ,T Z ] T , the transformation matrix is ​​a 4*4 matrix, where:

[0117]

[0118] Then use the transformation matrix to obtain the pixel point P s =(x s ,ys ,z s ,1) T Point P converted to the world coordinate system w =(x w ,y w ,z w ,1) T , so as to realize the first positioning of the target flower. The conversion formula is as follows:

[0119] P w =M*P s

[0120] Thus, with the help of the path planning algorithm, the pollination device moves to the vicinity of the target flower, and with the help of the depth camera to collect information, the custom feature extraction module in the improved YOLOv8 is used to effectively train the model for the flower type of the Amaryllis japonica, and the feature extraction part of YOLOv8 is optimized by the improved depth separable convolution and attention mechanism, thereby improving the accuracy and speed of detecting the Amaryllis japonica. After detecting the Amaryllis japonica, the world coordinates of the target flower are determined according to the pixel points of the target flower image taken by the depth camera, thereby realizing the first positioning of the target flower, and the pollination device reaches a more accurate position of the target flower according to the coordinate result of the first positioning, preparing for pollination.

[0121] Step S5: using a depth camera to perform a second positioning of the nearest target flower, identifying the various structures of the target flower, and obtaining the world coordinates of the various structures of the target flower;

[0122] Specifically, when the robot dog reaches the target area, it uses the FCN semantic segmentation algorithm to specifically identify the structure of each part of the target flower through the depth camera of the nearest flower, such as Figure 2 As shown, the petals 11, stigma 12, pollen sac 13, and style 14 of the Amaryllis japonica are identified;

[0123] After the recognition is completed, the method of first positioning the target flower is used to perform a second positioning of the target flower structure, and the coordinates of each structure in the world coordinate system are obtained. Assume that the coordinates of the petal 11 in the world coordinate system are (x whb ,y whb , z whb ), the coordinates in the image coordinate system are (x shb ,y shb , z shb ), similarly, the coordinates of the column head 12 are (x wzt ,y wzt , x wzt ), (x wzt ,y wzt , zwzt ); the coordinates of pollen sac 13 are (x whf ,y whf , z whf ), (x whf ,y whf , z whf ); the coordinates of style 14 are (x whz ,y whz , z whz ), (x whz ,y whz , z whz ), and then use the above conversion formula to calculate the coordinates of each part of the target flower structure in the world coordinate system, so that the pollination device can accurately distinguish the various parts of the target flower structure and achieve accurate pollination.

[0124] Step S6: performing a pollination operation on the target flower according to the acquired world coordinates of each part of the target flower;

[0125] Specifically, step S6 includes:

[0126] Step S61: The robot arm 2 establishes a relative coordinate system for positioning according to the relevant coordinates of the recorded flower, and then locates the flower according to P d =N -1 *P w Calculate by inverse transformation matrix N -1 Put the object in the world coordinate system P w The position in is converted to the relative coordinate system P d The relative coordinate system coordinates of the petal 11, stigma 12, pollen sac 13, and style 14 of the target flower are calculated as (x xhb ,y xhb , z xhb ),(x xzy ,y xzt , z xzt ),(x xhf ,y xhf , z xhf ),(x xhz ,y xhz , z xhz ); then suspend the flexible clip mechanism 6, the vibration module 8 and the blowing module 7 above the target flower;

[0127] Step S62: After receiving the clamping instruction, the flexible clamp mechanism 6 clamps the style 14 according to the relative coordinates (x xhz ,y xhz , z xhz), calculate its strength and opening and closing angle, and then drive the motor to clamp the style 14 and move it upward for a distance so that the pollen sac 13 is not buried by the petals; specifically including:

[0128] Since the flexible clamp mechanism 6 is composed of a motor system and a transmission system, let f be the input voltage frequency (Hz) and p be the number of magnetic pole pairs, the speed can be obtained as Let T be the motor torque (N*m), k T is the torque coefficient, so we can get T = k T n; Assuming the total transmission ratio of the transmission mechanism is i, the mechanical efficiency is η, and the distance from the transmission shaft of the clamp of the flexible clamp mechanism 6 to the contact of the style 14 is L, the clamping force generated at the clamp contact point after the motor output torque is transmitted is Furthermore, in order to ensure that the clamped style 14 is not damaged, the pressure of the clamp on the style 14 is set to P, which cannot exceed the ultimate compressive strength P that the style 14 can withstand. max , the area of ​​the clip and the style 14 is A, then

[0129] Next, assume that the lengths of the two arms of the clamp are L 1 and L 2 , the distance from the connection point to the driving point is L 3 , the driving displacement is x, then according to the cosine theorem, the opening and closing angle can be estimated as:

[0130]

[0131] Finally, the motor is driven to clamp the style 14 and move it upward for a certain distance, so that the pollen sac 13 is not buried by the petals 11.

[0132] Step S63: The turntable module 9 drives the vibration module 8 and the blowing module 7 to be located at appropriate positions by rotating, and the pollen is shaken to the flower groove by the vibration effect between the vibration module 8 and the pollen sac 13. The blowing module 7 generates gas through the air compressor 4 and uses the airflow duct 5 to transport the airflow, and the pollen is blown into the top of the style 14 through the blowing needle 71 to complete a pollination.

[0133] Preferably, the turntable module 9 drives the vibration module 8 and the blowing module 7 to be located at appropriate positions by rotating. Specifically, the vibration module 8 is located above the pollen sac at a position coordinate of (x xzt ,y xzt , z xzt +Δz); the air blowing module 7 is located above the middle of the style 14 and the pollen sac 13, and is represented by the following formula:

[0134] Next, the vibration frequency of the vibration module 8 is set to fz , suppose that the expected frequency under the premise that the pollen in the pollen sac 13 is successfully shaken off by the vibration module 8 and the structure of the pollen sac 13 is not damaged is f q The pulse signal generated by the PWM module affects its period and finally affects the vibration frequency f. z Assume that the clock source of the microcontroller PWM module is the system clock, the pre-division factor is set to 1, the period register is PERIOD, and the duty cycle register is DUTY.

[0135] The calculation formula of PWM period is: Among them, T clk is the clock period of the PWM module, f clk is the clock frequency of the PWM module, and the duty cycle is calculated as

[0136] Assume the wind speed of the blowing module is V, and the desired wind speed is V q ,in, V q Wherein, Q is the air volume, and A is the cross-sectional area of ​​the outlet of the air blowing needle 71. Wherein, d is the diameter of the outlet of the blowing needle 71 .

[0137] Step S64: After the pollination device completes one pollination, it continues to complete pollination according to the number of petals. The rotating disk module 9 rotates to place the vibration module 8, the blowing module 7 and the blowing needle 71 in the appropriate position again, thereby completing the pollination at the same coordinate P w =(x w ,y w ,z w ) T Pollination work of different petals nearby.

[0138] After completing the positioning of the target flowers, the robotic arm of the pollination device begins to perform pollination work, and through the cooperation of various modules of the pollination device, efficient and accurate self-pollination of the Amaryllis flower is achieved.

[0139] Step S7: After the pollination is completed, the pollination device is reset, and then steps S2-S6 are repeated to complete the pollination task of the next target flower.

[0140] Preferably, in step S7, Figure 1 As shown, the robot arm 2 in this embodiment has three joints. The homogeneous transformation matrix A between adjacent links can be obtained by the DH parameter method. i (i=1,2,3), the total transformation matrix from the base coordinate system to the end effector coordinate system is T: T=A 1 A 2 A3 . Wherein, the homogeneous transformation matrix A i The formula is:

[0141]

[0142] Among them, θ i is the joint angle of joint i, α i is the torsion angle of the connecting rod, o i is the length of the connecting rod, d i is the offset between the connecting rods.

[0143] At the same coordinate P w =(x w ,y w ,z w ) T After the pollination of different petals nearby is completed, and the robotic arm 2 and the pollination device are reset, the robot dog detects the surrounding environment again through the depth camera 10, finds the next area to be pollinated, and completes the identification and positioning of the next target flower again through steps S2-S6. At the same time, its position is determined through path planning, and it moves to the target area, ensuring that the pollination task of the next target flower can be carried out in time, thereby improving the pollination efficiency.

[0144] In the present application, the data measured by the inertial sensor built into the mechanical dog is combined with the Kalman filter to calculate the position of the mechanical dog itself, and then the D* algorithm is used to plan the path of the mechanical dog so that the pollination component carried by the mechanical dog can accurately reach a certain range near the target flower, and then the target flower is identified and located twice by using a depth camera to accurately identify and locate the coordinates of the various parts of the flower organ of the Amomum vulgare, so that the pollination device can accurately perform self-pollination operations on the Amomum vulgare, ensuring the effective transfer of pollen and improving the success rate of pollination, thereby avoiding omissions or unevenness that may occur in traditional artificial pollination, reducing human intervention, and improving pollination efficiency. Through automated pollination, the dependence on natural pollinators (such as bees and other insects) is reduced, which helps to maintain crop growth and yield when the number of pollinators is insufficient or the ecological environment is disturbed, and is suitable for large-scale agricultural production.

[0145] Example 3

[0146] like Figure 4 As shown, this embodiment also provides a pollination device of another structure, the device comprising:

[0147] A mechanical dog, which is set at the bottom of the pollination device, is used to control the moving direction and route of the device;

[0148] The expansion dock 24 and the laser radar 25 are arranged above the robot dog. The laser radar 25 can be used to detect obstacles, and the expansion dock 24 can be used to expand the robot arm 2;

[0149] A mechanical arm 2, mounted on the expansion dock 24, for receiving pollination signals and executing pollination instructions;

[0150] The end effector 15 is connected to the end of the robot arm 2 and is used to aim at the target flower;

[0151] A depth camera 10 is connected to the end effector 16 and is used to collect environmental information and collect images of the sand flower;

[0152] The flexible clamp 16 is connected in front of the depth camera 10 and is used to pollinate the Amaryllis japonica detected by the depth camera 10 .

[0153] Preferably, if Figure 4 As shown, the flexible clamp 16 includes a gripper 17 and a turntable 18, wherein the turntable 18 and the gripper 17 are arranged on the same bearing 20, a rotating bearing 26 is arranged at the center of the turntable 18 for controlling the rotation of the turntable 18, and the turntable 18 is located above the gripper 17; goose down 19 is arranged on the outer ring of the turntable 18 and inside the gripper 17, as the turntable 18 rotates, pollen adheres to the surface of the goose down 19, and at the same time, the gripper 17 moves the pollen upward along the style 14 until the goose down 19 of the turntable 18 delivers the pollen to the top of the style 14, thereby completing pollination.

[0154] In the device provided in this embodiment, the flexible clamp 16 can complete an embracing grip on the stigma 14 in the stamen, and then the turntable 18 is rotated to allow the pollen on the pollen sac 13 under the stigma of the pistil to adhere to the goose down 19 of the turntable 18, and then, driven by the robotic arm 2, the turntable 18 with the pollen attached is gently swept across the top of the stigma 14 in the stamen using the "push-pull method", thereby completing autonomous pollination, filling the technical gap in the robotic arm's pollination of Amomum villosum, and effectively increasing the yield of Amomum villosum.

[0155] Example 4

[0156] According to another pollination device of embodiment 1-2 and embodiment 3, Figure 6 As shown, another pollination method is also provided in this embodiment, the method comprising:

[0157] Step A1: Start the robot dog and initialize it, then start the depth camera 10, and use the depth camera 10 to collect environmental information in real time for path planning; wherein, in this embodiment, the depth camera 10 is RealSenseD435i.

[0158] Specifically, step A1 includes:

[0159] Step A11: Use the built-in inertial measurement unit (IMU) and odometer to determine the initial position coordinates (x 0 ,y 0 ,z 0 ), set the initial velocity to zero, and use the inertial measurement unit to measure the acceleration a of the robot dog on the x, y, and z axes x 、a y 、a z and angular velocity ω x ,ω y ,ω z ;

[0160] Step A12: Based on the improved A* algorithm, the robot dog uses the terrain information obtained by the depth camera 10 and the five cameras and infrared sensing detectors it carries with it, and then calculates the path to the approximate area of ​​flower distribution determined by the pre-scan based on its own position; specifically, it includes:

[0161] The robot dog determines the terrain type by image analysis by presetting characteristic thresholds of different terrains, and then calculates the path to the approximate area of ​​flower distribution determined by the pre-scan in combination with its own position;

[0162] Assume the current position of the robot dog is (X n , Y n , Z n ), the center position of the target area is (X g , Y g , Z g ), the target area is the approximate area of ​​flower distribution determined by the pre-scan. In the A* algorithm, the evaluation function of the node is f(n)=g(n)+h(n), where the actual path cost g(n) from the starting node s to the node n is calculated by taking into account the terrain cost C t (n), for example, the base value of the moving cost of ordinary ground is set to 1, the cost of soft land is set to 3, the cost of uneven area is set to 5, and the distance cost C d (n) and other costs of the robot dog's movement plus the turning cost C s (n) to express it, we can get:

[0163]

[0164] The distance cost can be calculated using the Euclidean distance formula:

[0165]

[0166] Where (X n-1 , Y n-1, Z n-1 ) is the coordinate of the previous node before reaching node n.

[0167] The heuristic estimated cost h(n) from the current node n to the target node g combines the distance and flower density information. The flower density ρ is obtained by scanning the surrounding small area in real time with a depth camera to estimate the number of flowers and the ratio of the area. Then, we can get:

[0168]

[0169] Among them, k is the weight coefficient, which is used to adjust the impact of flower density on the heuristic cost. The specific value can be determined according to the actual situation. For example, the optimal value can be determined by experimentally comparing the path planning effects under different values.

[0170] During the search process, the node with the smallest evaluation function value is selected for expansion, that is, As the next node to be searched, the path to the approximate area of ​​flower distribution determined by the pre-scan is calculated.

[0171] Step A13: The robot dog moves along the planned path and updates its own position coordinates according to the inertial measurement unit and odometer data at regular intervals during the movement.

[0172] Specifically, during the movement, the time interval can be set to 0.5 seconds, and then the position coordinates are updated according to the IMU and odometer data. The position update formula is as follows:

[0173] v x (t+Δt)=v x (t)+a x Δt

[0174] v y (t+Δt)=v y (t)+a y Δt

[0175] v z (t+Δt)=v z (t)+a z Δt

[0176] At the same time, the depth camera 10 is used to continuously monitor the environment. If an obstacle is found, the movement is immediately stopped, and the route is replanned to bypass the obstacle and then continue to move forward. The obstacle is an object detected by an image recognition algorithm and matches a preset obstacle model.

[0177] Therefore, in this embodiment, the robot dog performs path planning through the improved A* algorithm, can realize autonomous navigation in a complex flower field environment, avoid obstacles, and find the optimal path, thereby achieving accurate and efficient autonomous pollination.

[0178] Preferably, in step A2, the depth camera 10 is used to collect images in real time while the robot dog moves along the planned route, and the target flower is detected and located by a target detection algorithm;

[0179] Specifically, step A2 includes:

[0180] Step A21: The target flowers are detected using the improved YOLOv5 algorithm. When training the YOLOv5 algorithm, the network structure is optimized according to the morphological characteristics of the Amaryllis odoratus, and the attention layer for key features such as the style and pollen sac is added. The YOLOv5 algorithm is improved through data enhancement techniques (such as random illumination changes, angle rotation, etc.), such as adding an attention mechanism, thereby improving the generalization ability of the model and making the detection rate of the target flowers higher.

[0181] Step A22: Use the three-dimensional reconstruction algorithm to calculate the distance between the target flower and the depth camera, and then combine the camera's intrinsic parameter matrix and the pixel coordinates of the target flower on the image plane to calculate the coordinates of the target flower in the camera coordinate system. Then, use the pre-calibrated rotation matrix and translation vector of the camera relative to the robot dog to convert the coordinates of the target flower in the camera coordinate system into coordinates in the world coordinate system where the robot dog is located.

[0182] Specifically, when the target flower is detected, the depth camera 10 emits infrared light and sets the camera baseline length to b. The camera baseline length represents the distance between two infrared light emitting points or receiving points, which is a known parameter of the camera. The angles formed by the two infrared light receiving points and the object are α and β, respectively. These two parameters are measured by the optical system and sensor inside the camera.

[0183] Then, the distance between the object and the camera, that is, the depth value, is calculated according to the 3D reconstruction algorithm. The calculation formula is:

[0184]

[0185] Where, f is the focal length of the camera, (x 1 ,y 1 ) and (x r ,y r ) are the pixel coordinates of the image of the object at the left and right infrared light receiving points, which are obtained through the camera image sensor.

[0186] Assume that the center of the camera image plane is the coordinate origin (0, 0), the horizontal direction of the image plane is the x-axis, and the vertical direction is the y-axis. Then, the camera intrinsic parameter matrix k is obtained by camera calibration. The form of the intrinsic parameter matrix k is:

[0187]

[0188] Among them, f x and f y are the focal lengths in the x and y directions, c x and c y are the coordinates of the center of the image plane.

[0189] Assume that the pixel coordinates of the target flower on the image plane are (u, v), and combined with the depth value d calculated by the depth camera, we can get the pixel coordinates of the target flower in the camera coordinate system (X c , Y c , Z c ) is calculated by:

[0190] Z c =d

[0191]

[0192] Then, the coordinates are transformed to the world coordinate system where the robot dog is located (X w , Y w , Z w ), where the rotation matrix R is a 3×3 matrix, which represents the rotation relationship of the camera in the robot dog coordinate system and is obtained by pre-calibration; the translation vector t is a 3×1 vector, which represents the translation relationship of the camera in the robot dog coordinate system and is also a calibration parameter. The calculation formula for converting the coordinates of the target flower in the camera coordinate system to the coordinates in the world coordinate system where the robot dog is located is:

[0193]

[0194] Therefore, in this embodiment, the improved YOLOv5 algorithm can accurately detect the target flowers in a complex environment with a high recognition rate and a low false detection rate. In addition, through the depth camera and three-dimensional reconstruction technology, accurate distance measurement of the target flowers can be achieved, so that the spatial coordinates of each flower in the world coordinate system where the robot dog is located can be accurately calculated, so that the pollination device can perform precise operations on the flowers.

[0195] Preferably, in step A3, when the robot dog reaches within a certain range of the detected target flower, the robot arm 2 installed on the robot dog is activated and the target flower is re-positioned using the depth camera 10, thereby obtaining accurate position information of the flower relative to the robot arm 2; wherein, in this embodiment, the certain range of the target flower is set as a circular area with a radius of 1 meter with the target flower as the center according to the working distance and accuracy of the robot arm 2, and when the robot dog reaches within the circular area with a radius of 1 meter from the center of the target flower, it triggers a signal to activate the pollination robot arm.

[0196] Specifically, step A3 includes:

[0197] Step A31: After receiving the activation signal, the robot arm 2 performs an initialization operation through the joint angle sensor carried by itself to measure the initial joint angle vectors of all joints of the robot arm;

[0198] Assume that the robot arm has N joints, and the initial angle of the i-th joint measured by the joint angle sensor is θ i0 (i=1,2,…,N), then the initial joint angle vector of the robot arm is

[0199] Step A32: Use the depth camera to obtain an image of the target flower, and then match the feature point set of the target flower on the camera image plane with the pre-stored standard feature point module of Amaryllis japonica based on the target positioning algorithm of feature point matching. Suppose the feature point set of the target flower on the camera image plane is P = {p 1 , p 2 , ..., p m}, each feature point p j With coordinates (u j , v j ), j = 1, 2, ..., m; then by comparing with the pre-stored Yangchun sand flower standard feature point template T = {t 1 , t 2 , ..., t m} for matching, where the coordinates of the feature points of the standard template are (u j ′,v j ′);

[0200] Then the matching error is calculated, and the formula for calculating the matching error is:

[0201]

[0202] In this embodiment, the threshold e is set according to the actual positioning accuracy requirement. threshold Set to 5 pixels, when e is less than the set threshold e threshold If the error is less than 5 pixels, it means that the target flower is successfully located. Otherwise, the matching is repeated and the error is recalculated for judgment.

[0203] Step A33: After successful positioning, the coordinates of the target flower in the camera coordinate system are calculated based on the camera's intrinsic parameter matrix and depth information, where the depth information is the target flower feature point p j The corresponding depth value d j ;

[0204] For feature point p j, its coordinates in the camera coordinate system (X cj , Y cj , Z cj ) is calculated by:

[0205] Z cj =d j

[0206]

[0207] Among them, c x , c y , f x , f y are the parameters in the camera intrinsic parameter matrix k.

[0208] Then, the coordinates of the target flower in the camera coordinate system are transformed into the coordinate system of the robot base through the pre-calibrated transformation matrix M of the coordinates of the camera relative to the robot base, wherein the transformation matrix M is a 4×4 matrix containing rotation and translation information, which is obtained through calibration;

[0209] Then let the transformed coordinates be (X bj , Y bj , Z bj ), the precise position information of the target flower relative to the base of the robotic arm can be obtained by combining the above parameters, where the calculation formula is:

[0210]

[0211] Therefore, after the robotic dog in this embodiment reaches a certain position range of the target flower, it activates the robotic arm through an activation signal to prepare to perform the pollination task, and uses the depth camera again to more accurately locate the target flower. By converting the coordinates of the target flower in the camera coordinate system to the robotic arm base coordinate system, more accurate target flower position information is obtained, thereby further improving the accuracy of autonomous pollination and improving the pollination effect.

[0212] Step A4: When the robot arm approaches the flower, it moves toward the target flower at a speed of 0.1 m / s, while continuously using the depth camera 10 and the laser radar 25 for real-time monitoring. The depth camera acquires images at a frame rate of f=30 frames per second, and the position of the target flower is determined in each frame image by the target monitoring algorithm;

[0213] Suppose the coordinates of the center of the target flower detected in the k-th frame image on the image plane are (u k , v k ), calculate the real-time position (X) of the target flower in the coordinate system of the robot base according to the camera calibration parameters and the previously calculated depth information of the target flower in the coordinate system of the robot base. bk(k), Y nk (k), Z bk (k)), the calculation method is consistent with the above-mentioned coordinate conversion formula for converting the coordinates of the target flower in the camera coordinate system to the coordinate system of the robot base, and will not be repeated here.

[0214] Then, the joint angle adjustment of the robot arm is calculated based on the real-time position of the flower and the current position of the end effector, so as to fine-tune the position and posture of the robot arm and finally confirm the position of the flower.

[0215] Specifically, the coordinates of the current position of the end effector of the robot arm in the robot arm base coordinate system are (X ek , Y ek , Z ek ), the coordinates are calculated by the joint angles and kinematic model of the robot; then the kinematic model of the robot is assumed to be in, is the position coordinate of the end effector of the robot arm in the base coordinate system; is the robot joint angle vector; f is the robot kinematic function, which represents the relationship between the joint angle and the end effector position; then the kinematic model is inverted to obtain Then the calculation formula of the joint angle adjustment amount is obtained as follows:

[0216]

[0217] Where f is the kinematic function of the robot arm, which represents the relationship between the joint angle and the position of the end effector; is the current joint angle vector of the robotic arm at the kth frame image.

[0218] Preferably, the step A4 further includes: in the process of the robot arm 2 controlling the movement of each joint to achieve fine adjustment of the position and posture according to the calculated joint angle adjustment amount, the distance between the end effector and the target flower is calculated in real time, and the distance is compared with the preset precise operation distance. In this embodiment, the preset precise operation distance d is calculated. precision Set to 10 cm. If it is less than the preset precise operating distance, stop fine-tuning, complete the final confirmation of the flower position, and prepare for pollination; otherwise, continue fine-tuning until the calculated distance between the end effector of the robot arm and the target flower is less than the preset precise operating distance.

[0219] The calculation formula of the distance between the end effector 15 and the target flower is:

[0220]

[0221] Among them, (X ek , Yrk , Z ek ) represents the coordinates of the current position of the end effector 15 in the robot arm base coordinate system; (X bk (k), Y bk (k), Z bk (k)) represents the real-time position of the target flower in the coordinates of the robot base.

[0222] In the process of the robotic arm approaching the flower in this embodiment, the joint angle adjustment amount of the current position of the robotic arm end effector 15 relative to the real-time position of the target flower is calculated by the above formula, and the position and posture of the robotic arm 2 can be further fine-tuned until the distance between the robotic arm end effector 15 and the target flower is less than the preset precise operating distance, thereby ensuring that the robotic arm end effector 15 can obtain pollen from the anthers of the stamens and can accurately release the pollen onto the stigma above the anthers, thereby achieving precise pollination operations, ensuring that the pollen is effectively transferred to the style, improving the pollination success rate, and thereby improving the yield and quality of Amomum villosum.

[0223] Step A5: Identify and locate the position of the style 14, then use the robotic arm 2 to control the transmission bearing to rotate at a constant speed to drive the turntable 18 to make the pollen adhere to the goose down 19 of the turntable 18, and at the same time use the flexible clamp 16 to clamp the style 14 and send the pollen on the turntable 18 to the top of the style 14 to complete pollination.

[0224] Specifically, the following steps are included:

[0225] Step A51: After the robot arm 2 completes the final confirmation of the flower position, the depth camera 10 is used to perform secondary positioning and identification on the style of the target flower in a static position. After the depth camera 10 obtains the image containing the style, it performs image preprocessing, including graying, filtering (such as Gaussian filtering to remove noise), and other operations. The preprocessed image is set to I.

[0226] Then, an image analysis algorithm based on edge detection and shape analysis is used to identify the unique morphological features of the style. Edge detection is performed on image I, such as using the Canny edge detection algorithm to obtain an edge image E. In edge image E, straight line segments are detected by a shape analysis algorithm, such as Hough transform. Combined with the prior knowledge of the shape of the style of Amomum villosum, the style is usually slender, and constraints such as the length of the straight line segment and the angle range can be set to detect possible style edge segments. Suppose the set of detected style edge segments is L = {l 1 , l 2 , ..., l q}, for each line segment l i , whose endpoint coordinates on the image plane are (x i1 ,y i1 ) and (xi2 ,y i2 ).

[0227] Step A52: Calculate the central axis of the style according to the detected style edge line segments; assume that the parametric equation of the central axis of the style is y=ax+b, which is for a two-dimensional image. If three-dimensional space is considered, it can be expanded to a plane equation and the like. Then, the parameters a and b are determined by fitting the edge line segment set L by the least squares method. The goal of the least squares method is to minimize the error function Among them, d(l i , y = ax + b) represents the line segment l i The distance to the straight line y=ax+b, calculate the partial derivatives of e(a, b) with respect to a and b respectively and set them to zero, solve the system of simultaneous equations to obtain the values ​​of a and b, and thus determine the equation of the central axis of the style.

[0228] Step A53: According to the central axis of the style and the depth information of the depth camera 10, the position coordinates of the style in the three-dimensional space are determined. Select a number of points on the central axis equation (for example, select p points at equal intervals), and set the coordinates of the selected points on the image plane to (u i , v i ), i = 1, 2, ..., p; the coordinates of these points in the camera coordinate system are calculated through camera calibration and depth information (X ci , Y ci , Z ci ), the coordinate transformation formula is the same as that in the previous step and will not be repeated here.

[0229] Finally, by averaging or weighted averaging these coordinates, the final position coordinates of the style are obtained to ensure the accuracy of the pollination operation. For example, in this embodiment, the weighted average method is used, and the weight W is set i It is related to the distance from the point to the center of the image. The closer the distance to the center of the image, the greater the weight. The weight function can be set according to the specific situation, such as:

[0230]

[0231] Where D is the length of the image diagonal, the final position coordinates of the style are:

[0232]

[0233] Therefore, after the position of the target flower is determined, the pollen can be accurately released into the pistil 14, and the pollen attached to the goose down 19 of the turntable 18 can be gently released onto the pistil using the "push-pull method" under the rotation of the turntable 18, thereby achieving autonomous pollination of the Amomum villosum flower.

[0234] Further preferably, in step A5, the use of a flexible clamp to clamp the pistil includes: after the robot arm 2 receives the clamping instruction, the flexible clamp 16 of the end effector 15 is controlled to start the action. The driving system of the flexible clamp 16 is composed of a motor (such as a DC motor) and a transmission mechanism (such as a gear transmission, a connecting rod mechanism, etc.), and the rotation of the motor is converted into the opening and closing movement of the clamp through the transmission mechanism. Assume that the input voltage of the motor is U, the speed of the motor is n, and the torque of the motor is T. There is a certain relationship between them, which can be determined according to the motor characteristic curve, and can generally be approximately expressed as T=k T n, where k T is the torque coefficient, obtained through experimental tests.

[0235] The opening and closing angle α of the clamp 17 is related to the rotation angle θ of the motor through the transmission ratio i of the transmission mechanism, that is, α=iθ. The closing force F of the clamp 17 is related to the opening and closing angle α of the clamp 17, the structural parameters of the clamp 17 (such as the clamp length L, the clamp width W, etc.) and the elastic modulus E of the clamp material. The relationship can be established through mechanical analysis. For example, a simple approximation is F=k F (α 0 -α), where α 0 is the angle when the jaws are fully opened; k F It is a coefficient related to the structure and material of the gripper and is determined through experiments and mechanical calculations.

[0236] Then the current gripper pressure P is measured in real time by the pressure sensor current , and preset the target closing pressure P of the clamping jaws target In the process of clamping the style, the control system determines the error between the current clamping pressure and the target closing pressure. 1 =P target -P current To adjust the driving signal of the gripper (i.e. the input voltage U of the motor), a simple proportional control algorithm is used, and the driving signal adjustment amount △U=K p e 1 , where K p is the proportional coefficient, which is determined according to the characteristics of the gripper drive system and experiments.

[0237] Thus, the error is continuously updated during the control process until the current clamp pressure approaches the target closing pressure, ensuring that the clamp can firmly grasp the style without damaging it. target =15N (determined according to the bearing capacity of the style and the requirements for grip stability), initial P target =0.5N, if K p =0.5V / NV / N, then the first adjustment value △U = 0.5×(1.5-0.5) = 0.5v. After adjusting the motor input voltage, measure P again. current, repeat the above process until the clamping jaw pressure reaches the target value.

[0238] Then, the pollen is transferred. The robot arm 2 controls the rotating bearing 26 to drive the turntable 18 to move. The rotating bearing 26 is driven by a high-precision servo motor. The control of the servo motor is based on the feedback signal of the encoder inside it to achieve precise position and speed control. Set the target speed n in the control system target =5 rpm, and use the encoder to monitor the actual motor speed n in real time actual .

[0239] Assume the control voltage of the motor is U m , a simple proportional integral (PI) control algorithm is used to adjust the motor speed and control the voltage U m =K p ×(n target -n actual )+K i ×∫(n targer -n actual )dt, where K p is the proportionality coefficient; K i is the integral coefficient, which is determined according to the motor characteristics and experiments. For example, in this embodiment, the speed response curve of the motor under different control voltages is obtained through experimental testing, and K is determined according to the characteristics such as the slope of the curve and the steady-state error. p and K i If K p =0.2V / (rev / min), When n actual =4 rpm, U m =0.2×(5-4)+0.05×∫(5-4)dt=0.2+0.05t, where t is time. As time increases, the integral term gradually increases, causing the motor speed to approach the target speed faster.

[0240] Then, the goose down 19 covering the turntable 18 collects pollen in the pollen sac through its softness and adsorption. As the turntable 18 rotates, the pollen adheres to the surface of the goose down 19. In order to ensure the pollen collection effect, the material and density of the goose down can be optimized. Assume that the adsorption force F of the goose down is a It is related to factors such as the fiber length l, fiber diameter d, and density ρ of goose down (an empirical formula can be established through experiments, such as F a =k a l 2 ρd, where k a is the adsorption coefficient, which is determined through experiments. According to the characteristics of pollen and collection requirements, appropriate goose down parameters are selected to make F a Can effectively absorb pollen.

[0241] The pollen is transported upward to the style above the pollen sac. During the rotation of the turntable 18, the gripper 17 slightly moves the pollen upward along the style at a relatively slow and uniform speed. In this embodiment, the target speed is set to v target = 3 mm / s. The position and speed of the gripper are precisely controlled by the robot 2, and the control system of the robot 2 generates a control signal according to a preset movement trajectory (such as straight rise along the central axis of the style) and speed parameters.

[0242] The encoder (or other position sensor) at the joint of the robot arm is used to obtain the current position information of the gripper 17. The current position of the gripper 17 is x. currena , by comparing with the target position (determined according to the style length and pollination position requirements, such as 5 mm below the top of the style as the target position x target ) to compare and calculate the position error e x =x target -x current At the same time, a visual sensor (such as a small camera mounted on a robotic arm) or a distance sensor (such as an ultrasonic sensor) is used to monitor the relative position of the gripper 17 and the style 14 in real time to obtain the distance h between the gripper 17 and the surface of the style 14 in the vertical direction.

[0243] Assume that the driving signal of the control system to the gripper 17 is U g , according to the speed error e v =v target -v current (where v current The current speed of the gripper is measured or calculated by the sensor) to adjust the gripper's drive signal. A similar proportional control algorithm is used. The drive signal adjustment amount △U g =K pv ×e v (where K pv is the speed control proportional coefficient, which is determined according to the gripper drive system characteristics and experiments). For example, if K pv =0.3V / (mm / s), when v current =0.2 mm / s, △U g =0.3×(3-2)=0.3V, adjust the drive signal to make the gripper speed approach the target speed. At the same time, fine-tune the gripper position according to the change of distance h. If h deviates from the preset ideal distance (such as h ideal =2 mm), the position of the gripper 17 is corrected by adjusting the joint angle of the robotic arm 2 to ensure that the pollen is transported upward steadily until the goose down 19 of the turntable 18 delivers the pollen to the top of the style.

[0244] Then the pollen needs to be released to complete pollination. The turntable moves to the top of the style 14 through a preset position sensor or visual recognition, and the turntable is controlled to rotate in the opposite direction. Set the motor reversal time t reverse = 1 second. During the reversal process, the motor speed change is controlled by a trapezoidal speed curve. Assume that the initial motor speed is n start = 0 rpm, the acceleration phase time is t acc = 0.3 seconds, the speed at the end of the acceleration phase is n max =10 rpm, the deceleration time is t dec = 0.3 seconds, then the uniform speed stage time t const =t reverse -t acc -t dec = 0.4 seconds;

[0245] During the acceleration phase, the calculation formula for the motor speed n(t) is:

[0246] In the uniform speed stage n(t) = n max (t acc <t≤t acc +t const );

[0247] In the deceleration phase

[0248] Through such speed control, the motor gradually increases to a higher value during the reversal process, and then reverse At the end of the time, it gradually drops to zero, achieving the effect of slight shaking at the top of the style, ensuring that the pollen is effectively released and accurately reaches the top of the style.

[0249] After the pollination operation is completed, the pollination robot arm returns to its initial state according to the preset reset path. The preset reset path can be that the robot arm joints move back to the initial position in sequence according to a specific angle sequence. Suppose the robot arm joint angle vector is The initial joint angle vector is During the resetting process, each joint moves according to a certain speed curve (such as a linear speed curve or an exponential speed curve, etc., which is selected according to the requirements of the smoothness of the robot movement). For example, if a linear speed curve is used, the angle change formula of joint i is θ i (t) = θ i0 +v θi (tt start ), where v θi is the resetting speed of joint i, t start It is the reset start time until all joint angles return to their initial values, ready for the next pollination task.

[0250] After pollination is completed, the robot dog detects the surrounding environment again through the depth camera. The depth camera 10 obtains point cloud data of the surrounding environment and processes and analyzes the data. First, the data is filtered (such as statistical filtering to remove outliers, voxel filtering to reduce data density, etc.) to obtain more accurate environmental point cloud information.

[0251] Then, the target detection algorithm (such as a target detection network based on deep learning or a traditional target detection algorithm based on geometric features) is used to search for possible target flowers in the point cloud data. Suppose the point cloud set of the detected target flower is C = {c 1 , c 2 , ..., c q}, each point c j With three-dimensional coordinates (x j ,y j , z j ), calculate the center position of the point cloud set

[0252] According to the center position of the target flower and the current position of the robot dog (obtained through the robot dog's own positioning system, set it as (X d , Y d , Z d ))Distance

[0253]

[0254] and the working range of the robot dog (set as R work , determined by the mechanical structure and movement ability of the robot dog) to evaluate whether there is a suitable next pollination target. cd ≤R work , it is considered that there is a suitable target, and the robot dog moves to the vicinity of the target flower and repeats the above target flower identification and subsequent pollination steps; if d cd >R work , it performs corresponding operations according to the preset instructions, such as returning to the charging area after completing the pollination task of the entire flower field (achieved through the pre-set charging area position coordinates and path planning algorithm) or waiting for new instructions (communicating with the external control system to obtain new instructions).

[0255] Therefore, in this embodiment, the device proposed in Example 3 provides another pollination method for Amaryllis japonica, which realizes precise pollination operation, ensures the effective transfer of pollen, and improves the success rate of pollination. It can also avoid omissions or unevenness that may occur in traditional artificial pollination, reduce human intervention, and reduce dependence on natural pollinators (such as bees and other insects) through automated pollination. It helps to maintain crop growth and yield when the number of pollinators is insufficient or the ecological environment is disturbed, and is suitable for large-scale agricultural production.

[0256] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the technical solution of the present invention, and are not intended to limit the specific implementation methods of the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the claims of the present invention shall be included in the protection scope of the claims of the present invention.

Claims

1. A pollination device for the pollination problem of Amomum villosum, characterized in that: The device comprises: A chassis, the chassis is arranged above the back of the mechanical dog, and a mechanical arm, an alarm device and an air compressor are arranged above the chassis; The robotic arm comprises a robotic arm base and a plurality of intermediate robotic arms sequentially connected to the rear of the robotic arm base, wherein the plurality of intermediate robotic arms are connected via a plurality of hinges, and a vertically placed mounting platform is provided at the end of the robotic arm, and a flexible clip mechanism, a blowing module, a vibration module, a turntable module and a depth camera are sequentially installed on the mounting platform from bottom to top; wherein the blowing module and the vibration module are installed on the turntable module; The air compressor is connected to the blowing module through an air flow pipeline, and is used to provide an air source for the blowing module. The air source is blown out from a blowing needle arranged on the top of the blowing module through the air flow pipeline.

2. A pollination method for a pollination device for the pollination problem of Amomum villosum according to claim 1, characterized in that: The method comprises the following steps: S1: Place the robot dog in the area where pollination is required, start the robot dog, measure its acceleration and angular velocity through its built-in inertial sensor, and calculate its position in combination with Kalman filtering; S2: Use the model predictive control algorithm to control the robot dog’s posture; S3: Use the D* algorithm to plan the path of the robot dog; S4: When the robot dog moves to the vicinity of the target flower according to the planned path, the improved YOLOv8 is used to detect and identify the target flower of Amaryllis sinensis, and the target flower is positioned for the first time; S5: Use the depth camera to perform a second positioning of the nearest target flower, identify the various structures of the target flower, and obtain the world coordinates of the various structures of the target flower; S6: performing a pollination operation on the target flower according to the acquired world coordinates of each part of the target flower; S7: After the pollination is completed, the pollination device is reset, and then steps S2-S6 are repeated to complete the pollination task of the next target flower.

3. A pollination method for the pollination problem of Amomum villosum according to claim 2, characterized in that: The step S1 includes: obtaining the position estimation of the robot dog by integrating the acceleration and angular velocity data according to the Kalman filter equation, and the formula is as follows: in, is the position vector, is the velocity vector, is the acceleration vector.

4. A pollination method for the pollination problem of Amomum villosum according to claim 2, characterized in that: The step S2 includes: establishing a position coordinate system by establishing a kinematic model to obtain a body coordinate system and a world coordinate system, wherein the body coordinate system is located at the center of mass of the robot dog, the positive direction of the x-axis is the forward direction of the robot dog, the positive direction of the z-axis is perpendicular to the plane of the robot dog body and is vertically upward, and the y-axis is determined by the right-hand rule; the world coordinate system is located in the vertical direction of the projection of the body coordinate system on the ground, and the distance to the body coordinate system is h.

5. A pollination method for the pollination problem of Amomum villosum according to claim 2, characterized in that: The step S3 includes: S31: Using the robot dog's five cameras and infrared sensing detectors to sense the surrounding environment 360 degrees without blind spots, the collected three-dimensional terrain map is discretized into a three-dimensional grid structure, each small cube unit is set as a node, and the connection relationship between the nodes is determined. Then, the starting position of the robot dog is set as the starting node s, and the desired destination is set as the target node g; S32: according to the starting node and the target node, estimating the distance from the starting node to the target node by starting the function; S33: Take out the node n with the smallest evaluation function f(n) value from the priority queue OPEN; S34: For each neighboring node m∈Neighbors(n) of the retrieved node n, calculate the movement cost c∈(n,m) from the node n to the neighboring node m, and then calculate the new path cost t based on the movement cost; S35: Compare the new path cost t with the path cost g(m) of the adjacent node m. If t<g(m), update g(m)=t, indicating that a better path to m is found. If m is not in the OPEN queue, put m into the OPEN queue and repeat steps S31-S35 until the starting node is taken out or the OPEN queue is empty.

6. A pollination method for the pollination problem of Amomum villosum according to claim 2, characterized in that: In step S4, The improved YOLO v 8 includes: YOLO through improved depth-separable convolution and attention mechanism v 8. Optimize the feature extraction part to improve the accuracy and speed of detecting Amaryllis odoratus. The first positioning of the target flower comprises: S41: The depth camera eliminates lens distortion on the image by using its internal reference to perform distortion correction to obtain a distortion-free image, and obtains the camera's internal reference data from a configuration file of a color frame or a depth frame; S42: For each pixel point, convert its depth pixel value into an actual depth value Z; S43: Obtaining the three-dimensional coordinates of the pixel point in the camera coordinate system according to the camera's intrinsic parameter data and the actual depth value; S44: The rotation matrix and the translation vector of the robot dog are combined to form a transformation matrix, and then the three-dimensional coordinates of the pixel points to be obtained are converted into the world coordinate system using the transformation matrix, so as to achieve the first positioning of the target flower.

7. A pollination method for Amomum villosum according to claim 6, characterized in that: The step S5 includes: using the FCN semantic segmentation algorithm to specifically identify the various parts of the target flower structure, and then using the method of first positioning the target flower to perform a second positioning on the various parts of the target flower structure, and calculate the coordinates of each part of the structure in the world coordinate system.

8. A pollination method for the pollination problem of Amomum villosum according to claim 2, characterized in that: The step S6 includes: S61: The robot arm establishes a relative coordinate system for positioning according to the recorded relevant coordinates of the flower, and then calculates the relative coordinate system coordinates of the petals, stigma, pollen sac, and style of the target flower, and then suspends the flexible clamp mechanism, the vibration module, and the blowing module above the target flower; S62: After receiving the clamping instruction, the flexible clamp mechanism calculates its force and opening and closing angle according to the relative coordinates of the style, and then drives the motor to clamp the style and move upward for a distance so that the pollen sac is not buried by the petals; S63: The turntable module drives the vibration module and the blowing module to be located at appropriate positions by rotating, and the vibration between the vibration module and the pollen sac shakes the pollen to the groove of the flower; the blowing module generates gas through an air compressor and uses an air flow pipeline to transport the air flow, and blows the pollen into the top of the style through the blowing needle to complete a pollination; S64: According to the number of petals, the vibration module, the blowing module and the blowing needle are placed at appropriate positions again by rotating the turntable module, thereby completing the pollination work of different petals near the same coordinate.

9. A pollination method for the pollination problem of Amomum villosum according to claim 8, characterized in that: In step S63, the vibration module includes: using the pulse signal generated by the PWM module to affect its period and finally affect the vibration frequency f z , wherein the period calculation formula of the PWM module is: T clk is the clock period of the PWM module, f clk is the clock frequency of the PWM module; PERIOD is the period ratio register.

10. A pollination method for the pollination problem of Amomum villosum according to any one of claims 1 to 9, characterized in that: In step S7, resetting the pollination device includes resetting the robot arm through a total transformation matrix, and the total transformation matrix formula is: T=A1A2...A i Among them, A i is the homogeneous transformation matrix between adjacent intermediate manipulators obtained by the DH parameter method, and i is the number of manipulator joints.

Images