Vision-guided strawberry stolon recognition-cutting-propagation system and method

Abstract

The application discloses a strawberry stolon recognition-cutting-cuttage system and method based on visual guidance, and belongs to the field of agricultural intelligent equipment. The application comprises the following steps: controlling a robot to move to a working position and completing initialization; collecting RGB images and depth point cloud data and performing pretreatment; constructing a feature point recognition model, identifying stolon main stem growth direction vectors and stolon root feature points, and calculating the three-dimensional coordinates of the optimal cutting point; solving the inverse kinematics parameters of a mechanical arm and driving the mechanical arm to move to a target position; controlling an end effector to sequentially complete the cutting and clamping of the seedling; carrying the seedling to a medicine immersion box for quantitative medicine immersion treatment; and finally accurately cutting the seedling into a designated hole of a plug tray. The application realizes the full-process automation of the strawberry stolon seedling from recognition, cutting, medicine immersion to cutting, significantly reduces the labor intensity, improves the seedling screening and cutting efficiency and consistency, and is suitable for facility agriculture and large-scale strawberry production.

Description

Technical Field

[0001] This invention relates to the field of agricultural intelligent equipment technology, specifically to a vision-guided strawberry runner identification-cutting-propagation system and method. Background Technology

[0002] Strawberries, as an important economic crop, are characterized by their short growth cycle, high market demand, and high added value, making them widely used in facility agriculture and large-scale cultivation. In strawberry production, seedling propagation is a crucial step in ensuring planting scale and varietal consistency. Currently, the identification, selection, and cutting of strawberry seedlings mainly rely on manual labor: operators must judge the morphological characteristics of the runners based on experience, and then manually separate, position, and insert the seedlings. This process is labor-intensive, inefficient, and the quality of work is easily affected by the operator's experience.

[0003] In recent years, machine vision and image recognition technologies have made some progress in fields such as crop sorting and harvesting. For example, the invention patent CN117876645A, entitled "A Method for Detecting Mature Tomato Stems Based on Visual Deep Learning Algorithm," proposes a method to identify tomato stems by deriving the corresponding stem frame of a mature tomato in a 2D image and performing angle reasoning. Another invention patent CN114972979B, entitled "A Method for Detecting Plant Main Stems in Images Captured in a Greenhouse Environment," utilizes the strong correlation between the position and direction of line segments detected in an image to achieve the detection of plant main stems. However, for the identification of strawberry runners and automated cutting propagation, there is currently a lack of effective visual guidance methods and dedicated robot systems.

[0004] To address this, the present invention designs a robotic system based on visual recognition to identify and locate strawberry runners and complete the cutting operation. This system aims to reduce the intensity of manual labor, improve the efficiency of seedling selection and cutting, and thus meet the needs of modern facility agriculture and large-scale strawberry production for automated and intelligent equipment. Summary of the Invention

[0005] The purpose of this invention is to provide a vision-guided strawberry runner identification-cutting-propagation system and method. By integrating depth vision and multi-task recognition models, the system automatically detects the location of aerial roots and the growth direction of runners in seedlings, accurately calculates the optimal cutting point, and then a robotic arm sequentially completes the cutting, clamping, soaking in medicine, and seedling propagation in plug trays. This achieves fully automated operation of strawberry seedlings from identification to propagation, effectively improving seedling efficiency and quality, and meeting the needs of facility agriculture for intelligent equipment.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: In a first aspect, the present invention provides a visually guided method for identifying, cutting, and propagating strawberry runners, comprising the following steps: S1. Control the creeping stem cutting robot to move to the preset initial position and complete the initialization and calibration of the vision sensor and the robotic arm. S2. Acquire RGB images and depth point cloud data of the work area through a vision sensor, and perform random mirroring, cropping and flipping processing on the RGB images, and perform statistical filtering and mean filtering preprocessing on the point cloud data. S3. Construct a stolon feature point recognition model, process the collected image data, identify the aerial root feature points of the seedlings and the growth direction vector of the main stem of the stolons, and calculate the three-dimensional spatial coordinates of the best cutting point of the seedlings. S4. Calculate the inverse kinematics parameters of the robotic arm based on the three-dimensional spatial coordinates of the optimal cutting point, generate the movement and operation commands of the robotic arm, and send them to the controller. S5. The controller drives the robotic arm to move to the target position and controls the end effector to perform the cutting and gripping operations on the creeping stem seedlings in sequence. S6. Control the robotic arm to carry the seedlings it has gripped to the soaking tank station to complete the quantitative soaking treatment operation. S7. After soaking in the medicine, control the robotic arm to move to the seedling tray position and accurately insert the seedlings into the designated holes in the seedling tray.

[0007] Further, in step S3, the target categories of the stolon feature point recognition model include four categories: background, stolons, aerial roots of seedlings, and leaves of seedlings; the model includes a YOLOv11 target detection model and a MobileNet semantic segmentation model, wherein the feature pyramid network of the YOLOv11 model embeds an aerial root sensitive feature enhancement layer, and increases the weight allocation of aerial root texture features through a channel attention mechanism.

[0008] Furthermore, step S3 specifically includes: S31. Input the acquired RGB image into the target detection model to locate the target region containing the stolons; S32. Detect aerial root feature points and crop the region of interest image containing the stolon seedlings based on the feature point positions; S33. Input the image of the region of interest into the semantic segmentation model to perform pixel-level classification of the stem, leaves and background; S34. Binarize the semantically segmented image to generate an image mask that retains only the target features; S35. Map the image mask back to the original image coordinate system and perform spatial registration and alignment with the depth point cloud data; S36. Based on the registered depth data, extract the three-dimensional coordinate center point of the aerial roots. Principal component analysis was used to fit a straight line to the local three-dimensional point cloud of the main stem of the stolon, and the unit vector of the growth direction of the main stem was obtained. ; Calculate the three-dimensional coordinates of the optimal cutting point for the seedlings. The calculation formula is as follows: in, This represents the three-dimensional coordinate center point of the aerial roots of the seedling, and L represents the preset trimming distance. This represents the unit vector indicating the growth direction of the main stem of the stolons, with its direction set to deviate from the direction of the strawberry mother plant.

[0009] Furthermore, the specific method for calculating the inverse kinematic parameters of the robotic arm in step S4 includes: Step S41: Use a homogeneous transformation matrix to transform the three-dimensional spatial coordinates of the clipping point from the camera coordinate system to the base coordinate system of the robotic arm, and establish the desired pose matrix of the end effector. The coordinate transformation formula is: in, Let the cutting point be the three-dimensional spatial coordinates of the robotic arm's base coordinate system. To perform the homogeneous forward kinematic transformation matrix of the robotic arm in its current posture, This is the fixed extrinsic parameter matrix from the camera coordinate system to the end effector of the robotic arm. The three-dimensional spatial coordinates of the clipping point in the camera coordinate system; Step S42: A hybrid inverse kinematics solution strategy combining analytical methods and numerical iteration based on damped least squares is adopted. The numerical iteration formula is as follows: in, This is the incremental correction for the joint angle. Let λ be the pose error vector between the current end-effector pose and the desired pose, λ be the damping factor, I be the identity matrix, and J be the Jacobian matrix of the robotic arm. Step S43: Perform trajectory planning in joint space using a fifth-order polynomial interpolation algorithm. For any joint i, its motion trajectory angle function... Set as: Solve for the polynomial coefficients based on the boundary conditions at the start and end times. ~ It generates a discrete control command stream in which position, velocity, and acceleration are all continuous and smooth.

[0010] Furthermore, in step S4, if the target cutting point exceeds the maximum workspace threshold of the robotic arm, the collaborative control logic is triggered. First, the two-dimensional planar displacement that the moving chassis needs to compensate is calculated, and the chassis is driven to move into the effective envelope space. Then, the robotic arm is triggered to perform a smooth approximation action.

[0011] Furthermore, the spatial calibration relationship between the vision sensor and the robotic arm is as follows: in, These are the coordinates of the target point in the robot arm's base coordinate system. d is the pose matrix of the vision sensor in base coordinates, d is the distance from the target point to the vision sensor, K is the camera intrinsic parameter matrix, and u is the target pixel coordinate. , It is the position vector of the camera in the base coordinate system.

[0012] Secondly, the present invention provides a vision-guided strawberry runner identification-pruning-cutting system for performing the vision-guided strawberry runner identification-pruning-cutting method, comprising: The mobile chassis is equipped with a mounting platform to provide a fixed base for the robotic arm. The industrial computer, installed inside the mobile chassis, is used to collect and process image information acquired by the vision sensor, convert it into control signals, and send them to the controller. The controller is used to receive instructions from the industrial computer and drive the robotic arm and mobile chassis to perform (1) actions. The robotic arm is fixedly mounted on a mobile chassis and is used to complete the positioning, cutting, transportation and cutting of strawberry runners; A vision sensor, fixedly mounted on the end effector of the robotic arm, is used to acquire color images and depth information of the strawberry runners; The soaking tank, located above the mobile chassis and within the first working area along the direction of travel of the robotic arm, is used to soak the cut strawberry runners in a chemical solution. The seedling tray is positioned above the mobile chassis and within the second working area along the direction of travel of the robotic arm. It is used to support the cutting substrate and fix the strawberry runner cuttings. The second side is opposite to the first side. An end effector, installed at the end of a robotic arm, is used to perform cutting and gripping operations.

[0013] Furthermore, the robotic arm includes a base, shoulder joint, elbow joint, wrist joint and end effector interface connected in sequence. Each joint is driven by a servo motor to achieve position and posture adjustment in three-dimensional space. The horizontal field of view α of the vision sensor ranges from 70° to 90°, and the vertical field of view β ranges from 45° to 60°.

[0014] Furthermore, the immersion tank has a corrosion-resistant structure and is equipped with a liquid level sensor inside.

[0015] Furthermore, the cavity tray is provided with multiple regularly arranged insertion holes; the industrial control computer integrates obstacle avoidance and remote control functions to complete the operation of the mobile chassis in the gantry.

[0016] The beneficial effects of the above scheme are as follows: First, this invention automates the entire process of strawberry runner seedling identification and propagation, significantly reducing manual labor intensity. Through the coordinated operation of a moving chassis, a robotic arm, vision sensors, and an end effector, the system can autonomously complete the positioning, cutting, transportation, soaking in chemicals, and propagation of seedlings without continuous human intervention. This effectively reduces reliance on skilled workers in strawberry seedling cultivation, thereby lowering labor costs and improving operational efficiency.

[0017] Secondly, this invention proposes a precise pruning point calculation method based on the characteristic points of aerial roots and the growth direction vector of stolons. After acquiring the three-dimensional coordinates of the aerial roots of the seedlings using a visual sensor, the system uses principal component analysis to fit the growth direction of the main stem and sets a pruning distance along the direction deviating from the mother plant, thereby accurately calculating the optimal pruning position. This method avoids the arbitrariness of traditional manual pruning based on experience, ensuring that the pruning cut position of each seedling is reasonable. This is beneficial for the later rooting of the seedlings and reduces damage to the seedlings caused by improper pruning, thereby improving the survival rate and seedling quality.

[0018] Third, this invention incorporates a dynamic viewpoint adjustment and closed-loop correction mechanism. After the initial detection of the aerial root region, the system controls the robotic arm to dynamically adjust the orientation of the vision sensor, ensuring the camera's optical axis is as perpendicular as possible to the plane where the aerial root is located for a secondary close-up shot. This effectively eliminates perspective distortion and leaf occlusion caused by the initial fixed viewing angle. By matching the high-quality image acquired in the secondary acquisition with a pre-stored heatmap, minor positioning errors are corrected in real time, significantly improving the accuracy and robustness of identifying stolons and seedlings under complex lighting and occlusion conditions.

[0019] Fourth, this invention integrates soaking in disinfectant and tray propagation into the same robotic system, forming a complete operational chain. The cut seedlings are held in place by an end effector and sequentially transferred to a soaking tank for quantitative disinfection and immersion, before being precisely inserted into designated holes in the trays. The entire process is continuous and controllable, ensuring that each seedling receives consistent soaking time and insertion depth. Furthermore, the insertion angle and position can be set according to agronomic requirements, achieving standardized and digitalized seedling management, effectively improving the uniformity of seedling rooting and subsequent transplant survival rate.

[0020] Fifth, this invention exhibits excellent environmental adaptability and scalability. The joint calibration method of the visual sensor and the robotic arm, along with the hybrid inverse kinematics solution strategy, enables the system to adapt to different row spacings and light variations in greenhouse-grown strawberries. Furthermore, the proposed identification and propagation method is not only applicable to strawberry runner seedlings but can also be extended to other crops with runner propagation characteristics, providing technical support for automated seedling cultivation of various crops in facility agriculture. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the hardware system of the present invention; Figure 2 This is a diagram illustrating the system's operational scenario according to the present invention. Figure 3 This is a system workflow diagram of the present invention; Figure 4 This is a flowchart illustrating the feature recognition and extraction process of the target object in an embodiment of the present invention; Figure 5 This is a diagram of the hardware control system of the present invention. Detailed Implementation

[0022] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0023] It should be noted that, unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0024] This invention provides a vision-guided strawberry runner identification-cutting-propagation system and method, which can automatically complete the entire process of runner seedling identification, optimal cutting point positioning, seedling cutting and clamping, solution soaking, and plug tray propagation in a greenhouse strawberry growing environment.

[0025] I. System Structure like Figure 1 , Figure 2 , Figure 5As shown, the system of the present invention mainly includes a mobile chassis 1, an industrial computer 2, a controller 3, an execution robotic arm 4, a immersion tank 5, a pitting tray 6, a vision sensor 7, and an end effector 8.

[0026] The mobile chassis 1 is equipped with a mounting platform to provide a fixed base for the robotic arm 4, ensuring the movement accuracy and stability of the robotic arm during operation. An industrial control computer 2 is installed inside the mobile chassis 1. The industrial control computer 2 collects and processes image information acquired by the vision sensor 7, converting it into control signals that are sent to the controller 3. The industrial control computer 2 also integrates obstacle avoidance and remote control functions to enable the mobile chassis to move and operate autonomously within the greenhouse scaffolding. The controller 3, acting as a lower-level machine, has its signal input connected to the industrial control computer 2 and its signal output connected to the drive units of the robotic arm 4 and the mobile chassis 1. It receives position commands from the industrial control computer 2 and drives the corresponding components to move.

[0027] The robotic arm 4 is fixedly mounted on the mobile chassis 1 and is used to perform the positioning, cutting, transfer, and propagation of strawberry runners. The robotic arm 4 includes a base, shoulder joint, elbow joint, wrist joint, and end effector interface connected in sequence. Each joint is driven by a servo motor, enabling position and posture adjustment in three-dimensional space. The end effector 8 is mounted at the end of the robotic arm 4 and is used to perform cutting and gripping operations.

[0028] A vision sensor 7 is fixedly mounted on the end effector of the robotic arm 4, with its optical axis pointing towards the strawberry plant's working area. It is used to acquire real-time color images and depth information of the working area, enabling three-dimensional perception of the runner's spatial position. In this embodiment, the vision sensor 7 uses an Intel RealSense D456 depth camera, capable of simultaneously acquiring high-resolution RGB images and depth point cloud data, and obtaining image data aligned with the two-dimensional image and the three-dimensional point cloud. The horizontal field of view α and the vertical field of view β of the vision sensor 7 can be adjusted according to the scene, with an adjustment range satisfying α from 70° to 90° and β from 45° to 60°.

[0029] The soaking tank 5 is positioned above the mobile chassis 1 and within the first working area along the direction of travel of the robotic arm 4. It is used to soak the cut strawberry runner cuttings in a disinfectant solution to improve the survival rate of the cuttings. The soaking tank 5 is a corrosion-resistant structure, containing disinfectant or rooting promoter, and is equipped with a liquid level sensor to monitor the solution level in real time. An alarm is triggered or automatic replenishment occurs when the solution is insufficient. The seedling tray 6 is positioned above the mobile chassis 1 and within the second working area along the direction of travel of the robotic arm 4, opposite to the first side. The seedling tray 6 is used to support the cutting substrate and fix the strawberry runner cuttings. The seedling tray 6 has multiple regularly arranged insertion holes; in this embodiment, a 32-hole seedling tray is used. The substrate is a mixture of peat moss, vermiculite, and perlite in a certain volume ratio, and pre-moistened.

[0030] II. Specific Implementation Steps The following combination Figure 3 The flowchart describes in detail the strawberry runner identification-cutting-propagation method of the present invention.

[0031] Step S1: Initialization and Calibration After the system is powered on, the industrial control computer 2 controls the mobile chassis 1 to move to the preset initial working position in the elevated strawberry trellis of the greenhouse. Then, the vision sensor 7 and the robotic arm 4 are powered on and initialized. A preset calibration method is used to complete the spatial calibration between the vision sensor 7 and the robotic arm 4, establishing the mapping relationship between the camera coordinate system, the robotic arm end effector coordinate system, the robotic arm base coordinate system, and the world coordinate system. This ensures the accuracy of subsequent 3D visual positioning and precise robotic arm operations. In this embodiment, the spatial calibration relationship between the vision sensor and the robotic arm is as follows: in, These are the coordinates of the target point in the robot arm's base coordinate system. d is the pose matrix of the vision sensor in base coordinates, d is the distance from the target point to the vision sensor, K is the camera intrinsic parameter matrix, and u is the target pixel coordinate. , It is the position vector of the camera in the base coordinate system.

[0032] Step S2: Image Acquisition and Preprocessing The vision sensor 7 begins acquiring image data of the working area, obtaining RGB images and depth point cloud data including stolons, seedlings, aerial roots of seedlings, and leaves. To expand the sample size and alleviate the overfitting problem of the subsequent model, the industrial control computer 2 performs data enhancement processing on the acquired RGB images, including random mirroring, random cropping, and random flipping. At the same time, statistical filtering is used to remove outlier noise points from the depth point cloud data, and then mean filtering is used for smoothing to improve the point cloud quality.

[0033] Step S3: Construct a stolon feature point recognition model to identify aerial roots and growth direction, and calculate the optimal pruning point. This step is the most innovative part of this invention, and the specific process is as follows: Figure 4 As shown.

[0034] Before building the model, the dataset was first prepared and labeled. The collected raw images were labeled with information including strawberry runners, runner seedlings, aerial roots of the seedlings, trellises, and cultivation troughs. The labeling process was done manually, using the polygon annotation function of the labeling software to accurately delineate the irregular shapes of the runners, aerial roots of the seedlings, and leaves, generating JSON format label files.

[0035] A YOLOv11 object detection model and a MobileNet semantic segmentation model were constructed based on labeled data. Specifically, the JSON label files generated from the annotations were converted into TXT files required for YOLOv11 training, and then divided into training, validation, and test sets in an 8:1:1 ratio. Simultaneously, the labeled data was converted into a mask format for training the MobileNet semantic segmentation model. The object detection model and the semantic segmentation model were then trained using the above process.

[0036] Specifically, the constructed YOLOv11 model embeds an aerial root-sensitive feature enhancement layer in its feature pyramid network. This layer introduces a channel attention mechanism to increase the weight allocation of light yellow or white aerial root texture features, actively suppressing the feature responses of green background leaves and occlusions, thereby outputting high-confidence coarse localization bounding boxes for aerial roots under complex lighting conditions.

[0037] This invention combines an improved YOLOv11 object detection model with the MobileNet semantic segmentation model to design a stolon pruning point detection network model with dynamic viewpoint adjustment and loop closure correction. Specifically, it is executed according to the following sub-steps: Step S31: Simultaneously acquire RGB and depth images of the target area using a depth vision sensor to ensure accurate alignment of spatial coordinates and timestamps between the two types of images. Use an improved YOLOv11 target detection model to perform initial screening and inference on the RGB images, extracting regions of interest containing stolons and aerial roots; simultaneously, obtain the coarse normal vector of the plane containing the aerial roots based on the initial screening results.

[0038] Step S32: Enter the dynamic viewpoint adjustment stage. Based on the coarse normal vector from step S31, the industrial control computer generates attitude compensation commands to control the robotic arm 4 to dynamically adjust the attitude of the end-effector vision sensor 7, making the camera optical axis as perpendicular as possible to the local plane where the aerial roots are located. Then, a second close-up shot is taken, effectively eliminating perspective distortion and leaf occlusion under the initial fixed viewpoint. The image from the second shot is then input into the YOLOv11 object detection model to crop out a high-quality region of interest image containing the stolon seedlings.

[0039] Step S33: Perform closed-loop fine localization and correction. The high-quality region-of-interest image captured a second time is input into the MobileNet lightweight semantic segmentation model. Pixel-level classification of the stem, leaves, and background in the image is performed to accurately delineate the target spatial contour. Then, an image registration algorithm is used to match the second-acquired local features with a pre-stored standardized strawberry aerial root heatmap, correcting minor localization errors caused by the robotic arm's movements in real time.

[0040] Step S34: Binarize the image after semantic segmentation, set the pixels corresponding to the target category to 1 and the background to 0, and generate an image mask that retains only the target features.

[0041] Step S35: Align the binary mask with the original depth image in terms of spatial coordinates and filter the depth data to extract the three-dimensional coordinate center point of the aerial roots. Principal component analysis was used to fit a straight line to the local 3D point cloud of the main stem of the stolon, obtaining the unit vector of the growth direction of the main stem. The direction of this vector is set to deviate from the direction of the strawberry mother plant. Finally, according to the preset cutting distance L, in this embodiment L=0.3 cm, along... 3D coordinates of the optimal cutting point for the seedling. The calculation formula is: The formula indicates that the optimal cutting point is located at a distance L from the mother plant along the growth direction of the stolon, from the center of the aerial root.

[0042] Step S4: Inverse kinematics calculation and control command generation Based on the three-dimensional spatial coordinates of the cutting point obtained in step S3, the industrial control computer 2 calculates the inverse kinematic parameters of the robotic arm 4, generates robotic arm movement and operation commands, and sends them to the controller 3. Specifically, this includes the following sub-steps: Step S41: Use a homogeneous transformation matrix to transform the 3D spatial coordinates of the clipping point from the camera coordinate system to the base coordinate system of the robotic arm, and establish the desired pose matrix of the end effector. The coordinate transformation formula is: in, The optimal cutting point is defined in the three-dimensional space coordinates of the robotic arm's base coordinate system. To perform the homogeneous forward kinematic transformation matrix of the robotic arm in its current posture, This is the fixed extrinsic parameter matrix from the camera coordinate system to the end effector of the robotic arm. The three-dimensional spatial coordinates of the optimal cropping point in the camera coordinate system; Step S42: Inverse kinematics solution based on hybrid algorithm. To ensure real-time performance and solution accuracy, the system adopts a hybrid inverse kinematics solution strategy that combines analytical method with numerical iteration method based on damped least squares method.

[0043] First, a mathematical model of each link of the robotic arm is established using the standard DH parameter table. The link length *a*, link twist angle *α*, joint offset *d*, and joint rotation angle *θ* are defined. The initial values ​​of the analytical solutions for each joint angle are obtained through the algebraic-geometric separation method. .

[0044] When the target point is near a singular configuration or has multiple analytical solutions, a damped least squares method is introduced to construct a numerical iterative formula based on the Jacobian matrix J for approximation: in, This is the incremental correction for the joint angle. Let be the pose error vector between the current end-effector pose and the desired pose, λ be the damping factor, I be the identity matrix, and J be the Jacobian matrix of the robotic arm. Iterate until the error converges, and output the target angle sequence [θ1,θ2,...,θn] for each joint.

[0045] Step S43: A fifth-order polynomial interpolation algorithm is used for trajectory planning within the joint space to avoid mechanical vibration caused by sudden acceleration changes during the robotic arm's movement. For any joint i, its motion trajectory angle function... Set as: Set start time With arrival time The boundary conditions, including the velocity and acceleration being zero at the start and end times, are used to solve for the polynomial coefficients c0~c5, thereby generating a discrete control command flow that continuously and smoothly changes with time in terms of position, velocity, and acceleration.

[0046] In addition, before executing steps S41 to S43, the industrial control computer 2 first determines whether the target cutting point exceeds the maximum working space threshold of the robotic arm 4. If it does, the collaborative control logic is triggered, and the two-dimensional planar displacement (x, y) that the mobile chassis 1 needs to compensate is calculated first. The chassis is then driven to move into the effective envelope space, and the robotic arm 4 is triggered to perform a smooth approximation action, so that the end effector 8 can accurately reach the cutting point in the desired posture Ttarget.

[0047] Step S5: Cutting and clamping operations The controller 3 drives the robotic arm 4 to move along the planned trajectory to the target cutting point. Upon arrival, the end effector 8 first performs the cutting operation, severing the runner to separate the seedling from the mother plant; then, the end effector 8 maintains a clamping state, stably holding the cut seedling to prevent it from falling. In this embodiment, the end effector 8 adopts a structure similar to a combination of scissors and grippers, completing the clamping process simultaneously with cutting the runner.

[0048] In actual operation, a single mother plant may produce multiple seedlings, and the vision system can identify the location of these seedlings. The industrial control computer 2 generates a seedling operation sequence based on the spatial location of each seedling and its growth order on the stolons. The seedlings are then cut and picked up sequentially from far to near or from tip to base to avoid omissions or repetitive operations.

[0049] Step S6: Immersion in medicinal herbs The robotic arm 4 carries the seedlings from the cutting position to the soaking tank 5. The system controls the robotic arm to immerse the aerial root area at the base of the seedlings into the solution in the soaking tank. A liquid level sensor ensures that the solution submerges the aerial roots but not the leaves. After maintaining the preset soaking time, the seedlings are removed to promote rooting and improve the survival rate of subsequent cuttings.

[0050] Step S7: Cuttings Propagation After soaking in the medicinal solution, the robotic arm 4 carries the seedling to the working area of ​​the seedling tray 6. The industrial control computer 2 determines the insertion position of the current seedling according to the pre-set cutting plan. In this embodiment, the seedling tray is placed horizontally, and during cutting, the back arch of the seedling is positioned close to one corner of the seedling tray, with the seedling at a 45° angle to the substrate plane to prevent it from falling over while upright. The end effector 8 presses the base of the seedling into the substrate to the preset insertion depth, then releases the gripper to complete the cutting. The robotic arm 4 then returns to its initial position, ready for the next seedling cutting. The system repeats steps S3 to S7 until all identified seedlings have been cut.

[0051] Finally, it should be noted that any parts of this invention not described in detail are prior art. Those skilled in the art will understand that the above descriptions are merely preferred embodiments of the invention and are not intended to limit the invention. Although the invention has been described in detail with reference to the foregoing examples, those skilled in the art can still modify the technical solutions described in the foregoing examples or make equivalent substitutions for some of the technical features. All modifications and equivalent substitutions made within the spirit and principles of the invention should be included within the scope of protection of the invention.

Claims

1. A vision-guided method for identifying, pruning, and propagating strawberry runners, characterized in that... Includes the following steps: S1. Control the creeping stem cutting robot to move to the preset initial position and complete the initialization and calibration of the vision sensor and the robotic arm. S2. Acquire RGB images and depth point cloud data of the work area through a vision sensor, and perform random mirroring, cropping and flipping processing on the RGB images, and perform statistical filtering and mean filtering preprocessing on the point cloud data. S3. Construct a stolon feature point recognition model, process the collected image data, identify the aerial root feature points of the seedlings and the growth direction vector of the main stem of the stolons, and calculate the three-dimensional spatial coordinates of the best cutting point of the seedlings. S4. Calculate the inverse kinematics parameters of the robotic arm based on the three-dimensional spatial coordinates of the optimal cutting point, generate the movement and operation commands of the robotic arm, and send them to the controller. S5. The controller drives the robotic arm to move to the target position and controls the end effector to perform the cutting and gripping operations on the creeping stem seedlings in sequence. S6. Control the robotic arm to carry the seedlings it has gripped to the soaking tank station to complete the quantitative soaking treatment operation. S7. After soaking in the medicine, control the robotic arm to move to the seedling tray position and accurately insert the seedlings into the designated holes in the seedling tray.

2. The visually guided strawberry runner identification-cutting-propagation method according to claim 1, characterized in that, In step S3, the target categories of the stolon feature point recognition model include four categories: background, stolons, aerial roots of seedlings, and leaves of seedlings. The model includes a YOLOv11 target detection model and a MobileNet semantic segmentation model. The feature pyramid network of the YOLOv11 model embeds an aerial root sensitive feature enhancement layer, which increases the weight allocation of aerial root texture features through a channel attention mechanism.

3. The visually guided strawberry runner identification-cutting-propagation method according to claim 2, characterized in that, Step S3 specifically includes: S31. Input the acquired RGB image into the target detection model to locate the target region containing the stolons; S32. Detect aerial root feature points and crop the region of interest image containing the stolon seedlings based on the feature point positions; S33. Input the image of the region of interest into the semantic segmentation model to perform pixel-level classification of the stem, leaves and background; S34. Binarize the semantically segmented image to generate an image mask that retains only the target features; S35. Map the image mask back to the original image coordinate system and perform spatial registration and alignment with the depth point cloud data; S36. Based on the registered depth data, extract the three-dimensional coordinate center point of the aerial roots. Principal component analysis was used to fit a straight line to the local three-dimensional point cloud of the main stem of the stolon, and the unit vector of the growth direction of the main stem was obtained. ; Calculate the three-dimensional coordinates of the optimal cutting point for the seedlings. The calculation formula is as follows: in, This represents the three-dimensional coordinate center point of the aerial roots of the seedling, and L represents the preset trimming distance. This represents the unit vector indicating the growth direction of the main stem of the stolons, with its direction set to deviate from the direction of the strawberry mother plant.

4. The visually guided strawberry runner identification-cutting-propagation method according to claim 3, characterized in that, The specific method for calculating the inverse kinematics parameters of the robotic arm in step S4 includes: Step S41: Use a homogeneous transformation matrix to transform the three-dimensional spatial coordinates of the clipping point from the camera coordinate system to the base coordinate system of the robotic arm, and establish the desired pose matrix of the end effector. The coordinate transformation formula is: in, The three-dimensional spatial coordinates of the cutting point in the base coordinate system of the robotic arm. To perform the homogeneous forward kinematic transformation matrix of the robotic arm in its current posture, The fixed extrinsic parameter matrix from the camera coordinate system to the end effector of the robotic arm. The three-dimensional spatial coordinates of the clipping point in the camera coordinate system; Step S42: A hybrid inverse kinematics solution strategy combining analytical methods and numerical iteration based on damped least squares is adopted. The numerical iteration formula is as follows: in, This is the incremental correction for the joint angle. Let λ be the pose error vector between the current end-effector pose and the desired pose, λ be the damping factor, I be the identity matrix, and J be the Jacobian matrix of the robotic arm. Step S43: Perform trajectory planning in joint space using a fifth-order polynomial interpolation algorithm. For any joint i, its motion trajectory angle function... Set as: Solve for the polynomial coefficients based on the boundary conditions at the start and end times. ~ It generates a discrete control command stream in which position, velocity, and acceleration are all continuous and smooth.

5. The visually guided strawberry runner identification-cutting-propagation method according to claim 4, characterized in that, In step S4, if the target cutting point exceeds the maximum workspace threshold of the robotic arm, the collaborative control logic is triggered. First, the two-dimensional planar displacement that the moving chassis needs to compensate is calculated, and the chassis is driven to move into the effective envelope space. Then, the robotic arm is triggered to perform a smooth approximation action.

6. The method for identifying, pruning, and propagating strawberry runners based on vision guidance according to claim 1, characterized in that, The spatial calibration relationship between the vision sensor and the robotic arm is as follows: in, These are the coordinates of the target point in the robot arm's base coordinate system. d is the pose matrix of the vision sensor in base coordinates, d is the distance from the target point to the vision sensor, K is the camera intrinsic parameter matrix, and u is the target pixel coordinate. , It is the position vector of the camera in the base coordinate system.

7. A vision-guided strawberry runner identification-pruning-cutting system, used to perform the method according to any one of claims 1 to 6, characterized in that, include: The mobile chassis (1) is equipped with an installation platform to provide a fixed base for the robotic arm (4); The industrial computer (2) is installed inside the mobile chassis (1) to collect and process the image information acquired by the vision sensor (7), and convert it into control signals to be sent to the controller (3). The controller (3) is used to receive instructions from the industrial computer (2) and drive the robotic arm (4) and the mobile chassis (1) to perform actions. The robotic arm (4) is fixedly installed on the mobile chassis (1) to complete the positioning, cutting, transportation and cutting of strawberry runners; A vision sensor (7) is fixedly installed at the end bracket of the robotic arm (4) to acquire color images and depth information of the strawberry stolons; The soaking tank (5) is set above the mobile chassis (1) and located in the first working area along the direction of travel of the robotic arm (4), and is used to soak the cut strawberry runners in the solution. The seedling tray (6) is set above the mobile chassis (1) and located in the second side working area of ​​the robotic arm (4) along the direction of travel. It is used to carry the cutting substrate and fix the strawberry runner cuttings. The second side is opposite to the first side. An end effector (8) is installed at the end of the robotic arm (4) for performing cutting and gripping operations.

8. The vision-guided strawberry runner identification-pruning-cutting system according to claim 7, characterized in that, The robotic arm (4) includes a base, shoulder joint, elbow joint, wrist joint and end effector interface connected in sequence. Each joint is driven by a servo motor to realize position and posture adjustment in three-dimensional space. The horizontal field of view α of the vision sensor (7) is in the range of 70°~90° and the vertical field of view β is in the range of 45°~60°.

9. The vision-guided strawberry runner identification-pruning-cutting system according to claim 7, characterized in that, The immersion tank (5) is a corrosion-resistant structure and is equipped with a liquid level sensor inside.

10. The vision-guided strawberry runner identification-pruning-cutting system according to claim 7, characterized in that, The acupoint (6) is provided with a number of regularly arranged insertion holes; the industrial control computer (2) integrates obstacle avoidance and remote control functions to complete the operation of the mobile chassis in the gantry.

Images