Intelligent tea picking robot system of embedded edge computing architecture

Abstract

The invention relates to the technical field of intelligent tea-picking robot systems, and discloses an intelligent tea-picking robot system with an embedded edge computing architecture, which realizes localized parallel processing of visual data through a heterogeneous computing platform. And in combination with front-end multi-modal data acquisition, middle-layer YOLOv5 tender tip identification and positioning and a terminal millisecond-level response execution mechanism, the picking success rate and the system robustness in a complex tea garden scene are remarkably improved. A binocular vision navigation and tender tip point cloud positioning technology is adopted by the system, and the three-dimensional coordinates of picking points are accurately calculated; the end effector simulates the manual picking action, so that the tea tender shoots are prevented from being damaged; meanwhile, the tea growth state and pest and disease damage real-time monitoring function is achieved. The tea leaf picking machine solves the problems that a traditional tea leaf picking machine is insufficient in recognition precision and poor in adaptability, and efficient, accurate and automatic tea leaf picking is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent tea-picking robot systems, and specifically to an intelligent tea-picking robot system with an embedded edge computing architecture. Background Art

[0002] The tea-picking machines currently on the market generally have problems such as insufficient recognition accuracy, single picking standards, and poor adaptability, which seriously restrict the large-scale production and quality stability of famous and high-quality teas.

[0003] The current tea picking industry is still dominated by traditional manual picking and semi-mechanized equipment, and faces the following challenges: 1. The tea picking process is heavily reliant on manual labor, resulting in high labor costs. Although some tea plantations have adopted mechanized picking equipment, the majority of high-quality tea leaves still rely on manual screening and picking. Due to the highly seasonal nature of tea picking and the high concentration of labor demand, labor costs account for over 50% of total costs. Furthermore, the limited number of skilled tea pickers and the difficulty recruiting them in remote tea-growing areas further exacerbate production efficiency bottlenecks.

[0004] 2. Existing tea-picking machines have a low level of intelligence and lack recognition and picking accuracy. Currently, tea-picking machines on the market are mainly divided into two categories: large-scale harvesting machines and handheld / two-person lift-type machines. They have obvious defects. Although large-scale harvesting machines can increase the picking speed, they cannot distinguish the quality of tea leaves, resulting in the mixing of high-value young buds with old and dead leaves, and the proportion of high-quality tea is less than 40%. Although handheld / two-person lift-type tea-picking machines can improve picking accuracy, such as Figure 1 and Figure 2 As shown, manual assistance is still needed to judge the tea standards, and true automation cannot be achieved.

[0005] In addition, the existing mechanical visual recognition technology is backward and relies on traditional image processing algorithms. It has a high false detection rate in complex tea garden environments, is unable to identify overlapping leaves, and cannot adapt to changes in lighting. It takes an average of 15 seconds to pick a young bud, which is far lower than manual efficiency.

[0006] 3. Existing tea-picking machines can only perform basic plucking tasks and are unable to monitor the health of tea leaves in real time. For example, unqualified buds and leaves, such as those infested with insects, diseases, or defects, can be mixed into the harvested batch, affecting the overall quality of the tea. This can also affect the taste of the tea during processing and even lead to the downgrade of the entire batch.

[0007] Therefore, we propose an intelligent tea-picking robot system with an embedded edge computing architecture to solve the problems in the above background. Summary of the Invention

[0008] The present invention provides an intelligent tea-picking robot system with an embedded edge computing architecture, which can solve the problems in existing tea picking technology, such as low efficiency of manual picking and the inability of mechanical picking to guarantee quality.

[0009] In order to solve the above technical problems, the present invention provides the following technical solutions: An intelligent tea-picking robot system with an embedded edge computing architecture, comprising: A multimodal data acquisition module includes a binocular vision camera and an RGB sensor. The binocular vision camera is installed at a 60° angle to the horizontal direction. The binocular vision camera is used to fit the edge lines of the tea rows through OTSU threshold segmentation and improved Hough transform to guide the movement of the tea picking machine. Embedded edge computing platform, integrating high-performance processors and storage devices, for running lightweight YOLOv5 models; 3D positioning and navigation module, used to calculate the 3D coordinates of the picking point based on binocular vision technology; The robotic arm control module includes a five-degree-of-freedom robotic arm and an end effector; Tea health monitoring module, including a multispectral camera and data analysis unit; Closed-loop feedback module, including gravity sensor and communication unit.

[0010] Preferably, the embedded edge computing platform is configured as follows: an image pre-processing unit for performing illumination compensation and normalization processing on the collected image to eliminate the influence of ambient illumination changes; The model training unit adopts the Cutout data augmentation strategy to enhance the model's generalization ability to local occlusion by randomly generating occlusion boxes on the training images; The model optimization unit performs sparse training and channel pruning compression on the YOLOv5 model, reducing computational complexity and ensuring a full-process processing latency of <200ms.

[0011] Preferably, the three-dimensional positioning and navigation module includes: a point cloud generation unit and a coordinate calculation unit, wherein the point cloud generation unit generates three-dimensional point cloud data of tender shoots through stereo matching and triangulation based on the parallax method of binocular vision; and the point cloud generation unit uses a spatial constraint algorithm to remove background noise and mismatched points to optimize the point cloud quality; The coordinate calculation unit fits the edge lines of the tea rows through improved Hough transform, and calculates the precise three-dimensional coordinates of the picking points in combination with point cloud data, thereby achieving precise navigation of the tea picking machine.

[0012] Preferably, the robotic arm control module includes: The system includes a trajectory planning unit that uses a fifth-order Bezier curve to plan the picking path and achieves smooth motion control through the following methods: a1. Decomposing the picking process into three stages: acceleration, constant speed, and deceleration; a2. Using a fifth-order Bezier curve to interpolate the trajectory, making the velocity and acceleration exhibit the continuous characteristics of fourth-order and third-order Bezier curves, respectively; a3. Precise trajectory control is achieved by constraining control points, and midpoint continuity conditions are used to achieve smooth connection of multiple curve segments; It also includes an end-effector unit, which includes a pneumatically controlled rubber gripper and an air suction pipe, and performs the following operations: b1. The air pressure is adjusted by an air pump to control the end closure; b2. The gripper wrapped in natural rubber is used to gently grip the tender buds; b3. The picked tea leaves are directly transported to the collector using the air suction pipe.

[0013] Preferably, the tea health monitoring module includes: Multispectral imaging unit for synchronously capturing visible light and near-infrared images of tea leaves; Intelligent analysis unit, which runs a monitoring model optimized based on neural networks and genetic algorithms, and identifies the type and severity of pests and diseases through multispectral feature analysis; The result output unit is used to generate a visual report containing pest and disease information and growth status, and mark the location of abnormal buds and leaves.

[0014] Also included is an intelligent tea picking method based on the above system, comprising: S1. Use binocular vision cameras and RGB sensors to synchronously collect images of the tea garden environment, collect images of tea leaves at different times and lighting conditions, and classify the collected images as a dataset; S2. Dataset enhancement: Using the Cutout augmentation strategy algorithm, occlusion boxes are randomly generated on the image. The image will be partially occluded during model training, reducing the reliance on local features and enhancing the learning of global features. The Cutout strategy can enhance the robustness and generalization of the dataset to a certain extent. Process image data in real time on the embedded edge computing platform to annotate the young shoots in the image; S3, generate three-dimensional point cloud data based on binocular ranging technology and calculate the three-dimensional coordinates of the picking point; S4, planning the motion trajectory of the robotic arm and controlling the end effector to complete the gripping and cutting of the buds; S5. Monitor tea growth status and pest and disease conditions through multispectral imaging and screen out unqualified buds and leaves; S6. Real-time analysis of tea growth status through the tea health monitoring module; S7. Use the closed-loop feedback module to monitor the tea collection status and trigger a full warehouse recovery instruction.

[0015] Preferably, the image processing step includes: (1) Image preprocessing: perform illumination compensation and normalization on the captured images; (2) Tender shoot identification: An improved YOLOv5 model is used to perform multi-scale feature fusion to identify tender shoots that meet the standards; (3) Three-dimensional positioning: Based on binocular vision technology, the following steps are used to achieve precise positioning of tender shoots: generate a three-dimensional point cloud using the principle of triangulation; define and filter the point cloud pose for the one-bud-one-leaf or one-bud-two-leaf feature; extract the center point of the recognition frame as the target centroid for spatial positioning; calculate the precise distance between the picking point and the binocular camera; (4) Model optimization: Reduce computational complexity through model pruning and compression to ensure processing delay < 200ms.

[0016] Preferably, the coordinate calculation step includes:

[0017] First, based on the parallax principle of binocular vision, the depth information of the tea shoots is calculated using a stereo matching algorithm and triangulation methods to generate high-precision 3D point cloud data. During the point cloud data processing stage, a spatial constraint algorithm is used to effectively remove background noise and mismatched points. Point cloud filtering and pose calibration are performed based on the unique morphological characteristics of the tea shoots. Two OTSU threshold segmentation techniques are used to eliminate the interference of environmental shadows on positioning accuracy.

[0018] In the precise coordinate calculation stage, the improved Hough transform algorithm is used to accurately fit the edge lines of the tea rows. The three-dimensional spatial coordinates (X, Y, Z) of the picking points are calculated based on the optimized point cloud data, and the final picking position is optimized and adjusted by comprehensively considering the workspace constraints of the robotic arm.

[0019] Preferably, the picking action execution step includes: c1. using a fifth-order Bezier curve to plan a smooth motion trajectory, and realizing continuous acceleration trajectory control by constraining control points to ensure the smoothness of the movement of the robotic arm in the acceleration, uniform speed and deceleration stages; c2. Pneumatically controlled rubber grippers simulate manual, gentle gripping, with gripping force automatically adjusted based on the characteristics of the buds. c3. The picked tea leaves are directly transported to the collector using an air intake pipe. A gravity sensor at the bottom of the collector monitors the weight of the tea leaves in real time and triggers an automatic recovery command when the threshold is reached. The picking process satisfies the following kinematic constraints: 1) The velocity and acceleration at the starting and ending points of the motion are zero; 2) The velocity and acceleration curves during the motion process are continuous; 3) There is an extreme point in the acceleration rate of change to avoid mechanical vibration.

[0020] Preferably, the tea health monitoring step includes: first, synchronously collecting visible light and near-infrared band image data of tea leaves through a multispectral imaging system, and simultaneously performing targeted data collection in combination with the tea row grid area information provided by a visual navigation system; During the data processing phase, the OTSU threshold segmentation algorithm was used twice to effectively eliminate environmental shadow interference, and an edge point extraction algorithm based on spatial constraints was used to remove interference points caused by tea leaves with cavities. An improved Hough transform algorithm was then used to accurately fit the edge features of the tea leaves, providing an accurate positioning benchmark for subsequent analysis. Finally, the optimized neural network classification model is run to realize the intelligent identification of unqualified buds and leaves such as insect-infested buds and diseased buds.

[0021] In the result output phase, the system automatically generates a visual report including an assessment of the tea growth status, accurately marks the location of detected unqualified buds and leaves, and provides detailed classification statistics. It also triggers a graded early warning mechanism based on pre-set quality standards. The monitoring results will be fed back to the picking control system in real time, guiding the tea picking machine to automatically avoid areas with pests and diseases, and at the same time record the coordinates of abnormal areas for subsequent manual review and quality control.

[0022] Compared with the prior art, the present invention has the following beneficial effects: The present invention achieves localized parallel processing of visual data through a heterogeneous computing platform. Combining front-end multimodal data acquisition, middle-layer YOLOv5 young shoot identification and positioning, and terminal millisecond-level response actuators, it significantly improves the picking success rate and system robustness in complex tea garden scenarios. The system uses binocular visual navigation and young shoot point cloud positioning technology to accurately calculate the three-dimensional coordinates of the picking points; the end effector simulates manual picking movements to avoid damaging the young tea buds; and it also has real-time monitoring capabilities for tea growth status and pests and diseases. The present invention solves the problems of insufficient recognition accuracy and poor adaptability of traditional tea-picking machinery, achieving efficient, accurate, and automated tea picking. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a schematic diagram of a two-person lifting tea plucking machine; Figure 2 This is a schematic diagram of a handheld tea picking machine; Figure 3 This is a schematic diagram of the tea picking process of the intelligent tea picking robot system of the present invention; Figure 4 This is a diagram of the network structure for intelligent detection of tea buds using the improved YOLOv5 network in the present invention.

[0024] Figure 5 This is a schematic diagram of the results before improvement of the present invention, which has low density and severe out-of-focus blur.

[0025] Figure 6 This is a schematic diagram of the improved result of the present invention with low density and severe out-of-focus blur.

[0026] Figure 7 This is a schematic diagram of the results before improvement of the present invention with medium density, medium out-of-focus blur, and complex background.

[0027] Figure 8 This is a schematic diagram of the improved results of the present invention with medium density, medium out-of-focus blur, and complex background.

[0028] Figure 9 This is a schematic diagram of the results before improvement of the present invention with high density, severe out-of-focus blur, complex background, and the presence of multi-scale targets.

[0029] Figure 10 This is a schematic diagram of the improved results of the present invention with high density, severe out-of-focus blur, complex background, and the presence of multi-scale targets.

[0030] Figure 11 This is a schematic diagram of the binocular ranging principle in the present invention.

[0031] Figure 12 This is a flow chart of the three-dimensional reconstruction of tea shoots in the present invention.

[0032] Figure 13 This is a schematic diagram of the design of the rectangular coordinate robotic arm in the present invention.

[0033] Figure 14 This is a schematic diagram of tea leaf positioning, picking and collecting in the present invention.

[0034] Figure 15 Schematic diagram of velocity and acceleration curves under the modified trapezoidal law in the present invention.

[0035] Figure 16 This is a performance comparison table of tea bud intelligent detection models. DETAILED DESCRIPTION

[0036] The specific embodiments of the present invention are described in detail below, but it should be understood that the protection scope of the present invention is not limited by the specific embodiments.

[0037] Example 1:

[0038] See also Figure 1-15 The present invention provides an intelligent tea-picking robot system with an embedded edge computing architecture, comprising: A multimodal data acquisition module includes a binocular vision camera and an RGB sensor. The binocular vision camera is installed at a 60° angle to the horizontal direction. The binocular vision camera is used to fit the edge lines of the tea rows through OTSU threshold segmentation and improved Hough transform to guide the movement of the tea picking machine. Embedded edge computing platform, integrating high-performance processors and storage devices, for running lightweight YOLOv5 models; 3D positioning and navigation module, used to calculate the 3D coordinates of the picking point based on binocular vision technology; The robotic arm control module includes a five-degree-of-freedom robotic arm and an end effector; Tea health monitoring module, including a multispectral camera and data analysis unit; Closed-loop feedback module, including gravity sensor and communication unit.

[0039] The embedded edge computing platform is configured as: an image pre-processing unit for performing illumination compensation and normalization on the captured image to eliminate the influence of ambient illumination changes; The model training unit adopts the Cutout data augmentation strategy to enhance the model's generalization ability to local occlusion by randomly generating occlusion boxes on the training images; The model optimization unit performs sparse training and channel pruning compression on the YOLOv5 model, reducing computational complexity and ensuring a full-process processing latency of <200ms.

[0040] It should be specifically explained that the above-mentioned intelligent tea picking robot system is mainly divided into three stages for tea picking: identification stage, positioning stage and picking stage.

[0041] The recognition phase is divided into two parts: dataset establishment and model training.

[0042] 1. Establishment of young shoot image dataset

[0043] 1. Analysis of tea characteristics Tea leaves exhibit different color characteristics in different months, and since most tea gardens are open-air, climatic factors can cause the color characteristics of tea leaves to change. For example, changes in lighting conditions can also shift the threshold division between young and old leaves.

[0044] Therefore, we collected images classified according to different time and lighting conditions as a dataset. Since different tea varieties have different color and morphological characteristics, we mainly analyzed Huangshan Maofeng tea, and eventually generalized it to other tea varieties.

[0045] 2. Image acquisition According to the picking time, it is divided into spring tea, summer tea and autumn tea; according to the lighting conditions, it is divided into cloudy and sunny days; according to the occlusion situation, it is divided into "Y" type and "I" type, and images are collected for each type to create a data set.

[0046] In order to reduce the obstruction of young shoots, the camera installation angle was selected to be 60° from the horizontal.

[0047] 3. Dataset creation A large number of original tea tree images were taken on site to obtain tea image data, and the young shoots in the images were labeled (using the LabelImg visual image annotation tool) to generate an XML file of the original dataset.

[0048] 4. Dataset Augmentation The dataset was augmented using a cutout augmentation strategy algorithm, which randomly generates occlusion boxes on the image. This partially occludes the image during model training, reducing reliance on local features and enhancing learning of global features. Because tea buds are often occluded in natural environments, the cutout strategy can enhance the robustness and generalization of the dataset to a certain extent.

[0049] 2. Training and Evaluation of Tender Shoot Detection Model

[0050] 1. Model training refer to Figure 4 The YOLOv5 network architecture shown here performs intelligent tea bud detection. The YOLOv5-based tea bud detection network primarily consists of input, backbone, neck, prediction, and output. The input image size is 640*640*3. The final output is feature maps of three different scales: 80*80*18, 40*40*18, and 20*20*18.

[0051] 2. Model Evaluation Evaluate and compare the models used, such as Figure 16 As shown: Depend on Figure 16 It can be seen that the accuracy of the YOLOv5 model is 82.7% and the recall rate is 82.3%. The score is 82.5%, The YOLOv5 model performance is above average at 85.6%.

[0052] 3. Tender Shoot Detection Model Pruning and Compression The trained shoot detection model is subjected to sparse training, and the shoot detection model is pruned using channel pruning and layer pruning algorithms. The pruned model is then trained on data again.

[0053] Example 2: In combination with Example 1, it is further obtained that the three-dimensional positioning and navigation module includes: a point cloud generation unit and a coordinate calculation unit. The point cloud generation unit generates three-dimensional point cloud data of tender shoots through stereo matching and triangulation based on the parallax method of binocular vision; and the point cloud generation unit uses a spatial constraint algorithm to remove background noise and mismatched points to optimize the point cloud quality. The coordinate calculation unit fits the edge lines of the tea rows through improved Hough transform, and calculates the precise three-dimensional coordinates of the picking points in combination with point cloud data, thereby achieving precise navigation of the tea picking machine.

[0054] It is further obtained that the robotic arm control module includes: The system includes a trajectory planning unit that uses a fifth-order Bezier curve to plan the picking path and achieves smooth motion control through the following methods: a1. Decomposing the picking process into three stages: acceleration, constant speed, and deceleration; a2. Using a fifth-order Bezier curve to interpolate the trajectory, making the velocity and acceleration exhibit the continuous characteristics of fourth-order and third-order Bezier curves, respectively; a3. Precise trajectory control is achieved by constraining control points, and midpoint continuity conditions are used to achieve smooth connection of multiple curve segments; It also includes an end-effector unit, which includes a pneumatically controlled rubber gripper and an air suction pipe, and performs the following operations: b1. The air pressure is adjusted by an air pump to control the end closure; b2. The gripper wrapped in natural rubber is used to gently grip the tender buds; b3. The picked tea leaves are directly transported to the collector using the air suction pipe.

[0055] The tea health monitoring module includes: Multispectral imaging unit for synchronously capturing visible light and near-infrared images of tea leaves; Intelligent analysis unit, which runs a monitoring model optimized based on neural networks and genetic algorithms, and identifies the type and severity of pests and diseases through multispectral feature analysis; The result output unit is used to generate a visual report containing pest and disease information and growth status, and mark the location of abnormal buds and leaves.

[0056] An intelligent tea picking method based on the above system comprises: S1. Use binocular vision cameras and RGB sensors to synchronously collect images of the tea garden environment, collect images of tea leaves at different times and lighting conditions, and classify the collected images as a dataset; S2. Dataset enhancement: Using the Cutout augmentation strategy algorithm, occlusion boxes are randomly generated on the image. The image will be partially occluded during model training, reducing the reliance on local features and enhancing the learning of global features. The Cutout strategy can enhance the robustness and generalization of the dataset to a certain extent. Process image data in real time on the embedded edge computing platform to annotate the young shoots in the image; S2. Generate 3D point cloud data based on binocular ranging technology and calculate the 3D coordinates of the picking points; S3, planning the robot arm's motion trajectory and controlling the end effector to complete the bud gripping and cutting; S4. Monitor tea growth status and pest and disease conditions through multispectral imaging and screen out unqualified buds and leaves; S5. Real-time analysis of tea growth status through the tea health monitoring module; S6. Use the closed-loop feedback module to monitor the tea collection status and trigger a full warehouse recovery instruction.

[0057] The image processing step comprises: (1) Image preprocessing: perform illumination compensation and normalization on the captured images; (2) Tender shoot identification: An improved YOLOv5 model is used to perform multi-scale feature fusion to identify tender shoots that meet the standards; (3) Three-dimensional positioning: Based on binocular vision technology, the following steps are used to achieve precise positioning of tender shoots: generate a three-dimensional point cloud using the principle of triangulation; define and filter the point cloud pose for the one-bud-one-leaf or one-bud-two-leaf feature; extract the center point of the recognition frame as the target centroid for spatial positioning; calculate the precise distance between the picking point and the binocular camera; (4) Model optimization: Reduce computational complexity through model pruning and compression to ensure processing delay < 200ms.

[0058] The coordinate calculation step includes: First, based on the parallax principle of binocular vision, the depth information of the tea shoots is calculated using a stereo matching algorithm and triangulation methods to generate high-precision 3D point cloud data. During the point cloud data processing stage, a spatial constraint algorithm is used to effectively remove background noise and mismatched points. Point cloud filtering and pose calibration are performed based on the unique morphological characteristics of the tea shoots. Two OTSU threshold segmentation techniques are used to eliminate the interference of environmental shadows on positioning accuracy.

[0059] In the precise coordinate calculation stage, the improved Hough transform algorithm is used to accurately fit the edge lines of the tea rows. The three-dimensional spatial coordinates (X, Y, Z) of the picking points are calculated based on the optimized point cloud data, and the final picking position is optimized and adjusted by comprehensively considering the workspace constraints of the robotic arm.

[0060] What needs to be explained in detail is the positioning link: it is divided into two parts: tea picking machine positioning and tender shoot point cloud acquisition.

[0061] Tea picking machine visual navigation Using camera imaging principles, we perform offline internal and external parameter calibration to determine the mapping relationship between the camera imaging plane and the tea row plane. We then divide the grid based on the actual location information and display a virtual grid within the image. We then select the grid where the tea ridges are located, and use different thresholds for each grid to segment the image.

[0062] Two OTSU threshold segmentations were performed to remove shadows and ultimately segment the tea ridge area. A spatially constrained edge point extraction algorithm was used to remove interference from tea holes. An improved Hough transform was applied to the extracted edge points to create edge lines. This line was used to guide the movement of the tea picker, ensuring it stayed within the ridge.

[0063] Young shoot point cloud acquisition

[0064] 1. Point cloud acquisition Binocular Stereo Vision (Binocular Stereo Vision) technology is an important form of machine vision. It is based on the principle of visual difference and uses imaging equipment to obtain different images of the object being measured from different positions. The method then obtains the three-dimensional geometric information of the object by calculating the deviation values ​​of the corresponding points in the image information.

[0065] The principle of binocular ranging is modeled after the human eye's. To perceive the distance of an object, an observer needs to observe it from at least two angles. When the human eye observes an object, binocular disparity (also known as parallax) occurs. Binocular stereo vision technology, based on disparity theory, uses two identical cameras to capture positional images of the same object from various angles, obtaining image data pairs. Using various algorithms, corresponding points are matched, and the disparity information is used to construct stereo and depth information.

[0066] 2. Precise positioning of tender shoots The 3D point cloud-based algorithm intuitively provides a plant's hierarchical structure and accurately locates the position of young shoots. This is achieved by combining binocular ranging with a young shoot detection model. Binocular vision-based 3D reconstruction methods offer advantages such as low cost, ease of implementation, minimal impact from illumination, and good real-time performance. They are particularly well-suited for 3D reconstruction of plant canopies in outdoor environments and are widely applicable for crop location and harvesting, plant growth monitoring, and tree pruning. The binocular vision-based 3D reconstruction method for tea shoot canopies is unaffected by weather and can provide users with a 3D model of the tea shoot canopy's growth at any time.

[0067] Considering the one bud and two leaves or one bud and one leaf features of the tea bud shape, targeted point cloud pose coordinate definition and filtering processing are carried out in the three-dimensional coordinate conversion link of the visual module. At the same time, intelligent parameter setting for distance monitoring between the tea bud picking point and the binocular camera is carried out to ensure timely and accurate segmentation and decision-making timing during the binocular recognition and positioning process.

[0068] By analyzing the principle of triangulation ranging, a ranging experiment based on a binocular vision system was conducted. By combining a deep learning recognition system with a binocular camera system, the center point of the recognition frame was extracted as the target centroid, and the distance and spatial positioning of the tea target was performed. Finally, the ranging error was obtained, which can be used to effectively identify and locate the tea buds.

[0069] Example 3: In conjunction with the first embodiment, the picking action execution step includes: c1. using a fifth-order Bezier curve to plan a smooth motion trajectory, and realizing continuous acceleration trajectory control by constraining control points to ensure the smoothness of the movement of the robot arm in the acceleration, uniform speed and deceleration stages; c2. Pneumatically controlled rubber grippers simulate manual, gentle gripping, with gripping force automatically adjusted based on the characteristics of the buds. c3. The picked tea leaves are directly transported to the collector using an air intake pipe. A gravity sensor at the bottom of the collector monitors the weight of the tea leaves in real time and triggers an automatic recovery command when the threshold is reached. The picking process satisfies the following kinematic constraints: (1) The velocity and acceleration at the starting and ending points of the motion are zero; (2) The velocity and acceleration curves during the movement are continuous; (3) There is an extreme point in the rate of change of acceleration to avoid mechanical vibration.

[0070] Embodiment 9: In combination with Example 5, it is further obtained that the tea health monitoring step includes: first, synchronously collecting visible light and near-infrared band image data of tea leaves through a multispectral imaging system, and simultaneously performing targeted data collection in combination with the tea plantation grid area information provided by the visual navigation system; During the data processing phase, the OTSU threshold segmentation algorithm was used twice to effectively eliminate environmental shadow interference, and an edge point extraction algorithm based on spatial constraints was used to remove interference points caused by tea leaves with cavities. An improved Hough transform algorithm was then used to accurately fit the edge features of the tea leaves, providing an accurate positioning benchmark for subsequent analysis. Finally, the optimized neural network classification model is run to realize the intelligent identification of unqualified buds and leaves such as insect-infested buds and diseased buds.

[0071] In the result output phase, the system automatically generates a visual report including an assessment of the tea growth status, accurately marks the location of detected unqualified buds and leaves, and provides detailed classification statistics. It also triggers a graded early warning mechanism based on pre-set quality standards. The monitoring results will be fed back to the picking control system in real time, guiding the tea picking machine to automatically avoid areas with pests and diseases, and at the same time record the coordinates of abnormal areas for subsequent manual review and quality control.

[0072] What needs to be explained specifically is the picking process: Research on Key Technologies for Tea Picking Robots: Domestic and international scholars have conducted extensive research on the identification and positioning of tea buds, proposing a variety of bud recognition algorithms based on traditional image processing techniques, machine learning, and deep learning. These algorithms aim to improve recognition accuracy and positioning precision, providing accurate target position information for robotic arm motion control. Regarding tea-picking robotic arms, different structural forms, including serial robots, parallel robots, and Cartesian robots, have been applied to tea picking, each with its own unique characteristics and advantages. Serial robots offer a large workspace, simple structure, low manufacturing cost, and simplified control; parallel robots have a strong load capacity, high precision, and low end-of-line inertia; and Cartesian robots offer a large workspace and low manufacturing cost. Furthermore, research and analysis of the robotic arms' workspaces and end-of-line workload have been conducted to determine whether they meet the requirements of tea picking.

[0073] Study on the working of the robotic arm The tea bud picking device mainly consists of a mounting platform, an air pump connection port, and rubber picking fingers. The specific working principle is: after the end is installed on the moving platform, the robot arm sends the device to the picking point. Then, the air pressure is adjusted by the air pump to control the closure of the execution end at the lower end of the device. After the tea buds are clamped, they are cut and picked by the mechanical end effector. The tea leaves are then collected by the suction pipe of the end effector of the tea picking robot. The inlet end of the suction pipe of the end effector of the tea picking robot is located above the mechanical claw, and the outlet end is located in the tea collector. The picked tea leaves will be sucked into the suction pipe and transported to the inside of the tea collector through the pipe.

[0074] The tea-picking robot's tea collector is equipped with a gravity sensor at the bottom, which monitors the weight of the collected tea leaves. If the weight reaches a predetermined value, the robot enters a recovery state, transporting the collected tea leaves to a warehouse or temporary storage point. The tea leaves in the collector are then manually collected. Compared to a robotic arm directly clamping the tea leaves into the tea collector, the robot's suction-based transport of the tea leaves significantly reduces picking time because it eliminates the need for reciprocating motion between the tea leaves and the collector. The automatic collection of young tea buds significantly reduces labor costs during tea picking, allowing for timely collection and processing of young tea buds to prevent deterioration and flavor loss.

[0075] End effector position interpolation calculation During the end-of-line picking motion, it's important to consider the smooth operation of position, velocity, and acceleration. This section uses a fifth-order Bezier curve to interpolate the picking trajectory. The corresponding velocity and acceleration are fourth-order and third-order Bezier curves, respectively, both of which are continuous and meet the requirements of motion. Furthermore, compared to fifth-order polynomial trajectory planning, fifth-order Bezier trajectory planning can achieve trajectory control by constraining control points. Based on the midpoint continuity control condition, it can achieve continuity between two curves, facilitating the connection of multiple Bezier curves.

[0076] The Bezier curve is a type of parametric curve. The n-order Bezier curve defined by a+1 control points q1, q2, q3, and q4 can be recursively expressed by the following formula:

[0077] 1-1

[0078] Where t[0,1];Bi,n(t) is the Bezier curve coefficient under the t parameter, which is defined as:

[0079] 1-2

[0080] Its k-th order derivative is:

[0081] (t) 1-3

[0082] When n=6, the fifth-order Bessel end motion position can be obtained according to the formula

[0083] 1-4

[0084] The preliminary design plan of this rectangular coordinate robot arm is as follows Figure 13 This solution uses a cross structure, with the x-axis located above the y-axis. The x / y robotic arm is installed face-down, and the y-axis is balanced by the left and right mounting positions on the x-axis. The z-axis is raised and lowered using the previous rack and pinion mechanism, with claw fingers at the end. Equipment technical parameters: The x / y / z axis working strokes are 400mm / 400mm / 100mm respectively. The x / y axis is positive when the slide moves away from the respective drive motor end, and the z axis is positive when it moves vertically downward. The x / y axis moving speed is 200mm / s, and the z axis is not required for the time being; The x / y / z axis positioning accuracy is 0.1mm, and the repeatability is ±0.05mm.

[0085] Trajectory analysis of picking movement In the process of the end picker from starting to move to completing the leaf bud collection, it generally goes through three movement processes: acceleration, constant speed, and deceleration. The path that the end picker passes through during these three movement processes is represented by an interpolation function. In order to ensure that the speed of the end picker is consistent and there is no rigid impact during the movement, the motion trajectory should be a smooth curve. In the linear interpolation method, a small "transition section" is added to each corner on the motion path. This curve is usually selected as a parabola or a sine function curve to ensure a smooth transition of the driving joint angular displacement and angular velocity without speed jumps.

[0086] The trajectory of the end-point picker and the selection of the transition curve directly impact the design and research of the automatic tea plucking machine, determining the robot's kinematic performance during tea picking, such as smoothness, operational safety, and overall energy consumption. During the tea plucking process, the robot performs the picking and collection operations between suitable leaf buds and the collection device, depending on the task at hand. Figure 14 The figure shows the basic process of a single picking operation. In process 1, the global camera of the vision system identifies the X and Y coordinates of a suspected bud by color, driving the end-point picker to the suspected bud's location. In process 2, the binocular camera determines whether the suspected bud is suitable for picking based on features such as bud shape and color. If it is not suitable, the process returns to process 1 and moves to the next suspected bud. If it is suitable, the bud shape features are used to determine the X, Y, and Z coordinates of the picking point. These coordinates are then sent to the control board, controlling the end-point picker to approach the leaf bud and pick it.

[0087] At the same time, for each linear interpolation and circular interpolation motion trajectory, the motion law of the end picker is set as: stop-acceleration-deceleration-stop. Figure 15 According to the picking motion requirements of the automatic tea picking machine, the trajectory of the displacement function should meet the following conditions: (1) The velocity and acceleration of the end picker should be zero at the starting and ending points of its picking movement; (2) The velocity-time curve and acceleration-time curve of the end picker must be continuous; (3) The first-order derivative curve of the end picker's motion acceleration with respect to time should have a maximum / minimum value, otherwise it will cause vibration of the end picker.

[0088] In summary, through the precise identification and positioning of tea leaves in the early stage, the end-effector position interpolation range and trajectory analysis were obtained to ensure the optimal working state under different conditions, providing a guarantee for the entire process of intelligent tea picking.

[0089] The above disclosures are only a few specific embodiments of the present invention. However, the embodiments of the present invention are not limited thereto. Any changes that can be conceived by those skilled in the art should fall within the scope of protection of the present invention.

Claims

1. An intelligent tea-picking robot system with an embedded edge computing architecture, characterized in that: include: A multimodal data acquisition module includes a binocular vision camera and an RGB sensor. The binocular vision camera is installed at a 60° angle to the horizontal direction. The binocular vision camera is used to fit the edge lines of the tea rows through OTSU threshold segmentation and improved Hough transform to guide the movement of the tea picking machine. Embedded edge computing platform, integrating high-performance processors and storage devices, for running lightweight YOLOv5 models; 3D positioning and navigation module, used to calculate the 3D coordinates of the picking point based on binocular vision technology; The robotic arm control module includes a five-degree-of-freedom robotic arm and an end effector; Tea health monitoring module, including a multispectral camera and data analysis unit; Closed-loop feedback module, including gravity sensor and communication unit.

2. The system according to claim 1, wherein: The embedded edge computing platform is configured as: an image pre-processing unit for performing illumination compensation and normalization on the captured image to eliminate the influence of ambient illumination changes; The model training unit adopts the Cutout data augmentation strategy to enhance the model's generalization ability to local occlusion by randomly generating occlusion boxes on the training images; The model optimization unit performs sparse training and channel pruning compression on the YOLOv5 model, reducing computational complexity and ensuring a full-process processing latency of <200ms.

3. The system according to claim 1, wherein: The three-dimensional positioning and navigation module includes: a point cloud generation unit and a coordinate calculation unit. The point cloud generation unit generates three-dimensional point cloud data of tender shoots through stereo matching and triangulation based on the parallax method of binocular vision; and the point cloud generation unit uses a spatial constraint algorithm to remove background noise and mismatched points to optimize the point cloud quality. The coordinate calculation unit fits the edge lines of the tea rows through improved Hough transform, and calculates the precise three-dimensional coordinates of the picking points in combination with point cloud data, thereby achieving precise navigation of the tea picking machine.

4. The system according to claim 1, wherein: The robotic arm control module includes: The system includes a trajectory planning unit that uses a fifth-order Bezier curve to plan the picking path and achieves smooth motion control through the following methods: a1. Decomposing the picking process into three stages: acceleration, constant speed, and deceleration; a2. Using a fifth-order Bezier curve to interpolate the trajectory, making the velocity and acceleration exhibit the continuous characteristics of fourth-order and third-order Bezier curves, respectively; a3. Precise trajectory control is achieved by constraining control points, and midpoint continuity conditions are used to achieve smooth connection of multiple curve segments; It also includes an end-effector unit, which includes a pneumatically controlled rubber gripper and an air suction pipe, and performs the following operations: b1. The air pressure is adjusted by an air pump to control the end closure; b2. The gripper wrapped in natural rubber is used to gently grip the tender buds; b3. The picked tea leaves are directly transported to the collector using the air suction pipe.

5. The system according to claim 1, wherein: The tea health monitoring module is configured as follows: Multispectral imaging unit for synchronously capturing visible light and near-infrared images of tea leaves; Intelligent analysis unit, which runs a monitoring model optimized based on neural networks and genetic algorithms, and identifies the type and severity of pests and diseases through multispectral feature analysis; The result output unit is used to generate a visual report containing pest and disease information and growth status, and mark the location of abnormal buds and leaves.

6. An intelligent tea picking method based on the system according to any one of claims 1 to 5, characterized in that: include: S1. Use binocular vision cameras and RGB sensors to synchronously collect images of the tea garden environment, collect images of tea leaves at different times and lighting conditions, and classify the collected images as a dataset; S2. Dataset enhancement: Using the Cutout augmentation strategy algorithm, occlusion boxes are randomly generated on the image. The image will be partially occluded during model training, reducing the reliance on local features and enhancing the learning of global features. The Cutout strategy can enhance the robustness and generalization of the dataset to a certain extent. Process image data in real time on the embedded edge computing platform to annotate the young shoots in the image; S3, generate three-dimensional point cloud data based on binocular ranging technology and calculate the three-dimensional coordinates of the picking point; S4, planning the motion trajectory of the robotic arm and controlling the end effector to complete the gripping and cutting of the buds; S5. Monitor tea growth status and pest and disease conditions through multispectral imaging and screen out unqualified buds and leaves; S6. Real-time analysis of tea growth status through the tea health monitoring module; S7. Use the closed-loop feedback module to monitor the tea collection status and trigger a full warehouse recovery instruction.

7. The method according to claim 6, characterized in that Said S2 includes: (1) Image preprocessing: perform illumination compensation and normalization on the captured images; (2) Tender shoot identification: An improved YOLOv5 model is used to perform multi-scale feature fusion to identify tender shoots that meet the standards; (3) Three-dimensional positioning: Based on binocular vision technology, the following steps are used to achieve precise positioning of tender shoots: generate a three-dimensional point cloud using the principle of triangulation; define and filter the point cloud pose for the one-bud-one-leaf or one-bud-two-leaf feature; extract the center point of the recognition frame as the target centroid for spatial positioning; calculate the precise distance between the picking point and the binocular camera; (4) Model optimization: Reduce computational complexity through model pruning and compression to ensure processing delay < 200ms.

8. The method according to claim 6, characterized in that Said S3 includes: First, based on the parallax principle of binocular vision, the depth information of the tea shoots is calculated through a stereo matching algorithm and triangulation methods to generate high-precision 3D point cloud data. During the point cloud data processing stage, a spatial constraint algorithm is used to effectively remove background noise and mismatched points. At the same time, point cloud filtering and pose calibration are performed based on the unique morphological characteristics of the tea shoots. The interference of environmental shadows on positioning accuracy is eliminated through two OTSU threshold segmentation techniques. In the precise coordinate calculation stage, the improved Hough transform algorithm is used to accurately fit the edge lines of the tea rows. The three-dimensional spatial coordinates (X, Y, Z) of the picking points are calculated based on the optimized point cloud data, and the final picking position is optimized and adjusted by comprehensively considering the workspace constraints of the robotic arm.

9. The method according to claim 6, characterized in that Said S4 comprises: c1. using a fifth-order Bezier curve to plan a smooth motion trajectory, realizing continuous acceleration trajectory control by constraining control points, and ensuring the smoothness of the motion of the manipulator in the acceleration, uniform speed and deceleration stages; c2. Pneumatically controlled rubber grippers simulate manual, gentle gripping, with gripping force automatically adjusted based on the characteristics of the buds. c3. The picked tea leaves are directly transported to the collector using an air intake pipe. A gravity sensor at the bottom of the collector monitors the weight of the tea leaves in real time and triggers an automatic recovery command when the threshold is reached. And the following kinematic constraints are satisfied in S4: 1) The velocity and acceleration at the starting and ending points of the motion are zero; 2) The velocity and acceleration curves during the motion process are continuous; 3) There is an extreme point in the acceleration rate of change to avoid mechanical vibration.

10. The method according to claim 6, characterized in that The tea health monitoring steps include: first, synchronously collecting visible light and near-infrared band image data of tea leaves through a multispectral imaging system, and simultaneously performing targeted data collection in combination with the tea plantation grid area information provided by a visual navigation system; During the data processing phase, the OTSU threshold segmentation algorithm was used twice to effectively eliminate environmental shadow interference, and an edge point extraction algorithm based on spatial constraints was used to remove interference points caused by tea leaves' cavities. An improved Hough transform algorithm was then used to accurately fit the edge features of the tea leaves, providing an accurate positioning benchmark for subsequent analysis. Finally, the optimized neural network classification model is run to achieve intelligent identification of insect-infested buds, diseased buds and unqualified buds and leaves; In the result output phase, the intelligent tea-picking robot system with embedded edge computing architecture automatically generates a visual report containing an assessment of the tea growth status, accurately marks the location of detected unqualified buds and leaves, and conducts detailed classification statistics, and triggers a graded early warning mechanism based on preset quality standards; The monitoring results will be fed back to the picking control system in real time, guiding the tea picking machine to automatically avoid areas with pests and diseases, and at the same time record the coordinates of abnormal areas for subsequent manual review and quality control.

Images