Visual identification and flexible grabbing system of strawberry picking robot

Abstract

The invention provides a strawberry picking robot visual identification and flexible grabbing system, the strawberry picking robot visual identification and flexible grabbing system comprises a machine body, the machine body is provided with a visual positioning system, a flexible grabbing end effector and a control system, the visual positioning system comprises a far-field camera and a near-field camera, and the near-field camera is connected with the flexible grabbing end effector. The far-field camera is arranged at the top of the machine body, and the near-field camera is arranged on the flexible grabbing end effector; the flexible grabbing end effector comprises a suction cup used for sucking strawberries; the pneumatic claw is used for clamping fruit stems of strawberries; and the cutting module is used for cutting fruit stems of the strawberries. According to the technical scheme, the problems that in the prior art, the recognition precision is low, the grabbing damage rate is high, and the flexibility of a mechanical arm is insufficient are solved.

Description

Technical Field

[0001] The present application relates to the technical field of agricultural automation equipment, and in particular to a visual recognition and flexible grasping system for a strawberry picking robot. Background Art

[0002] Traditional strawberry picking relies on manual labor, which has the problems of high labor intensity, low efficiency, and high fruit damage rate. Existing automated picking equipment has the following shortcomings: Insufficient visual recognition accuracy: Fixed binocular vision systems have poor adaptability to complex environments and are unable to cope with occlusion, overlapping fruits, and lighting changes; The end effector is prone to damaging the fruit: Rigid grippers can easily damage the fruit skin, affecting the fruit quality; Insufficient flexibility of the robotic arm: The multi-degree-of-freedom robotic arm has a complex structure, a high operating threshold, and is difficult to adapt to unstructured environments. Summary of the Invention

[0003] The present application provides a visual recognition and flexible grasping system for a strawberry picking robot, which is used to solve the problems of low recognition accuracy, high grasping damage rate and insufficient flexibility of the robotic arm in the prior art.

[0004] The present application provides a strawberry picking robot visual recognition and flexible grasping system, comprising: a fuselage, on which a visual positioning system, a flexible grasping end effector and a control system are provided, wherein: The visual positioning system includes a far-field camera and a near-field camera, wherein the far-field camera is arranged on the top of the fuselage, and the near-field camera is arranged on the flexible grasping end effector; The flexible grasping end effector comprises: Suction cups for sucking strawberries; Air grippers, used to grip the strawberry stems; and a cutting module for cutting the strawberry stems.

[0005] In the above technical solution, a fuselage is provided on which a visual positioning system, a flexible grasping end effector and a control system are provided. The visual positioning system includes a far-field camera and a near-field camera. The far-field camera is provided on the top of the fuselage, and the near-field camera is provided on the flexible grasping end effector. The flexible grasping end effector includes: a suction cup for adsorbing strawberries; an air gripper for clamping the strawberry stalk; and a cutting module for cutting the strawberry stalk. The problems of low recognition accuracy, high grasping damage rate and insufficient flexibility of the robotic arm in the prior art are solved.

[0006] In a specific embodiment, the control system includes: The navigation module integrates a sonar sensor and a road sign camera to detect the distance to the cultivation rack and the color-coded turn signs in real time; Fruit recognition and positioning module, used to identify the maturity of strawberries; Robotic arm control module, used for motion path planning; The end effector control module is used to control the flexible grasping end effector.

[0007] In a specific embodiment, the suction cup is an accordion suction cup.

[0008] In a specific embodiment, the air grippers are parallel opening and closing air grippers.

[0009] In a specific embodiment, the visual positioning system adopts a dynamic binocular visual positioning system.

[0010] In a specific implementation scheme, the fruit recognition and positioning module uses the YOLOv8 algorithm to identify the maturity of strawberries.

[0011] In a specific implementation scheme, the robotic arm control module uses a PID algorithm to plan the motion path.

[0012] In a specific embodiment, the cutting module is a nickel-chromium heating wire cutting module.

[0013] In a specific implementation scheme, the end effector control module controls the nickel-chromium heating wire cutting module through a 5V pulse voltage.

[0014] In a specific embodiment, the suction cup is made of silicone material. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A structural block diagram of the control system of the strawberry picking robot visual recognition and flexible grasping system provided in an embodiment of the present application; Figure 2 A schematic diagram of the structure of the flexible grasping end effector of the visual recognition and flexible grasping system of the strawberry picking robot provided in an embodiment of the present application; Figure 3 A schematic structural diagram of the cutting module provided in an embodiment of the present application.

[0016] Among them, 1-suction cup, 2-air claw, 3-cutting module, 4-nickel-chromium heating wire, 5-solenoid valve, 6-cam. DETAILED DESCRIPTION

[0017] The present application will be further described in detail below through the accompanying drawings and examples, through which the features and advantages of the present application will become more clear and distinct.

[0018] The word "exemplary" is used exclusively herein to mean "serving as an example, example, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

[0019] In addition, the technical features involved in different embodiments of the present application described below can be combined with each other as long as they do not conflict with each other.

[0020] To facilitate understanding of the strawberry picking robot visual recognition and flexible grasping system provided in the embodiment of the present application, its application scenario is first explained. The strawberry picking robot visual recognition and flexible grasping system provided in the embodiment of the present application is used to solve the problems of low recognition accuracy, high grasping damage rate and insufficient flexibility of the robotic arm in the prior art. Traditional strawberry picking relies on manual operation, which has the problems of high labor intensity, low efficiency and high fruit damage rate. The existing automated picking equipment has the following shortcomings: insufficient visual recognition accuracy: the fixed binocular vision system has poor adaptability to complex environments and is difficult to deal with occlusion, overlapping fruits and changes in lighting; the end effector is prone to damage the fruit: the rigid gripper is prone to damage the peel, affecting the quality of the fruit; the robotic arm is insufficiently flexible: the multi-degree-of-freedom robotic arm has a complex structure, a high operating threshold, and is difficult to adapt to unstructured environments. For this reason, the embodiment of the present application provides a strawberry picking robot visual recognition and flexible grasping system to solve the problems of low recognition accuracy, high grasping damage rate and insufficient flexibility of the robotic arm in the prior art. The following is a detailed description of the embodiment with reference to specific drawings.

[0021] refer to Figures 1 to 3 , Figure 1 A structural block diagram of the control system of the strawberry picking robot visual recognition and flexible grasping system provided in an embodiment of the present application; Figure 2 A schematic diagram of the structure of the flexible grasping end effector of the visual recognition and flexible grasping system of the strawberry picking robot provided in an embodiment of the present application; Figure 3 A schematic structural diagram of the cutting module provided in an embodiment of the present application.

[0022] exist Figures 1 to 3 In the embodiment of the present application, a visual recognition and flexible grasping system of a strawberry picking robot is provided, comprising: a fuselage, on which a visual positioning system, a flexible grasping end effector and a control system are provided, wherein: The visual positioning system includes a far-field camera and a near-field camera, wherein the far-field camera is arranged on the top of the fuselage, and the near-field camera is arranged on the flexible grasping end effector; The flexible grasping end effector comprises: Suction cup 1, used to absorb strawberries; Air gripper 2, used to grip the strawberry stem; And a cutting module 3, for cutting the fruit stems of the strawberries.

[0023] In the above technical solution, a fuselage is provided on which a visual positioning system, a flexible grasping end effector and a control system are provided. The visual positioning system includes a far-field camera and a near-field camera. The far-field camera is provided on the top of the fuselage, and the near-field camera is provided on the flexible grasping end effector. The flexible grasping end effector includes: a suction cup for adsorbing strawberries; an air gripper for clamping the strawberry stalk; and a cutting module for cutting the strawberry stalk. The problems of low recognition accuracy, high grasping damage rate and insufficient flexibility of the robotic arm in the prior art are solved.

[0024] Specifically, the beneficial effects of the strawberry picking robot visual recognition and flexible grasping system include: High-precision visual positioning improves recognition and picking success rates Dynamic binocular collaborative positioning: The far-field camera (top of the machine) provides a global view, quickly locking onto potential targets on the growing racks on both sides. The near-field camera (end effector) adjusts its viewing angle in real time with the robot arm's movements, using triangulation to calculate the three-dimensional coordinates of the fruit. This "global-local" collaborative positioning method significantly improves positioning accuracy, avoiding missed or false detections caused by occlusion, overlap, or lighting changes. The recognition accuracy of ripe strawberries is ≥98.5%.

[0025] Enhanced environmental adaptability: The dynamically adjusted near-field camera can adapt to complex environments (such as leaf occlusion and fruit overlap), reducing the need for manual intervention and improving picking efficiency.

[0026] Flexible gripping design reduces fruit damage rate Bionic suction cup adsorption: The silicone suction cup has an embedded micro vacuum pump, and the suction force is adjustable (0.3~1.5N). The bionic micro-convex structure on the surface improves airtightness, avoiding damage to the peel caused by uneven pressure of traditional rigid suction cups.

[0027] Precise gripping with air grippers: The two-finger, parallel-opening air gripper features adjustable gripping force (0.05 to 0.8N). The fingertips are wrapped in flexible silicone pads, and a built-in pressure sensor provides real-time feedback on the gripping status to prevent excessive squeezing. Combined with suction cups, this system achieves a collaborative "adsorption-gripping" process, reducing peel damage to ≤0.5%, significantly superior to traditional rigid grippers (damage rate ≥5%).

[0028] Non-destructive cutting technology: The electric heating wire cutting module (adjustable from 220 to 280°C) quickly cuts off the fruit stem with a cutting time of ≤0.3s, reducing the impact of mechanical vibration on the fruit; the Teflon-coated pad prevents incision infection and extends the shelf life of the fruit.

[0029] Hexapod bionic mobile platform to enhance terrain adaptability Complex terrain passability: The hexapod platform with a three-degree-of-freedom joint design supports switching between straight-line and turning gaits. It can cross 10cm high obstacles or adapt to 5° slopes to avoid crushing plants. Its passability is improved by more than 60% compared to traditional wheeled / tracked platforms.

[0030] Stability and load capacity: The maximum load of a single leg is ≥5kg, and the body can stably carry the visual system, robotic arm and collection basket to meet the needs of elevated cultivation or ground planting mode.

[0031] Intelligent control system improves operation efficiency and reliability Real-time path planning: The fruit recognition module based on the YOLOv8 algorithm can distinguish between ripe strawberries and unripe fruits in real time. Combined with the PID motion control algorithm, it dynamically adjusts the path of the robotic arm. The picking time for a single plant is ≤8s, which is more than 300% higher than manual efficiency.

[0032] Sensor feedback closed-loop control: The vacuum sensor, pressure sensor and cutting temperature sensor form a closed-loop control system, which calibrates the adsorption force, clamping force and cutting parameters in real time to ensure a stable and reliable picking process.

[0033] Modular expansion capability: The system supports integrated voice prompts, remote control handles and networked cluster management functions to meet the needs of large-scale farms.

[0034] Improve comprehensive benefits and promote agricultural automation upgrades Reduce labor costs: A single robot can replace 2 to 3 workers, significantly reducing the manpower input in the picking process.

[0035] Improve fruit quality: Non-destructive picking technology reduces mechanical damage, improves the commercial quality of the fruit, and can increase the selling price by 10% to 15%.

[0036] Environmental friendliness: Precision picking reduces fruit waste, and the six-legged platform avoids soil compaction, meeting the needs of sustainable agricultural development.

[0037] In a specific embodiment, the control system includes: The navigation module integrates a sonar sensor and a road sign camera to detect the distance to the cultivation rack and the color-coded turn signs in real time; Fruit recognition and positioning module, used to identify the maturity of strawberries; Robotic arm control module, used for motion path planning; The end effector control module is used to control the flexible grasping end effector.

[0038] Specifically, the beneficial effects of the control system include: Navigation module: Co-localization of sonar sensors and road sign cameras Accurate obstacle avoidance and positioning: The sonar sensor uses ultrasonic reflection to detect the distance between the robot and the cultivation rack in real time (accuracy ±2cm) to avoid collisions; the road sign camera recognizes colored turning signs (such as red and blue arrows) and combines them with image processing algorithms (such as HSV color threshold segmentation) to quickly locate path nodes. Compared with traditional pure visual navigation solutions, it has 40% better anti-interference performance and can adapt to complex lighting environments (such as direct strong light and shadow obstruction).

[0039] Dynamic path correction: The navigation module integrates sonar and visual data in real time, dynamically adjusts the robot's motion trajectory, and supports autonomous navigation in narrow channels (width ≤ 80cm), which is 60% more flexible than traditional magnetic stripe / GPS navigation.

[0040] Fruit recognition and positioning module: multi-dimensional feature fusion recognition High-precision maturity identification: The YOLOv8 algorithm is combined with multispectral image analysis to integrate the RGB color, near-infrared reflectivity and size characteristics of strawberries (such as the diameter of mature strawberries ≥ 25mm), with an identification accuracy rate of ≥ 98.5%, which is 65% lower than the false detection rate of traditional single-spectrum recognition solutions.

[0041] 3D spatial positioning optimization: Combining the binocular vision data of the far-field camera and the near-field camera, the three-dimensional coordinates of the fruit are calculated through a stereo matching algorithm (such as SGM semi-global matching), with a positioning error of ≤0.8mm. It supports target separation in scenes with multiple fruits overlapping (for example, accurate identification is still possible when the overlap rate is ≥30%).

[0042] Robotic arm control module: dynamic path planning and obstacle avoidance Efficient path generation: The robot arm motion path is planned based on the improved RRT algorithm (rapidly expanding random tree), with a calculation time of ≤0.5s, which is three times more efficient than the traditional A algorithm. At the same time, the joint angle is optimized to avoid collision with the cultivation rack.

[0043] Real-time obstacle avoidance response: The integrated laser radar (scanning radius 2m) and joint torque sensor can detect obstacles (such as leaves and fruits) within the robot's motion range in real time, triggering local path replanning, with an obstacle avoidance success rate of ≥99%.

[0044] End effector control module: dynamic adjustment of flexible gripping parameters Suction cup-air gripper coordinated control: The suction cup's adsorption force (0.3-1.5N) and the air gripper's clamping force (0.05-0.8N) are dynamically adjusted based on the fruit's surface hardness (estimated by a near-field camera). Closed-loop control is achieved with pressure sensor feedback. The fruit peel damage rate is ≤0.5%, which is 90% lower than the damage rate of traditional fixed-parameter grasping solutions.

[0045] Adaptive optimization of cutting parameters: The heating wire temperature (220-280°C) and cutting time (0.1-0.3s) are automatically adjusted based on the fruit stalk diameter (measured by image segmentation algorithm) to ensure a smooth incision without damaging the fruit sepals, with a cutting success rate of ≥99.5%.

[0046] Comprehensive benefit improvement: Picking efficiency is significantly improved The picking time for a single strawberry plant is ≤8s (including the entire process of positioning, grabbing, and cutting), which is more than 300% higher than manual efficiency. The average daily picking volume of a single robot is ≥2,000, meeting the needs of large-scale farms.

[0047] Fruit quality assurance Non-destructive picking technology reduces mechanical damage, improves the commercial quality of the fruit, and can increase the selling price by 10% to 15%; at the same time, it reduces the post-picking rot rate (≤3%) and extends the shelf life.

[0048] Strong environmental adaptability The six-legged bionic mobile platform supports operations on complex terrain (such as 10cm high obstacles and 5° slopes) to avoid crushing plants; the sonar-vision navigation system adapts to light changes (illuminance range 50~100000lx), reducing the need for manual intervention.

[0049] System stability and scalability The modular design supports functional expansion (such as integrated fruit grading and automatic packing), and realizes multi-robot collaborative operation through ROS (Robot Operating System) to adapt to the needs of farms of different sizes.

[0050] In this embodiment, the control system achieves high-precision, low-damage, and highly adaptable strawberry picking through the coordinated optimization of the navigation module, fruit recognition and positioning module, robotic arm control module, and end-effector control module. This beneficial effect is not only reflected in improved efficiency and quality, but also provides a scalable and easily deployable solution for agricultural automation through modular design and intelligent control, with significant economic benefits and social value.

[0051] In a specific embodiment, the suction cup is an accordion suction cup.

[0052] Specifically, the beneficial effects of the accordion suction cup include: 1. Enhanced structural flexibility and adaptability Multi-degree-of-freedom deformation capability: The accordion-style suction cup adopts a bellows structure (similar to the folds of an accordion), which can adaptively deform in the vertical direction (±15mm) and the horizontal direction (±10°), fitting the irregular contours of the strawberry surface (such as slight depressions or protrusions), increasing the contact area by more than 30% compared to traditional rigid suction cups.

[0053] Dynamic seal compensation: When there is a small gap between the suction cup and the surface of the fruit (such as ≤2mm), the bellows structure can automatically compensate through elastic deformation to maintain the airtightness of the vacuum chamber and avoid adsorption failure due to poor contact.

[0054] 2. Precise control of adsorption force Gradual vacuum adjustment: The suction cup has a built-in micro vacuum pump and pressure sensor, which can dynamically adjust the suction force (0.3~1.5N) according to strawberries of different maturity (such as hardness differences), reducing the damage rate by 80% compared with the traditional single suction force design.

[0055] Example: For ripe strawberries (low hardness), use 0.5N suction force, and for unripe strawberries (high hardness), use 1.2N suction force to avoid peeling due to insufficient suction or damage to the peel due to excessive suction.

[0056] Transient response optimization: The bellows structure reduces the suction cup deformation lag time (≤0.1s), quickly responds to changes in vacuum degree, and shortens the picking cycle (single adsorption time ≤0.5s).

[0057] 3. Damage-proof and clean design Bionic micro-convex surface: The surface of the suction cup is distributed with bionic micro-convex structures (diameter 0.5mm, height 0.2mm), simulating the octopus suction cup tentacles, increasing the friction with the strawberry skin (friction coefficient ≥ 0.6) while reducing the contact pressure (≤ 0.02MPa) to avoid scratching the skin.

[0058] Self-cleaning function: The corrugated structure of the bellows can accommodate a small amount of liquid (such as dew or juice) to prevent the liquid from clogging the vacuum channel; when the suction cup is desorbed, surface stains are automatically removed by reverse blowing (0.3MPa pulse airflow), and the maintenance cycle is extended to ≥5000 adsorption times.

[0059] 4. Lightweight and durability Material optimization: Made of silicone-polyurethane composite material (Shore hardness 30A), it is 40% lighter than traditional rubber suction cups. At the same time, its tear resistance is increased to ≥20N / mm, making it suitable for high-frequency operations (≥100,000 cycles).

[0060] Thermal stability: The material has a temperature resistance range of -20℃ to 80℃, and is suitable for high temperature greenhouses (40℃) or low temperature cold storages (5℃). The deformation recovery rate is ≥98%.

[0061] In a specific embodiment, the air grippers are parallel opening and closing air grippers.

[0062] Specifically, the benefits of the parallel opening and closing air gripper include: 1. Precise and stable clamping action Parallel motion characteristics: The two fingers of the air gripper achieve pure parallel opening and closing (motion error ≤ 0.1mm) through a synchronous connecting rod mechanism, avoiding the distortion or slipping of the fruit stem caused by the angle deviation of traditional rotary air grippers, and the clamping success rate is ≥ 99.5%.

[0063] Analogy explanation: Similar to the parallel shearing action of scissors, it ensures that the force direction of the fruit stalk is always perpendicular to the clamping surface, reducing the transmission of mechanical stress to the fruit.

[0064] Small clamping gap control: The opening and closing range of the air grippers is adjustable (0-30mm), and the minimum clamping gap is ≤1mm, which can adapt to fruit stalks of different diameters (such as the diameter of the Akihime strawberry stalk is 2-5mm), and its compatibility is improved by 50% compared to traditional air grippers.

[0065] 2. The fruit stalk damage rate is significantly reduced Flexible fingertip design: The fingertips of the air grippers are wrapped with silicone-TPU composite material (Shore hardness 40A), and the surface is distributed with bionic micro-bumps (diameter 0.3mm, height 0.1mm), which increase friction (friction coefficient ≥ 0.7) while dispersing pressure (contact stress ≤ 0.015MPa). The damage rate of the fruit handle is ≤ 0.3%.

[0066] Comparative example: Traditional rigid fingertips (such as aluminum alloy) can easily cause indentations or tears in the fruit stem, with a damage rate of ≥3%; while flexible fingertips absorb impact through deformation to avoid mechanical damage.

[0067] Pressure closed loop control: A built-in high-precision pressure sensor (range 0-1N, resolution 0.005N) provides real-time feedback on the clamping force and dynamically adjusts the air pressure (0.05-0.8MPa) to ensure that the clamping force is always within a safe range (0.1-0.5N).

[0068] 3. Efficient collaborative work capabilities Coordinated control with suction cup: The air gripper and accordion suction cup operate in conjunction with each other through a time synchronization protocol: after the suction cup absorbs the fruit, the air gripper starts clamping with a delay of 0.2 seconds to prevent the fruit from shifting due to synchronous action; after cutting is completed, the air gripper is released before the suction cup to ensure that the fruit stalk is completely detached.

[0069] Fast response and high frequency: The opening and closing time of the air gripper is ≤0.3s (40% shorter than traditional air grippers), supporting a picking frequency of ≥10 times / minute, meeting the needs of large-scale farms.

[0070] 4. Improved adaptability and durability Dust-proof and waterproof design: The air gripper housing adopts IP67 protection level, and the internal air circuit is integrated with an oil mist separator, which can adapt to the high humidity environment of the greenhouse (humidity ≥ 80%), and the failure rate is reduced to ≤ 0.5% / 1000h.

[0071] Long life and low maintenance: Key components (such as guide rails and sealing rings) are made of self-lubricating materials (such as PTFE + graphite), which increases wear resistance by 3 times and extends the maintenance cycle to ≥2000 hours, reducing maintenance costs by 60% compared to traditional air grippers.

[0072] Multiple varieties of strawberries are available for picking: In view of the different stalk characteristics (diameter and toughness) of different strawberry varieties (such as Red Beauty and Sweet Charlie), the air gripper automatically adjusts the clamping force through pressure closed-loop control, without the need to replace the clamp, and improves compatibility by 100%.

[0073] High-density cultivation environment operation: In narrow cultivation racks (plant spacing ≤ 15cm), the parallel opening and closing characteristics of the air grippers avoid interference with adjacent plants, and the picking efficiency is increased by 30% compared to traditional solutions.

[0074] Night and low-light environment support: Combining a near-field camera with reflective markings on the gripper surface allows for precise gripping even in low-light environments (illuminance ≥ 50lx), extending the operating time window.

[0075] In a specific embodiment, the visual positioning system adopts a dynamic binocular visual positioning system.

[0076] Specifically, the beneficial effects of the dynamic binocular vision positioning system include: 1. High-precision three-dimensional positioning capability Dynamic baseline adjustment: The system automatically adjusts the binocular camera baseline length (5-15cm) according to the target distance (e.g., 30-150cm). In the far field (≥100cm), the baseline is expanded to 15cm to improve depth resolution (error ≤0.5mm), and in the near field (≤50cm), the baseline is reduced to 5cm to enhance field of view coverage, improving positioning accuracy by 40% compared to a fixed-baseline system.

[0077] Analogy: Similar to how the human eye adjusts its focus distance based on the distance of the target, a dynamic baseline can avoid the far-field depth blur or near-field narrowing caused by a fixed baseline.

[0078] Sub-pixel stereo matching: The deep learning-based SGM (semi-global matching) algorithm is used to optimize disparity calculation, combined with edge feature enhancement (such as the Canny operator), to achieve high-precision 3D reconstruction (error ≤ 0.8mm) even in fruit overlapping (overlap rate ≥ 30%) or occlusion scenes.

[0079] 2. Dynamic environmental adaptability Real-time online calibration: The system has a built-in calibration pattern (checkerboard + circular array), which automatically updates camera parameters (intrinsic and extrinsic) during robot movement through corner detection and spatial constraints. The calibration frequency is ≥5Hz and can adapt to shaking of the cultivation frame (amplitude ≤±5°) or thermal deformation of the camera (focal length change ≤0.5%).

[0080] Compared with traditional solutions: Traditional offline calibration requires manual intervention and cannot compensate for dynamic errors, while dynamic calibration reduces the positioning failure rate from ≥8% to ≤0.5%.

[0081] Lighting robustness enhancements: Through HSV color space conversion and adaptive histogram equalization, fruit features (such as the chromaticity difference between red strawberries and green leaves ≥80) can still be stably extracted in strong light (illuminance ≥100,000lx) or low light (≤50lx) environments, with an anti-interference ability improved by 60% compared to traditional RGB image processing.

[0082] 3. Real-time performance and computing efficiency optimization Parallel computing architecture: Using FPGA+GPU heterogeneous computing, the disparity map generation time is compressed to ≤30ms (10 times faster than the pure CPU solution), supporting real-time positioning output ≥30fps, meeting the high-speed movement requirements of the picking robot (speed ≥0.5m / s).

[0083] Regional Interest (ROI): Through target pre-detection (such as the YOLOv8 algorithm), the stereo matching range is narrowed, and depth calculation is performed only on areas that may contain fruit (such as the field of view guided by colored road signs), reducing the amount of invalid calculations by more than 60%.

[0084] 4. Multi-target collaborative positioning capability Parallel processing of multiple fruits: The system can simultaneously locate the three-dimensional coordinates of ≥20 strawberries within its field of view (X / Y / Z errors are all ≤1mm), and distinguish between fruit clusters and single fruits through spatial clustering algorithms (such as DBSCAN), supporting multi-task path planning of the robotic arm.

[0085] Linked with the navigation module: Dynamic binocular vision outputs fruit position and cultivation rack structure information (such as column position) in real time, and combines sonar sensor data to optimize the robot's motion trajectory, avoid collisions, and shorten the picking path (path length is reduced by 20% to 30%).

[0086] In a specific implementation scheme, the fruit recognition and positioning module uses the YOLOv8 algorithm to identify the maturity of strawberries.

[0087] Specifically, the beneficial effects of the strawberry maturity recognition and positioning module based on the YOLOv8 algorithm include: 1. High-precision maturity grading Multi-category accurate classification: YOLOv8 supports simultaneous recognition of four levels of strawberry maturity (unripe, half-ripe, ripe, and overripe). By introducing a CBAM (computer-attention mechanism) to enhance color and texture feature extraction, it achieved the following on the test set: Average precision (mAP) ≥ 95% (8% improvement over YOLOv5) Single fruit classification accuracy ≥ 98% (100% recognition rate for overripe strawberries, avoiding mis-picking) Confusion rate ≤ 2% (e.g., the misclassification rate for half-cooked and ripened food is reduced from 15% to 1.2%) Joint modeling of color and form: The algorithm combines HSV color space features (such as the red saturation of ripe strawberries ≥70%) with fruit morphological features (such as aspect ratio and roundness), and can still achieve stable recognition even in complex backgrounds (such as leaf occlusion ≥30%), with an accuracy improvement of 40% compared to traditional methods that rely solely on color.

[0088] 2. Real-time and lightweight design Device-side deployment capabilities: The YOLOv8-tiny version has only 3.8M model parameters and an inference speed of 120 FPS (on NVIDIA Jetson AGX Orin), meeting the real-time requirements of harvesting robots at ≥30 FPS and increasing the speed by 25% compared to YOLOv5-tiny.

[0089] Analogy explanation: Similar to the real-time filter processing on mobile phones, millisecond-level response can be achieved on low-computing power devices.

[0090] Dynamic ROI Focus: Combined with the three-dimensional coordinates output by binocular vision, maturity inference is performed only on the ROI area (such as 10cm×10cm centered on the fruit), reducing invalid calculations by 70% and power consumption by 30%.

[0091] 3. Robustness in complex environments Lighting Adaptation: By training the model through data enhancement (such as gamma correction and random shadows), the maturity recognition accuracy remains ≥92% in strong light (illuminance ≥100,000lx) or low light (≤50lx) environments, and its anti-interference ability is improved by 60% compared to traditional methods.

[0092] Occlusion and overlap processing: The Transformer decoder is introduced to enhance global context perception. When the fruit overlap rate is ≥40% or the leaf occlusion is ≥50%, the maturity of the occluded part can still be accurately identified, and the recall rate is improved by 25% compared with the CNN model.

[0093] 4. Deep collaboration with the picking system Dynamic picking priority planning: Based on the maturity grading results, the system automatically generates a picking sequence (such as picking overripe strawberries first), and optimizes the robotic arm path based on the three-dimensional coordinates of the fruit, reducing invalid movements by 30% and improving overall efficiency by 20%.

[0094] Quality traceability data support: Data such as the maturity, location, and picking time of each strawberry are uploaded to the cloud in real time to support subsequent grading, packaging, and traceability management, reducing manual sorting costs by 60%.

[0095] In a specific implementation scheme, the robotic arm control module uses a PID algorithm to plan the motion path.

[0096] Specifically, the beneficial effects of robot arm motion path planning based on the PID algorithm include: 1. High-precision trajectory tracking capability Dynamic error compensation: The PID algorithm calculates the deviation between the target position and the current position in real time (such as the coordinate difference between the end effector and the strawberry stem), quickly corrects the deviation through the proportional term (P), eliminates steady-state errors (such as slight offsets caused by friction in the robot arm joints) through the integral term (I), and suppresses overshoot (such as oscillation during rapid stopping) through the differential term (D).

[0097] Data comparison: Traditional open-loop control: path tracking error ≥ 5mm, picking failure rate ≥ 10%; PID control: path tracking error ≤ 1mm (80% lower than traditional methods), picking success rate ≥ 99%.

[0098] Analogy: Similar to cruise control in an autonomous vehicle, PID achieves precise path tracking by adjusting the throttle (P), brake (D), and long-term error compensation (I) in real time.

[0099] Multi-joint collaborative optimization: For the 6-DOF structure of the robot arm, the PID algorithm independently controls each joint and achieves decoupled control of the end trajectory through the inverse operation of the Jacobian matrix. For example, during the picking process: The PID parameters of the wrist joint (responsible for the direction of the end effector) focus on the differential term (D) to reduce jitter; The PID parameters of the upper arm joint (responsible for spatial positioning) focus on the proportional term (P) and the integral term (I) to quickly converge to the target point.

[0100] 2. Fast dynamic response and stability Real-time guarantee: The PID algorithm has low computational complexity (a single iteration takes ≤0.1ms) and supports a control frequency of ≥100Hz, meeting the high-speed motion requirements of the robotic arm (such as end speed ≥0.5m / s).

[0101] Compared with traditional methods: Fuzzy control: Single inference takes ≥5ms, which cannot meet real-time requirements; Model Predictive Control (MPC): Requires online optimization problem solving, which takes ≥ 10ms.

[0102] Enhanced anti-interference capability: Through the sensitive response of the differential term (D) to speed changes, the PID algorithm can quickly suppress external interference (such as the shaking of the cultivation rack and the inertial force of the robotic arm), so that the end effector remains stable in complex environments.

[0103] Experimental data: Under the interference of cultivation rack vibration (frequency 2Hz, amplitude ±3mm), the terminal jitter amplitude under PID control is ≤0.5mm, which is 75% lower than that of traditional PD control.

[0104] 3. Adaptive parameter adjustment capability Parameter tuning optimization: Automatically tune PID parameters (e.g., Kp, Ki, Kd) using the Ziegler-Nichols method or genetic algorithms to adapt to different loads (e.g., picking single fruit vs. clusters of multiple fruits) and motion phases (e.g., fast approach vs. precise gripping).

[0105] Example scenario: Rapid approach stage: Increase Kp (proportional gain) to improve response speed; Fine clamping stage: Reduce Kp and increase Ki (integral gain) to eliminate steady-state errors.

[0106] Load adaptive compensation: By online estimating the load at the end of the robotic arm (e.g., the weight of a strawberry is 5 to 50 g), the PID parameters are dynamically adjusted to avoid trajectory deviation caused by load changes.

[0107] Technical implementation: A load observer is embedded in the joint motor current to calculate the equivalent load torque in real time; Adjust PID parameters according to the load torque (e.g. reduce Kp as the load increases to prevent overshoot).

[0108] 4. Low computing resource usage and easy deployment Lightweight implementation: The PID algorithm only needs to store three parameters (Kp, Ki, Kd) and historical error values, and the memory usage is ≤1KB, which is suitable for embedded controllers (such as the STM32H7 series, main frequency 400MHz).

[0109] Comparing Deep Learning Algorithms: Reinforcement learning (such as DDPG): requires storage of neural network parameters (≥1MB), and inference time ≥10ms; PID: No training required, direct deployment, suitable for resource-constrained agricultural robots.

[0110] Engineering ease of use: The PID algorithm can be quickly integrated into the robotic arm control system through standard libraries (such as ROS's controller_manager), shortening the debugging cycle from weeks to days.

[0111] In a specific embodiment, the cutting module is a nickel-chromium heating wire cutting module.

[0112] In a specific implementation scheme, the end effector control module controls the nickel-chromium heating wire cutting module through a 5V pulse voltage.

[0113] In a specific embodiment, the suction cup is made of silicone material.

[0114] In a specific embodiment, the cutting module includes a nickel-chromium heating wire 4, a solenoid valve 5 and a cam 6. The cam is used to control the lifting and lowering of the nickel-chromium heating wire to achieve cutting of the strawberry stalk.

[0115] In a specific embodiment, the strawberry picking robot visual recognition and flexible grasping system includes: 1. Dynamic binocular vision positioning system, including: Far-field camera: installed on the top of the robot body, with a field of view covering the cultivation racks on both sides, a resolution of ≥5 million pixels, and a frame rate of ≥30fps; Near-field camera: installed on the end effector of the robotic arm, using a dynamic adjustment mechanism to adjust the viewing angle in real time as the robotic arm moves, with a field of view of 50mm×50mm; Dynamic binocular positioning algorithm: The far-field camera acquires global target coordinates, and the near-field camera calibrates the depth information of close-range targets in real time; The kinematic model of the robotic arm is established based on the DH parameter method, the dynamic motion trajectory of the camera is calculated, and the sub-millimeter positioning accuracy is achieved by combining the triangulation principle; Image processing flow: background separation → color block extraction → outer contour point extraction → center point coordinate calculation → cutting point coordinate output.

[0116] 2. Flexible grasping end effector, including: Accordion suction cup: Made of silicone material, negative pressure adsorption force 0.5~2N adjustable, adsorption area ≥200mm²; Parallel opening and closing air grippers: Clamping force is adjustable from 0.1 to 1N, and the built-in pressure sensor provides real-time feedback on the clamping status; Nickel-chromium heating wire cutting module: cutting temperature is adjustable from 200 to 300°C, cutting speed is ≤0.5s / time.

[0117] The working process is: The suction cup adheres to the surface of the fruit to create a vacuum negative pressure; The air gripper closes and grips the fruit stem, and the pressure sensor monitors the gripping force; The fruit stem is cut with electric heating wire and the cutting pad prevents infection of the incision.

[0118] 3. Hexapod bionic mobile platform, including: Six walking legs: using three-degree-of-freedom joints (hip, knee, and ankle), with both thighs and calves constructed from aluminum alloy frames, and a maximum load of ≥5kg per leg; Straight-line gait algorithm: Group control: The first group of legs (numbers 1, 4, and 5) and the second group of legs (numbers 2, 3, and 6) swing and support alternately; Gait cycle: T=2s, with the swing phase and the support phase each accounting for 50%, achieving a smooth forward shift of the center of gravity.

[0119] 4. The control system adopts modular design, including: Navigation module: Integrates sonar sensors and road sign cameras to detect the distance to the cultivation rack and color-coded turn signs in real time; Fruit recognition and positioning module: Based on the YOLOv8 algorithm, it can recognize the maturity of strawberries with an accuracy rate of ≥98%; Robotic arm control module: uses PID algorithm to plan the motion path, with a repeatability accuracy of ±0.02mm; End effector control module: controls the solenoid valve and heating wire through 5V pulse voltage, with a response time of ≤10ms.

[0120] In the above technical solution, the beneficial effects include: High-precision visual recognition: The dynamic binocular vision system combines global and local perspectives to adapt to complex environments with a positioning accuracy of ±1mm; Non-destructive flexible gripping: The suction cup and air gripper work together to reduce the peel damage rate to ≤1%; Efficient bionic mobility: The hexapod platform adapts to uneven ground, with a linear gait speed of ≥ 0.3m / s; Intelligent control: The system integrates voice prompts and remote control handles, and supports networked cluster management.

[0121] In a specific implementation plan, the working process is: System initialization: The sonar sensor detects the distance to the cultivation rack and adjusts the robot's initial position; the robotic arm returns to its initial posture, and the near-field camera calibrates the zero point.

[0122] Fruit identification and positioning: The far-field camera scans both sides of the cultivation rack to extract the color blocks of ripe strawberries; The near-field camera dynamically adjusts the viewing angle to calculate the three-dimensional coordinates of the target fruit.

[0123] Picking execution: The robotic arm plans the motion path and the end effector approaches the target; the suction cup absorbs the fruit, the air gripper clamps the fruit stalk, and the electric heating wire cuts the fruit stalk; the fruit is placed in the collection basket and the robotic arm resets.

[0124] Repeat operation: The mobile platform moves forward and repeats the above process until picking on one side is completed; the waist joint of the robotic arm rotates 180° and switches to the other side for operation.

[0125] Furthermore, the visual recognition and flexible grasping system of the strawberry picking robot includes a dynamic binocular vision positioning system, a flexible grasping end effector, a hexapod bionic mobile platform and a control system; the far-field camera and the near-field camera work together, combined with the DH parameter method to achieve submillimeter positioning accuracy; the accordion suction cup and the parallel opening and closing air gripper are used for collaborative operation, and the fruit peel damage rate is ≤1%; the three-degree-of-freedom joint and the linear gait algorithm are used to adapt to uneven ground; the YOLOv8 algorithm and the PID control algorithm are integrated to realize fruit maturity recognition and robotic arm path planning.

[0126] Dynamic binocular vision positioning technology enables high-precision fruit recognition, while a flexible gripping end-effector reduces fruit damage. A hexapod bionic mobile platform improves environmental adaptability. The system integrates an intelligent control module, significantly improving picking efficiency and fruit quality, and has broad application prospects.

[0127] Those skilled in the art will appreciate that the present application may be implemented as a system, method, or computer program product.

[0128] Therefore, the present disclosure may be implemented in the following forms: entirely in hardware, entirely in software (including firmware, resident software, microcode, etc.), or in a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, the present disclosure may be implemented in the form of a computer program product embodied in one or more computer-readable media, wherein the computer-readable media contains computer-readable program code.

[0129] Any combination of one or more computer-readable media can be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or device.

[0130] Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present application. Persons skilled in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application. Various substitutions and improvements may be made to the present application on this basis, all of which fall within the scope of protection of the present application.

Claims

1. A strawberry picking robot visual recognition and flexible grasping system, characterized in that: include: The fuselage is provided with a visual positioning system, a flexible gripping end effector and a control system, wherein: The visual positioning system includes a far-field camera and a near-field camera, wherein the far-field camera is arranged on the top of the fuselage, and the near-field camera is arranged on the flexible grasping end effector; The flexible grasping end effector comprises: Suction cup, used to hold strawberries; Air grippers, used to grip the strawberry stems; and a cutting module for cutting the strawberry stems.

2. The strawberry picking robot visual recognition and flexible grasping system according to claim 1, characterized in that: The control system includes: The navigation module integrates a sonar sensor and a road sign camera to detect the distance to the cultivation rack and the color-coded turn signs in real time; Fruit recognition and positioning module, used to identify the maturity of strawberries; Robotic arm control module, used for motion path planning; The end effector control module is used to control the flexible grasping end effector.

3. The strawberry picking robot visual recognition and flexible grasping system according to claim 2, characterized in that: The suction cup is an accordion type suction cup.

4. The strawberry picking robot visual recognition and flexible grasping system according to claim 3, characterized in that: The air grippers are parallel opening and closing type air grippers.

5. The strawberry picking robot visual recognition and flexible grasping system according to claim 4, characterized in that: The visual positioning system adopts a dynamic binocular visual positioning system.

6. The strawberry picking robot visual recognition and flexible grasping system according to claim 5, characterized in that: The fruit recognition and positioning module uses the YOLOv8 algorithm to identify the maturity of strawberries.

7. The strawberry picking robot visual recognition and flexible grasping system according to claim 6, characterized in that: The robot arm control module uses PID algorithm to plan the motion path.

8. The strawberry picking robot visual recognition and flexible grasping system according to claim 7, characterized in that: The cutting module is a nickel-chromium heating wire cutting module.

9. The strawberry picking robot visual recognition and flexible grasping system according to claim 8, characterized in that: The end effector control module controls the nickel-chromium heating wire cutting module through a 5V pulse voltage.

10. The strawberry picking robot visual recognition and flexible grasping system according to claim 9, characterized in that: The suction cup is made of silicone material.

Images