Automatic picking device for agricultural planting and method thereof

Abstract

The invention relates to an automatic picking device for agricultural planting and a method thereof, and belongs to the technical field of agricultural robots and automation. A base of the multi-degree-of-freedom mechanical arm is vertically fixed to the center position of the ground moving platform; the six-dimensional force / torque sensor is mounted at the tail end of a wrist joint of the multi-degree-of-freedom mechanical arm in series; the multi-mode sensing clamp holder is connected to the six-dimensional force / torque sensor, the multi-mode sensing clamp holder comprises a clamp holder base, the clamp holder base is connected with the six-dimensional force / torque sensor, and the multi-mode sensing clamp holder is connected with the six-dimensional force / torque sensor; and the stereoscopic vision camera is arranged at the front end of the gripper base, and the operation adaptability to fruits in different states is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of agricultural robots and automation, and particularly to an automatic picking device for agricultural planting and a method thereof. BACKGROUND

[0002] At present, the existing automatic picking device for agricultural planting usually adopts fixed mechanical action to perform picking operation. However, this mode is difficult to adapt to the differences in the form and connection state between different crop individuals. This fixed operation mode is prone to cause two main problems: one is that it may cause damage to the fruits, affecting their commodity value; the other is that it may cause damage to the plants, affecting their subsequent growth. Therefore, how to design an automatic picking device and method with self-adaptive ability to adapt to the characteristics of different crops, so as to reduce the damage to fruits and plants, has become a technical problem to be solved in this field.

[0003] However, in the related art, with the progress of automation technology and robot technology, although some automatic picking devices have appeared, these devices still have some problems or weaknesses in actual application: the picking device in the prior art is usually difficult to identify and quantify the firmness of the fruit connection, that is, the toughness, and cannot dynamically adjust the picking strategy according to the specific physical characteristics of the fruit. This fixed strategy without perception leads to unstable operation success rate and high damage rate when facing fruits of different maturity, different varieties or different growth states, and needs to be improved.

[0004] The above information disclosed in the above background section is only used to strengthen the understanding of the background of the present disclosure, and therefore it can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0005] The purpose of the present application is to provide an automatic picking device for agricultural planting and a method thereof to solve the problems raised in the above background.

[0006] The technical solution of the present application is as follows:

[0007] a ground mobile platform;

[0008] a multi-degree-of-freedom manipulator, a base of the multi-degree-of-freedom manipulator being vertically fixed to a center position of the ground mobile platform;

[0009] a six-axis force / torque sensor, the six-axis force / torque sensor being installed in series at the end of a wrist joint of the multi-degree-of-freedom manipulator;

[0010] a multi-modal perception gripper, the multi-modal perception gripper being connected to the six-axis force / torque sensor, the multi-modal perception gripper comprising:

[0011] a gripper base connected with the six-dimensional force / torque sensor;

[0012] a stereo vision camera arranged at the front end of the gripper base;

[0013] two oppositely arranged flexible wrapping fingers, the roots of which are mounted at the front end of the gripper base;

[0014] a high-frequency vibration cutting knife fixed on the gripper base and mounted between the roots of the two flexible wrapping fingers.

[0015] Preferably, the inside of the gripper base is integrated with a high-frequency micro-vibration motor for driving the multi-modal perception gripper to produce weak vibration.

[0016] Preferably, the inside of each flexible wrapping finger is paved with a flexible film pressure sensor array, the flexible wrapping finger is a hollow structure, and is connected with an air pipe to control its bending wrapping.

[0017] Preferably, the ground mobile platform comprises a rectangular chassis frame, and each corner of the chassis frame is mounted with an omnidirectional wheel driven by an independent motor.

[0018] An automatic picking method for agricultural planting, comprising the following steps:

[0019] a step of obtaining target three-dimensional space information and performing preliminary positioning, which is used to drive the stereo vision camera to obtain image data, and calculate and generate three-dimensional space coordinates of target fruits based on the image data, so as to drive the multi-degree-of-freedom mechanical arm to move the multi-modal perception gripper to a preliminary position of the target fruits;

[0020] a step of obtaining contact pressure distribution data and performing posture correction picking, which is used to drive the multi-degree-of-freedom mechanical arm to descend, obtain pressure distribution data of the initial contact point by the flexible film pressure sensor array inside the flexible wrapping finger, correct the posture of the multi-modal perception gripper according to the pressure distribution data, and control the flexible wrapping finger to stably pick the target fruits;

[0021] a step of obtaining fruit stem physical characteristic data and evaluating the toughness, which is used to drive the high-frequency micro-vibration motor inside the gripper base to produce sweep frequency vibration, and monitor vibration force signal feedback in real time by the six-dimensional force / torque sensor to obtain fruit stem physical characteristic data, and calculate and generate a fruit stem toughness index quantifying separation difficulty based on the fruit stem physical characteristic data;

[0022] The index is equivalent to a beam structure with elasticity and damping, and the controller is based on the resonance frequency point causing the maximum feedback amplitude and the half-power bandwidth of the resonance peak, and the energy dissipation characteristics of the stem are calculated by the preset physical model, and the numerical value is a dimensionless value, which is used to quantify the difficulty of separating the stem;

[0023] A step of generating and executing an adaptive separation strategy is generated, which is used to compare the stem toughness index with a preset threshold to match the optimal separation strategy, and to control the multi-degree-of-freedom robot arm and the high-frequency vibration cutting knife to perform picking according to the optimal separation strategy.

[0024] Preferably, in the step of obtaining the physical characteristic data of the stem and evaluating the toughness, the physical characteristic data of the stem includes a resonance frequency point causing the maximum feedback amplitude, which is used for subsequent separation steps.

[0025] Preferably, before driving the high-frequency micro-vibration motor, the method further comprises:

[0026] A step of performing physical characteristic pre-labeling of the fruit is performed, which is used to drive the multi-degree-of-freedom robot arm to perform a preset slight shaking action, and the pressure response delay time and pressure peak decay rate are collected by the flexible film pressure sensor array to generate a damping compensation coefficient for correcting the stem toughness index.

[0027] Preferably, the step of generating and executing an adaptive separation strategy comprises:

[0028] The stem toughness index is compared with a first threshold and a second threshold;

[0029] If the stem toughness index is lower than the first threshold, the optimal separation strategy is set to a twisting separation strategy;

[0030] Under this strategy, the controller will drive the wrist of the multi-degree-of-freedom robot arm to perform a slow rotation with a slight upward pulling action, and use shear stress to naturally break the stem;

[0031] If the stem toughness index is between the first threshold and the second threshold, the optimal separation strategy is set to a vibration-assisted separation strategy;

[0032] Under this strategy, the controller will drive the high-frequency vibration cutting knife to vibrate at the previously measured resonance frequency, while driving the robot arm to apply a continuous but smaller pulling force, and use resonance energy concentration to break the stem connection until it is separated;

[0033] If the stem toughness index is higher than the second threshold, the optimal separation strategy is set to a precise cutting strategy.

[0034] Preferably, during the implementation of the optimal separation strategy, the six-dimensional force / torque sensor continuously monitors the separation force and sets the determination basis for successful picking when the separation force reading suddenly drops to only the weight of the fruit and the gripper itself.

[0035] The present application provides an automatic picking device and method for agricultural planting by improving the prior art, which has the following improvements and advantages compared with the prior art:

[0036] 1. The present application cooperates with a stereo vision camera, a flexible thin film pressure sensor array and a six-dimensional force / torque sensor, so that the device can obtain the three-dimensional coordinates of the target, correct the grasping posture and evaluate the fruit stem toughness, which allows the picking action to be adjusted according to the characteristics of different fruits, improving the adaptability of the device to different state fruits;

[0037] 2. The present application quantifies the separation difficulty by calculating the fruit stem toughness index and selects the most suitable separation strategy accordingly, for example, for fruit stems with low toughness, twist separation is used to reduce physical damage; for fruit stems with moderate toughness, a high-frequency vibration cutting knife driven by a resonance frequency point is used for vibration-assisted separation to efficiently destroy the connection area; for high toughness fruit stems, precise cutting is used. This adaptive control avoids unnecessary violent operation, thereby reducing physical damage to the fruit and the plant while ensuring success rate;

[0038] 3. The ground mobile platform of the present application adopts an omni-directional wheel design, which has high mobility and can realize multi-directional translation and in-place rotation, reducing the space required for operation in narrow passages and improving the deployment efficiency of the overall work. BRIEF DESCRIPTION OF DRAWINGS

[0039] The present application will be further explained in conjunction with the drawings and examples:

[0040] Figure 1 is a schematic diagram of the overall structure of the device;

[0041] Figure 2 is a schematic diagram of the structure of the ground mobile platform;

[0042] Figure 3 is a schematic diagram of the structure of the ground mobile platform;

[0043] Figure 4 is a schematic diagram of the method flow structure of the present application.

[0044] In the drawings:

[0045] 100, ground mobile platform; 200, multi-degree-of-freedom manipulator; 210, six-axis force / torque sensor; 300, multi-modal perception gripper; 310, gripper base; 311, high-frequency micro-vibration motor; 320, stereo vision camera; 330, flexible wrapping finger; 331, flexible thin film pressure sensor array; 340, high-frequency vibration cutting knife. DETAILED DESCRIPTION

[0046] In order to make the objects, technical solutions and advantages of the present application clearer, further detailed description will be made to the present application in combination with specific embodiments.

[0047] Embodiment 1:

[0048] Please refer to Figures 1-3 The present application provides an automatic picking device for agricultural planting, comprising:

[0049] The ground mobile platform 100;

[0050] The multi-degree-of-freedom manipulator 200 is vertically fixed to the center position of the ground mobile platform 100.

[0051] The six-axis force / torque sensor 210 is installed in series at the wrist joint end of the multi-degree-of-freedom manipulator 200.

[0052] The multi-modal perception gripper 300 is connected to the six-axis force / torque sensor 210, and the multi-modal perception gripper 300 comprises:

[0053] The gripper base 310 is connected to the six-axis force / torque sensor 210.

[0054] The stereo vision camera 320 is arranged at the front end of the gripper base 310.

[0055] Two oppositely arranged flexible wrapping fingers 330 are installed at the front end of the gripper base 310.

[0056] The high-frequency vibration cutting knife 340 is fixed to the gripper base 310, and the installation position is between the roots of the two flexible wrapping fingers 330.

[0057] The embodiment provides an automatic picking device for agricultural planting. The picking device in the prior art generally adopts fixed mechanical action, and it is difficult to adapt to the differences in individual shapes and connection states of crops, so that fruit damage or plant damage is caused. The automatic picking device for agricultural planting solves the above problems through cooperative work of various components. A ground moving platform 100 provides a moving capability of the whole device between ridges. A multi-degree-of-freedom mechanical arm 200, for example, a six-axis articulated robot, is fixed to the ground moving platform 100, and the function of the six-axis articulated robot is to deliver a work end to a target position. A six-dimensional force / torque sensor 210, for example, a Mini40 model of ATI Industrial Automation Company, is connected in series at the end of the mechanical arm, and is used for measuring small force and torque changes in the work process. A multi-modal perception gripper 300 is the core of execution and perception, and integrates a stereo vision camera 320 for acquiring three-dimensional information of a target, a flexible wrapping finger 330 for soft grasping, and a high-frequency vibration cutting knife 340 for separation work. The combination of the structure enables the device to not only move and execute actions, but also has visual and force perception capabilities, thereby providing a hardware foundation for subsequent implementation of adaptive picking actions based on perception, and improving work adaptability to fruits in different states.

[0058] A high-frequency micro vibration motor 311 is integrated in the inside of the gripper base 310, and the high-frequency micro vibration motor 311 is used for driving the multi-modal perception gripper 300 to generate weak vibration.

[0059] The high-frequency micro vibration motor 311 in the embodiment, for example, a hollow cup vibration motor with a diameter of 4 mm, is integrated in the inside of the gripper base 310. The high-frequency micro vibration motor 311 is used for providing a controllable weak vibration source for the multi-modal perception gripper 300. When the motor works, mechanical vibration generated by the motor is transmitted to the grasped fruit through the gripper base 310 and further transmitted to the fruit stem. The vibration is not directly used for separation, but is used as a physical detection means to actively excite the system formed by the fruit and the fruit stem. By analyzing the response of the system to the excitation, data about the physical characteristics of the fruit stem can be obtained, and necessary input information for subsequent evaluation of the firmness of the connection, that is, the toughness, is provided.

[0060] A flexible film pressure sensor array 331 is laid on the inner side of each flexible wrapping finger 330. The flexible wrapping finger 330 is a hollow structure, and is connected with an air pipe to control bending and wrapping.

[0061] The flexible wrapping fingers 330 in this embodiment adopt a hollow silica gel structure and are connected to a miniature electrically controlled proportional pressure valve, such as the PVQ series valve from SMC, through an air pipe; by precisely controlling the gas pressure entering the inside, the fingers can be bent to different degrees, thereby achieving gentle wrapping of fruits of different sizes; the flexible thin film pressure sensor array 331, such as the FlexiForce sensor array from Tekscan, is laid on the inner side surface of each finger in contact with the fruit. The function of the flexible thin film pressure sensor array 331 is to detect the contact pressure and its distribution between the fingers and the fruit surface in real time during the grabbing process; these pressure data have two main uses: one is to judge whether the grabbing posture is accurate after initial contact and to provide the basis for fine-tuning of the mechanical arm to achieve centralized and stable holding of the fruit; the other is to monitor the size of the grabbing force during the wrapping process to ensure that the force is sufficient to stably fix the fruit, but will not cause damage due to excessive squeezing.

[0062] The ground mobile platform 100 comprises a rectangular chassis frame, and an omnidirectional wheel driven by an independent motor is installed at each corner of the chassis frame.

[0063] The ground mobile platform 100 in this embodiment is composed of a rectangular chassis frame, and an omnidirectional wheel driven by an independent motor is installed at each corner of the chassis frame; the purpose of this design is to give the entire device high mobility; in an agricultural planting environment, such as a greenhouse or a field, the work channel is often narrow and the path is complex; the application of omnidirectional wheels enables the ground mobile platform 100 to realize various movement modes such as forward movement, backward movement, lateral translation, and in-place rotation without changing its own orientation; this movement ability reduces the space required by the device during travel and positioning, enabling it to more conveniently shuttle between crop rows and quickly and accurately reach the designated work point, thereby improving the overall deployment efficiency of the work.

[0064] Embodiment 2:

[0065] Please refer to Figure 4 An automatic picking method for agricultural planting, comprising the following steps:

[0066] The step of obtaining target three-dimensional space information and performing preliminary positioning is used to drive the stereo vision camera 320 to obtain image data, and to calculate and generate the three-dimensional space coordinates of the target fruit based on the image data, and then to drive the multi-degree-of-freedom mechanical arm 200 to move the multi-modal perception gripper 300 to a preliminary position of the target fruit;

[0067] The step of acquiring contact pressure distribution data and performing posture correction grabbing is used to drive the multi-degree-of-freedom robot arm 200 to descend, and the flexible film pressure sensor array 331 inside the flexible wrapping finger 330 acquires the pressure distribution data of the initial contact point, and then the posture of the multi-modal perception gripper 300 is corrected according to the pressure distribution data, and the flexible wrapping finger 330 is controlled to stably grasp the target fruit;

[0068] The step of acquiring fruit stem physical characteristic data and evaluating toughness is used to drive the high-frequency micro-vibration motor 311 inside the gripper base 310 to generate sweep frequency vibration, and the vibration force signal feedback is monitored in real time by the six-axis force / torque sensor 210 to acquire the fruit stem physical characteristic data, and then the fruit stem toughness index quantifying the separation difficulty is calculated based on the fruit stem physical characteristic data;

[0069] The step of generating and executing an adaptive separation strategy is used to compare the fruit stem toughness index with a preset threshold to match the optimal separation strategy, and the multi-degree-of-freedom robot arm 200 and the high-frequency vibration cutting knife 340 are controlled according to the optimal separation strategy to perform picking.

[0070] The embodiment provides an automatic picking method for agricultural planting, which is executed by a controller such as an NVIDIA Jetson AGXXavier platform;

[0071] In the step of acquiring target three-dimensional space information and performing preliminary positioning, the stereo vision camera 320 collects an image pair of left and right views; the controller runs an instance segmentation algorithm to identify and segment the pixel region of the target fruit from the image; the three-dimensional space coordinates of the fruit center point are calculated by the triangulation principle in combination with the internal and external parameters of the two cameras; the coordinates are used to provide an initial target point for the multi-degree-of-freedom robot arm 200, so that the multi-modal perception gripper 300 can be quickly moved to a preliminary position above the fruit to complete coarse positioning.

[0072] In the step of acquiring contact pressure distribution data and performing posture correction grabbing, the multi-degree-of-freedom robot arm 200 controls the multi-modal perception gripper 300 to slowly descend until the fingertips of the flexible wrapping finger 330 slightly contact the surface of the fruit; at this time, the flexible film pressure sensor array 331 inside the finger is triggered and generates pressure distribution data of the initial contact point; the controller analyzes the data, and if the data shows that the contact point deviates from the central region of the finger, the controller calculates the necessary pose adjustment amount and drives the robot arm to perform micron-level translation or rotation to correct the posture of the gripper; the purpose of posture correction is to ensure that the subsequent wrapping action can be uniformly applied to the most stable part of the fruit; after the correction is completed, the controller instructs the proportional pressure valve to inflate the finger, and the overall pressure value fed back by the pressure sensor array reaches the preset stable grasping threshold.

[0073] In the step of acquiring the physical characteristic data of the fruit stalk and evaluating the toughness, the high-frequency micro-vibration motor 311 inside the gripper base 310 is activated and performs a sweep-frequency vibration from low frequency to high frequency. This vibration is transmitted to the fruit stalk through the gripper and the fruit; in this process, the six-axis force / torque sensor 210 installed on the wrist of the mechanical arm monitors the force and torque signal feedback generated by the vibration in real time at a high sampling rate; the controller analyzes these feedback signals and extracts the physical characteristic data of the fruit stalk from them; the core of these data is to determine the resonance frequency point that can cause the maximum force feedback; the controller further deduces based on the signal amplitude and phase information near the resonance frequency point through a pre-set physical model, and the deduction process is: the fruit stalk is equivalent to a beam structure with elasticity and damping, the relationship between the vibration input energy and the response energy measured by the sensor reflects the energy dissipation characteristics of the structure, and these characteristics are directly related to the internal fiber structure and water content of the fruit stalk; the controller synthesizes these characteristics to generate a dimensionless value, i.e. the fruit stalk toughness index, which is used to quantify the difficulty of separating the fruit stalk.

[0074] In order for those skilled in the art to be able to implement it, an exemplary calculation method is provided here, which does not constitute the only limitation of the present application; the calculation of the fruit stalk toughness index can be based on a model that parameterizes the key characteristics of the vibration response; in order to ensure that the index is a dimensionless value for threshold comparison, normalization processing of the measured values is required, for example, the index can be calculated by the following formula:

[0075] ;

[0076] Wherein:

[0077] is the fruit stalk toughness index, which is a dimensionless value;

[0078] is an empirical calibration coefficient, the value of which is obtained by calibrating the experimental measurements of known toughness samples;

[0079] is the amplitude peak value of the force signal feedback measured by the six-axis force / torque sensor 210 at the resonance frequency point ;

[0080] is a reference force value obtained by calibration, for example, the average amplitude peak value measured during calibration, which has the same dimension as ;

[0081] is the half-power bandwidth of the resonance peak, i.e. the amplitude drops to The difference between the two corresponding frequency points. This parameter reflects the damping size of the system. Generally, the stronger the connection and the more energy dissipation of the system, the wider the resonance peak, and the greater the value;

[0082] is the measured resonance frequency point, which is used to normalize the half-power bandwidth, so that the ratio becomes a dimensionless parameter;

[0083] Alternatively, in another optional embodiment, the controller can integrate a pre-trained neural network model; the model takes the complete frequency response curve data collected by the six-dimensional force / torque sensor 210 as input, including amplitude spectrum and phase spectrum, and directly outputs the corresponding fruit stalk toughness index through model inference calculation.

[0084] In the step of generating and executing the adaptive separation strategy, the controller compares the calculated fruit stalk toughness index with the preset threshold value stored in the strategy library; the result of the comparison is used to match the current optimal separation strategy from a plurality of pre-set separation strategies; for example, a lower toughness index will match a gentle separation action, and a higher index will match an action that requires the use of tools; once the strategy is selected, the controller will generate corresponding motion instructions to coordinate the action trajectory of the multi-degree-of-freedom robot arm 200 and the working state of the high-frequency vibration cutting knife 340 to perform the picking operation. The significance of this step is that the picking action is no longer fixed and unchanging, but is dynamically adjusted according to the results of each perception evaluation to adapt to the connection strength of different fruit stalks, thereby minimizing physical damage to the fruit and the plant while ensuring success rate.

[0085] In the step of obtaining fruit stalk physical characteristic data and evaluating toughness, the fruit stalk physical characteristic data includes the resonance frequency point that can cause the maximum feedback amplitude, which is used in the subsequent separation step.

[0086] The resonance frequency point in this embodiment is a key parameter in the fruit stalk physical characteristic data. The process of obtaining this frequency point is as follows: during the sweep frequency vibration, the controller continuously analyzes the amplitude spectrum of the feedback signal of the six-dimensional force / torque sensor 210, and records the vibration frequency corresponding to the peak amplitude as the resonance frequency point; this resonance frequency point is not only used to evaluate toughness, but also directly applied to the subsequent separation execution link; in some separation strategies, such as the vibration-assisted separation strategy for medium toughness fruit stalks, the controller will drive the high-frequency vibration cutting knife 340 to vibrate at this resonance frequency, the purpose of which is to use the resonance effect to most efficiently transmit vibration energy to the weakest delamination area of the fruit stalk and fruit connection, causing local stress concentration, thereby facilitating fatigue fracture of the fruit stalk under the action of a smaller external pulling force, and achieving low-damage separation.

[0087] Before driving the high-frequency micro-vibration motor 311, the method further comprises:

[0088] The step of pre-labeling the physical characteristics of the fruit is performed to drive the multi-degree-of-freedom robot arm 200 to perform a preset slight shaking action, and the flexible diaphragm pressure sensor array 331 collects the pressure response delay time and the pressure peak decay rate to calculate and generate a damping compensation coefficient for correcting the fruit stalk toughness index.

[0089] Before performing the fruit stalk toughness evaluation, the embodiment adds the step of pre-labeling the physical characteristics of the fruit; the purpose of this step is to separate and quantify the influence of the physical characteristics of the fruit itself, such as mass and hardness, on the subsequent vibration response measurement, so that the finally calculated toughness index more accurately reflects only the characteristics of the fruit stalk. The process is as follows: after stably grabbing the fruit, the controller drives the wrist joint of the multi-degree-of-freedom robot arm 200 to perform a preset slight shaking action with a specific trajectory and speed. During this period, the flexible diaphragm pressure sensor array 331 collects pressure data on the inside of the fingers at a high frequency; the controller calculates the equivalent inertia and damping of the fruit by analyzing the time difference between the time when the robot arm sends a motion instruction and the time when the pressure on the pressure sensor array reaches a peak, i.e., the pressure response delay time, and the decay rate of the pressure peak after the shaking stops. The derivation logic is as follows: the larger the mass or the softer the texture of the fruit, the more lagged the motion response will be, and the faster the energy dissipation will be. The controller calculates and generates a damping compensation coefficient based on these data. In the subsequent steps of obtaining the physical characteristics of the fruit stalk and evaluating the toughness, this damping compensation coefficient is used to correct the original feedback signal of the six-axis force / torque sensor 210. The correction method is to subtract the energy dissipation part caused by the characteristics of the fruit itself in the calculation model, so that the finally derived fruit stalk toughness index excludes the interference of the characteristics of the fruit.

[0090] To make the scheme more specific, the preset slight shaking action can be a simple harmonic vibration in the horizontal direction with a frequency of 5 Hz and an amplitude of 2 mm. The calculation process of the damping compensation coefficient can be specified as follows: the controller fits the decay process of the pressure peak after the shaking stops into an exponential decay function:

[0091] ;

[0092] Thus, the pressure decay constant ;

[0093] The damping compensation coefficient can be calculated by a model that combines the physical quantities representing the delay and the decay into a dimensionless parameter: ; wherein: is the pressure decay constant obtained by exponential fitting; is the measured pressure response delay time, here, Since the two quantities have dimensions of and respectively, they become dimensionless products, w is a dimensionless weighting factor calibrated by experiments.

[0094] Thereafter, in calculating the final pedicel toughness index, the original pedicel toughness index containing fruit interference is first calculated according to the above method , and then the final pedicel toughness index after correction is obtained in the manner of .

[0095] The steps of generating and executing the adaptive separation strategy include:

[0096] Comparing the pedicel toughness index with the first threshold value and the second threshold value;

[0097] If the pedicel toughness index is lower than the first threshold value, the optimal separation strategy is set as the twisting separation strategy;

[0098] If the pedicel toughness index is between the first threshold value and the second threshold value, the optimal separation strategy is set as the vibration-assisted separation strategy;

[0099] If the pedicel toughness index is higher than the second threshold value, the optimal separation strategy is set as the precise cutting strategy;

[0100] Under this strategy, the controller will fine-tune the multi-degree-of-freedom robot arm 200 based on the accurate pose of the fruit determined by the stereo vision camera 320 and the tactile pressure data, so that the cutting edge of the high-frequency vibration cutting knife 340 is accurately aligned with the pedicel, the vibration is started, and a quick cutting action is performed to achieve physical cutting of the pedicel.

[0101] The steps of generating and executing the adaptive separation strategy in the embodiment are embodied as a three-layer decision structure, which judges the pedicel toughness index based on two preset threshold values, the first threshold value and the second threshold value;

[0102] If the calculated pedicel toughness index is lower than the first threshold value, it indicates that the pedicel connection is very weak and easy to separate; at this time, the optimal separation strategy is set as the twisting separation strategy; the controller will drive the wrist of the multi-degree-of-freedom robot arm 200 to perform a slow rotation combined with a slight upward pulling action, and use the shear stress to make the pedicel naturally break;

[0103] If the pedicel toughness index is between the first threshold value and the second threshold value, it indicates that the pedicel has a standard strength of connection; at this time, the optimal separation strategy is set as the vibration-assisted separation strategy. The controller will drive the high-frequency vibration cutting knife 340 to vibrate at the previously measured resonance frequency, while driving the robot arm to apply a continuous but small pulling force, and use the resonance energy concentration to destroy the pedicel connection until it separates.

[0104] ​If the stem toughness index is higher than the second threshold value, it indicates that the stem connection is very tough. At this time, the optimal separation strategy is set to the precise cutting strategy; the controller will fine-tune the multi-degree-of-freedom robot arm 200 based on the accurate pose of the fruit determined by the stereo vision camera 320 and the tactile pressure data, so that the cutting edge of the high-frequency vibration cutter 340 is accurately aligned with the stem, the vibration is started, and a quick cutting action is performed to achieve physical cutting of the stem.

[0105] This interval-based strategy selection enables the picking device to adopt a targeted and less resource-consuming and physically damaging separation method for fruits of different maturity or varieties. It should be noted that the first threshold value and the second threshold value are not fixed values; they are empirical values obtained through a large number of physical picking experiments on specific picking objects, such as tomatoes of specific varieties and specific maturity stages, combined with corresponding stem toughness index measurement data, statistical analysis and calibration. These threshold values are stored in the parameter database of the controller, and can be selectively called or fine-tuned according to the type of crop to be picked before actual operation, to ensure the accuracy of the separation strategy selection.

[0106] During the execution of the optimal separation strategy, the six-axis force / torque sensor 210 continuously monitors the separation force, and sets the event node of the sudden drop of the separation force reading to only the weight of the fruit and the gripper as the basis for determining the success of picking.

[0107] The six-axis force / torque sensor 210 in the present embodiment also monitors the separation state during the entire process of executing the separation strategy; no matter what separation strategy is executed, the separation force acting on the sensor will disappear at the moment when the stem is disconnected from the plant; this will be reflected in the sensor readings, i.e. the value of the force or torque will suddenly and significantly decrease and stabilize at a new baseline; this new stable baseline value corresponds to the weight of the picked fruit and the multi-modal perception gripper 300 itself; the controller continuously compares the real-time sensor readings with this known baseline value; when it detects that the readings suddenly decrease and stabilize at this baseline value, the controller determines this event node as the sign of successful picking; this determination basis serves as a clear termination signal for the picking action; once the determination is successful, the controller immediately stops the current robot arm and cutter action, avoiding the damage to the plant caused by the unnecessary action after separation, and enabling the device to immediately enter the next process, such as placing the fruit into the collection box.

[0108] It should be noted that the above examples are only used to illustrate the technical solutions of the present application but not limit the present application. Although the present application is described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or equivalently replaced, without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. An automatic harvesting device for agricultural planting, characterized in that, include: Ground mobile platform (100); A multi-degree-of-freedom robotic arm (200) has its base vertically fixed at the center of the ground mobile platform (100); A six-dimensional force / torque sensor (210) is connected in series at the end of the wrist joint of the multi-degree-of-freedom robotic arm (200); A multimodal sensing gripper (300) is connected to the six-dimensional force / torque sensor (210), and the multimodal sensing gripper (300) includes: A gripper base (310) is connected to the six-dimensional force / torque sensor (210); A stereo vision camera (320) is disposed at the front end of the gripper base (310); Two flexible wrap-around fingers (330) arranged opposite each other, the roots of which are mounted on the front end of the gripper base (310); A high-frequency vibration cutting blade (340) is fixed on the clamp base (310) and installed between the roots of the two flexible wrapped fingers (330).

2. The automatic harvesting device for agricultural planting according to claim 1, characterized in that, The gripper base (310) integrates a high-frequency micro vibration motor (311), which is used to drive the multimodal sensing gripper (300) to generate weak vibrations.

3. The automatic harvesting device for agricultural planting according to claim 1, characterized in that, Each of the flexible wrapped fingers (330) has a flexible thin film pressure sensor array (331) laid on its inner side. The flexible wrapped fingers (330) are hollow structures and are connected to air tubes to control their bending and wrapping.

4. The automatic harvesting device for agricultural planting according to claim 1, characterized in that, The ground mobile platform (100) includes a rectangular chassis frame, and each of the four corners of the chassis frame is equipped with an omnidirectional wheel driven by an independent motor.

5. An automatic harvesting method for agricultural planting, applied to the automatic harvesting device for agricultural planting as described in claim 1, characterized in that, Includes the following steps: The steps of acquiring target three-dimensional spatial information and performing preliminary positioning are used to drive the stereo vision camera (320) to acquire image data, and calculate and generate the three-dimensional spatial coordinates of the target fruit based on the image data, thereby driving the multi-degree-of-freedom robotic arm (200) to move the multimodal sensing gripper (300) to the preparatory position of the target fruit. The steps of acquiring contact pressure distribution data and performing posture correction grasping are used to drive the multi-degree-of-freedom robotic arm (200) to descend, and the flexible thin film pressure sensor array (331) inside the flexible wrapped finger (330) to acquire the pressure distribution data of the initial contact point, and then correct the posture of the multimodal sensing gripper (300) according to the pressure distribution data, and control the flexible wrapped finger (330) to stably grasp the target fruit. The steps of acquiring physical characteristic data of the fruit stalk and evaluating toughness are as follows: the steps are used to drive the high-frequency micro vibration motor (311) inside the clamp base (310) to generate sweep frequency vibration, and the six-dimensional force / torque sensor (210) monitors the vibration force signal feedback in real time to obtain physical characteristic data of the fruit stalk, and then calculates and generates a fruit stalk toughness index that quantifies the separation difficulty based on the physical characteristic data of the fruit stalk. The steps of generating and executing an adaptive separation strategy are as follows: the steps are used to compare the fruit stalk toughness index with a preset threshold to match the optimal separation strategy, and to coordinate the multi-degree-of-freedom robotic arm (200) and the high-frequency vibration cutting blade (340) to perform picking according to the optimal separation strategy.

6. The automatic harvesting method for agricultural planting according to claim 5, characterized in that, In the steps of acquiring physical characteristic data of the fruit stalk and evaluating its toughness, the physical characteristic data of the fruit stalk includes the resonant frequency point that can cause the maximum feedback amplitude, and the resonant frequency point is used in the subsequent separation step.

7. The automatic harvesting method for agricultural planting according to claim 6, characterized in that, Before driving the high-frequency micro vibration motor (311), the method further includes: The step of pre-calibrating the physical properties of the fruit is performed. The step is used to drive the multi-degree-of-freedom robotic arm (200) to perform a preset micro-shaking motion, and the flexible thin film pressure sensor array (331) collects the pressure response delay time and pressure peak decay rate to calculate and generate a damping compensation coefficient for correcting the toughness index of the fruit stalk.

8. The automatic harvesting method for agricultural planting according to claim 5, characterized in that, The steps for generating and executing the adaptive separation strategy include: The pedicel toughness index is compared with a first threshold and a second threshold. If the fruit stalk toughness index is lower than the first threshold, then the optimal separation strategy is set to the torsion separation strategy; If the fruit stalk toughness index is between the first threshold and the second threshold, then the optimal separation strategy is set as a vibration-assisted separation strategy. If the pedicel toughness index is higher than the second threshold, then the optimal separation strategy is set as the precise cutting strategy.

9. The automatic harvesting method for agricultural planting according to claim 8, characterized in that, During the execution of the optimal separation strategy, the six-dimensional force / torque sensor (210) continuously monitors the separation force and sets the event node where the separation force reading suddenly drops to only the weight of the fruit and the gripper itself as the criterion for successful picking.

Images