Fruit picking method and device, computer equipment and storage medium

Abstract

The invention provides a fruit picking method and device, computer equipment and a storage medium, and belongs to the field of agricultural picking. The method comprises the steps that under the condition that a clamping jaw picks fruits, magnetic field changes in the fruit contact process are obtained in real time, and three-dimensional contact force is determined through a pre-established mapping relation model from external force to magnetic field signals; based on the three-dimensional contact force, the maturity and the contact state of the fruits are estimated through a pre-established multi-field collaborative mapping model, and the multi-field collaborative mapping model comprises a viscoelastic contact mechanical model, a friction cone model and a damage energy model; according to the estimated maturity and the contact state, the clamping force compensation amount is calculated in combination with priori clamping force estimation and tactile feedback; the current clamping force of the clamping jaw is adjusted in real time based on the clamping force compensation amount, and stable grabbing and picking of fruits are achieved. Therefore, when the clamping jaws slightly slide or the softness changes, the force can be rapidly and smoothly adjusted, and stable grabbing and lossless picking are achieved.

Description

Technical Field

[0001] This invention belongs to the field of agricultural harvesting, specifically relating to a fruit harvesting method, apparatus, computer equipment, and storage medium. Background Technology

[0002] Fruit harvesting is a core step in intelligent production within protected horticulture, and the method of grasping directly impacts harvesting efficiency, fruit integrity, and subsequent commercial quality. Traditional manual harvesting relies on experience to adjust finger pressure, resulting in low efficiency and poor consistency. Automated harvesting devices, using rigid grippers or negative pressure suction cups, are prone to causing skin damage and flesh injury, especially noticeable in soft fruits like strawberries, tomatoes, and kiwis. For fruits with significant ripening variations, such as mangoes, avocados, and peaches, existing automated harvesting devices lack adaptive ripening control mechanisms, often resulting in the problem of "unripe fruits being difficult to pick, while ripe fruits are easily damaged," failing to meet the demand for high-quality, damage-free harvesting.

[0003] Currently, automated fruit harvesting technologies are mainly divided into three categories: mechanical gripping, vision-assisted detection, and tactile sensing-driven. Mechanical gripping harvests fruit through rigid or semi-flexible clamping, but lacks flexibility adjustment and is prone to causing localized compression damage. Vision-assisted detection mainly judges the harvesting time based on the fruit's appearance (color, shape, size, etc.), but is not sensitive enough to the fruit's internal hardness and ripeness. Tactile sensing-driven technologies rely on pressure or hardness sensors for feedback control, which can reduce damage to some extent, but is limited by insufficient sensing dimensions, mostly focusing on normal pressure monitoring, and is difficult to effectively perceive and model complex contact states such as tangential slippage and adhesion transition.

[0004] Therefore, existing harvesting methods all suffer from insufficient sensory dimensions, poor signal coupling, and poor robustness, resulting in limitations such as difficulty in simultaneously achieving slip suppression and damage minimization. Summary of the Invention

[0005] To address the aforementioned problems, the present invention provides a fruit harvesting method, apparatus, computer equipment, and storage medium.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A fruit-picking method is applied to a gripper made of flexible magnetic material, wherein the flexible magnetic material on the surface of the gripper generates a measurable magnetic field change when subjected to force; the method includes: When the gripper picks the fruit, the magnetic field changes during the contact process with the fruit are acquired in real time, and the three-dimensional contact force is determined by a pre-established mapping relationship model from external force to magnetic field signal. The three-dimensional contact force includes normal force and tangential force. Based on the three-dimensional contact force, the maturity and contact state of the fruit are estimated by a pre-established multi-field cooperative mapping model, wherein the multi-field cooperative mapping model includes a viscoelastic contact mechanics model, a friction cone model, and a damage energy model. Based on the estimated maturity and contact state, combined with prior clamping force estimation and tactile feedback, the clamping force compensation amount is calculated. The clamping force of the gripper is adjusted in real time based on the clamping force compensation amount to achieve stable fruit grasping and picking.

[0007] Optionally, the viscoelastic contact mechanics model uses the Kelvin-Voigt model to describe the fruit's mechanical response, with the expression as follows: ; in, For contact force, For compression deformation, For equivalent elastic stiffness, It is the viscous damping coefficient; The friction cone model is based on a predetermined friction coefficient. When tangential force With normal force The proportion satisfies At that time, determine whether to trigger slippage; The damage energy model uses the applied energy. As a damage assessment indicator, when energy exceeds a threshold At that time, fruit damage was determined.

[0008] Optionally, the step of calculating the clamping force compensation amount based on the estimated maturity and contact state, combined with prior clamping force estimation and tactile feedback, includes: The prior gripping force for grasping is estimated based on the multi-field cooperative mapping model; the prior gripping force is the median of the minimum normal gripping force required to complete the grasping and the maximum normal gripping force required to avoid damage. During the gripping process, the actual normal gripping force fed back by the gripper in real time is compared with the prior gripping force to obtain the force residual; and a real-time gripping force compensation amount is generated based on the change of the force residual. The slip risk index and damage degree index are calculated in real time using a digital twin system, and the two indices are used as constraints to correct and limit the clamping force compensation amount.

[0009] Optionally, adjusting the current clamping force of the gripper in real time based on the clamping force compensation amount includes: Based on the fruit's ripeness, an initial clamping force is set on top of the prior clamping force; The initial clamping force is controlled collaboratively using a fast channel and a slow channel; the fast channel performs initial gripping based on the initial clamping force at the moment of contact, and the slow channel performs closed-loop fine-tuning of the initial clamping force according to the clamping force compensation amount; the clamping force output by the closed-loop fine-tuning is constrained between the minimum normal clamping force and the maximum normal clamping force.

[0010] Optionally, performing closed-loop fine-tuning of the initial clamping force based on the clamping force compensation amount includes: When the slip risk index increases and the damage index is less than the preset threshold, the control gain is increased to enhance the clamping force and suppress slip. When the damage level index approaches the preset threshold, reduce the control gain or pause the force application.

[0011] A fruit-harvesting device, applied to the fruit-harvesting method according to claim 1, the device comprising: The acquisition module is used to acquire the magnetic field changes during the contact process with the fruit in real time when the gripper picks the fruit, and to determine the three-dimensional contact force through a pre-established mapping relationship model between external force and magnetic field signal. The three-dimensional contact force includes normal force and tangential force. The estimation module is used to estimate the maturity and contact state of the fruit based on the three-dimensional contact force through a pre-established multi-field cooperative mapping model, wherein the multi-field cooperative mapping model includes a viscoelastic contact mechanics model, a friction cone model, and a damage energy model. The calculation module is used to calculate the clamping force compensation amount based on the estimated maturity and contact state, combined with the prior clamping force estimate and tactile feedback. The picking module is used to adjust the current gripping force of the gripper in real time based on the clamping force compensation amount, so as to achieve stable grasping and picking of the fruit.

[0012] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described fruit-harvesting method.

[0013] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the aforementioned fruit picking method.

[0014] The fruit harvesting method provided by this invention has the following beneficial effects: This invention achieves a fundamental breakthrough at the perception level by employing a magnetic composite flexible material gripper with a specific cross-shaped microstructure array. It can transform complex contact mechanics information into measurable three-dimensional magnetic field changes, thereby calculating independent normal and tangential forces. This directly overcomes the shortcomings of traditional sensing methods, such as single-dimensional sensing and signal coupling, providing a comprehensive and accurate mechanical information foundation for precise control. Secondly, by introducing a multi-field collaborative mapping model that integrates viscoelastic contact mechanics, friction cones, and damage energy models, this method can deeply estimate the internal physical properties of the fruit (such as maturity) and real-time contact state (such as slip risk) based on multi-dimensional mechanical information. This elevates the interpretation of simple force signals to a comprehensive understanding of the fruit's biomechanical characteristics, significantly enhancing the system's adaptability (i.e., robustness) to differences in individual fruits and maturity levels. Then, by combining prior estimation with real-time tactile feedback to calculate the dynamic clamping force compensation, an adaptive closed-loop control system with close linkage between perception, decision-making and execution was constructed. This enabled the gripper to make rapid and smooth force adjustments when faced with slight slippage or changes in softness of the fruit. Ultimately, it effectively coordinated the two originally contradictory goals of "inhibiting fruit slippage" and "minimizing mechanical damage" and achieved a high level of unity between stable gripping and non-destructive harvesting. Attached Figure Description

[0015] To more clearly illustrate the embodiments and design schemes of the present invention, the accompanying drawings required for this embodiment will be briefly described below. The drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of a magnetoelastic preparation process according to an exemplary embodiment of the present invention.

[0017] Figure 2 This is a schematic diagram of a magnetic field disturbance-signal transduction according to an exemplary embodiment of the present invention.

[0018] Figure 3 This is a flowchart illustrating a fruit harvesting method according to an exemplary embodiment of the present invention.

[0019] Figure 4 This is a schematic diagram of tactile information transmission according to an exemplary embodiment of the present invention.

[0020] Figure 5 This is a block diagram of a fruit picking device provided by the present invention according to an exemplary embodiment. Detailed Implementation

[0021] To enable those skilled in the art to better understand and implement the technical solutions of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention.

[0022] This invention proposes an overall solution of "flexible three-dimensional tactile sensor + force-shape-field-information multi-field collaborative mapping + prior-measured assimilation + digital twin-driven posterior compensation + hierarchical adaptive force control", which specifically addresses the main shortcomings of existing technologies. It forms a closed loop from perception, modeling, estimation to control, improving the stability, robustness and maturity discrimination ability of fruit picking without damage, and has significant technological progress and industrial application value.

[0023] The technical solutions provided by the various embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0024] First, the fruit picking method provided by this invention is applied to a gripper made of flexible magnetic material. The flexible magnetic material on the surface of the gripper is regularly distributed, such as in a cross-shaped microstructure array, which generates a measurable magnetic field change when subjected to force.

[0025] In one embodiment, the present invention can use high-energy-product permanent magnet microparticles (NdFeB neodymium iron boron powder) mixed with a flexible elastomer (PDMS silicone rubber) to form a flexible magnetic material. Based on magnetic circuit simulation, the pole spacing, thickness, and magnetization direction layout of the magnet are optimized, and a controllable microstructure array is constructed inside the flexible sensor, enabling the sensor to generate a measurable magnetic field change when subjected to force. This flexible tactile device is fabricated using a 3D printing sacrificial template method, such as... Figure 1 The fabrication process of the magnetoelasticity is shown as follows: First, a rigid template with a pre-defined channel structure is printed. Then, elastomer ink containing magnetic particles is filled into the template voids under vacuum and cured. Subsequently, the template is dissolved and removed to obtain a magnetically sensitive flexible sensor with a precise internal microstructure. The sensor structure design of this invention has self-decoupling characteristics: through geometric encoding of the microstructure and magnetic circuit correction, the magnetic field signals generated under normal pressure and tangential shear force are spatially distinguished, achieving independent sensing of normal and tangential forces. Figure 2 As shown, a cross-shaped soft magnetic structure is arranged in the sensing layer, which allows vertical pressing to mainly cause changes in the magnetic flux in the vertical direction of the sensor, while tangential sliding mainly causes changes in the magnetic flux in the horizontal direction. The Hall element is used to detect the changes in the magnetic field, thereby achieving decoupling of the force component in the signal.

[0026] Based on the gripper constructed with the aforementioned sensor, the specific steps of this invention are as follows: Figure 3 As shown, it includes: S101. When the gripper is picking fruit, the magnetic field changes during the contact process with the fruit are acquired in real time, and the three-dimensional contact force is determined by a pre-established mapping relationship model between external force and magnetic field signal.

[0027] The three-dimensional contact force includes normal force and tangential force.

[0028] In this step, a force-magnetic field calibration experiment is performed on the sensor in the gripper: under a known force standard (including combinations of normal and tangential forces of different magnitudes), multi-axis magnetic field signals of the sensor are acquired to establish the force-magnetic field calibration. To magnetic field signal The mapping relationship model, i.e., the force-magnetic field coupling mapping function The inverse mapping model is obtained through the least squares fitting method. It is used for real-time calculation of contact force. Calibration results show that the response of each component force to the magnetic signal has good linear independence and a small cross-coupling term, which can realize real-time decoupled measurement of three-dimensional contact force.

[0029] Based on a pre-trained mapping model, the magnetic field changes during fruit contact can be obtained in real time when the gripper picks the fruit, thereby determining the three-dimensional contact force.

[0030] S102. Based on this three-dimensional contact force, the ripeness and contact state of the fruit are estimated through a pre-established multi-field cooperative mapping model.

[0031] The multi-field cooperative mapping model includes a viscoelastic contact mechanics model, a friction cone model, and a damage energy model.

[0032] In this step, a series of indentation-shear linkage tests were first carried out on typical fruit samples under a constant temperature and humidity experimental environment. Multi-channel data such as fruit surface deformation, force / displacement, and sensor signals were collected simultaneously to establish a positive mapping model of force-deformation-physical field-sensor signal.

[0033] Specifically, such as Figure 4 As shown, a sensor gripper was used to press the fruit surface at different speeds and depths to obtain the indentation depth-force curves, and the corresponding sensor signal changes were recorded. Subsequently, a tangential force was applied until the fruit initially slipped in the gripper, and the slippage critical force and corresponding sensor signal thresholds were recorded. Furthermore, the force-displacement hysteresis curves and phase difference were measured during the loading-unloading cycle to characterize the viscoelastic properties and energy loss of the fruit contact. Based on the obtained data, a physical model was established including the following:

[0034] Viscoelastic contact mechanics model: Treating the local contact deformation of the fruit as a viscoelastic body, its mechanical response can be approximated using the Kelvin-Voigt viscoelastic model, i.e., the contact force satisfies , For compression deformation, For equivalent elastic stiffness, This represents the viscous damping coefficient. The equivalent stiffness is obtained by performing linear regression on the initial climbing segment of the indentation curve. The viscous damping coefficient is determined by the hysteresis width and phase hysteresis of the loading / unloading curve. (For example, by fitting the Maxwell model or calculating the storage modulus and loss modulus). This viscoelastic model reveals the elastic recovery and viscous buffering effect of fruit tissue under stress: elastic elements store energy to support the load, while viscous elements delay the transmission of force and smooth the impact, which helps to explain the stress differences of fruit under different loading rates.

[0035] Friction cone theory and slip triggering mechanism: Determining the friction coefficient of clamping contact based on shear linkage test data. When tangential contact force Exceeding the friction cone limit (i.e.) When the fruit begins to slide, relative slippage will occur. By measuring the normal and tangential forces at the onset of fruit slippage, the applicability of the friction cone theory in finger-fruit contact was verified, and a slippage criterion was defined accordingly: for example, when the tangential force ratio is monitored in real time to be close to a critical value (such as... When the sensor detects a high-frequency micro-vibration signal, it determines that slippage is imminent. At this time, the slippage tactile sensing module outputs a slippage trigger signal, providing a basis for subsequent control adjustments. The slippage threshold measured in experiments is related to the surface characteristics of the fruit: the skin of highly mature fruits is softer, resulting in a larger contact area and effective friction. The skin of mature fruits increases in size, but at the same time, the softer skin is more prone to localized adhesion-slip transitions; the skin of immature fruits is hard and smooth. Slightly lower but less prone to deformation. Control strategies need to take these factors into account and dynamically assess slip risk at different times.

[0036] Damage Energy Model: Based on the critical conditions for slight indentation or internal bruising of the fruit observed in multiple experiments, a fruit damage criterion is established. The energy input into the fruit during the contact process is considered. As one of the damage evaluation indicators, it was found that when the applied energy or peak stress exceeds a certain threshold... When the pressure is applied for several seconds, irreversible damage occurs to the fruit's cellular tissue. The relationship between the area of ​​the indentation on the fruit's epidermis and the applied force / time is measured using an image-based method to obtain the damage energy threshold and damage discrimination model. For example, for a typical kiwifruit sample, when a normal force exceeding 12-15 N is applied for several seconds, browning and pressure marks begin to appear inside; below this force range, no tissue damage is detected even when the fruit is fully clamped. Based on this, a safe contact force range and a reversible deformation range are defined, meaning the fruit is subjected to force that does not exceed its elastic limit and the energy accumulation does not exceed the threshold. The conditions are called the stable and reversible operating region; beyond this region, the system will enter an irreversible region where damage is possible. Based on this, the present invention uses the damage threshold and slip threshold as important physical boundary conditions for designing the gripping force control strategy.

[0037] Based on the above experimental data, a multi-field cooperative positive mapping model was established, encompassing external forces (fruit deformation, damage, etc.), internal physical parameters (adhesion, friction, etc.), and sensor-observed signals. This model clearly defines the stable operating domain (signal-force correspondence within a high-confidence, safe range) and the reversible operating domain (short-term overload behavior that slightly exceeds the stable range but does not cause permanent damage) of the flexible tactile sensor during fruit grasping. This provides a priori basis for state estimation and control decisions in subsequent steps. Furthermore, through the analysis of parameters from fruits at different maturity levels, this step provides a basis for maturity identification: for example, the equivalent stiffness identified using the Kelvin-Voigt model. It can be used as an indicator of maturity among the same variety. The lower the value, the softer and riper the fruit, and the control strategy can be adjusted accordingly based on this stratification.

[0038] S103. Based on the estimated maturity and contact state, combined with the prior clamping force estimation and tactile feedback, calculate the clamping force compensation amount.

[0039] Based on the viscoelastic constitutive model of the fruit and the friction-adhesion boundary conditions established in the preceding steps, a priori mechanical model of the contact between the fingertip and the fruit is constructed to estimate the required gripping force range before grasping. For example, using the identified equivalent stiffness and mass of the fruit, the minimum normal force required to complete the grasping and load-bearing can be estimated a priori. (make (Greater than the weight of the fruit and the inertial force generated by the operation) and the recommended force after considering the safety margin. Simultaneously, the maximum permissible normal force is determined based on the fruit's maturity level and surface characteristics. (Given by the damage threshold) and the corresponding safe range of tangential force. This yields the a priori feasible region of the clamping force: i.e. and .

[0040] During the grasping action, the sensor provides real-time feedback on the actual contact force. And contact states (including events such as microslippage and adhesion). Tactile observations are compared with prior estimates to analyze the differences between the two. The differences may arise from several factors, including: model errors (inconsistencies between prior viscoelastic parameters or friction coefficients and the actual values ​​of the fruit), sensor noise and bias (sensor zero-point drift or measurement noise), environmental disturbances (such as instantaneous force changes caused by branch and leaf collisions), and individual differences (differences in hardness and surface characteristics between individual fruits). This invention utilizes a residual panel for real-time monitoring. Over time, the main sources of mismatch are identified using change point detection and consistency checks: if the residuals show a continuous deviation trend, it may be due to model parameter deviation, requiring updates to the fruit's equivalent stiffness or friction coefficient; if the residuals exhibit high-frequency random fluctuations, it is mainly noise interference, and robustness can be improved through filtering; if the residuals show a sharp change at a specific moment, it indicates a sudden environmental disturbance or slippage event, and the corresponding compensation mechanism should be triggered. Using these diagnostic results, posterior assimilation is implemented under physical prior constraints: that is, adjusting model parameters or control outputs based on observation errors to generate a grasping force compensation amount with uncertainty assessment. For example, if the model is detected to have underestimated the softness of the fruit (the residual is positive and gradually increases), the target clamping force can be appropriately increased. When slip signs are detected (a sharp increase in residuals accompanied by tangential vibration), within the specified time... Instantly increase clamping force under the premise To suppress slippage; conversely, if the force feedback is found to be higher than the prior prediction for a long time and close to the damage threshold, it may be due to external forces (such as handle tension) not considered by the model or measurement drift, and subsequent applied forces should be reduced or limited.

[0041] To utilize the above information more systematically, this invention constructs a finger-fruit interaction digital twin system: a virtual model synchronized with the actual grasping process runs in real time in the controller, the prior model is updated by the current fruit parameters detected by the sensors, and the predicted grasping state index is output. The digital twin continuously calculates the grasping feasible domain boundary, slippage risk index, and damage degree index based on the updated model: slippage risk index... Defined as the ratio of the current tangential force to the friction limit. A level close to 1 indicates a risk of slippage; damage severity index It can be based on the proportion of the current applied energy to the damage threshold. Alternatively, the peak compressive stress can be determined as a percentage of the threshold; a value close to 1 indicates potential damage. The digital twin system provides these indicators as constraints and adjustment references to the control module, ensuring that control decisions consider both grasping stability and fruit safety. Finally, step three generates a compensation interface that can be called by the control side, outputting real-time updated grasping force adjustments. and constraint-setting-scheduling boundaries (e.g.) , Maximum allowed (e.g., rate of change), laying the foundation for adaptive scheduling of subsequent control laws.

[0042] S104. Based on the clamping force compensation amount, the current clamping force of the gripper is adjusted in real time to achieve stable grasping and picking of the fruit.

[0043] Building upon the preceding steps, this step constructs a prototype and system of a flexible grasping end-effector integrating sensing and control for non-destructive harvesting. This involves interface integration between the sensing, estimation, and control modules to form a real-time closed-loop control system. Based on the boundary conditions and maturity stratification strategy provided in step three, a tactile feedback-driven adaptive scheduling control algorithm for grasping force is designed as follows: First, an initial grasping force reference value is set according to the maturity category. (For fruits with high maturity, a lower initial value is used to ensure safety; for fruits with low maturity, a relatively higher value is used to overcome the harder skin of the fruit.) The controller consists of two coordinated components: a fast channel and a slow channel. The fast channel provides feedforward force control commands based on a priori estimation model, such as rapidly increasing the clamping force to the priori estimated safe median value at the moment of contact. To establish a stable grip as early as possible; the slow channel performs closed-loop force control adjustments based on tactile feedback and force update rules, continuously fine-tuning. To approximate the optimal gripping force, the two channels coordinate via a gain scheduler: when an increased risk of slippage is detected but there is still a damage margin, the gain of the fast channel is increased, making the control law more aggressive in increasing the gripping force; when the damage index is detected to be close to the threshold, any further force is reduced or paused, and the upper-level decision-making is notified to reassess the operating strategy (performing repositioning gripping if necessary). Throughout the gripping process, the controller ensures that the output execution instructions always meet the constraint boundaries (not exceeding...). No less than Furthermore, the rate of force change is limited to prevent shocks. The control algorithm supports control law reconstruction and smooth switching: when switching between different control modes according to real-time operating conditions, force spikes are avoided through smooth interpolation. For example, when switching from "force holding mode" to "slip suppression mode", the target force is gradually increased while the damping is increased to avoid oscillations; when switching back from "force application mode" to "hold mode", the gain is slowly reduced to stabilize the force value at the new level.

[0044] The sampling frequency can reach 1~2 kHz, which is higher than the frequency range of human hand touch (the controller update cycle is, for example, 1~5 ms), ensuring that emergency force compensation is completed within tens of milliseconds after slippage occurs. At the same time, the bandwidth of the force control output is adjusted to avoid overshoot, and an impedance control effect is formed on the flexible actuator (such as pneumatic drive or servo motor + elastic fingertip system), so that the gripper exhibits a certain degree of compliance with the fruit. While ensuring stable gripping, the contact force is controlled at the lowest possible level, and the fruit is prevented from slipping through the rapid response, thus achieving the goal of "neither dropping the fruit nor damaging it".

[0045] By employing the aforementioned method and utilizing a magnetic composite flexible material gripper with a specific cross-shaped microstructure array, a fundamental breakthrough is achieved at the perception level. This gripper can transform complex contact mechanics information into measurable three-dimensional magnetic field changes, thereby calculating independent normal and tangential forces. This directly overcomes the shortcomings of traditional sensing methods, such as single-dimensional sensing and signal coupling, providing a comprehensive and accurate mechanical information foundation for precise control. Secondly, by introducing a multi-field collaborative mapping model that integrates viscoelastic contact mechanics, friction cones, and damage energy models, this method can deeply estimate the internal physical properties of the fruit (such as maturity) and real-time contact state (such as slippage risk) based on multi-dimensional mechanical information. This elevates the interpretation of simple force signals to a comprehensive understanding of the biomechanical characteristics of the fruit, significantly enhancing the system's adaptability (i.e., robustness) to differences in individual fruits and maturity levels. Then, by combining prior estimation with real-time tactile feedback to calculate the dynamic clamping force compensation, an adaptive closed-loop control system with close linkage between perception, decision-making and execution was constructed. This enabled the gripper to make rapid and smooth force adjustments when faced with slight slippage or changes in softness of the fruit. Ultimately, it effectively coordinated the two originally contradictory goals of "inhibiting fruit slippage" and "minimizing mechanical damage" and achieved a high level of unity between stable gripping and non-destructive harvesting.

[0046] Based on the above method, the present invention also provides a simulation test to verify the effect of the aforementioned steps of the present invention.

[0047] In addition, to verify the effectiveness and engineering feasibility of the present invention, virtual simulation and experimental testing were conducted. First, multiple fruit sample models were constructed in a digital twin simulation environment, corresponding to different maturity and size characteristics. Typical picking operation scenarios (such as fruit grasping tasks in a greenhouse environment) were set up, and the effects of the adaptive force control strategy and other benchmark strategies were compared and evaluated. Key indicators under each strategy were recorded in the simulation, including: damage area (the proportion of simulated fruit pressure exceeding the threshold volume or energy), slippage rate (the proportion of slippage occurrences to the total number of grasping attempts), number of re-grasping attempts (the number of times a fruit needs to be re-grasped due to slippage), fruit drop rate (the proportion of fallen fruit that could not be recovered), single fruit picking cycle, and grasping success rate. Subsequently, preliminary physical experiments were conducted in a laboratory and greenhouse environment: a flexible three-finger gripper prototype was built, with each finger embedded with the tactile sensor of the present invention, and installed at the end effector of a six-degree-of-freedom robot. Various fruits, such as strawberries, tomatoes, and kiwis, were selected and stratified into test groups according to high, medium, and low maturity levels, with each group undergoing no fewer than 20 picking and grasping experiments. The experimental group used the estimation + compensation + adaptive scheduling control strategy of this invention, while the control group adopted strategies such as constant force without feedback, prior estimation only, and prior estimation + simple feedback, to highlight the role of the key steps of this invention. During the experiment, a high-speed camera and sensing system recorded the grasping force curve, sensor signals, fruit status (whether slipping or damaged), and harvesting results. The experimental results showed that the method of this invention can significantly reduce fruit damage and slippage rates. Taking the high-maturity kiwifruit group as an example, compared with constant force control, adaptive force control reduced the average epidermal indentation area by about 40%, and the slippage rate dropped from 20% to below 5%, essentially eliminating re-grabbing and drop. In the low-maturity group, this method, by compensating for the overestimated force, avoided applying excessive clamping force to unripe, hard fruits, and no epidermal cracking occurred. Similar positive effects were achieved under different fruit varieties and scenarios. Comparison of various statistical indicators showed that the complete strategy of prior estimation + posterior compensation + scheduling (i.e., the scheme of this invention) exhibited a higher success rate and stability than the case of using only part of the strategy. For example, compared to using only prior model estimation, adding tactile feedback compensation reduces the injury rate by about half, and further enabling dual-channel scheduling further reduces the injury rate and almost eliminates slippage. These results verify the necessity and effectiveness of the collaborative work of the modules in this invention.

[0048] The above tests and evaluations demonstrate that the flexible three-dimensional tactile sensing-driven adaptive force control method of this invention possesses superior technical performance: it achieves real-time sensing and independent decoupling of the gripper's three-dimensional contact force with the fruit, can adaptively adjust the gripping force to fruits of different hardness / ripeness, significantly reduces harvesting damage and slippage risks, and has adaptability across varieties and complex environments. All aspects of the entire system, from theoretical model to control implementation, have been verified, demonstrating a rigorous and complete approach from theory to practice, and possessing engineering feasibility and innovation for application in actual agricultural harvesting robots.

[0049] Secondly, the present invention also provides a fruit picking device, such as... Figure 5 As shown, it includes: The acquisition module 201 is used to acquire the magnetic field changes during the fruit contact process in real time when the gripper picks the fruit, and to determine the three-dimensional contact force through a pre-established mapping relationship model between external force and magnetic field signal. The three-dimensional contact force includes normal force and tangential force.

[0050] The estimation module 202 is used to estimate the maturity and contact state of the fruit based on the three-dimensional contact force through a pre-established multi-field co-mapping model, wherein the multi-field co-mapping model includes a viscoelastic contact mechanics model, a friction cone model, and a damage energy model.

[0051] The calculation module 203 is used to calculate the clamping force compensation amount based on the estimated maturity and contact state, combined with the prior clamping force estimation and tactile feedback.

[0052] The picking module 204 is used to adjust the current gripping force of the gripper in real time based on the gripping force compensation amount, so as to achieve stable grasping and picking of the fruit.

[0053] Using the aforementioned device, a fundamental breakthrough is achieved at the sensing level by employing grippers made of flexible magnetic materials. This allows for the transformation of complex contact mechanics information into measurable three-dimensional magnetic field changes, enabling the calculation of independent normal and tangential forces. This directly overcomes the shortcomings of traditional sensing methods, such as single-dimensional sensing and signal coupling, providing a comprehensive and accurate mechanical information foundation for precise control. Secondly, by introducing a multi-field collaborative mapping model that integrates viscoelastic contact mechanics, friction cones, and damage energy models, this method can deeply estimate the internal physical properties of the fruit (such as maturity) and real-time contact state (such as slippage risk) based on multi-dimensional mechanical information. This elevates the interpretation of simple force signals to a comprehensive understanding of the fruit's biomechanical characteristics, significantly enhancing the system's adaptability (i.e., robustness) to differences in individual fruits and maturity levels. Then, by combining prior estimation with real-time tactile feedback to calculate the dynamic clamping force compensation, an adaptive closed-loop control system with close linkage between perception, decision-making and execution was constructed. This enabled the gripper to make rapid and smooth force adjustments when faced with slight slippage or changes in softness of the fruit. Ultimately, it effectively coordinated the two originally contradictory goals of "inhibiting fruit slippage" and "minimizing mechanical damage" and achieved a high level of unity between stable gripping and non-destructive harvesting.

[0054] The present invention also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 The steps provided are for the fruit picking method.

[0055] This invention also provides a computer device. At the hardware level, the computer device includes a processor, an internal bus, a network interface, memory, and non-volatile memory, and may also include other hardware required for various operations. The processor reads the corresponding computer program from the non-volatile memory into the memory and then executes it to achieve the above-mentioned functions. Figure 1 The steps provided are for the fruit picking method.

[0056] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0057] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0058] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0059] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0060] It should be noted that the specific embodiments described above enable those skilled in the art to more fully understand the present invention, but do not limit the present invention in any way. Therefore, although the present invention has been described in detail in this specification, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the present invention; and all technical solutions and improvements that do not depart from the spirit and scope of the present invention are covered within the protection scope of the patent of the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A method for harvesting fruit, characterized in that, A gripper made of a flexible magnetic material, wherein the flexible magnetic material on the surface of the gripper generates a measurable change in magnetic field when subjected to force; the method includes: When the gripper picks the fruit, the magnetic field changes during the contact process with the fruit are acquired in real time, and the three-dimensional contact force is determined by a pre-established mapping relationship model from external force to magnetic field signal. The three-dimensional contact force includes normal force and tangential force. Based on the three-dimensional contact force, the maturity and contact state of the fruit are estimated by a pre-established multi-field cooperative mapping model, wherein the multi-field cooperative mapping model includes a viscoelastic contact mechanics model, a friction cone model, and a damage energy model. Based on the estimated maturity and contact state, combined with prior clamping force estimation and tactile feedback, the clamping force compensation amount is calculated. The clamping force of the gripper is adjusted in real time based on the clamping force compensation amount to achieve stable fruit grasping and picking.

2. The method according to claim 1, characterized in that, The viscoelastic contact mechanics model uses the Kelvin-Voigt model to describe the fruit's mechanical response, and the expression is: ； in, For contact force, For compression deformation, For equivalent elastic stiffness, It is the viscous damping coefficient; The friction cone model is based on a predetermined friction coefficient. When tangential force With normal force The proportion satisfies At that time, determine whether to trigger slippage; The damage energy model uses the applied energy. As a damage assessment indicator, when energy exceeds a threshold At that time, fruit damage was determined.

3. The method according to claim 1, characterized in that, The calculation of the clamping force compensation amount based on the estimated maturity and contact state, combined with prior clamping force estimation and tactile feedback, includes: The prior gripping force for grasping is estimated based on the multi-field cooperative mapping model; the prior gripping force is the median of the minimum normal gripping force required to complete the grasping and the maximum normal gripping force required to avoid damage. During the gripping process, the actual normal gripping force fed back by the gripper in real time is compared with the prior gripping force to obtain the force residual; and a real-time gripping force compensation amount is generated based on the change of the force residual. The slip risk index and damage degree index are calculated in real time using a digital twin system, and the two indices are used as constraints to correct and limit the clamping force compensation amount.

4. The method according to claim 3, characterized in that, Real-time adjustment of the current clamping force of the gripper based on the clamping force compensation amount includes: Based on the fruit's ripeness, an initial clamping force is set on top of the prior clamping force; The initial clamping force is controlled collaboratively using a fast channel and a slow channel; the fast channel performs initial gripping based on the initial clamping force at the moment of contact, and the slow channel performs closed-loop fine-tuning of the initial clamping force according to the clamping force compensation amount; the clamping force output by the closed-loop fine-tuning is constrained between the minimum normal clamping force and the maximum normal clamping force.

5. The method according to claim 4, characterized in that, Closed-loop fine-tuning of the initial clamping force based on the clamping force compensation amount includes: When the slip risk index increases and the damage index is less than the preset threshold, the control gain is increased to enhance the clamping force and suppress slip. When the damage level index approaches the preset threshold, reduce the control gain or pause the force application.

6. A fruit-harvesting device, characterized in that, The apparatus used in the fruit harvesting method according to claim 1 includes: The acquisition module is used to acquire the magnetic field changes during the contact process with the fruit in real time when the gripper picks the fruit, and to determine the three-dimensional contact force through a pre-established mapping relationship model between external force and magnetic field signal. The three-dimensional contact force includes normal force and tangential force. The estimation module is used to estimate the maturity and contact state of the fruit based on the three-dimensional contact force through a pre-established multi-field cooperative mapping model, wherein the multi-field cooperative mapping model includes a viscoelastic contact mechanics model, a friction cone model, and a damage energy model. The calculation module is used to calculate the clamping force compensation amount based on the estimated maturity and contact state, combined with the prior clamping force estimate and tactile feedback. The picking module is used to adjust the current gripping force of the gripper in real time based on the clamping force compensation amount, so as to achieve stable grasping and picking of the fruit.

7. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1 to 5.

8. A computer device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in any one of claims 1 to 5.

Images