A fruit grasping method, device, and storage medium

Abstract

The application provides a fruit grabbing method and device and a storage medium, and belongs to the technical field of mechanical arm control. The method comprises the following steps: an RGB-D depth camera installed at the end of a mechanical arm collects images of a region to be grabbed to obtain a target fruit RGB image and a target fruit depth image; a multi-task deep learning network is constructed, and feature analysis is performed on the target fruit RGB image and the target fruit depth image by using the multi-task deep learning network to obtain a fruit visual physical property feature matrix; and current limiting threshold analysis is performed on the fruit visual physical property feature matrix to obtain a motor armature current limiting threshold. The application eliminates the decoupling drawbacks of perception semantics and action execution, simultaneously, completely cancels external fragile sensors, eliminates torque sharp peak pulsation caused by hard cutting, realizes full-closed-loop flexible non-destructive wrapping grabbing for soft and hard heterogeneous fruits, and realizes flexible, high success rate and non-destructive picking grabbing for heterogeneous biological fruits.

Description

Technical Field

[0001] This invention mainly relates to the field of robotic arm control technology, specifically to a fruit grasping method, device, and storage medium. Background Technology

[0002] In the fields of automated agricultural harvesting and end effector control, utilizing computer vision to locate target fruits in unstructured environments in two or three dimensions is fundamental to achieving automated grasping. Common technologies typically employ single-stage target detection networks (such as the standard YOLO network) to extract the spatial coordinates of the fruit's bounding box center, and then transmit these coordinates as position input variables to the robotic arm control system. The closing of the end effector (such as a two- or multi-finger gripper) usually relies on a servo motor or stepper motor drive. At the control layer, the maximum instantaneous closing torque of the gripper is limited by restricting the maximum current input to the motor to prevent the drive motor from burning out due to overload.

[0003] In existing technologies, visual recognition and action execution are treated as two completely independent, sequential linear flows. However, this approach suffers from a fatal flaw: a complete separation between perceptual semantics and dynamic constraints. In nature, agricultural fruits exhibit drastically different biorheological properties, pulp tissue hardness, and compressive modulus limits at different developmental stages (for example, green-ripe tomatoes have firm tissue and high compressive limits; while fully ripe tomatoes have highly softened pulp and a drastically decreased tissue compressive limits). This is because the execution end employs a blind, static, and singular constant upper limit of torque. If the global torque limit is set too high (to accommodate hard fruits or prevent slippage during grasping), when faced with highly mature soft fruits, the constant mechanical closing torque will instantly exceed the biological tissue compressive strength limit modulus of the mature fruit, causing the pulp tissue to be irreversibly crushed by physical compression at the moment of grasping, resulting in serious economic losses and cross-contamination. If the global torque limit is blindly reduced in order to protect ripe fruit, the clamping force provided by the gripper will not be able to overcome the static friction and dynamic acceleration disturbance when grasping hard, heavy, unripe fruit (or smooth fruit). This will cause the fruit to slip frequently during the retraction of the robotic arm, greatly reducing the success rate of the picking task.

[0004] To achieve flexible gripping to some extent, existing technologies attempt to add flexible thin-film pressure sensors or array tactile sensors to the outer side of the fingertips of electric grippers. Although this introduces physical force feedback, it suffers from the following drawbacks in complex, unstructured agricultural environments: discontinuous control and extremely poor engineering robustness. Static physical thresholds cannot adapt to the heterogeneity of biological hardness: their feedback judgment benchmark is still a manually preset static global constant, and the algorithm cannot know the specific softness or hardness of the fruit being grasped. When faced with field conditions where "ripe fruit" and "green fruit" are mixed, static thresholds still cannot reconcile the nonlinear inverse relationship between the dynamics of "green fruit slipping" and "ripe fruit being crushed".

[0005] Discontinuous hard cutoff disrupts the smoothness of torque transition: When the tactile sensor triggers an over-limit condition, the system directly employs discontinuous control logic that forcibly cuts off the input pulse. Under the influence of nonlinear electromagnetic dynamics, this can easily lead to a sudden step drop in the motor armature current, generating high-frequency oscillating tailing torque pulsations. Such instantaneous torque impacts can cause transient overloads at the contact surface, still posing a very high risk of localized damage to the ripe fruit.

[0006] External sensors increase end-effector load and degrade hardware robustness: In harsh field environments with high temperature, high humidity, dust, and scratching from branches and leaves, external tactile membranes are extremely susceptible to calibration drift, signal distortion, and even physical damage due to physical wear, moisture penetration, or snagging on branches and leaves. Once sensor feedback fails, the entire force closed-loop control chain immediately collapses, which can easily lead to serious accidents caused by continuous overload and crushing of the fruit. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to provide a fruit grasping method, device and storage medium to address the shortcomings of the prior art.

[0008] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: A fruit grasping method, comprising the following steps: An RGB-D depth camera installed at the end of the robotic arm acquires images of the area to be grasped, resulting in an RGB image of the target fruit and a depth map of the target fruit. A multi-task deep learning network is constructed, and feature analysis is performed on the RGB image and depth map of the target fruit through the multi-task deep learning network to obtain the visual physical property feature matrix of the fruit. The current limiting threshold is obtained by performing current limiting threshold analysis on the visual physical property feature matrix of the fruit; By importing the real-time armature current, feedback force analysis is performed on the visual physical property feature matrix of the fruit and the real-time armature current to obtain the real-time virtual contact feedback force. The real-time virtual contact feedback force is optimized and analyzed based on the real-time armature current and the motor armature current limit threshold, and the robotic arm is controlled to grasp based on the optimization analysis results.

[0009] Another technical solution of the present invention to solve the above-mentioned technical problems is as follows: A fruit grasping device, comprising: The image acquisition module is used to acquire images of the area to be grasped by an RGB-D depth camera installed at the end of the robotic arm, so as to obtain the RGB image and depth map of the target fruit. The feature analysis module is used to construct a multi-task deep learning network, and to perform feature analysis on the RGB image of the target fruit and the depth map of the target fruit through the multi-task deep learning network to obtain the visual physical property feature matrix of the fruit. The current limit threshold analysis module is used to perform current limit threshold analysis on the visual physical property feature matrix of the fruit to obtain the motor armature current limit threshold. The feedback force analysis module is used to import the real-time armature current, perform feedback force analysis on the visual physical property feature matrix of the fruit and the real-time armature current, and obtain the real-time virtual contact feedback force. The optimization analysis module is used to perform optimization analysis on the real-time virtual contact feedback force based on the real-time armature current and the motor armature current limit threshold, and control the robotic arm to grasp based on the optimization analysis results.

[0010] Based on the above-mentioned fruit grasping method, the present invention also provides a fruit grasping system.

[0011] Another technical solution of the present invention to solve the above-mentioned technical problems is as follows: a fruit grasping system, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the fruit grasping method as described above.

[0012] Based on the above-described fruit grasping method, the present invention also provides a computer-readable storage medium.

[0013] Another technical solution of the present invention to solve the above-mentioned technical problems is as follows: a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the fruit-grabbing method as described above.

[0014] The beneficial effects of this invention are as follows: An RGB-D depth camera installed at the end of a robotic arm acquires images of the target fruit's RGB image and depth map of the target fruit. A multi-task deep learning network analyzes the features of the RGB image and depth map to obtain a visual property feature matrix of the fruit. Current limiting threshold analysis of the visual property feature matrix yields a motor armature current limiting threshold. Feedback force analysis of the visual property feature matrix and real-time armature current yields a real-time virtual contact feedback force. The real-time virtual contact feedback force is optimized based on the real-time armature current and motor armature current limiting thresholds. The robotic arm is then controlled to grasp the fruit based on the optimization results. This eliminates the decoupling drawbacks of perception semantics and action execution. Simultaneously, it completely eliminates the need for external fragile sensors, eliminating torque spikes caused by hard truncation. This achieves fully closed-loop, flexible, and non-destructive grasping of fruits with varying hardness and softness, and also enables flexible, high-success-rate, and non-destructive grasping of heterogeneous biological fruits. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating the fruit-grabbing method provided in an embodiment of the present invention. Figure 2 This is an architecture diagram of the multi-task deep learning network for the fruit-grabbing method provided in this embodiment of the invention; Figure 3 This is a block diagram of the underlying sensorless virtual force feedback adaptive control logic of the fruit grasping method provided in the embodiments of the present invention; Figure 4 This is a schematic diagram of the underlying driving circuit H-bridge complementary conduction low-loss follow current attenuation topology for the fruit grasping method provided in this embodiment of the invention. Figure 5 A comparison curve of motor current / torque in the traditional hard truncation mode of the fruit grasping method provided in the embodiments of the present invention and the slow decay mode of the present invention. Figure 6 The system architecture diagram of the cross-modal decoupling mapping system based on visual maturity discrimination and hardness adaptation for the fruit grasping method provided in the embodiments of the present invention is shown. Figure 7 This is the main flowchart of the fruit grasping method provided in the embodiment of the present invention; Figure 8 This is a block diagram of a fruit-grabbing device provided in an embodiment of the present invention. Detailed Implementation

[0016] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0017] Figure 1This is a flowchart illustrating a fruit-grabbing method provided in an embodiment of the present invention.

[0018] like Figure 1 As shown, a method for grasping fruit includes the following steps: S1: The RGB-D depth camera installed at the end of the robotic arm acquires images of the area to be grasped, and obtains the RGB image and depth map of the target fruit. S2: Construct a multi-task deep learning network, and perform feature analysis on the RGB image of the target fruit and the depth map of the target fruit through the multi-task deep learning network to obtain the visual physical property feature matrix of the fruit. S3: Perform current limiting threshold analysis on the visual physical property feature matrix of the fruit to obtain the motor armature current limiting threshold; S4: Import the real-time armature current, perform feedback force analysis on the visual physical property feature matrix of the fruit and the real-time armature current, and obtain the real-time virtual contact feedback force. S5: Optimize and analyze the real-time virtual contact feedback force based on the real-time armature current and the motor armature current limit threshold, and control the robotic arm to grasp based on the optimization analysis results.

[0019] It should be understood that after the robotic arm end effector guides the RGB-D camera to the preset imaging pose of the picking area (i.e., the grasping area), the visual perception brain triggers an image acquisition sequence to acquire a high-resolution RGB image of the target fruit (i.e., the target fruit RGB image) and a time-aligned depth map (i.e., the target fruit depth map).

[0020] In the above embodiments, an RGB-D depth camera installed at the end of the robotic arm acquires images of the target fruit's RGB image and depth map of the target fruit. A multi-task deep learning network analyzes the features of the target fruit's RGB image and depth map to obtain a visual physical property feature matrix of the fruit. The current limiting threshold of the fruit's visual physical property feature matrix is ​​analyzed to obtain a motor armature current limiting threshold. The feedback force of the fruit's visual physical property feature matrix and real-time armature current is analyzed to obtain a real-time virtual contact feedback force. The real-time virtual contact feedback force is optimized based on the real-time armature current and motor armature current limiting threshold. The robotic arm is then controlled to grasp the fruit based on the optimization analysis results. This eliminates the decoupling drawbacks of perception semantics and action execution. At the same time, it completely eliminates the need for external fragile sensors and eliminates torque spike pulsations caused by hard truncation. This achieves fully closed-loop flexible and non-destructive grasping of fruits with different textures, and also realizes flexible, high-success-rate, and non-destructive grasping of heterogeneous biological fruits.

[0021] Optionally, as an embodiment of the present invention, such as Figure 1 and 2 As shown, the multi-task deep learning network includes a Backbone feature extraction subnetwork, a Neck feature fusion subnetwork, a position regression branch detection head, and a maturity classification branch detection head; The process of performing feature analysis on the RGB image and depth map of the target fruit using the multi-task deep learning network to obtain the visual physical property feature matrix of the fruit includes: The Backbone feature extraction subnetwork is used to extract features from the RGB image of the target fruit to obtain multiple original fruit features; The original fruit features are fused using the Neck feature fusion subnetwork to obtain the original fruit fused features; The location regression branch detection head is used to predict the location regression of the original fruit fusion features, and the two-dimensional box of the fruit, the box confidence, the fruit size, and the fruit variety are obtained. Import the camera intrinsic parameter matrix, perform coordinate system transformation on the fruit 2D bounding box, the bounding box confidence, the target fruit depth map, and the camera intrinsic parameter matrix to obtain the fruit 3D coordinates; The maturity classification branch detection head performs a maturity classification prediction analysis on the original fruit fusion features to obtain a confidence probability vector. The fruit visual property feature matrix includes the fruit size, the fruit variety, the fruit three-dimensional coordinates, and the confidence probability vector.

[0022] Specifically, the RGB image (i.e., the RGB image of the target fruit) is input into a pre-trained improved YOLO single-stage multi-task deep learning network (i.e., a multi-task deep learning network). This network (i.e., the multi-task deep learning network) shares the same backbone feature extraction network (i.e., the backbone feature extraction sub-network) and feature fusion network (i.e., the Neck feature fusion sub-network), but at the output end, the conventional detection head is decoupled and reconstructed, and two independent prediction branches are set up in parallel.

[0023] It should be understood that the position regression and target localization branch (i.e., the position regression branch detection head) outputs the coordinates of the two-dimensional center point of the target fruit (i.e., the two-dimensional bounding box of the fruit) and the width and height of the detection box. and background confidence score (i.e., frame confidence).

[0024] Specifically, the developmental stage corresponding to the highest confidence score among the three is selected as the final semantic label for the target fruit. Simultaneously, combining the depth image data (i.e., the depth map of the target fruit) acquired by the RGB-D depth camera, the target center coordinates are determined using the camera intrinsic parameter matrix. (i.e., the two-dimensional frame of the fruit) is converted into the absolute pose in three-dimensional space in the camera coordinate system. (i.e., the three-dimensional coordinates of the fruit).

[0025] It should be understood that the visual physical property features (i.e., the fruit visual physical property feature matrix) include not only the maturity semantic label of the target fruit, but also at least one of the maturity confidence probability vector (i.e., confidence probability vector), the target fruit geometric size (i.e., fruit size), the target variety parameters (i.e., fruit variety), and the physical property prior data established by historical compression experiments.

[0026] In the above embodiments, a visual physical property feature matrix of the fruit is obtained by performing feature analysis on the RGB image and depth map of the target fruit through a multi-task deep learning network. This eliminates the drawback of decoupling between perceptual semantics and action execution. At the same time, it completely eliminates the need for external fragile sensors and eliminates the torque spike pulsation caused by hard truncation.

[0027] Optionally, as an embodiment of the present invention, the maturity classification branch detection head includes a 1×1 convolutional layer, a batch normalization layer, a SiLU activation function layer, a 3×3 convolutional layer group, a global pooling layer, a hidden layer, a Dropout layer, a classification prediction layer, and a Softmax activation function layer. The process of performing maturity classification prediction analysis on the original fruit fusion features using the maturity classification branch detection head to obtain the confidence probability vector includes: The original fruit fusion features are extracted by the 1×1 convolutional layer to obtain the fruit fusion features to be processed. The batch normalization layer is used to normalize the fusion features of the fruit to be processed, and the normalized fruit fusion features are obtained. The normalized fruit fusion features are reduced in dimension by the SiLU activation function layer to obtain the reduced fruit fusion features. The 3×3 convolutional layer group is used to extract features from the dimensionality-reduced fruit fusion features to obtain the processed fruit fusion features. The processed fruit fusion features are pooled using the global pooling layer to obtain pooled fruit fusion features. The original hidden fruit features are obtained by performing dimensionality reduction on the pooled fruit fusion features through the hidden layer. The original hidden features of the fruit are randomly discarded through the Dropout layer to obtain the hidden features of the fruit after discarding. The classification prediction layer predicts the hidden features of the discarded fruit to obtain multiple maturity category score features; The softmax activation function layer is used to normalize the score features of all maturity categories, and the confidence probability vector is obtained by combining all the normalization results.

[0028] Specifically, a more detailed maturity semantic classification branch (i.e., a maturity classification branch detection head) is developed: this branch is specifically trained on the rheological characteristics of the developmental cycle of a particular fruit (such as a tomato). The output layer does not perform a simple "result / no result" binary classification, but instead outputs a discrete one-dimensional nonlinear one-hot encoded vector. .in, Represents the confidence level of "green maturity (hard)". Represents the confidence level of "color change period (semi-hard)". This represents the confidence level for "maturity (soft)".

[0029] It should be understood that the maturity semantic classification branch outputs a probability distribution vector (i.e., a confidence probability vector) indicating whether the target fruit belongs to different maturity stages, as shown in the following formula: , in, This indicates that the target fruit belongs to the first... Confidence level at each maturity stage This refers to the number of ripening stage categories. In a preferred embodiment, when the target fruit is a tomato, , representing the confidence levels of the target fruit belonging to the green ripening stage, color-changing stage, and ripening stage, respectively.

[0030] In the above embodiments, the maturity classification branch detection head is used to predict and analyze the maturity classification of the original fruit fusion features to obtain the confidence probability vector, which eliminates the drawback of decoupling between perceptual semantics and action execution. At the same time, the external fragile sensor is completely eliminated, and the torque spike pulsation caused by hard truncation is eliminated.

[0031] Optionally, as an embodiment of the present invention, the process of performing current limitation threshold analysis on the visual physical property feature matrix of the fruit to obtain the armature current limitation threshold of the motor includes: Based on the fruit size and the fruit variety, a set of baseline limit clamping forces is extracted from a preset prior mapping library. The safe clamping force is obtained by calculating the reference limit clamping force set and the confidence probability vector using the first formula, which is: , in, For safe clamping force, For the first Each confidence level probability For the first One benchmark limit clamping force; By importing the efficiency of the gripper transmission mechanism, the equivalent transmission radius of the gripper mechanism, and the no-load friction current, the safety clamping force, the efficiency of the gripper transmission mechanism, the equivalent transmission radius of the gripper mechanism, and the no-load friction current are calculated using the second equation to obtain the motor armature current limiting threshold. The second equation is: , in, This is the armature current limiting threshold for the motor. For safe clamping force, Let be the equivalent transmission radius of the gripper mechanism. To improve the efficiency of the gripper drive mechanism. The torque constant of the drive motor, This is the no-load triboelectric current.

[0032] Specifically, the host computer loads the built-in property mapping model. The property mapping model is based on the maturity confidence probability vector of the target fruit. Fruit size Target variety parameters and preset compression experimental data Output the maximum safe clamping force of the target fruit. and the corresponding current limit value The mapping relationship is expressed as follows: , In the formula, This is the maturity confidence probability vector; The fruit diameter, volume, or equivalent envelope size estimated from a visual inspection box, instance segmentation mask, or depth point cloud; These are the physical property correction parameters corresponding to fruit varieties, batches, or origins. The prior data of hardness, elastic modulus, yield force or destructive force obtained through a pre-set compression test; The property mapping function can be any one of the following: table lookup interpolation, piecewise fitting, multiple regression, neural network regression, or fuzzy controller.

[0033] It should be understood that a hard switch can also be performed without directly using the maximum probability maturity label, but instead, a confidence-weighted calculation can be performed based on the upper limit of the compression experiment safety force corresponding to different maturity stages, as shown in the following formula: , in, Indicates the first The upper limit of safe clamping force is adjusted according to fruit size and variety parameters at each maturity stage. Through the above confidence weighting method, even if the target fruit is in the transitional state between the green ripening stage, the color changing stage and the ripening stage, the system can still output a continuously changing upper limit of safe clamping force, avoiding abrupt changes in the underlying torque threshold caused by discrete label jumps.

[0034] Specifically, the upper limit of the safe clamping force is determined based on the parameters of the gripper transmission mechanism. (i.e., the safety clamping force) is converted into the current limit value of the gripper drive motor. (That is, the armature current limiting threshold of the motor). Its conversion relationship is expressed as: , In the formula, The equivalent working arm or equivalent transmission radius of the gripper mechanism; To improve the efficiency of the gripper transmission mechanism; This is the torque constant of the drive motor; This can be the no-load friction current, static friction compensation current, or non-contact baseline current. Therefore, the physical properties of the fruit obtained from the visual layer can be continuously mapped to the current safety boundary of the underlying actuator.

[0035] Specifically, to facilitate engineering deployment, the physical property mapping model can be pre-constructed as a maturity-hardness-current limit calibration table. For example, for tomatoes of a specific variety and size range, the green-ripe stage, color-changing stage, and maturity stage can be calibrated with different upper limits of safe clamping force and current limits: the green-ripe stage corresponds to a higher upper limit of safe clamping force, the color-changing stage corresponds to a medium upper limit of safe clamping force, and the maturity stage corresponds to a lower upper limit of safe clamping force. This discrete calibration table is only a preferred embodiment; in actual operation, the calibration values ​​can still be interpolated or corrected based on maturity confidence, fruit size, and variety parameters.

[0036] It should be understood that when the maximum confidence level of the maturity classification branch output is lower than the preset confidence threshold, a conservative protection mode is entered. In the conservative protection mode, this invention does not directly use the torque boundary corresponding to the maximum probability maturity label, but instead reduces the upper limit of the safe clamping force. This may trigger secondary image sampling, reduce the gripper closing speed, or reduce the robotic arm retraction acceleration to avoid damage to ripe fruit or slippage of unripe fruit due to misjudgment of maturity.

[0037] Specifically, through a continuous property mapping mechanism, this invention no longer simply maps maturity labels to fixed clamping force thresholds. Instead, it inputs maturity confidence levels, fruit size, varietal differences, and prior data from compression experiments into the property mapping model, thereby obtaining safe clamping force upper limits and current limits that better reflect the individual differences of the target fruit. This mechanism enhances adaptability to fruits of different varieties, sizes, and ripening stages, and reduces the risk of crushing or slippage caused by a single static threshold.

[0038] In the above embodiments, current limiting threshold analysis is performed on the visual physical property feature matrix of the fruit to obtain the armature current limiting threshold of the motor. This can output a continuously changing upper limit of the safe clamping force, avoiding abrupt changes in the bottom torque threshold caused by discrete label jumps. It also avoids damage to ripe fruit or slippage of unripe fruit due to misjudgment of maturity, enhances the adaptability to fruits of different varieties, sizes, and different maturity transition states, and reduces the risk of damage or slippage caused by a single static threshold.

[0039] Optionally, as an embodiment of the present invention, such as Figure 3 As shown, the process of performing feedback force analysis on the visual physical property feature matrix of the fruit and the real-time armature current to obtain the real-time virtual contact feedback force includes: The three-dimensional coordinates of the fruit are transformed to obtain the global coordinates of the robotic arm base; The end effector gripper of the robotic arm is driven according to the global coordinates of the robotic arm base; Extract the fruit diameter from the fruit size; The diameter of the fruit is calculated using the third equation to obtain the outer diameter of the envelope, and the end effector gripper of the robotic arm is controlled based on the outer diameter of the envelope. The third equation is as follows: , in, The outer diameter of the envelope, The diameter of the fruit. Pre-set safety redundancy gap; By importing the equivalent transmission radius of the gripper mechanism, the gripper motion friction force, and the gripper transmission mechanism efficiency, the real-time armature current, and the gripper motion friction force are calculated using the fourth equation to obtain the real-time virtual contact feedback force. The fourth equation is: , in, For real-time virtual contact feedback force, Let be the equivalent transmission radius of the gripper mechanism. To improve the efficiency of the gripper drive mechanism. The torque constant of the drive motor, For real-time armature current, This refers to the frictional force of the gripper's movement.

[0040] It should be understood that the hand-eye calibration matrix is ​​used to... The three-dimensional coordinates of the fruit are transformed into the coordinate system of the robotic arm base, driving the six-degree-of-freedom robotic arm to move so that the axis of the end effector gripper coincides with the center of the fruit. Before the gripper approaches the fruit, the drive motor controls the gripper to pre-open to the outer diameter of the envelope, based on the measured fruit diameter w. (in (with a pre-set safety redundancy gap) to complete the collision-free spatial envelope of the fruit.

[0041] It should be understood that after the envelope pre-shaping is completed, the underlying execution controller initiates the gripper closing control sequence. The drive motor operates in current-position composite control mode, using constant deceleration pulses to drive the fingertips of both grippers to move towards the center.

[0042] It should be understood that the torque is calculated internally in real time without physical sensors. During the closed stroke, the current sampling resistor in the underlying execution controller operates at a high-frequency clock cycle (sampling rate). Real-time capture of the real-time armature current of the drive motor Specifically, utilizing the electromagnetic torque conversion principle of a permanent magnet DC servo motor, the motor's output electromagnetic torque... It has a strictly proportional linear mapping relationship with the armature current, as shown in the following equation: , In the formula, is the inherent torque constant of the motor.

[0043] The transmission ratio via a gripper drive mechanism (such as a ball screw or rack and pinion) (i.e., the efficiency of the gripper transmission mechanism) and the effective lever arm radius (i.e., the equivalent transmission radius of the gripper mechanism) The underlying controller can calculate the real-time virtual contact feedback force of the gripper on the fruit surface through pure internal current without the need for any external thin-film pressure sensors. As shown in the following formula: , In the formula, The gripper mechanism rotates with the motor speed The changing internal inherent static and dynamic friction compensation terms (i.e., the friction of the gripper movement).

[0044] In the above embodiments, feedback force analysis is performed on the visual physical property feature matrix of the fruit and the real-time armature current to obtain the real-time virtual contact feedback force. This completely eliminates the need for external fragile sensors, eliminates the torque spike pulsation caused by hard truncation, and realizes a fully closed-loop flexible and non-destructive wrapping and grasping of fruits with different textures, as well as flexible, high success rate and non-destructive picking of fruits with different textures.

[0045] Optionally, as an embodiment of the present invention, such as Figure 4 and 5 As shown, the process of optimizing the real-time virtual contact feedback force based on the real-time armature current and the motor armature current limit threshold, and controlling the robotic arm to grasp based on the optimization analysis results, includes: S51: Determine whether the real-time armature current is greater than or equal to the motor armature current limit threshold. If yes, execute S52; otherwise, control the robotic arm to close. S52: Update the real-time armature current according to the motor armature current limit threshold to obtain the updated real-time armature current; S53: Control the motor of the robotic arm according to the updated real-time armature current and preset motor control rules; S54: Update the real-time virtual contact feedback force according to the updated real-time armature current, and control the robotic arm to grasp according to the update result.

[0046] It should be understood that when the tips of the grippers are not in contact with the fruit surface, (i.e., real-time armature current) is only used to overcome internal friction and is maintained at an extremely low level. The moment the fingertip touches the fruit surface, the motor tends to stall, and the armature current... (i.e., the real-time armature current) begins to rise sharply, and the calculated feedback force (That is, the real-time virtual contact feedback force) increases synchronously. The underlying execution controller will calculate the force in real time. (i.e., real-time armature current) and the hardness cutoff threshold established by visual semantics (i.e., the motor armature current limit threshold) is compared at a high frequency of milliseconds.

[0047] Specifically, adaptive stepless torque smooth hold control. Once detected... (That is, when the grasping force reaches the critical point of the compressive modulus of the fruit at the current maturity level), the controller does not execute the traditional "cut off the drive pulse" hard cutoff logic, but instead adaptively switches the control state to "magnetic linkage locked constant current source torque holding mode" (i.e., preset motor control rules), as follows: The master controller writes an adaptive duty cycle modulation instruction to the PWM control register of the servo driver, forcibly locking and clamping the current supplied to the motor at the current value. At a constant value. At the same time, in order to block the armature current pulsation caused by the huge back electromotive force (Back-EMF) generated at both ends of the winding due to the sudden deceleration and stall of the motor, the bottom drive circuit activates the "complementary conduction current low loss attenuation path (Current Decay Modulation)", which uses the full conduction of the internal H-bridge low-side MOSFET to form a freewheeling loop, and rapidly attenuates and dissipates the high-frequency oscillation waveform of the impact torque induced by the back EMF within milliseconds.

[0048] The gripper motor maintains a constant cutoff current. Maintaining a stable flexible static friction clamping force in a pulsation-free state The robotic arm then performs the retraction and picking action. Throughout the entire grasping stroke, the torque transition is extremely smooth, without any high-frequency torque peak pulsations, completely eliminating the technical risks of localized crushing or breaking of fragile ripe fruit from the underlying physical actuator.

[0049] In the above embodiments, the real-time virtual contact feedback force is optimized and analyzed based on the real-time armature current and the motor armature current limit threshold, and the robotic arm is controlled to grasp the object based on the optimization analysis results, thus completely eliminating the technical risks of local crushing and crushing of fragile ripe fruit.

[0050] Optionally, as another embodiment of the present invention, the present invention first acquires fruit images using an RGB-D camera and inputs them into an improved YOLO multi-task deep learning network, which outputs in parallel the spatial absolute pose and the probability vector of maturity confidence. Then, it loads a property mapping model to perform weighted calculations on maturity confidence, fruit size, and variety, outputting the upper limit of the safe clamping force and converting it into the current limit value of the motor. When the gripper closes, the armature current is sampled at high frequency, and the real-time virtual contact feedback force of the gripper on the fruit is calculated in reverse dynamic calculation. When the capture current reaches the limit value, it adaptively switches to the magnetic flux locking constant current source torque holding mode, and simultaneously turns on the low-side MOSFET of the H-bridge to conduct the low-loss freewheeling circuit, dissipating the high-frequency oscillation of the stalled back EMF. The present invention completely eliminates the external fragile sensor, eliminates the torque spike pulsation caused by hard truncation, and realizes a fully closed-loop flexible non-destructive wrapping and gripping for fruits with different textures.

[0051] Optionally, as another embodiment of the present invention, this proposal aims to address significant deficiencies in the prior art by providing a flexible, non-destructive fruit grasping system and method capable of deeply decoupling and mapping high-level visual semantics with low-level control constraints, achieving adaptive hardness and softness due to fruit heterogeneity without the need for external, fragile hardware force sensors, and providing a smooth and continuous torque transition. Specific technical problems to be solved include: (i) How to construct a multi-task collaborative feature output detection network head, which simultaneously and in parallel computes and outputs the spatial location bounding box of the target fruit and the subdivided biological maturity level semantic labels within a single deep learning network architecture, thus solving the problem that traditional visual perception only outputs spatial coordinates and lacks material property semantics.

[0052] (ii) How to establish a maturity-elastic modulus-stiffness prior mapping library based on cross-modal features, and mathematically map discrete computer vision maturity semantic labels to continuous physical boundary constraints of the maximum allowable clamping torque at the hardware level. This solves the technical problem of a one-size-fits-all torque control benchmark that cannot adapt to the heterogeneity of biological hardness.

[0053] (III) How to design an adaptive torque closed-loop control algorithm and smooth holding mechanism based on real-time monitoring of drive motor current, so as to adaptively adjust the closing torque by monitoring the motor armature current without the need for an external physical pressure sensor, and to reach the point where the closing torque is adjusted. It achieves smooth torque switching and maintenance without step change, solving the technical problems of easy damage to traditional external sensors and high-frequency torque pulsation caused by hard cut-off.

[0054] Optionally, as another embodiment of the present invention, the present invention eliminates the drawbacks of decoupling perception semantics and action execution by constructing a perception brain that integrates multi-task output, a cross-modal physical property feature prior mapping library, and low-level electrodynamic closed-loop servo control at the execution end. Under the premise of not requiring external physical pressure sensors, the present invention achieves flexible, high-success-rate, and non-destructive picking and grasping of heterogeneous biological fruits.

[0055] Alternatively, as another embodiment of the present invention, such as Figure 6 As shown, the flexible non-destructive grasping system provided by this invention mainly consists of the following core functional modules and logical connections to the physical hardware carrier: I. Advanced Multimodal Perception Brain Module: The physical carrier includes an RGB-D depth camera mounted on the end of a robotic arm and an edge computing terminal (embedded GPU computing platform) connected to it via a high-frequency data bus. This module is used to acquire unstructured field point clouds and 2D images in real time, and output the spatial geometric 3D coordinates of the fruit and refined maturity semantic labels in parallel.

[0056] II. Main Control Decision and Strategy Jump Module: The physical carrier is a central processing unit or an industrial host computer. This module integrates a maturity-compressive elastic modulus and hardness prior mapping library, which is used to receive discrete semantic tags output by the sensory brain, and dynamically calculate and generate the maximum allowable closing clamping torque and current cutoff safety boundary of the target fruit being grasped according to the preset mapping matrix.

[0057] III. Underlying Flexible Torque Servo Execution Module: The physical carrier includes a robotic arm controller, a servo driver, and an electric two-finger / multi-finger end effector mechanism with an integrated built-in current sampling resistor (shunt). This module receives the torque cutoff boundary from the host computer and constructs a sensorless virtual force feedback closed loop by sampling the armature current of the drive motor in real time at high frequency, driving the gripper to smoothly close and hold.

[0058] Alternatively, as another embodiment of the present invention, such as Figure 7 As shown, the overall implementation process of the method provided by this invention is mainly carried out through the following mutually closed-loop coupled steps in a progressive manner: Step S301: Multi-task visual feature extraction and multi-dimensional feature space output.

[0059] Step S302: Mapping of cross-modal physical property characteristic domains and establishment of dynamic torque threshold.

[0060] Step S303: Kinematic 3D spatial pose alignment and gripper envelope pre-shaping.

[0061] Step S304: Adaptive torque closed-loop servo control based on real-time monitoring of drive motor current.

[0062] Step S305: Non-destructive, stepless torque maintenance and back EMF vibration damping defense execution.

[0063] Optionally, as another embodiment of the present invention, the core of the present invention proposal lies in breaking the disconnect between traditional visual detection and low-level torque execution. By combining multi-task high-level semantic recognition with low-level current closed-loop servo control without physical sensors, a cross-modal physical property feature mapping mechanism is established, as follows: 1. A deep decoupling mapping architecture for cross-modal "visual high-level semantics - underlying physical constraints" This invention designs a multi-task collaborative feature output detection network head. At the output of a single-stage target detection network, a localization regression branch and a segmented maturity semantic classification branch are set in parallel. This directly solves and outputs a discrete one-dimensional nonlinear one-hot encoded vector to establish the fruit maturity label. Simultaneously, an internal "maturity-compressive elastic modulus and hardness prior mapping library" is established, and a transformation operator is used... Discrete visual semantic labels are dynamically mapped in real time to the maximum allowable armature current cutoff limit of the underlying actuator hardware. and ultimate safety clamping force This system achieves cross-modal linkage from "visual perception semantics" to "underlying dynamic physical boundary constraints" at the system architecture level. This enables the system to adapt to the heterogeneity of hardness in organisms and automatically switch physical defense boundaries for fruits at different developmental stages, fundamentally overcoming the technical bottleneck of damage and slippage caused by traditional single-torque "one-size-fits-all" control.

[0064] 2. Sensorless Virtual Force Feedback Control Mechanism Based on High-Frequency Sampling of Drive Motor Current This invention completely eliminates the need for external physical sensors. During the gripper closing sequence, it utilizes the current sampling resistor within the underlying execution controller to... High-frequency sampling rate captures real-time armature current of the drive motor Combining the electromagnetic torque conversion principle of permanent magnet DC servo motors, the gripper transmission ratio, and the effective lever arm, an internal calculation operator is constructed that includes inherent static and dynamic friction compensation terms. This allows for real-time reverse derivation and calculation of the real-time virtual contact feedback force between the gripper and the fruit surface. A highly robust, sensorless internal force closed-loop circuit was constructed. Millisecond-level contact force variation tracking was achieved without increasing the end load or introducing external, easily damaged hardware, completely solving the reliability challenge of system failure caused by the susceptibility of physical sensors to failure in the harsh environment of Daejeon.

[0065] 3. Adaptive flux linkage locking torque smooth maintenance and back EMF vibration damping and defense control The underlying execution controller of this invention detects Upon the moment of stall tendency, instead of executing a logic shutdown that disrupts the dynamic continuity, the controller adaptively switches to "magnetic linkage locked constant current source torque holding mode." The main controller uses PWM duty cycle modulation commands to forcibly clamp the armature current to the current... At a constant value; simultaneously, to block the huge back electromotive force generated at both ends of the winding due to sudden deceleration and stall of the motor, the drive circuit activates the "complementary conduction current low-loss attenuation path," utilizing the full conduction of the low-side MOSFETs of the H-bridge to form a freewheeling circuit, rapidly attenuating and dissipating the high-frequency oscillation waveform of the impact torque induced by the back electromotive force within milliseconds. This achieves a smooth transition without step jumps in the gripping force from "dynamic centripetal approach" to "stable static friction clamping." It eliminates the high-frequency spike pulsation of electromagnetic torque caused by hard-cutoff control commands, ensuring that the physical pressure of the contact surface is always maintained within the biological pressure resistance threshold, and completely eliminating the technical risk of transient overload damage at the underlying physical actuator level.

[0066] Optionally, as another embodiment of the present invention, the present invention reconstructs and sets up a "localization regression branch" and a "subdivision maturity semantic classification branch" in parallel at the output of a visual deep learning network, and uses the Softmax output layer to calculate a discrete one-dimensional nonlinear one-hot encoding vector, thereby accurately locking the specific developmental stage label of the target fruit. Furthermore, relying on the system's built-in "maturity-compressive elastic modulus and hardness prior mapping library," cross-modal transformation operators are utilized. Discrete visual semantic features are converted into the maximum allowable armature current cutoff limit of the underlying actuator motor in real time and seamlessly. and ultimate safety clamping force This allows the control system to perceive the physical properties of the target fruit before it even contacts it. The system can automatically and precisely switch physical defense boundaries for tomatoes at different stages: "green ripening (firm)," "color changing (semi-firm)," and "ripe (soft)." When facing fragile ripe fruit, it can strictly compress the upper limit of the current cutoff to ensure that the gripping force does not exceed the cell rupture limit; when facing hard, heavy green fruit, it can release a higher upper limit of torque, completely solving the technical bottleneck of localized pressure damage or dynamic slippage caused by traditional single-threshold "one-size-fits-all" control.

[0067] Optionally, as another embodiment of the present invention, the present invention completely eliminates the external physical force-sensitive element of the gripper, and instead adds a high-frequency current sampling resistor inside the underlying execution controller, so as to... High-frequency sampling rate for real-time capture of motor armature current Utilizing the strictly proportional linear mapping principle of electromagnetic torque and current of a permanent magnet DC servo motor, and integrating the system's built-in internal torque inverse calculation operator, which includes the gripper transmission ratio, effective lever arm radius, and internal inherent static and dynamic friction compensation terms that vary with motor speed, the real-time virtual contact feedback force of the gripper fingertips on the fruit surface is derived in reverse and in real time. This approach elevates "external physical hardware contact sensing" to "internal high-frequency numerical calculation of electromagnetic current." Because it eliminates the need for any external hardware sensors, the end effector does not generate parasitic loads and is completely immune from the technical risks of zero-point calibration drift, high-frequency noise distortion, and even physical damage caused by high temperatures, high humidity, mud mist, dust, and scraping from dense branches and leaves in field conditions. This significantly improves the engineering robustness, reliability, and lifespan of the entire control chain in harsh agricultural field environments, while reducing maintenance costs.

[0068] Optionally, as another embodiment of the present invention, the present invention reconstructs the current limiting triggering mechanism of the underlying actuator. When the sampled current... Reaching the hardness cutoff threshold for visual semantic establishment At this time, the controller abandons the discontinuous logic hard cutoff of the traditional forced shutdown pulse, and instead adaptively switches to "magnetic flux locked constant current source torque holding mode", using PWM duty cycle modulation command to clamp the armature current steplessly at the current level. At a constant value, the drive circuit simultaneously activates the "complementary conduction current low-loss attenuation path," utilizing the full conduction of the low-side MOSFETs of the H-bridge to form a low-resistance freewheeling circuit. This rapidly attenuates and dissipates the enormous back electromotive force (Back-EMF) generated at the winding ends during the moment of motor stall within milliseconds. This maintains the high-order continuity of the control signal throughout the entire contact dynamics stroke of the system. Through magnetic flux locking and the H-bridge freewheeling vibration damping mechanism, the high-frequency spike pulsation of the back EMF and electromagnetic torque caused by the hard-cutoff control command is completely eliminated, avoiding transient local pressure overload on the contact surface caused by torque oscillation, and ensuring perfectly smooth fluctuations in the current waveform and gripping force. This provides optimal flexible shock absorption and damage-free protection for the fragile fruit without reducing gripping efficiency, while also significantly reducing fatigue damage to the precision reducer gears of the robotic arm.

[0069] Alternatively, as another embodiment of the present invention, compared with the prior art, the present invention enables the system to have the ability to anticipate the future full-time-domain position evolution of a swaying target before the robotic arm initiates its approach stroke. This overcomes the cascading delay caused by camera inference time and the physical inertia of the robotic arm joints from a mathematical perspective. This completely eliminates the overshoot and high-frequency reciprocating oscillation of the control system, improves the alignment success rate of the spatial envelope window to an industrial-grade robust standard, and avoids defects such as the robotic arm accidentally scratching the fruit, hitting surrounding branches and leaves, or frequently missing its target.

[0070] Figure 8 This is a block diagram of a fruit grasping device provided in an embodiment of the present invention.

[0071] Alternatively, as another embodiment of the present invention, such as Figure 8 As shown, a fruit-grabbing device includes: The image acquisition module is used to acquire images of the area to be grasped by an RGB-D depth camera installed at the end of the robotic arm, so as to obtain the RGB image and depth map of the target fruit. The feature analysis module is used to construct a multi-task deep learning network, and to perform feature analysis on the RGB image of the target fruit and the depth map of the target fruit through the multi-task deep learning network to obtain the visual physical property feature matrix of the fruit. The current limit threshold analysis module is used to perform current limit threshold analysis on the visual physical property feature matrix of the fruit to obtain the motor armature current limit threshold. The feedback force analysis module is used to import the real-time armature current, perform feedback force analysis on the visual physical property feature matrix of the fruit and the real-time armature current, and obtain the real-time virtual contact feedback force. The optimization analysis module is used to perform optimization analysis on the real-time virtual contact feedback force based on the real-time armature current and the motor armature current limit threshold, and control the robotic arm to grasp based on the optimization analysis results.

[0072] Optionally, as an embodiment of the present invention, the multi-task deep learning network includes a Backbone feature extraction subnetwork, a Neck feature fusion subnetwork, a position regression branch detection head, and a maturity classification branch detection head; The feature analysis module is specifically used for: The Backbone feature extraction subnetwork is used to extract features from the RGB image of the target fruit to obtain multiple original fruit features; The original fruit features are fused using the Neck feature fusion subnetwork to obtain the original fruit fused features; The location regression branch detection head is used to predict the location regression of the original fruit fusion features, and the two-dimensional box of the fruit, the box confidence, the fruit size, and the fruit variety are obtained. Import the camera intrinsic parameter matrix, perform coordinate system transformation on the fruit 2D bounding box, the bounding box confidence, the target fruit depth map, and the camera intrinsic parameter matrix to obtain the fruit 3D coordinates; The maturity classification branch detection head performs a maturity classification prediction analysis on the original fruit fusion features to obtain a confidence probability vector. The fruit visual property feature matrix includes the fruit size, the fruit variety, the fruit three-dimensional coordinates, and the confidence probability vector.

[0073] Optionally, another embodiment of the present invention provides a fruit-grabbing system, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the fruit-grabbing method described above. This system can be a computer or similar system.

[0074] Optionally, another embodiment of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the fruit-grabbing method as described above.

[0075] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0076] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described apparatus and unit can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0077] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0078] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of the present invention, depending on actual needs.

[0079] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0080] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0081] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for grasping fruit, characterized in that, Includes the following steps: An RGB-D depth camera installed at the end of the robotic arm acquires images of the area to be grasped, resulting in an RGB image of the target fruit and a depth map of the target fruit. A multi-task deep learning network is constructed, and feature analysis is performed on the RGB image and depth map of the target fruit through the multi-task deep learning network to obtain the visual physical property feature matrix of the fruit. The current limiting threshold is obtained by performing current limiting threshold analysis on the visual physical property feature matrix of the fruit; By importing the real-time armature current, feedback force analysis is performed on the visual physical property feature matrix of the fruit and the real-time armature current to obtain the real-time virtual contact feedback force. The real-time virtual contact feedback force is optimized and analyzed based on the real-time armature current and the motor armature current limit threshold, and the robotic arm is controlled to grasp based on the optimization analysis results.

2. The fruit grasping method according to claim 1, characterized in that, The multi-task deep learning network includes a backbone feature extraction sub-network, a Neck feature fusion sub-network, a position regression branch detection head, and a maturity classification branch detection head; The process of performing feature analysis on the RGB image and depth map of the target fruit using the multi-task deep learning network to obtain the visual physical property feature matrix of the fruit includes: The Backbone feature extraction subnetwork is used to extract features from the RGB image of the target fruit to obtain multiple original fruit features; The original fruit features are fused using the Neck feature fusion subnetwork to obtain the original fruit fused features; The location regression branch detection head is used to predict the location regression of the original fruit fusion features, and the two-dimensional box of the fruit, the box confidence, the fruit size, and the fruit variety are obtained. Import the camera intrinsic parameter matrix, perform coordinate system transformation on the fruit 2D bounding box, the bounding box confidence, the target fruit depth map, and the camera intrinsic parameter matrix to obtain the fruit 3D coordinates; The maturity classification branch detection head performs a maturity classification prediction analysis on the original fruit fusion features to obtain a confidence probability vector. The fruit visual property feature matrix includes the fruit size, the fruit variety, the fruit three-dimensional coordinates, and the confidence probability vector.

3. The fruit grasping method according to claim 2, characterized in that, The maturity classification branch detection head includes a 1×1 convolutional layer, a batch normalization layer, a SiLU activation function layer, a 3×3 convolutional layer group, a global pooling layer, a hidden layer, a Dropout layer, a classification prediction layer, and a Softmax activation function layer. The process of performing maturity classification prediction analysis on the original fruit fusion features using the maturity classification branch detection head to obtain the confidence probability vector includes: The original fruit fusion features are extracted by the 1×1 convolutional layer to obtain the fruit fusion features to be processed. The batch normalization layer is used to normalize the fusion features of the fruit to be processed, and the normalized fruit fusion features are obtained. The normalized fruit fusion features are reduced in dimensionality by the SiLU activation function layer to obtain the dimensionality-reduced fruit fusion features. The 3×3 convolutional layer group is used to extract features from the dimensionality-reduced fruit fusion features to obtain the processed fruit fusion features. The processed fruit fusion features are pooled using the global pooling layer to obtain pooled fruit fusion features. The original hidden fruit features are obtained by performing dimensionality reduction on the pooled fruit fusion features through the hidden layer. The original hidden features of the fruit are randomly discarded through the Dropout layer to obtain the hidden features of the fruit after discarding. The classification prediction layer predicts the hidden features of the discarded fruit to obtain multiple maturity category score features; The softmax activation function layer is used to normalize the score features of all maturity categories, and the confidence probability vector is obtained by combining all the normalization results.

4. The fruit grasping method according to claim 2, characterized in that, The process of performing current limitation threshold analysis on the visual physical property feature matrix of the fruit to obtain the armature current limitation threshold of the motor includes: Based on the fruit size and the fruit variety, a set of baseline limit clamping forces is extracted from a preset prior mapping library. The safe clamping force is obtained by calculating the reference limit clamping force set and the confidence probability vector using the first formula, which is: ， in, For safe clamping force, For the first Each confidence level probability For the first One benchmark limit clamping force; By importing the efficiency of the gripper transmission mechanism, the equivalent transmission radius of the gripper mechanism, and the no-load friction current, the safety clamping force, the efficiency of the gripper transmission mechanism, the equivalent transmission radius of the gripper mechanism, and the no-load friction current are calculated using the second equation to obtain the motor armature current limiting threshold. The second equation is: ， in, This is the armature current limiting threshold for the motor. For safe clamping force, Let be the equivalent transmission radius of the gripper mechanism. To improve the efficiency of the gripper drive mechanism. The torque constant of the drive motor, This is the no-load triboelectric current.

5. The fruit grasping method according to claim 2, characterized in that, The process of performing feedback force analysis on the visual physical property feature matrix of the fruit and the real-time armature current to obtain the real-time virtual contact feedback force includes: The three-dimensional coordinates of the fruit are transformed to obtain the global coordinates of the robotic arm base; The end effector gripper of the robotic arm is driven according to the global coordinates of the robotic arm base; Extract the fruit diameter from the fruit size; The diameter of the fruit is calculated using the third equation to obtain the outer diameter of the envelope, and the end effector gripper of the robotic arm is controlled based on the outer diameter of the envelope. The third equation is as follows: ， in, The outer diameter of the envelope, The diameter of the fruit. Pre-set safety redundancy gap; By importing the equivalent transmission radius of the gripper mechanism, the gripper motion friction force, and the gripper transmission mechanism efficiency, the real-time armature current, and the gripper motion friction force are calculated using the fourth equation to obtain the real-time virtual contact feedback force. The fourth equation is: ， in, For real-time virtual contact feedback force, Let be the equivalent transmission radius of the gripper mechanism. To improve the efficiency of the gripper drive mechanism. The torque constant of the drive motor, For real-time armature current, This refers to the frictional force of the gripper's movement.

6. The fruit grasping method according to claim 1, characterized in that, The process of optimizing the real-time virtual contact feedback force based on the real-time armature current and the motor armature current limit threshold, and controlling the robotic arm to grasp based on the optimization analysis results, includes: S51: Determine whether the real-time armature current is greater than or equal to the motor armature current limit threshold. If yes, execute S52; otherwise, control the robotic arm to close. S52: Update the real-time armature current according to the motor armature current limit threshold to obtain the updated real-time armature current; S53: Control the motor of the robotic arm according to the updated real-time armature current and preset motor control rules; S54: Update the real-time virtual contact feedback force according to the updated real-time armature current, and control the robotic arm to grasp according to the update result.

7. A fruit-grabbing device, characterized in that, include: The image acquisition module is used to acquire images of the area to be grasped by an RGB-D depth camera installed at the end of the robotic arm, so as to obtain the RGB image and depth map of the target fruit. The feature analysis module is used to construct a multi-task deep learning network, and to perform feature analysis on the RGB image of the target fruit and the depth map of the target fruit through the multi-task deep learning network to obtain the visual physical property feature matrix of the fruit. The current limit threshold analysis module is used to perform current limit threshold analysis on the visual physical property feature matrix of the fruit to obtain the motor armature current limit threshold. The feedback force analysis module is used to import the real-time armature current, perform feedback force analysis on the visual physical property feature matrix of the fruit and the real-time armature current, and obtain the real-time virtual contact feedback force. The optimization analysis module is used to perform optimization analysis on the real-time virtual contact feedback force based on the real-time armature current and the motor armature current limit threshold, and control the robotic arm to grasp based on the optimization analysis results.

8. The fruit grasping device according to claim 7, characterized in that, The multi-task deep learning network includes a backbone feature extraction sub-network, a Neck feature fusion sub-network, a position regression branch detection head, and a maturity classification branch detection head; The feature analysis module is specifically used for: The Backbone feature extraction subnetwork is used to extract features from the RGB image of the target fruit to obtain multiple original fruit features; The original fruit features are fused using the Neck feature fusion subnetwork to obtain the original fruit fused features; The location regression branch detection head is used to predict the location regression of the original fruit fusion features, and the two-dimensional box of the fruit, the box confidence, the fruit size, and the fruit variety are obtained. Import the camera intrinsic parameter matrix, perform coordinate system transformation on the fruit 2D bounding box, the bounding box confidence, the target fruit depth map, and the camera intrinsic parameter matrix to obtain the fruit 3D coordinates; The maturity classification branch detection head performs a maturity classification prediction analysis on the original fruit fusion features to obtain a confidence probability vector. The fruit visual property feature matrix includes the fruit size, the fruit variety, the fruit three-dimensional coordinates, and the confidence probability vector.

9. A fruit-grabbing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the fruit-grabbing method as described in any one of claims 1 to 6.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the fruit-grabbing method as described in any one of claims 1 to 6.

Images