Article soft gripping control method based on visual and overall curved shape feedback

Abstract

This invention discloses a method for soft object gripping control based on visual and overall bending shape feedback, including: acquiring the allowed contact area, prohibited area, local geometry, and gripping constraints of the fruit; establishing a reference relationship between air pressure and the free shape of the entire finger without contact; matching the free shape with the allowed contact area and planning the initial air pressure of each finger; reconstructing the actual overall shape and comparing it with the free shape under the same pressure point by point to identify contact; predicting the contact area, normal contact force, maximum local contact pressure, and reliability using an offline calibration model; independently adjusting the air pressure of each finger and determining stable gripping; executing fruit removal after stabilization, and depressurizing or repositioning in case of abnormalities. This invention can solve the problem that existing technologies are difficult to accurately reflect the contact position and degree of constraint of the entire soft finger, and are prone to insufficient gripping force or excessive local pressure.

Description

Technical Field

[0001] This invention relates to the field of electronic information technology, and more specifically to a soft-body clamping control method for objects based on visual and overall bending shape feedback. Background Technology

[0002] With the development of flexible materials, 3D vision, and intelligent control technologies, pneumatic soft grippers have been widely used, especially in fruit picking, because they can form an envelope contact with objects through flexible deformation. However, different fruits vary in size, shape, ripeness, and surface pressure resistance, and the gripping process is also affected by fruit eccentricity, nonlinearity of soft materials, manufacturing errors, and dynamic differences in the air path.

[0003] Existing control schemes typically preset a uniform air pressure based on the type or size of the fruit, or control based solely on information such as the total bending angle and single-point curvature. This makes it difficult to accurately reflect the contact position and degree of constraint of the entire soft finger, and can easily lead to problems such as insufficient clamping force or excessive local pressure. Summary of the Invention

[0004] In view of this, the present invention provides a soft object clamping control method based on visual and overall bending shape feedback, in order to solve the problem that the prior art is difficult to accurately reflect the contact position and degree of constraint of the entire soft finger, and is prone to insufficient clamping force or excessive local pressure.

[0005] A method for soft object gripping control based on vision and overall bending shape feedback includes: S1. Use a 3D vision device to acquire 3D surface data of the object and surrounding objects, and determine the target permissible contact sub-region, prohibited area, local geometric parameters and clamping safety parameters corresponding to each pneumatic soft finger in a common coordinate system. S2. For any pneumatic soft finger, select a continuous material representative line from the root to the fingertip on the surface of the side facing the object. Use the arc length of the representative line from the root when it is not deformed as the material coordinate. Establish a reference relationship between the air cavity pressure and the free shape position vector when there is no external contact in the finger root coordinate system. S3. Based on the free shape position vector and the target allowed contact sub-region and restricted area, transform the free shape of each pneumatic soft finger to the common coordinate system, determine the first contact prediction air pressure that makes the finger contact side reach the corresponding target allowed contact sub-region and maintain a safe gap with the restricted area, and add pressure margin within the allowed maximum air chamber pressure to obtain the initial target air pressure of each finger. S4, control each pneumatic soft finger to increase the pressure to the initial target air pressure, obtain the actual measured air chamber pressure and shape detection data of multiple material positions along the finger length direction, reconstruct the actual shape curve in the corresponding finger root coordinate system, and compare it with the free shape curve under the same actual measured air chamber pressure point by point, and identify the contact state according to the shape deviation. S5. After the contact between the objects is identified in step S4, the actual measured air cavity pressure, shape deviation distribution characteristics, normalized position of the maximum deviation, and local geometric parameters are combined to form a contact state feature vector, which is then input into the pre-calibrated contact state prediction model to obtain the predicted contact area, predicted normal contact force, predicted maximum local contact pressure, and prediction confidence. S6. Based on the predicted contact area, predicted normal contact force, predicted maximum local contact pressure and prediction reliability, and combined with the clamping safety parameters, the pressure of each pneumatic soft finger is adjusted independently, and the clamping is determined to be stable when the stable clamping conditions are met. S7, after determining that the clamping is stable in step S6, performs the disengagement action, and performs the protection action when an abnormality occurs.

[0006] The soft object clamping control method based on vision and overall bending shape feedback provided by the present invention has the following beneficial effects: 1. This invention first transforms the free-form curve to the common coordinate system of the fruit, and then uses the target permissible contact area and the prohibited area to jointly constrain the initial contact air pressure, which can reduce accidental contact with the fruit stem, defects, and branches during the initial planning stage. Specifically, through spatial transformation and distance constraints, prohibited areas are avoided before initial contact, reducing accidental contact with non-target areas; furthermore, by comparing the actual shape with the free shape point by point, the invention accurately identifies whether contact occurs in the permissible or prohibited area, thus providing accurate spatial location information for subsequent force control. The combination of the above spatial constraints and contact positioning effectively solves the technical problem of easily accidentally contacting non-target areas such as the fruit stem and branches due to the lack of accurate contact position feedback.

[0007] 2. This invention compares the actual continuous shape of the entire finger represented by the same material coordinate system with the free continuous shape under the same pressure, without equating multiple air chambers to multiple curvature detection segments. By aligning the root coordinate system, the influence of the overall movement of the robotic arm on shape deviation can be eliminated. In step S2, a continuous free shape reference for the entire finger is established in the material coordinate system. In step S4, the actual shape is reconstructed on the same material coordinate grid and compared point by point, thereby obtaining the shape deviation distribution along the entire active segment, rather than local curvature information at discrete points. The above-mentioned global shape comparison strategy effectively solves the technical problem that it is difficult to accurately reflect the contact position and degree of constraint of the entire finger when relying solely on the total bending angle or single-point curvature information.

[0008] 3. This invention characterizes the degree of external constraint and its spatial position along the entire finger by using average deviation, maximum deviation, normalized position of maximum deviation, and piecewise deviation features when sufficient samples are available. This information, along with the local geometry of the fruit, is input into a prediction model that has undergone offline modeling, threshold calibration, and independent testing. This reduces the difficulty in distinguishing different contact states when using only air pressure or a single total bending amount. This invention maps the spatial distribution characteristics of shape deviation into key mechanical quantities such as contact area, normal force, and local pressure. Through multi-dimensional feature fusion and model prediction strategies, it effectively solves the technical problem of insufficient control decision-making basis due to the difficulty in distinguishing different contact states caused by single feedback information.

[0009] 4. This invention uses the predicted contact area as the minimum effective contact constraint, allowing the contact area to directly participate in the determination of stable clamping; the maximum local contact pressure serves as the fruit damage constraint, while utilizing the model calibration domain and prediction reliability to avoid unconstrained extrapolation. In step S6, the predicted values ​​are compared with the minimum effective contact area, the minimum stable contact force, and the maximum allowable local contact pressure, respectively, and the model calibration domain and reliability index are introduced as prerequisites for performing pressure adjustment actions. This multi-objective constraint and model validity check mechanism effectively solves the technical problem that when only air pressure or force is used as the sole constraint, it is difficult to simultaneously prevent fruit damage (local pressure control) and ensure clamping stability (contact area and force control).

[0010] 5. This invention independently adjusts the pressure of each of the three fingers while simultaneously checking the contact area, normal contact force, maximum local contact pressure, force imbalance, and frictional bearing capacity. This is beneficial for balancing damage prevention and stable clamping under conditions of fruit asymmetry and differences in finger manufacturing. This invention allows each finger to be differentiated according to its local contact state (geometric curvature, contact force magnitude, local pressure), thereby achieving a stable clamping state with balanced force distribution without exceeding limits. It effectively solves the technical problem of difficulty in adapting to differences in finger contact due to uniform air pressure control, resulting in some fingers experiencing excessive force while others are under-clamped. Attached Figure Description

[0011] Figure 1 This is a flowchart illustrating a software gripping control method for objects based on visual and overall bending shape feedback. Figure 2 A schematic diagram of an exemplary three-finger pneumatic soft fruit-picking end effector; Figure 3 A schematic diagram of the structure of an exemplary single five-cavity pneumatic soft finger; Figure 4 This is a comparison chart of numerical calculation results between traditional uniform air pressure regulation and the three-finger independent air pressure regulation of the present invention under the equivalent working condition of fruit eccentric contact. Figure 5The graph shows a comparison of the time history of the maximum single-finger normal force in the numerical example between traditional unified air pressure regulation and the three-finger independent air pressure regulation of the present invention. Detailed Implementation

[0012] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain embodiments of the present invention, and should not be construed as limiting the present invention.

[0013] Please see Figure 1 The present invention provides a method for soft object clamping control based on visual and overall bending shape feedback, the method comprising steps S1 to S7: S1 uses a 3D vision device to acquire 3D surface data of the object and surrounding objects, and determines the target permissible contact area, prohibited area, local geometric parameters and clamping safety parameters corresponding to each pneumatic soft finger in a common coordinate system.

[0014] In this embodiment, fruit is used as an example for illustration. The method is applied to an end effector including a three-dimensional vision device, a controller, and three independently adjustable pneumatic soft fingers. Each pneumatic soft finger is equipped with a pressure sensor and a bending shape detection mechanism. The three pneumatic soft fingers are arranged circumferentially around the central axis of the fruit picking device, and the circumferential angle between two adjacent pneumatic soft fingers is 120°. Each finger is inflated, pressurized, and deflated by a proportional pressure valve, and the controller outputs independent pressure commands to the three proportional pressure valves.

[0015] Specifically, such as Figure 2 As shown, the end effector includes a circular base 1, three tilting mounts 2, a guide connecting block 3, and a pneumatic soft finger 4. The tilting mounts 2 determine the finger installation direction, and the guide connecting block 3 positions and fixes the finger root; the coordinate system of the finger root is obtained from the common coordinate system through installation posture calibration. The transformation relationship is used in subsequent steps S2 and S3.

[0016] In step S1, the center, principal axis direction, contour dimensions, and local surface normal of the fruit are determined based on the three-dimensional point cloud of the fruit. The longitudinal local normal curvature is determined along the longitudinal projection direction of the corresponding pneumatic soft finger and its orthogonal direction. and transverse local curvature The prohibited area is defined as the fruit tree, surface defects, foliage obstruction, and interference areas with adjacent fruits, along with the safety expansion boundary; based on the variety, maturity, or firmness grade of the target fruit, the equivalent firmness of the fruit is selected from a parameter library established through experiments. Equivalent static friction coefficient Minimum effective contact area Minimum stable contact force Maximum permissible local contact pressure Expected fruit load and maximum allowable air chamber pressure Specifically, , , , , , The parameters for fruit removal or disturbance loads are selected from the calibrated parameter library.

[0017] S2. For any pneumatic soft finger, select a continuous material representative line from the root to the fingertip on the surface of the side facing the object. Use the arc length of the representative line from the root when it is not deformed as the material coordinate. Establish a reference relationship between the air cavity pressure and the free shape position vector when there is no external contact in the coordinate system of the finger root.

[0018] Among them, with Figure 2 A coordinate system is established at the finger root defined by the guide connecting block 3 shown, using material coordinates. This refers to the entire finger. Select the pressure increase or decrease branch based on the direction of pressure change; actual pressure. When located between adjacent calibration nodes, the interpolated free-shape position vector is obtained by interpolating the same material coordinates. Unconstrained extrapolation is not performed when the actual pressure exceeds the calibrated and verified range.

[0019] Specifically, the material coordinates at the boundary between the fixed root and the moving section, and at the end of the moving section, are respectively denoted as... and , That is, the initial total length of the finger and the length of its active segment. Set along the active section One fixed material location:

[0020] in, Indicates the first segment set along the moving part of the pneumatic soft finger Material coordinates for a fixed material location; ; .

[0021] Specifically, in this embodiment, the initial total length of a single finger... The dimensions are 136mm in length, 40mm in width, and 7mm in thickness for the continuous bottom layer; the material coordinates of the fixed root length and the boundary between the fixed root and the movable section are also specified. Both are 26mm, and the length of the movable section is... The active segment comprises five repeating air chamber units with a unit pitch of 22 mm. Each unit, along its length, includes a 10 mm interval segment, a 2 mm rear wall, an 8 mm air chamber, and a 2 mm front wall. The internal dimensions of a single air chamber are 8 mm × 34 mm × 19 mm, with side and top wall thicknesses of 3 mm, and a total finger height of 29 mm. These dimensions represent one feasible structural example and are not the only limitation on the finger structure to which the control method is applicable.

[0022] like Figure 3 As shown, the first air chamber 40, the second air chamber 41, the third air chamber 42, the fourth air chamber 43, and the fifth air chamber 44 are connected to the air inlet 45 through internal air passages. Each finger is driven by a single air pressure, and the pressure of each air chamber is not adjusted independently; the three fingers are adjusted independently through their respective proportional pressure valves. The bending shape detection is performed on the entire moving section, and the local curvature of the corresponding section of each air chamber is not used as the feedback quantity.

[0023] The air cavity body and the continuous bottom layer can be molded, vacuum degassed, cured and then bonded separately, or they can be molded as one piece; if the upper and lower layers are bonded together, the bonding and post-curing should be completed before free inflation calibration.

[0024] In the silicone matrix implementation with a Shore A hardness of 50, the Mooney-Rivlin parameters are: the first material constant of the Mooney-Rivlin model. The second material constant of the Mooney-Rivlin model Material bulk modulus related parameters The material density is The corresponding initial equivalent Young's modulus is approximately 2.848 MPa and the initial Poisson's ratio is approximately 0.483.

[0025] The Mooney-Rivlin strain energy density function used in this embodiment ,in, , The first and second invariants of the right Cauchy-Green tensor under isochoric deformation. Volume ratio; corresponding initial shear modulus initial bulk modulus .

[0026] Seven fixed visual markers are set for each finger's active segment, namely... The undeformed coordinates of each point are saved according to the material coordinate formula. In this embodiment, these coordinates are 26.00mm, 44.33mm, 62.67mm, 81.00mm, 99.33mm, 117.67mm, and 136.00mm, respectively. The side marker points are converted into representative points of the continuous material on the contact side through non-contact calibration. When a point is temporarily obstructed, spline reconstruction is only performed if there are at least four valid points, with valid points near the root and fingertip; otherwise, pressure holding or repositioning is required. The seven detection points are distributed according to material coordinates and do not correspond one-to-one with the five air chambers.

[0027] Under ordered pressure nodes, the spatial coordinates of each fixed material position are obtained through a free inflation test without external contact, or the spatial coordinates are obtained through a finite element analysis without external contact corrected by the free inflation test, and a free shape curve is fitted.

[0028] The above interpolation relationship applies to situations where the pressure changes monotonically and there is no significant hysteresis.

[0029] When actual pressure When linear interpolation is used:

[0030]

[0031] in, For the first An ordered pressure calibration node; For the first An ordered pressure calibration node; These are the linear interpolation weighting coefficients; Indicates the first Each finger under actual pressure Below, material coordinates The interpolated free-shape position vector at the location; Indicates the first One finger in Below, material coordinates The free-form position vector at the location; Indicates the first One finger in Below, material coordinates The free-form position vector at that location; If there are distinguishable differences in the free shape at the same pressure node during the pressurization and depressurization processes, reference relationships for the pressurization and depressurization branches are established separately. During online control, the corresponding branch is selected for interpolation based on the current direction of pressure change.

[0032] Specifically, when establishing the free-form reference library, the pressure is gradually increased from 0 to 0.13 MPa. After each pressure node stabilizes, the data is recorded, and the non-contact inflation and depressurization cycle is repeated. The pressure boosting branch and depressurization branch are saved respectively. During the online pressure boosting stage, the pressure boosting branch is queried first.

[0033] S3. Based on the free shape position vector and the target allowed contact sub-region and restricted area, transform the free shape of each pneumatic soft finger to a common coordinate system, determine the first contact prediction air pressure that makes the finger contact side reach the corresponding target allowed contact sub-region and maintain a safe gap with the restricted area, and add pressure margin within the allowed maximum air chamber pressure to obtain the initial target air pressure of each finger.

[0034] Among them, according to The free shape is determined by the coordinate system at the base of the finger. Transform to common coordinate system ,in, Indicates the first Freeform transformation of each finger to a common coordinate system The position vector after that, and These are the rotation matrix and translation vector, respectively; the candidate pressure set is denoted as... The minimum distance from the finger contact side to the target's permissible contact area is denoted as... From zero boost to The minimum gap between the area occupied by the finger on the path and the restricted area after safety expansion is denoted as . The initial contact predicted pressure and the initial target pressure are respectively:

[0035]

[0036] in, For the first The first touch of a finger predicts air pressure. For the first The initial target air pressure for one finger. To allow for contact distance tolerance, To prevent entry into the safety gap, This is the initial pressure margin. When any of the above feasible sets is empty, the target allowed contact sub-region is redefined or the fruit picking device is adjusted.

[0037] Specifically, in Internal addition get If no pressure is available, adjust the gripper position or repeat S1; press each of the three fingers individually. Independent supercharging.

[0038] In this embodiment, the allowable contact distance tolerance Determined by a combination of point cloud error, marker point reconstruction error, and free reference library error; prohibited safety gap. The error exceeds the composite error value. Contact confirmation time. Take the shortest duration that can be stably resolved, and the stable holding time. Greater than All of the above parameters have been determined through equipment calibration.

[0039] S4, control each pneumatic soft finger to increase the pressure to the initial target air pressure, obtain the actual measured air chamber pressure and shape detection data of multiple material positions along the finger length direction, reconstruct the actual shape curve in the corresponding finger root coordinate system, and compare it with the free shape curve under the same actual measured air chamber pressure point by point, and identify the contact state according to the shape deviation.

[0040] Among them, the sampling time The actual measured air chamber pressure is recorded as and define the shape deviation vector. Average shape deviation Maximum shape deviation and the location of maximum deviation :

[0041]

[0042]

[0043]

[0044] in, Indicates the first Each finger at a moment Material coordinates The actual shape and position vector at that location; Indicates the first A pneumatic soft finger at all times The actual measured air chamber pressure; Indicates the first Each finger is under pressure Time and material coordinates The free-form position vector at that location; express The differential; when or If the corresponding contactless threshold is exceeded and a preset confirmation time is maintained, and the... When the candidate contact position obtained by transforming the actual material point from the finger root coordinate system to the common coordinate system is located in the target allowed contact sub-region and the shape deviation direction is consistent with the target allowed contact sub-region, it is determined to be target fruit contact; if it exceeds the threshold but the candidate contact position or shape deviation direction is inconsistent with the target allowed contact sub-region, it is determined to be abnormal contact constraint.

[0045] S5. After the object contact is identified in step S4, the actual measured air cavity pressure, shape deviation distribution characteristics, normalized position of maximum deviation, and local geometric parameters are combined to form a contact state feature vector, which is then input into the pre-calibrated offline contact state prediction model to obtain the predicted contact area, predicted normal contact force, predicted maximum local contact pressure, and prediction confidence.

[0046] Among them, let the position of the maximum deviation be normalized. And constitute the contact state feature vector:

[0047] in, Indicates the first Each finger at a moment The contact state feature vector, Indicates transpose; Pre-calculated according to the equivalent hardness of the fruit and equivalent static friction coefficient Divide the parameter groups into sets and establish prediction models for each set, then output the prediction vectors online. Prediction credibility index and model calibration domain Internal and external judgment results, among which For the first Each finger at a moment The predicted contact area For the first Each finger at a moment Predicted normal contact force, For the first Each finger at a moment The predicted maximum local contact pressure.

[0048] In this embodiment, the offline contact sample is based on the equivalent hardness of fruit. and equivalent static friction coefficient Grouping, covering air cavity pressure, normalized contact location, within the planned harvesting area. and The contact points cover the middle of the active segment and near the fingertips. The samples are divided into modeling samples, threshold calibration samples, and independent test samples based on independent operating conditions.

[0049] Specifically, the contact state prediction model is a quadratic response surface model, through... Six inputs are used to construct a quadratic response surface; the first... The modeling sample mean and standard deviation of each input feature are denoted as follows: and ,in , and according to standardization, For the first The standardized values ​​of the input features For the first The original values ​​of each input feature; the basis function vector is composed of constant terms, linear terms, quadratic terms, and pairwise interaction terms of the standardized input. Let the modeling sample basis function matrix, label matrix, and model coefficient matrix be denoted as follows: , and The results were obtained through ridge regression:

[0050]

[0051] in, Let be the ridge regression regularization parameter. Input scalar number is Number of time base functions ; for An identity matrix of order 1; Indicates at time For the first Standardized feature vectors of individual fingers The basis function vector obtained after evaluation; Let the Euclidean distance from the online standardized feature vector to the nearest modeling sample be denoted as . And make the prediction credibility index for:

[0052] in, To standardize the distance scale parameter in the feature space, Model parameters, model calibration domain, and confidence threshold are determined by threshold calibration samples and verified by independent test samples.

[0053] In this embodiment, each output corresponds to One basis function coefficient. The number of samples for modeling is greater than 28 and... Full column order; Selected through internal cross-validation of the modeling samples. Threshold calibration samples determined. , Independent test samples are used for final error verification; if the error does not meet the preset control accuracy, the number of samples is increased or the applicable range of the model is reduced.

[0054] The first The target per finger is allowed contact sub-region denoted as . The safety margin for local contact pressure is denoted as , will the The single inflation step size of one finger is denoted as... and will the Each finger at a moment The air pressure command is denoted as .

[0055] S6, based on the predicted contact area, predicted normal contact force, predicted maximum local contact pressure and prediction reliability, and combined with the clamping safety parameters, performs independent pressure adjustment on each pneumatic soft finger, and determines stable clamping when the stable clamping conditions are met.

[0056] Each pneumatic soft finger makes contact with the target fruit, resulting in an average normal contact force. And define the force imbalance index. and frictional bearing margin :

[0057]

[0058] in, Indicates the total number of fingers in the pneumatic software; To prevent zero force, ; For safety reasons, ; The fruit gravity load is calculated along the expected slip direction; Perform the following controls sequentially for each finger: model out of bounds or The pressure increase will stop when the pressure drops below a preset confidence threshold. Stress reduction; and Pressure holding or pressure reduction, among which To preset the upper limit of force imbalance; or Furthermore, the difference between the maximum local contact pressure and the predicted maximum local contact pressure is allowed to be greater than the preset local pressure safety margin, the prediction reliability is not lower than the preset reliability threshold, and the next pressure does not exceed the preset local pressure safety margin. When pressurization occurs, increase the pressure in small increments no larger than the preset single pressurization step size; otherwise, maintain the pressure. When all fingers simultaneously meet the following conditions: predicted contact area is not less than the minimum effective contact area, predicted normal contact force is not less than the minimum stable contact force, predicted maximum local contact pressure is not greater than the maximum allowable local contact pressure, force imbalance index is not greater than the preset upper limit, and prediction reliability is not lower than the preset reliability threshold. If the preset holding time is maintained, it is considered a stable clamping condition. The shape is reconstructed and safety conditions are checked after each update.

[0059] In this embodiment, the bending shape detection mechanism includes visual markers and an image acquisition device arranged at at least four different material locations along the side or non-contact side of each pneumatic soft finger; the first... One finger The coordinates of each visual marker point are converted into the coordinates of continuous material representative points at the same material cross-section, and then solved under the constraints of root location and root tangential direction:

[0060] in, Indicates the first The actual shape curve function of each finger; This represents the entire shape curve function to be solved; In the optimization problem, the material coordinates are... The position vector to be solved at; For the detection point sequence number index; Indicates the first Each finger at a moment , No. The coordinates of the continuous material representative point on the contact side after conversion of the visual marker points; For smoothing coefficients, ; yes Material coordinates The second derivative, It is a function In coordinates The value at that point represents the three-dimensional spatial position vector of the material point on the finger. This solution simultaneously satisfies the preset root position boundary conditions and root tangential boundary conditions.

[0061] S7, after determining that the clamping is stable in step S6, performs the disengagement action, and performs the protection action when an abnormality occurs.

[0062] Among these measures, fruit removal is carried out after stabilization, and continuous monitoring is conducted. Shape deviation, contact condition , And local contact pressure. If the pressure exceeds the limit, stop air intake and release pressure; if contact is lost, abnormal contact, or multi-finger coordination is abnormal, stop fruit detachment and perform pressure release, release, or robotic arm retraction; if the model goes out of bounds or the credibility is unqualified, stop pressurization. After repositioning, clear the old state and re-execute from step S1.

[0063] To verify the feasibility of this invention, simulation tests were conducted. Under the same initial pose, material parameters, contact geometry, and air pressure loading time history, numerical calculations were performed to compare the traditional unified air pressure regulation with the three-finger independent air pressure regulation of this invention. In the simulation test, the fruit was approximated with isotropic linear elasticity, with an elastic modulus of 0.50 MPa and a Poisson's ratio of 0.45; the soft fingers used the elastic parameters of the material in step S2. The equivalent local radii of curvature at the contact points between the three fingers and the fruit were 24 mm, 30 mm, and 20 mm, respectively. The initial contact air pressures were 0.018 MPa, 0.026 MPa, and 0.033 MPa, respectively, and the air pressure-injection coefficients were 55 mm / MPa, 48 mm / MPa, and 58 mm / MPa, respectively, to characterize the differences in fruit eccentricity, local geometry, and finger response. The equivalent elastic modulus of both contacting fingers was calculated using the following formula:

[0064] Among them, subscript Indicates fruit, subscript Indicates soft fingers, Indicates the elastic modulus. Let represent Poisson's ratio; substituting the initial tangential elastic modulus of the soft finger (2.848 MPa), the initial Poisson's ratio (0.483), and the aforementioned fruit parameters (elastic modulus 0.50 MPa, Poisson's ratio 0.45), we obtain the equivalent elastic modulus. It is 0.53642 MPa.

[0065] The gas path uses a time constant The first-order pressure dynamics are calculated, and the non-contact interference is calculated based on the difference between the actual air pressure and the initial contact air pressure:

[0066]

[0067] in, For the first Rate of change of air chamber pressure per finger For the first The air pressure command for each finger For the first The actual air chamber pressure of each finger For the first The amount of non-contact interference per finger, For the first The air pressure-inflation coefficient of each finger.

[0068] No. The amount of pressure applied by each finger Using natural frequency Damping ratio Second-order dynamics:

[0069] in, Angular frequency, For the first Rate of change of the amount of pressure applied by each finger contact For the first The acceleration of the amount of pressure applied by each finger.

[0070] Normal contact force Contact area and maximum local contact pressure The traditional linear elastic Hertzian contact relationship is then used for calculation.

[0071] Traditional unified barometric pressure regulation and the three-finger independent barometric pressure regulation of this invention use the same initial state and model parameters, both at 0. The pressure is smoothly increased within 0.50 seconds, with a calculation termination time of 2.00 seconds. In traditional unified pressure regulation, the target pressure for all three fingers is 0.045 MPa, which is the minimum common pressure required to achieve a steady-state normal contact force of 1.85 N for the third finger that makes contact last. Independent pressure regulation for the three fingers starts at 0.50 seconds and adjusts the pressure of each finger separately with an update cycle of 0.02 seconds, with a target normal contact force range of 1.85 N. 2.15N, minimum effective contact area is 40mm² 2 The maximum allowable upper limit of local contact pressure is 0.070 MPa, the normal single pressure adjustment amount is 0.0005 MPa, and the single pressure reduction amount when the local contact pressure exceeds the allowable upper limit is 0.0010 MPa.

[0072] The numerical simulation results are as follows: Figure 4 and Figure 5 As shown. By Figure 4 It is known that under the traditional uniform air pressure regulation strategy, because the three fingers use the same target air pressure, and the fruit's eccentricity leads to significant differences in the contact geometry and pressing response of each finger, the normal contact forces of the three fingers are severely uneven in steady state, amounting to 6.341 N, 3.412 N, and 1.857 N respectively, with the ratio of the maximum force to the minimum force reaching 3.41 times. However, under the independent air pressure regulation strategy of the three fingers in this invention, the controller adjusts the air pressure command according to the predicted contact area, normal contact force, and maximum local contact pressure of each finger. In steady state, the normal contact forces of the three fingers are 1.879 N, 2.044 N, and 2.094 N respectively, with the ratio of the maximum force to the minimum force being only 1.11 times, and the force on the three fingers tends to be more balanced. This indicates that the present invention, through an independent pressure regulation mechanism, can effectively overcome the problem of uneven force caused by the fruit's eccentricity or differences in finger manufacturing, avoiding overloading of some fingers while others are under-gripped.

[0073] Further as Figure 4 As shown, under traditional unified air pressure regulation, the common target air pressure is increased to compensate for insufficient contact force of the third finger, resulting in a steady-state maximum local contact pressure of 0.08495 MPa for the first finger, exceeding the allowable upper limit of 0.070 MPa, posing a risk of damaging the fruit. In contrast, under the independent air pressure regulation of the three fingers in this invention, the local contact pressure of each finger is controlled within the allowable range, with a steady-state maximum local contact pressure of 0.06631 MPa, meeting the safety constraints. Therefore, this invention, by using the predicted maximum local contact pressure as a feedback constraint and executing the highest priority decompression action, can effectively prevent local overpressure damage to the fruit.

[0074] Further as Figure 4 As shown, the steady-state mean of the force imbalance index under traditional uniform air pressure regulation is as high as 1.159, indicating a severe imbalance in the force distribution among the three fingers. In contrast, the steady-state mean of the force imbalance index under independent air pressure regulation of this invention is 0.1074, a reduction of 90.7%. This demonstrates that by continuously monitoring the force imbalance index and adjusting the pressure of each finger in a targeted manner during the clamping process, this invention can significantly improve the coordinated force distribution among the three fingers.

[0075] comprehensive Figure 4 and Figure 5 The comparative results show that, under the aforementioned equivalent parameters and eccentric contact conditions, compared with traditional unified air pressure regulation, the three-finger independent air pressure regulation reduces the steady-state maximum single-finger normal force from 6.341 N to 2.094 N, a reduction of approximately 67.0%, effectively preventing single-finger overload; the steady-state maximum local contact pressure decreases from 0.08495 MPa to 0.06631 MPa, a reduction of approximately 21.9%, meeting the requirements for preventing fruit damage; the force imbalance index decreases from 1.159 to 0.1074, a reduction of approximately 90.7%, significantly improving the force balance of the three fingers; and the peak value of the maximum single-finger normal force decreases from 6.341 N to 5.271 N. These numerical results fully verify the significant superiority of this invention over the traditional unified air pressure control method from three dimensions: force balance, local pressure safety, and collaborative stability.

[0076] In summary, the object gripping control method based on vision and overall bending shape feedback according to the above embodiments has the following beneficial effects: 1. This invention first transforms the free-form curve to the common coordinate system of the fruit, and then uses the target permissible contact area and the prohibited area to jointly constrain the initial contact air pressure, which can reduce accidental contact with the fruit stem, defects, and branches during the initial planning stage. Specifically, through spatial transformation and distance constraints, prohibited areas are avoided before initial contact, reducing accidental contact with non-target areas; furthermore, by comparing the actual shape with the free shape point by point, the invention accurately identifies whether contact occurs in the permissible or prohibited area, thus providing accurate spatial location information for subsequent force control. The combination of the above spatial constraints and contact positioning effectively solves the technical problem of easily accidentally contacting non-target areas such as the fruit stem and branches due to the lack of accurate contact position feedback.

[0077] 2. This invention compares the actual continuous shape of the entire finger represented by the same material coordinate system with the free continuous shape under the same pressure, without equating multiple air chambers to multiple curvature detection segments. By aligning the root coordinate system, the influence of the overall movement of the robotic arm on shape deviation can be eliminated. In step S2, a continuous free shape reference for the entire finger is established in the material coordinate system. In step S4, the actual shape is reconstructed on the same material coordinate grid and compared point by point, thereby obtaining the shape deviation distribution along the entire active segment, rather than local curvature information at discrete points. The above-mentioned global shape comparison strategy effectively solves the technical problem that it is difficult to accurately reflect the contact position and degree of constraint of the entire finger when relying solely on the total bending angle or single-point curvature information.

[0078] 3. This invention characterizes the degree of external constraint and its spatial position along the entire finger by using average deviation, maximum deviation, normalized position of maximum deviation, and piecewise deviation features when sufficient samples are available. This information, along with the local geometry of the fruit, is input into a prediction model that has undergone offline modeling, threshold calibration, and independent testing. This reduces the difficulty in distinguishing different contact states when using only air pressure or a single total bending amount. This invention maps the spatial distribution characteristics of shape deviation into key mechanical quantities such as contact area, normal force, and local pressure. Through multi-dimensional feature fusion and model prediction strategies, it effectively solves the technical problem of insufficient control decision-making basis due to the difficulty in distinguishing different contact states caused by single feedback information.

[0079] 4. This invention uses the predicted contact area as the minimum effective contact constraint, allowing the contact area to directly participate in the determination of stable clamping; the maximum local contact pressure serves as the fruit damage constraint, while utilizing the model calibration domain and prediction reliability to avoid unconstrained extrapolation. In step S6, the predicted values ​​are compared with the minimum effective contact area, the minimum stable contact force, and the maximum allowable local contact pressure, respectively, and the model calibration domain and reliability index are introduced as prerequisites for performing pressure adjustment actions. This multi-objective constraint and model validity check mechanism effectively solves the technical problem that when only air pressure or force is used as the sole constraint, it is difficult to simultaneously prevent fruit damage (local pressure control) and ensure clamping stability (contact area and force control).

[0080] 5. This invention independently adjusts the pressure of each of the three fingers while simultaneously checking the contact area, normal contact force, maximum local contact pressure, force imbalance, and frictional bearing capacity. This is beneficial for balancing damage prevention and stable clamping under conditions of fruit asymmetry and differences in finger manufacturing. This invention allows each finger to be differentiated according to its local contact state (geometric curvature, contact force magnitude, local pressure), thereby achieving a stable clamping state with balanced force distribution without exceeding limits. It effectively solves the technical problem of difficulty in adapting to differences in finger contact due to uniform air pressure control, resulting in some fingers experiencing excessive force while others are under-clamped.

[0081] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A method for soft object gripping control based on visual and overall bending shape feedback, characterized in that, include: S1. Use a 3D vision device to acquire 3D surface data of the object and surrounding objects, and determine the target permissible contact sub-region, prohibited area, local geometric parameters and clamping safety parameters corresponding to each pneumatic soft finger in a common coordinate system. S2. For any pneumatic soft finger, select a continuous material representative line from the root to the fingertip on the surface of the side facing the object. Use the arc length of the representative line from the root when it is not deformed as the material coordinate. Establish a reference relationship between the air cavity pressure and the free shape position vector when there is no external contact in the finger root coordinate system. S3. Based on the free shape position vector and the target allowed contact sub-region and restricted area, transform the free shape of each pneumatic soft finger to the common coordinate system, determine the first contact prediction air pressure that makes the finger contact side reach the corresponding target allowed contact sub-region and maintain a safe gap with the restricted area, and add pressure margin within the allowed maximum air chamber pressure to obtain the initial target air pressure of each finger. S4, control each pneumatic soft finger to increase the pressure to the initial target air pressure, obtain the actual measured air chamber pressure and shape detection data of multiple material positions along the finger length direction, reconstruct the actual shape curve in the corresponding finger root coordinate system, and compare it with the free shape curve under the same actual measured air chamber pressure point by point, and identify the contact state according to the shape deviation. S5. After the contact between the objects is identified in step S4, the actual measured air cavity pressure, shape deviation distribution characteristics, normalized position of the maximum deviation, and local geometric parameters are combined to form a contact state feature vector, which is then input into the pre-calibrated contact state prediction model to obtain the predicted contact area, predicted normal contact force, predicted maximum local contact pressure, and prediction confidence. S6. Based on the predicted contact area, predicted normal contact force, predicted maximum local contact pressure and prediction reliability, and combined with the clamping safety parameters, the pressure of each pneumatic soft finger is adjusted independently, and the clamping is determined to be stable when the stable clamping conditions are met. S7, after determining that the clamping is stable in step S6, performs the disengagement action, and performs the protection action when an abnormality occurs.

2. The soft-body gripping control method for objects based on vision and overall bending shape feedback according to claim 1, characterized in that, In step S1, the item is a fruit. The center, principal axis direction, contour size, and local surface normal of the fruit are determined based on the three-dimensional point cloud of the fruit. The longitudinal local normal curvature is determined along the longitudinal projection direction of the corresponding pneumatic soft finger and its orthogonal direction. and transverse local curvature The prohibited area is defined as the fruit tree, surface defects, foliage obstruction, and interference areas with adjacent fruits, along with the safety expansion boundary; based on the variety, maturity, or firmness grade of the target fruit, the equivalent firmness of the fruit is selected from a parameter library established through experiments. Equivalent static friction coefficient Minimum effective contact area Minimum stable contact force Maximum permissible local contact pressure Expected fruit load and maximum allowable air chamber pressure .

3. The soft-body clamping control method for objects based on vision and overall bending shape feedback according to claim 1, characterized in that, In step S2, the material coordinates at the boundary between the fixed root and the moving section and at the end of the moving section are respectively denoted as... and Length of active segment Set along the active section One fixed material location: in, Indicates the first [section] set along the moving segment of the pneumatic soft finger. Material coordinates for a fixed material location; ; ; Under ordered pressure nodes, the spatial coordinates of each fixed material position are obtained through a free inflation test without external contact, or the spatial coordinates are obtained through a finite element analysis without external contact corrected by the free inflation test, and a free shape curve is fitted. When actual pressure When linear interpolation is used: in, For the first An ordered pressure calibration node; For the first An ordered pressure calibration node; These are the linear interpolation weighting coefficients; Indicates the first Each finger under actual pressure Below, material coordinates The interpolated free-shape position vector at the location; Indicates the first One finger in Below, material coordinates The free-form position vector at that location; Indicates the first One finger in Below, material coordinates The free-form position vector at that location; If there are distinguishable differences in the free shape at the same pressure node during the pressurization and depressurization processes, reference relationships for the pressurization and depressurization branches are established separately. During online control, the corresponding branch is selected for interpolation based on the current direction of pressure change.

4. The soft-body clamping control method for objects based on vision and overall bending shape feedback according to claim 3, characterized in that, In step S3, according to The free shape is determined by the coordinate system at the base of the finger. Transform to common coordinate system ,in, Indicates the first Freeform transformation of each finger to a common coordinate system The position vector after that, and These are the rotation matrix and translation vector, respectively; the candidate pressure set is denoted as... The minimum distance from the finger contact side to the target's permissible contact area is denoted as... From zero boost to The minimum gap between the area occupied by the finger on the path and the restricted area after safety expansion is denoted as . The initial contact predicted pressure and the initial target pressure are respectively: in, For the first The first touch of a finger predicts air pressure. For the first The initial target air pressure for one finger. To allow for contact distance tolerance, To prevent entry into the safety gap, This is the initial pressure margin. When any of the above feasible sets is empty, the target allowed contact sub-region is redefined or the fruit picking device is adjusted.

5. The soft-body gripping control method for objects based on vision and overall bending shape feedback according to claim 4, characterized in that, In step S4, the sampling time is... The actual measured air chamber pressure is recorded as and define the shape deviation vector. Average shape deviation Maximum shape deviation and the position of maximum deviation : in, Indicates the first Each finger at a moment Material coordinates The actual shape and position vector at that location; Indicates the first A pneumatic soft finger at all times The actual measured air chamber pressure; Indicates the first Each finger is under pressure Time and material coordinates The free-form position vector at that location; express The differential; when or If the corresponding contactless threshold is exceeded and a preset confirmation time is maintained, and the... When the candidate contact position obtained by transforming the actual material point from the finger root coordinate system to the common coordinate system is located in the target allowed contact sub-region and the shape deviation direction is consistent with the target allowed contact sub-region, it is determined to be the target fruit contact.

6. The soft-body gripping control method for objects based on vision and overall bending shape feedback according to claim 5, characterized in that, In step S5, the position of the maximum deviation is normalized. And constitute the contact state feature vector: in, Indicates the first Each finger at a moment The contact state feature vector, Indicates transpose; Pre-calculated according to the equivalent hardness of the fruit and equivalent static friction coefficient Divide the parameter groups into sets and establish prediction models for each set, then output the prediction vectors online. Prediction credibility index and model calibration domain Internal and external judgment results, among which For the first Each finger at a moment The predicted contact area For the first Each finger at a moment Predicted normal contact force, For the first Each finger at a moment The predicted maximum local contact pressure.

7. The soft-body clamping control method for objects based on vision and overall bending shape feedback according to claim 6, characterized in that, In step S5, the contact state prediction model is a quadratic response surface model; the first... The modeling sample mean and standard deviation of each input feature are denoted as follows: and ,in , and according to standardization, For the first The standardized values ​​of the input features For the first The original values ​​of each input feature; the basis function vector is composed of constant terms, linear terms, quadratic terms, and pairwise interaction terms of the standardized input. Let the modeling sample basis function matrix, label matrix, and model coefficient matrix be denoted as follows: , and The results were obtained through ridge regression: in, Let be the ridge regression regularization parameter. Input scalar number is Number of time base functions ; for An identity matrix of order 1; Indicates at time For the Standardized feature vectors of individual fingers The basis function vector obtained after evaluation; Let the Euclidean distance from the online standardized feature vector to the nearest modeling sample be denoted as . And make the prediction credibility index for: in, To standardize the distance scale parameter in the feature space, Model parameters, model calibration domain, and confidence threshold are determined by threshold calibration samples and verified by independent test samples.

8. The soft-body clamping control method for objects based on vision and overall bending shape feedback according to claim 7, characterized in that, In step S6, after each pneumatic soft finger makes contact with the target fruit, the average normal contact force is set... And define the force imbalance index. and frictional bearing margin : in, Indicates the total number of fingers in the pneumatic software; To prevent the removal of zero force, ; For safety reasons, ; The fruit gravity load is calculated along the expected slip direction; Perform the following controls sequentially for each finger: model out of bounds or The pressure increase will stop when the pressure drops below a preset confidence threshold. Stress reduction; and Pressure holding or pressure reduction, among which To preset the upper limit of force imbalance; or Furthermore, the difference between the maximum local contact pressure and the predicted maximum local contact pressure is allowed to be greater than the preset local pressure safety margin, the prediction reliability is not lower than the preset reliability threshold, and the next pressure does not exceed the preset local pressure safety margin. When pressurization occurs, increase the pressure in small increments no larger than the preset single pressurization step size; otherwise, maintain the pressure. When all fingers simultaneously meet the following conditions: predicted contact area is not less than the minimum effective contact area, predicted normal contact force is not less than the minimum stable contact force, predicted maximum local contact pressure is not greater than the maximum allowable local contact pressure, force imbalance index is not greater than the preset upper limit, and prediction reliability is not lower than the preset reliability threshold. When the preset holding time is maintained, it is determined to be a stable clamping.

9. The soft-body gripping control method for objects based on vision and overall bending shape feedback according to claim 8, characterized in that, Each pneumatic soft finger is equipped with a pressure sensor and a bending shape detection mechanism. The bending shape detection mechanism includes visual markers located at at least four different material positions along the side or non-contact side of each pneumatic soft finger, as well as an image acquisition device. One finger The coordinates of each visual marker point are converted into the coordinates of continuous material representative points at the same material cross-section, and then solved under the constraints of root location and root tangential direction: in, Indicates the first The actual shape curve function of each finger; This represents the entire shape curve function to be solved; In the optimization problem, the material coordinates are... The position vector to be solved at; For the detection point sequence number index; Indicates the first Each finger at a moment , No. The coordinates of the continuous material representative point on the contact side after conversion of the visual marker points; For smoothing coefficients, ; yes Material coordinates The second derivative, It is a function In coordinates The value at that location.

10. The soft-body gripping control method for objects based on vision and overall bending shape feedback according to claim 9, characterized in that, The number of pneumatic soft fingers is three, and each pneumatic soft finger is equipped with an air pressure sensor; The three pneumatic soft fingers are arranged circumferentially around the central axis of the fruit picking device, with a circumferential angle of 120° between two adjacent pneumatic soft fingers; each finger is inflated, pressurized and deflated by a proportional pressure valve, and the controller outputs independent air pressure commands to the three proportional pressure valves.

Images