Active lossless grabbing gripper and multi-mode touch sensing method thereof

Abstract

The invention relates to the field of robot arms, in particular to an active lossless grabbing paw and a multi-mode touch sensing method.The active lossless grabbing paw comprises a support, a rolling ball is installed at the upper end of the support, and the rolling ball is composed of a rolling ball framework composed of eight curved bars and a sensing layer film; the side end of the support supports transmission of a rolling ball framework belt pulley through a rolling bearing, and the belt pulley is driven by a stepping motor on the back face of the support. The claw adopts an active compliant control system, the posture and the position of the claw can be timely adjusted according to the contact condition, if the claw is blocked by branches, the claw can actively bend or avoid and smoothly reach the position of cones for picking, and the claw can automatically adjust the grabbing mode according to the shape and the size of the cones through active compliant control, so that the picking efficiency is improved. Fruit epidermis damage and internal tissue damage caused by overexertion are avoided, it is ensured that the fruits are stably grabbed, and the grabbing success rate and the nondestructive rate are increased.

Description

Technical Field

[0001] The present invention relates to the field of robotic arms, and particularly to an active non-destructive grasping claw and its multi-modal tactile perception method. Background Art

[0002] The grasping and sorting processes of crops have a decisive impact on their yield and quality, and are directly related to the market competitiveness of agricultural products. The traditional manual sorting method not only has a large labor intensity and low efficiency, but also easily causes fruit damage due to improper grasping, affecting the crop quality and sales value. Moreover, with the increase in labor costs, the economic burden of manual sorting is also increasing.

[0003] Traditional crop grasping claws mostly adopt rigid structures or simple flexible drives, and there are some problems: in terms of structure, some active claws place the drive motor inside the rolling ball, resulting in limited volume of the rolling ball, making it difficult to adapt to interaction objects of different sizes, and difficult for heat dissipation and maintenance; in terms of force perception, existing tactile sensors are mostly based on a single modality, unable to accurately extract three-dimensional contact forces and force feedback under dynamic motion interference. The claw depends on the preset clamping force, making it difficult to dynamically adapt to the shape and hardness of fruits, and easily causing epidermal damage or internal tissue damage.

[0004] Therefore, the present invention proposes an active claw structure with an external motor and a multi-modal tactile perception method. When grasping and sorting crops, highly integrated sensors and advanced artificial intelligence algorithms are used to accurately identify the maturity level and implement non-destructive grasping, so as to reduce fruit losses caused by inadvertent human operations, help significantly improve the sorting and grasping efficiency and crop quality, and promote the sustainable development of agriculture and the modernization process of agricultural mechanization and intelligentization in China. Summary of the Invention

[0005] In order to overcome the above technical problems, the purpose of the present invention is to provide an active non-destructive grasping claw and its multi-modal tactile perception method to solve the problems in the background art.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] An active non-destructive grasping claw includes a bracket. At the upper end of the bracket, a rolling ball is installed. The rolling ball consists of a rolling ball skeleton composed of eight curved rods and an induction layer film. The side end of the bracket supports the transmission of the rolling ball skeleton pulley through a rolling bearing. The pulley is driven by a stepper motor on the back of the bracket, thereby driving the rolling ball skeleton to rotate around a fixed axis. On the other side of the bracket, a camera device is installed. The camera device extends into the rolling ball to capture the image returned from the induction layer. The lower end of the bracket is fixedly installed on a servo bracket. The servo bracket is connected to a servo. There are three combinations of the bracket, the servo bracket and the servo, and they are evenly fixed on the base through the servo.

[0008] As a further solution of the present invention, the rolling ball is wrapped with an induction layer film. The outer layer of the induction layer is made into a spherical shape, and raised marking points are provided on the inner side. A high-resolution camera is selected to capture the image returned from the induction layer. The induction layer is aligned with the camera, and a uniform light source is selected to evenly illuminate the induction layer of the sensor to ensure that the illumination condition of the induction layer remains stable when the external environment changes. After preparing the induction layer, it is necessary to determine how the captured image corresponds to the undulations on the surface of the induction layer. Different-shaped objects need to be selected for testing. To ensure accurate measurement, the original image captured by the camera is not directly used. First, an image without object contact is recorded as a reference, and then the actual measured image is subtracted from the reference image, and the difference is used for analysis. When an object touches the sensor, the marking points on the inner side of the induction layer will move according to the force received on the surface. By observing the movement of the marking points, the form of the force can be analyzed. When the object starts to slide, the markings on the sensor will also move accordingly. Usually, when the object starts to slide, the marking points at the edge move first, and the marking points at the center move later. The uneven degree of the movement of these markings is measured by "entropy". If this value suddenly becomes larger, it means that the object is sliding. Finally, a comprehensive test of the sensor is carried out, and adjustments and optimizations are made.

[0009] As a further solution of the present invention, the gripper adopts an active compliant control system; the mechanical models such as the physical properties, joints, and driving mechanisms of the gripper are transformed into transfer functions, and MATLAB is used to simulate the above transfer functions to help verify the accuracy and practicability of the model, as well as predict the performance under different control strategies; Sliding mode control is an effective nonlinear control method that can handle system uncertainties and disturbances, and the control parameters are determined by the empirical verification method; Impedance control can make the gripper have better interaction ability and environmental adaptability; The Ziegler-Nichols method is applied for automatic parameter tuning; Intelligent control technology is used to obtain data through simulation and train the control model; All models are evaluated, and according to the simulation and evaluation results, the control strategy with the best performance is selected; The selected control strategy is implemented on a single-chip microcomputer for actual application testing.

[0010] As a further solution of the present invention, the visualization of the gripper camera uses hardware devices such as a high-definition camera, an image sensor, a video encoder, and a data transmission module, as well as corresponding software platforms and algorithms for image processing and analysis; The camera is connected and integrated with a computer or a mobile device to achieve data sharing and interaction; A variety of sensing devices, such as camera visual tactile sensors, are used to collect multi-modal sensing data in the environment, including information such as the shape, texture, and position of the object; The matplotlib library is used to achieve the visual presentation of the data, and the original image, the processed image, and the detected target boxes and labels are displayed.

[0011] As a further solution of the present invention, the multi-modal force sensing method specifically relates to a visual tactile sensor based on image processing, which calculates contact force information by acquiring real-time image data of the contact area. This process starts from image acquisition, captures the real-time image of the contact area between the rolling ball and the fruit through a built-in camera. First, preprocess the original image, including noise filtering and brightness equalization, to improve the robustness of subsequent processing. Subsequently, use a pre-stored reference image library and coding markers (such as laser-etched black dot matrices) for pixel-level alignment to eliminate the image displacement error caused by the rotation of the rolling ball. After completing the coding matching, unfold the curved tactile surface image into a planar rectangular image through specular reflection, and combine the camera calibration parameters and the external parameter matrix to map the two-dimensional pixel coordinates to the three-dimensional sensor coordinate system to establish a surface geometry model. Next, use photometric stereo technology to analyze the image differences under red, green, and blue LED light sources, calculate the surface normal vector field of the contact area, convert the normal vector field into a depth image through the Poisson integral algorithm, quantify the deformation amount of the elastomer surface, and based on the known undeformed surface geometry, superimpose the deformation information of the depth map to reconstruct the three-dimensional contour of the contacted object. In the marker tracking stage, detect the displacement change of the black marker, and combine the stochastic optimization algorithm to match the marker positions of the reference image and the current image to generate a displacement field. This algorithm solves the problems of marker loss or noise interference through a probability model and neighborhood smoothing constraints. Finally, statistically analyze the mean and direction of the displacement field to calculate the shear force (Fx, Fy) and the normal force (Fz), and form a complete force vector output.

[0012] As a further solution of the present invention, the multi-modal force sensing method involves closed-loop control technology, which integrates the above force vector with the control system to achieve high-precision operation. This closed-loop control process takes the force vector (Fx, Fy, Fz) and the contact position output by the tactile sensor as inputs, and the error detection module analyzes the difference between the target value and the actual value in real time: if the contact position deviates from the preset center, the low-level kinematic controller is triggered; if the shear force exceeds the safety threshold, the high-level task controller is activated. The low-level controller maps the target object speed to the roller angular velocity and the bracket angle based on the rolling ball geometry model and the fruit curvature, and the high-level controller dynamically adjusts the strategy according to the task requirements. The actuator drives the motor according to the control instruction to adjust the motion states of the rolling ball and the bracket, thereby changing the object pose. The updated object state is fed back in real time through the tactile sensor to form a closed-loop control.

[0013] A multi-modal tactile sensing method for an active non-destructive grasping gripper, which specifically includes the following steps:

[0014] S1. Image acquisition and preprocessing: Capture the real-time image of the contact area between the rolling ball (1) and the object through the camera device (11), perform noise filtering and brightness equalization processing, and subtract the non-contact reference image to extract the displacement characteristics of the marker points;

[0015] S2. Three - dimensional force calculation:

[0016] S21. Analyze the image differences under red, green, and blue LED light sources using photogrammetric stereology technology, calculate the surface normal vector field of the contact area, generate a depth image through Poisson integration, and quantify the deformation of the sensing layer thin film (8);

[0017] S22. Track the displacement field of the marked points through the optical flow method, combine the stochastic optimization algorithm to match the reference image and the real - time image, and calculate the normal force (Fz) and shear forces (Fx, Fy);

[0018] S23. Integrate the pressure distribution on the contact surface by the pressure sensor array, and extract the total contact force and moment vectors (Mx, My, Mz);

[0019] S3. Inertial compensation and data fusion: Compensate the inertial force error caused by the gripper movement in the measurement unit data, and generate the three - dimensional force vectors (Fx, Fy, Fz) and moment vectors (Mx, My, Mz) according to the confidence of each modal data through the weighted fusion algorithm;

[0020] S4. Closed - loop control: Based on the force vectors and the contact position, trigger the hierarchical control strategy through the error detection module: The low - level kinematic controller adjusts the angular velocity of the rolling ball and the opening and closing angles of the bracket according to the rolling ball geometric model; The high - level task controller dynamically corrects the grasping force threshold, and the actuator drives the stepper motor (3) and the servo (6) to adjust the gripper posture, and the tactile sensor provides real - time feedback to form a closed - loop.

[0021] During the image pre - processing process, use the feature matching algorithm (SIFT / SURF / ORB) to perform pixel - level alignment on the marked points to eliminate the image displacement error caused by the rotation of the rolling ball; Unfold the curved tactile surface into a planar image through the specular reflection model, and establish a three - dimensional sensor coordinate system in combination with the camera calibration parameters.

[0022] The weighted fusion algorithm realizes the real - time smooth output of the force vector through sliding window filtering; In the hierarchical control strategy, when the shear force exceeds the safety threshold, the high - level controller activates the anti - slip mode or adjusts the grasping force threshold.

[0023] Advantages of the present invention:

[0024] The rolling of the rolling ball is driven by the stepper motor behind the bracket, and the opening and closing of the three brackets are driven by the servo under the bracket, which can better lock and clamp the cone, and reduce the damage to the cone. The external drive design enables the size of the rolling ball to be adjustable to adapt to various fruits with different diameters;

[0025] The sensing layer film wrapped around the rolling ball, combined with the camera inside the rolling ball, realizes force perception. It allows the gripper to accurately determine the position of the fruit by sensing the contact force with the fruit, avoiding grasping mistakes caused by inaccurate position judgment. At the same time, it can accurately adjust the grasping force according to the characteristics of the fruit such as size and hardness, ensuring the grasping quality. And with the help of force perception, the gripper can accurately judge whether it has grasped the fruit, reducing empty grasping operations, optimizing the grasping process, saving time and energy, and improving the operation efficiency;

[0026] Significantly improve the robustness of force perception through multimodal data complementarity. The pressure sensor quantifies local stress concentration, the IMU suppresses motion drift, and the adaptive calibration mechanism ensures long-term reliability, realizing accurate dynamic tracking of contact force in complex environments;

[0027] The gripper adopts an active compliant control system, which allows the gripper to adjust its posture and position in a timely manner according to the contact situation. For example, when encountering branch obstacles, it can actively bend or avoid, smoothly reach the position of the fruit for picking. And through active compliant control, the gripper can automatically adjust the grasping method according to the shape and size of the fruit, avoiding damage to the fruit skin and internal tissues caused by excessive force, ensuring a firm grasp of the fruit, and improving the grasping success rate and non-damage rate. Brief Description of the Drawings

[0028] The present invention will be further described below with reference to the accompanying drawings.

[0029] Figure 1 It is a schematic diagram of the overall structure of an active non-destructive grasping gripper and its multimodal tactile perception method of the present invention;

[0030] Figure 2 It is a cross-sectional view of the tip of the finger clip of an active non-destructive grasping gripper and its multimodal tactile perception method of the present invention:

[0031] Figure 3 It is a processing flow chart of image to force of an active non-destructive grasping gripper and its multimodal tactile perception method of the present invention;

[0032] Figure 4 It is a force-control closed-loop flow chart of an active non-destructive grasping gripper and its multimodal tactile perception method of the present invention.

[0033] In the figure: 1. Rolling ball; 2. Bracket; 3. Stepper motor; 4. Pulley; 5. Servo bracket; 6. Servo; 7. Base; 8. Film; 9. Rolling ball skeleton; 10. Rolling bearing; 11. Camera device. Detailed Embodiments

[0034] Next, in combination with the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0035] Embodiment 1:

[0036] As Figure 1 - Figure 2 shown, an active non-destructive grasping gripper and its multi-modal tactile perception method include a bracket 2. A rolling ball 1 is installed at the upper end of the bracket 2. The rolling ball 1 is composed of a rolling ball skeleton 9 formed by six curved rods and an induction layer film 8. The side end of the bracket 2 supports the rolling ball skeleton 9 and the transmission of a pulley 4 through a rolling bearing 10. The pulley 4 is driven by a stepping motor 3 on the back of the bracket 2, so as to drive the rolling ball skeleton 9 to rotate around a fixed axis. A camera device 11 is installed on the other side of the bracket 2. The camera device 11 extends into the rolling ball to capture the image returned from the induction layer. The lower end of the bracket 2 is fixedly installed on a servo bracket 5. The servo bracket 5 is connected to a servo 6. The combination of the bracket 2, the servo bracket 5 and the servo 6 has three in total, and is evenly and fixedly installed on the base through the servo 6.

[0037] As Figure 3 shown, an active non-destructive grasping gripper and its multi-modal tactile perception method include visual tactile force calculation: The camera device 11 captures the real-time image of the contact area of the rolling ball 1, and preprocesses the collected multi-modal information through OpenCV tools, including video decoding, image enhancement, and other operations; Based on the pre-stored reference image library, a feature matching algorithm is used to perform pixel-level alignment of the marked points to eliminate the image displacement caused by the rotation of the rolling ball; The mirror reflection model is used to unfold the curved tactile surface into a planar image, and combined with the camera calibration parameters, a three-dimensional sensor coordinate system is established; Through the difference of multi-spectral images under red, green, and blue LED light sources, a photometric stereo technique is used to calculate the surface normal vector field, and a depth image is generated based on the Poisson integral algorithm to quantify the elastic body deformation; At the same time, the displacement field of the marked points is tracked by the optical flow method, and the marked positions of the reference image and the real-time image are matched by a random optimization algorithm, and the displacement mean value and direction are statistically calculated. Finally, the normal force (Fz) and shear forces (Fx, Fy) are output.

[0038] As Figure 4As shown in the figure, an active non-destructive grasping gripper and its multi-modal tactile perception method include multi-modal data fusion and closed-loop control: the pressure sensor array collects the pressure distribution data of each node on the contact surface in real time, and after normalization, the total contact force and torque are calculated by integration; the measurement unit module synchronously collects the acceleration and angular velocity of the gripper, eliminates mechanical vibration noise, and compensates for the influence of inertial force on force perception; after the three-modal data (vision-tactile, pressure, measurement unit) are aligned by time stamps, they are input into the weighted fusion module, and a three-dimensional force vector (Fx, Fy, Fz) and torque vector (Mx, My, Mz) are generated according to the confidence weights; if the shear force exceeds the threshold or the pressure distribution is abnormal, a hierarchical control strategy is triggered: the low-level kinematic controller adjusts the roller angular velocity and bracket angle according to the rolling ball geometric model, and the high-level task controller dynamically corrects the grasping force threshold; the actuator drives the stepper motor 3 and the servo 6 to adjust the rolling ball movement and bracket opening and closing, and the tactile sensor real-time feedback updates the state to form a closed-loop control.

[0039] Embodiment 2

[0040] A multi-modal tactile perception method for an active non-destructive grasping gripper, the method specifically includes the following steps:

[0041] S1. Image acquisition and preprocessing: Capture the real-time image of the contact area between the rolling ball (1) and the object through the camera device (11), perform noise filtering, brightness equalization processing, and subtract the non-contact reference image to extract the displacement characteristics of the marked points;

[0042] S2. Three-dimensional force calculation:

[0043] S21. Analyze the image differences under the red, green, and blue LED light sources by using photogrammetric stereoscopy technology, calculate the surface normal vector field of the contact area, generate a depth image through Poisson integration, and quantify the deformation of the sensing layer film (8);

[0044] S22. Track the displacement field of the marked points by using the optical flow method, match the reference image and the real-time image by combining the stochastic optimization algorithm, and calculate the normal force (Fz) and shear forces (Fx, Fy);

[0045] S23. Integrate the pressure distribution on the contact surface by the pressure sensor array, and extract the total contact force and torque vector (Mx, My, Mz);

[0046] S3. Inertial compensation and data fusion: The measurement unit data compensates for the inertial force error caused by the movement of the gripper, and through the weighted fusion algorithm, a three-dimensional force vector (Fx, Fy, Fz) and torque vector (Mx, My, Mz) are generated according to the confidence of each modal data;

[0047] S4. Closed-loop control: Based on the force vector and the contact position, the hierarchical control strategy is triggered by the error detection module: the lower-level kinematic controller adjusts the angular velocity of the rolling ball and the opening / closing angle of the bracket according to the geometric model of the rolling ball; the higher-level task controller dynamically corrects the grasping force threshold, and the actuator drives the stepper motor (3) and the servo motor (6) to adjust the hand gesture, and the tactile sensor feeds back in real time to form a closed loop.

[0048] During the image preprocessing process, the feature matching algorithm (SIFT / SURF / ORB) is used to perform pixel-level alignment of the marker points to eliminate the image displacement error caused by the rotation of the rolling ball; the curved tactile surface is unfolded into a planar image through the specular reflection model, and the three-dimensional sensor coordinate system is established in combination with the camera calibration parameters.

[0049] The weighted fusion algorithm realizes the real-time smooth output of the force vector through sliding window filtering; in the hierarchical control strategy, when the shear force exceeds the safety threshold, the higher-level controller activates the anti-slip mode or adjusts the grasping force threshold.

[0050] Embodiment III

[0051] Before implementation, place this hand on the robotic arm, turn on the hand system, self-check the stepper motor 3, servo motor 6 and sensors, and initialize the control software. Use the vision sensor to obtain the position, shape and attitude information of the cone, plan the grasping path, and guide the hand to approach the target cone. The robotic arm drives the hand to approach the fruit in a low-force mode, and the rolling ball 1 pre-rotates to avoid sliding friction. When the hand approaches the cone at a certain distance, switch to the force sensing mode, and the servo motor drives the fingers to open to an appropriate angle to grasp the cone. At the moment when the rolling ball of the hand contacts the cone, the camera 11 captures the marked displacement image, and the force sensor starts to collect force data, including contact force, friction force, torque and other information, and calculates and feeds back the data through the control algorithm, fuses the multi-modal data to generate a three-dimensional force vector. If the detected shear force exceeds the limit, adjust the rotational speed of the rolling ball and the opening / closing angle of the bracket. During the grasping and moving process, the hand continuously adjusts the grasping force within the safety threshold according to the force feedback until the fruit is released after being moved to the target position. For example, when encountering external interference, the hand automatically adjusts the grasping force to ensure that the cone does not fall off. According to the target placement position information, the hand quickly and stably moves to the designated location, slowly reduces the grasping force, and releases the cone to the designated position to complete the grasping task. After release, the hand returns to the initial position to prepare for the next grasping task. At the same time, record and analyze the data during the grasping process to provide a basis for subsequent optimization.

[0052] In the description of this specification, the descriptions referring to terms such as "one embodiment", "example", "specific example", etc. mean that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in a suitable manner in any one or more embodiments or examples.

[0053] The above content is only an illustration and explanation of the present invention. Those skilled in the art to which this technology belongs can make various modifications or supplements to the described specific embodiments or use similar methods for substitution, as long as they do not deviate from the invention or exceed the scope defined by this claim book, they should all fall within the protection scope of the present invention.

Claims

1. An active non-destructive grasping gripper, characterized in that, The invention comprises a base (7) and three finger clamping units, each of which is evenly installed on the base (7). Each finger clamping unit comprises a rolling ball module, a driving module and a sensing module. The rolling ball module is composed of a rolling ball skeleton (9) composed of six curved rods and a sensing layer film (8). The inner side of the sensing layer film (8) is evenly embedded with a micro piezoresistive sensor to form a high-density pressure detection grid. Each sensor node is connected to a signal conditioning circuit through a flexible wire. The driving module comprises an external stepping motor (3) on the back of a bracket (2), which drives the rolling ball skeleton (9) to rotate on a fixed axis through a pulley (4). The lower end of the bracket (2) is connected to a steering gear (6) through a steering gear bracket (5), and the steering gear (6) drives the bracket (2) to open and close. The sensing module comprises a built-in camera device (11) on the other side of the bracket (2). The camera device (11) extends into the rolling ball skeleton (9) and is used to capture the mark displacement image of the sensing layer film (8). The steering gear bracket (5) is integrated with a measuring unit to collect acceleration, angular velocity and posture data of the gripper in real time.

2. The active non-destructive grasping claw according to claim 1, wherein The rolling ball skeleton (9) is connected to the bracket (2) via a rolling bearing (10), so as to realize the translation and rotation of the rolling ball (1) in the horizontal and vertical directions; the external design of the stepping motor (3) enables the size of the rolling ball (1) to be flexibly adjusted according to the target object, so as to adapt to grasping objects of different diameters.

3. The active non-destructive grasping gripper according to claim 2, wherein The outer side of the sensing layer film (8) is a spherical curved surface, and a raised marking point is arranged on the inner side. The camera device (11) uses a high-resolution camera, combined with uniform light source illumination, to ensure the stability and contrast of the contact area image.

4. The multimodal tactile perception method according to claim 1, characterized in that: The three finger clamping units are evenly fixed to the base (7) via a steering gear (6) to form a symmetrical opening and closing structure. The rotation of the rolling ball (1) of a single finger clamping unit and the opening and closing of the bracket (2) are independently controlled to achieve dynamic adjustment of the grasping posture.

5. The multimodal tactile perception method according to claim 1, wherein: The sensing layer thin film (8) is made of an elastic material, and its deformation is linearly positively correlated with the contact force. The detection accuracy of the micro piezoresistive sensor reaches 0.1 N, and the spatial resolution of the pressure detection grid is not less than 1 mm 2 .

6. A multi-modal tactile perception method for the active non-destructive grasping gripper according to any one of claims 1-5, characterized in that, The method specifically comprises the following steps: S1, image acquisition and preprocessing: using a camera device (11) to capture a real-time image of the contact area between the rolling ball (1) and the object, performing noise filtering and brightness equalization processing, and performing subtraction with a non-contact reference image to extract the displacement characteristics of the marking point; S2. Three-dimensional force calculation: S21, using photometric stereo technology to analyze the image differences under the red, green and blue LED light sources, calculating the surface normal vector field of the contact area, generating a depth image through Poisson integral, and quantifying the deformation of the sensing layer film (8); S22, track the displacement field of the marker points by optical flow method, match the reference image with the real-time image by combining the random optimization algorithm, and calculate the normal force (Fz) and shear force (Fx, Fy); S23, the pressure sensor array integrates the pressure distribution on the contact surface to extract the total contact force and moment vector (Mx, My, Mz); S3, inertia compensation and data fusion: the measurement unit data compensates the inertia force error caused by the gripper movement, and generates three-dimensional force vectors (Fx, Fy, Fz) and torque vectors (Mx, My, Mz) according to the confidence of each modal data through the weighted fusion algorithm; S4. Closed-loop control: Based on the force vector and the contact position, the hierarchical control strategy is triggered by the error detection module: the low-level kinematic controller adjusts the angular velocity of the rolling ball and the opening / closing angle of the bracket according to the rolling ball geometric model; the high-level task controller dynamically corrects the grasping force threshold, and the actuator drives the stepper motor (3) and the servo (6) to adjust the hand claw posture, and the tactile sensor feeds back in real time to form a closed loop.

7. The multimodal tactile perception method according to claim 6, wherein: During the image preprocessing process, the feature matching algorithm (SIFT / SURF / ORB) is used to perform pixel-level alignment of the marked points to eliminate the image displacement error caused by the rotation of the rolling ball; the curved tactile surface is unfolded into a planar image through the specular reflection model, and the three-dimensional sensor coordinate system is established in combination with the camera calibration parameters.

8. The multimodal tactile perception method according to claim 6, characterized in that: The weighted fusion algorithm realizes the real-time smooth output of the force vector through sliding window filtering; in the hierarchical control strategy, when the shear force exceeds the safety threshold, the high-level controller activates the anti-slip mode or adjusts the grasping force threshold.

Images