Robot jaw control method and related system and storage medium and program product

Abstract

The invention particularly relates to the technical field related to robots, and provides a robot clamping jaw control method, a related system, a storage medium and a program product. The method comprises the following steps: acquiring image information of a target object through an image acquisition unit deployed on a clamping jaw body; obtaining respective motor current information of the at least two motor drivers through the at least two motor drivers; a first contact force value is calculated based on visual deformation information of the target object through a digital processing unit, a second contact force value is calculated based on motor current information, then the first contact force value and the second contact force value are fused based on a preset fusion algorithm, and therefore a final estimated contact force value is obtained. The final estimated contact force value is used for clamping force control associated with the target object. In this way, multi-source and multi-mode sensing information is effectively fused to obtain more accurate and more reliable contact force estimation.

Description

Technical Field

[0001] This application relates to the field of robotics technology, specifically to the field of robot end effector technology, and particularly to robot gripper control methods and related systems, storage media, and program products. Background Technology

[0002] As a key component for robots to perform complex tasks, the level of intelligence in robotic grippers directly affects the overall performance of the robot. Existing control schemes for robotic grippers still have many problems in terms of visual perception, force control, adaptive grasping, intuitiveness of remote operation, and the accuracy of force perception. For example, schemes that separate vision and force control have low integration, resulting in large system response delays and difficulty in meeting the demands of high-speed, high-precision operations. Furthermore, static force control schemes lack adaptability, relying on preset fixed force thresholds for control and lacking the ability to dynamically adjust the grasping strategy based on the object's type, material, shape, and other characteristics. This makes them difficult to adapt to diverse workpieces and complex working environments, and often requires manual intervention when dealing with unknown or fragile objects. Finally, schemes using direct force sensors have limitations. High-precision multi-dimensional force sensors can provide accurate force feedback, but they are typically expensive, structurally fragile, and bulky, and their reliability may decrease under certain extreme environments (such as strong electromagnetic interference and high temperatures). For example, indirect force estimation methods suffer from shortcomings in accuracy and robustness. While these methods estimate contact forces indirectly through motor current, encoder data, or visual information, single-modal indirect force estimation often has limited accuracy. For instance, estimations based on motor current are susceptible to interference from friction, inertia, and the nonlinear characteristics of the motor, while vision-based estimations (such as object deformation analysis) may be affected by changes in lighting, occlusion, and the optical properties of the object itself, resulting in poor robustness under complex dynamic conditions. Furthermore, remote force manipulation methods often lack perception or are not intuitive. In scenarios requiring remote operation of robot grippers (such as hazardous environments or telemedicine), operators often struggle to accurately perceive the interaction forces between the gripper and the environment. Moreover, remote operating systems may lack fine force feedback, or the feedback may be untimely or unintuitive, leading to operator misjudgments and potential damage to the manipulated object or the gripper itself. Finally, virtual reality interaction methods are insufficient in terms of precise force adjustment design, focusing on position control or simple on / off operations, lacking an effective mechanism for finely and intuitively setting and adjusting the gripper's target clamping force. Summary of the Invention

[0003] To this end, this application provides a robot gripper control method and related systems, storage media and program products. By deeply integrating visual perception and dynamic force control, it has the ability to accurately estimate contact force based on multimodal information, and has rich expansion capabilities, such as combining virtual reality technology to realize intuitive and accurate remote force control operation, so as to meet the needs of increasingly complex industrial and service scenarios.

[0004] In a first aspect, this application provides a control method for a robot gripper. The control method includes: acquiring image information of a target object via an image acquisition unit deployed on the gripper body, wherein the gripper body includes at least two gripping members and at least two motor drivers corresponding to each of the at least two gripping members, the at least two motor drivers being used to drive the at least two gripping members to move relative to each other in order to grip the target object, and the image information of the target object including at least visual deformation information of the target object when it is gripped by the at least two gripping members; acquiring motor current information of each of the at least two motor drivers via the at least two motor drivers; calculating a first contact force value based on the visual deformation information of the target object via a digital processing unit, and calculating a second contact force value based on the motor current information; then, fusing the first contact force value and the second contact force value based on a preset fusion algorithm to obtain a final estimated contact force value, the final estimated contact force value being used for gripping force control associated with the target object.

[0005] The first aspect of this application realizes an intelligent gripper system and control method that integrates visual recognition, dynamic force control, multimodal force estimation and virtual reality remote control force feedback. It is particularly suitable for industrial automation, medical assistance, service robots and other scenarios that require adaptive gripping of diverse workpieces, precision force control operations or remote operation in complex environments. It also realizes the effective fusion of multi-source and multimodal sensor information to obtain more accurate and reliable contact force estimation.

[0006] In one possible implementation of the first aspect of this application, elastic elements are respectively deployed on one or more of the at least two clamping members, and when the target object is clamped by the at least two clamping members, the elastic elements of each of the one or more clamping members deform due to contact with the target object. The control method further includes: acquiring the deformation of the elastic elements of each of the one or more clamping members through the image acquisition unit, thereby acquiring elastic element deformation information associated with the target object, and the elastic element deformation information associated with the target object is used to calibrate the calculation of the first contact force value.

[0007] In one possible implementation of the first aspect of this application, the digital processing unit calculates the first contact force value using an artificial intelligence model, wherein the learning and optimization of the artificial intelligence model is based on the visual deformation information of the clamped object acquired by the image acquisition unit and the deformation information of the elastic element associated with the clamped object.

[0008] In one possible implementation of the first aspect of this application, the elastic element of each of the one or more clamping members includes patterns or markings for measuring the deformation of the elastic element of each of the one or more clamping members.

[0009] In one possible implementation of the first aspect of this application, the image information of the target object further includes overall image data of the target object, and the control method further includes: using the digital processing unit, based on the overall image data of the target object, to determine the object category of the target object, as well as the position and orientation of the target object using an object recognition algorithm; using the digital processing unit, based on the object category of the target object, to determine force control parameters of the target object for clamping force control associated with the target object by accessing a material property database, wherein the material property database includes load characteristics and force control parameters corresponding to different object categories based on prior knowledge; using the digital processing unit, based on the object category of the target object and the position and orientation of the target object, to determine initial gripping parameters of the target object by accessing the material property database, wherein the initial gripping parameters of the target object include initial contact force and a safety force threshold.

[0010] In one possible implementation of the first aspect of this application, the control method further includes: searching historical data for the object category, position, and orientation of objects that have been gripped by the robot gripper; when it is detected that the object category of the gripped object in the historical data is the same as the object category of the target object and the initial gripping parameters of the gripped object are similar to the initial gripping parameters of the target object, the force control parameters of the gripped object are used as the force control parameters of the target object.

[0011] In one possible implementation of the first aspect of this application, multidimensional sensors are respectively deployed on the at least two clamping members, the multidimensional sensors of the at least two clamping members being used to provide direct force data, and the control method further includes: fusing the final estimated contact force value and the direct force data to update the final estimated contact force value.

[0012] In one possible implementation of the first aspect of this application, when the object category of the target object is a vulnerable object type, the lower of the final estimated contact force value and the direct force data is used to update the final estimated contact force value; or, when the object category of the target object is a non-vulnerable object type, the weighted average of the final estimated contact force value and the direct force data or the higher of the final estimated contact force value and the direct force data is used to update the final estimated contact force value.

[0013] In one possible implementation of the first aspect of this application, the multidimensional sensors of each of the at least two grippers include a position sensor for detecting the actual position of the robot gripper and a force sensor for detecting the magnitude of the gripping force.

[0014] In one possible implementation of the first aspect of this application, the digital processing unit includes a visual force estimation submodule, a motor current force estimation submodule, and a data fusion submodule, wherein the visual force estimation submodule is used to calculate the first contact force value based on the visual deformation information of the target object, the motor current force estimation submodule is used to calculate the second contact force value based on the motor current information, and the data fusion submodule is used to fuse the first contact force value and the second contact force value to obtain the final estimated contact force value.

[0015] In one possible implementation of the first aspect of this application, the final estimated contact force value is used for clamping force control associated with the target object, comprising: controlling the at least two motor drivers through a closed-loop force control algorithm based on the final estimated contact force value, thereby driving the at least two clamping members to move relative to each other in order to apply a clamping force to the target object.

[0016] In one possible implementation of the first aspect of this application, the gripper body has the shape of a human hand, and the image acquisition unit is located at the web of the human hand so as to capture the inner side of the gripper body.

[0017] In one possible implementation of the first aspect of this application, the preset fusion algorithm is a dynamic weighted average algorithm, a Kalman filter algorithm, or a machine learning-based fusion model.

[0018] In one possible implementation of the first aspect of this application, the gripper body includes a parallel two-finger gripping structure, a multi-finger gripping structure, or a gripping structure of a specific shape, wherein the multi-finger gripping structure includes a three-finger or five-finger gripping structure.

[0019] In one possible implementation of the first aspect of this application, the gripper body includes a force control actuator, the at least two motor drivers are part of the force control actuator, the force control actuator includes a transmission mechanism composed of multiple links, one or more of the multiple links have a slip detection unit embedded inside, the slip detection unit is used to detect high-frequency vibration of the contact surface caused by the slight slip of the gripped object to determine whether slip has occurred, and when the slip detection unit detects that slip has occurred, the digital processing unit executes a gradient force compensation algorithm to suppress slip.

[0020] In one possible implementation of the first aspect of this application, the visual deformation information of the target object is obtained through a deformation feature extraction algorithm. The deformation feature extraction algorithm includes obtaining the object contour of the target object through image segmentation, and then extracting the quantified object deformation features of the target object through contour analysis, template matching or key point tracking. The quantified object deformation features include the amount of compression in the direction of force, the rate of change of area of ​​a specific region or the relative displacement of a preset marker point.

[0021] In one possible implementation of the first aspect of this application, the preset fusion algorithm includes a dynamic weighted average between the first contact force value and the second contact force value, wherein the weighting parameter of the first contact force value has a high confidence level when visual conditions are good, and the weighting parameter of the second contact force value has a high confidence level when the motor is running smoothly.

[0022] Secondly, embodiments of this application also provide a computer device, the computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a method according to any of the above-mentioned implementations.

[0023] Thirdly, embodiments of this application also provide a computer-readable storage medium storing computer instructions that, when executed on a computer device, cause the computer device to perform a method according to any of the above-described implementations.

[0024] Fourthly, embodiments of this application also provide a computer program product, the computer program product including instructions stored on a computer-readable storage medium, which, when executed on a computer device, cause the computer device to perform a method according to any of the above-described aspects.

[0025] Fifthly, this application provides a control system for a robot gripper. The control system includes: an image acquisition unit deployed on the gripper body for acquiring image information of a target object, wherein the gripper body includes at least two gripping members and at least two motor drivers corresponding to each of the at least two gripping members, the at least two motor drivers being used to drive the at least two gripping members to move relative to each other in order to grip the target object, and the image information of the target object including at least visual deformation information of the target object when it is gripped by the at least two gripping members; the at least two motor drivers being used to acquire motor current information of each of the at least two motor drivers; and a digital processing unit for calculating a first contact force value based on the visual deformation information of the target object, and calculating a second contact force value based on the motor current information, and then fusing the first contact force value and the second contact force value based on a preset fusion algorithm to obtain a final estimated contact force value, the final estimated contact force value being used for gripping force control associated with the target object.

[0026] The fifth aspect of this application realizes an intelligent gripper system and control method that integrates visual recognition, dynamic force control, multimodal force estimation and virtual reality remote control force feedback. It is particularly suitable for industrial automation, medical assistance, service robots and other scenarios that require adaptive gripping of diverse workpieces, precision force control operations or remote operation in complex environments. It also realizes the effective fusion of multi-source and multimodal sensor information to obtain more accurate and reliable contact force estimation. Attached Figure Description

[0027] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 A flowchart illustrating a control method for a robot gripper according to a first embodiment of this application; Figure 2 A flowchart illustrating a second embodiment of a control method for a robot gripper provided in this application. Figure 3 A schematic diagram of a gripper body showing the position of an image acquisition unit, provided for an embodiment of this application; Figure 4 A schematic diagram of a control system for a robot gripper provided in an embodiment of this application; Figure 5This is a schematic diagram of the structure of a computing device provided in an embodiment of this application. Detailed Implementation

[0029] The embodiments of this application will now be described in further detail with reference to the accompanying drawings.

[0030] It should be understood that in the description of this application, "at least one" means one or more, and "multiple" means two or more. In addition, the words "first," "second," etc., unless otherwise stated, are used only for the purpose of distinguishing descriptions and should not be construed as indicating or implying relative importance or order.

[0031] Figure 1 This is a flowchart illustrating a control method for a robot gripper according to a first embodiment of this application. Figure 1 As shown, the control method includes the following steps.

[0032] Step S101: Obtain image information of the target object through an image acquisition unit deployed on the gripper body. The gripper body includes at least two gripping members and at least two motor drivers corresponding to the at least two gripping members. The at least two motor drivers are used to drive the at least two gripping members to move relative to each other in order to grip the target object. The image information of the target object includes at least the visual deformation information of the target object when it is gripped by the at least two gripping members.

[0033] Step S103: Obtain the motor current information of each of the at least two motor drivers.

[0034] Step S105: The digital processing unit calculates a first contact force value based on the visual deformation information of the target object and a second contact force value based on the motor current information. Then, based on a preset fusion algorithm, the first contact force value and the second contact force value are fused to obtain a final estimated contact force value. The final estimated contact force value is used for clamping force control associated with the target object.

[0035] See Figure 1Robot grippers are key components for robots to perform complex tasks, and their intelligence level directly affects the overall performance of the robot. To precisely control the gripping force applied to objects by the robot gripper, various methods can be adopted. For example, a static force control method can be used, which controls the gripping force by setting a fixed force threshold. However, considering the complexity of the robot's task environment—objects to be gripped may have different categories, materials, and shapes—this static force control method is difficult to adapt to diverse workpieces and complex operating environments. Therefore, a dynamic force control method is needed, which adapts to the actual task execution. Dynamic force control requires timely monitoring of the actual gripping force. For this, vision-based and force-control-based monitoring methods can be used. However, this faces challenges in coordinating multimodal inputs and integrating vision and force control solutions, while high-speed and high-precision operation standards require low system response latency. Direct force sensors can be used to monitor forces, providing accurate force feedback. However, these sensors are typically deployed on the robot's end effector, which is in direct contact with the object. They are usually expensive, structurally fragile, and bulky, and their reliability may decrease in extreme environments (such as strong electromagnetic interference or high temperatures). Indirect force estimation methods can also be employed, such as estimating contact forces indirectly through motor current, encoder data, or visual information. However, indirect force estimation methods have limited accuracy and are susceptible to interference. For example, estimations based on motor current are easily affected by friction, inertia, and the nonlinear characteristics of the motor, while vision-based estimations (such as object deformation analysis) may be affected by changes in lighting, occlusion, and the optical properties of the object itself, resulting in poor robustness under complex dynamic conditions. Remote force manipulation can also be used, such as having an operator remotely control the robot gripper. However, in scenarios requiring remote operation of the robot gripper (such as hazardous environments or telemedicine), operators often struggle to accurately perceive the interaction forces between the gripper and the environment. Furthermore, remote operating systems may lack fine force feedback, or the feedback may be untimely or unintuitive, leading to operator misjudgment and potential damage to the manipulated object or the gripper itself. Virtual reality interaction can also be used, but it lacks precision in force adjustment, focusing on position control or simple on / off operation, and lacks an effective mechanism for finely and intuitively setting and adjusting the gripping force of the gripper target. It can be seen that although various solutions exist for precisely controlling the gripping force applied to an object by a robot gripper, each solution has its limitations, and conflicts may exist between multimodal inputs. These factors increase the demand for computing resources and may also cause higher processing latency. This means that a balanced design should be implemented across multiple monitoring methods to create a dynamic force control solution that optimizes the accuracy, anti-interference, and user customization of gripping force monitoring.The following detailed description, in conjunction with specific embodiments of this application, illustrates how the control method, electronic device, medium, and system for robot grippers provided in this application address the problems of low integration of vision and force control, poor adaptability, large system latency, strong dependence on direct force sensors, insufficient accuracy and robustness of indirect force estimation, and unintuitive remote operation force sensing and adjustment in existing robot grippers.

[0036] Continue reading Figure 1The control method for a robot gripper according to the first embodiment can be applied to a force-controlled gripper system based on vision recognition. The force-controlled gripper system includes a force-controlled actuator, which includes at least two relatively movable gripping members disposed on the gripper body and a drive source for driving the gripping members to move and providing feedback on motor current information. In some embodiments, the force-controlled actuator includes an optional direct force sensing unit, specifically including: a servo motor, as a power source, driving the gripper to move according to a control signal; a position sensor, such as a photoelectric encoder or magnetic encoder, detecting the actual position of the gripper in real time and feeding back the position information to the control system; a force sensor, installed at the contact point between the gripper and the workpiece, monitoring the magnitude of the gripping force to ensure precise force control; and a control system, receiving feedback signals from the position sensor and the force sensor, processing them through an algorithm, and adjusting the motor drive signal in real time to achieve dual control of position and force. An image acquisition unit, disposed on the gripper body, is used to acquire image information of the target object and monitor the visual deformation information of the target object during the gripping process. A digital processing unit, located externally and connected to the image acquisition unit and force control actuator, executes an object recognition algorithm to identify the category of the target object. It stores a material property database that records the correspondence between different object categories and one or more sets of preset force control parameters (e.g., parameters for calculating the initial contact force, parameters for calculating the maximum allowable force, recommended force control parameters, etc.). Exemplary force control parameters can be proportional, integral, or derivative parameters, also known as PID control parameters. Furthermore, the digital processing unit may include multiple modules for executing corresponding calculation tasks. The digital processing unit may include a contact force estimation module, which operates within the unit. This module includes: a visual force estimation submodule, used to estimate a first contact force value based on the visual deformation information of the target object monitored by the image acquisition unit; and a motor current force estimation submodule, used to estimate a second contact force value based on the motor current information fed back from the drive source, using a preset motor and transmission system model and performing disturbance compensation. The digital processing unit may further include a data fusion submodule for fusing the first contact force value and the second contact force value (and optionally the force value from a direct force sensor) according to a preset fusion algorithm (e.g., a dynamic weighted average algorithm, a Kalman filter algorithm, or a machine learning-based fusion model) to generate a final estimated contact force. In some embodiments, the force-controlled gripper system may further include a remote virtual reality (VR) interaction unit, including a VR display device worn by the user and a force feedback input device operated by the user, the remote VR interaction unit being bidirectionally connected to the digital processing unit via a communication link.In some embodiments, the material property database records the correspondence between different object categories and one or more sets of preset force control parameters. In some embodiments, the preset force control parameters include at least parameters for calculating the initial contact force, parameters for calculating the maximum allowable force, and / or a recommended set of PID controller parameters.

[0037] Continue reading Figure 1First, a visual force estimation submodule is used to estimate the contact force based on visual information (object deformation information), and a motor current force estimation submodule is used to estimate the contact force based on motor current information. Then, the estimated contact force values ​​are fused, optionally incorporating position information from a position sensor and clamping force values ​​from a direct force sensor, and the final estimated contact force is obtained through a data fusion submodule. Thus, based on a preset fusion algorithm, the first and second contact force values ​​are fused to obtain the final estimated contact force value. Here, the preset fusion algorithm can be a dynamic weighted average algorithm, a Kalman filter algorithm, or a machine learning-based fusion model. A dynamic weighted average algorithm means that each weighting parameter can be dynamically adjusted according to the real-time confidence level of each sensor source. For example, a higher confidence level may be set for visual information when visual conditions are good, or a higher confidence level may be set for motor current information when the motor is running smoothly. The final estimated contact force obtained in this way represents the dynamic monitoring result of the gripping force on the robot gripper, and it already takes into account the characteristics of the object's shape (deformation information). Then, using a material property database and a pre-trained model, suitable initial grasping parameters and force control parameters can be dynamically generated for different objects (PID control algorithm or other automatic control algorithm can be used). An exemplary preset fusion algorithm using a dynamic weighted average algorithm can be expressed by the following formula: Ffused = (wvis X Fvision + wcur X Fcurrent + wsen X Fsensor ) / (wvis + wcur + wsen ). Where, Ffused represents the final estimated contact force, Fvision represents the contact force estimated based on visual information with a weight coefficient of wvis, Fcurrent represents the contact force estimated based on motor current information with a weight coefficient of wcur, and Fsensor represents the position information from the position sensor and the gripping force value from the direct force sensor with a weight coefficient of wsen. Furthermore, the weight coefficients wvis, wcur, and wsen can be dynamically adjusted according to the real-time confidence level of each sensor source. For example, when visual conditions are good (sufficient lighting, no obstructions, and significant object deformation), increase wvis. When the motor runs smoothly and the model fit is high, increase wcur. When the direct force sensor is working properly, wsen can have a high baseline value. Confidence assessment can be based on signal quality, noise level, model residuals, etc. Furthermore, using a Kalman filter, the contact force is treated as the system state variable, and Fvision, Fcurrent, and Fsensor are treated as observations. State transition equations (describing the dynamic changes of force) and observation equations (describing the relationship between observations and the true force, as well as noise characteristics) are established. Thus, through the prediction and iterative update of the Kalman filter, the optimal estimate of the contact force, Ffused, is obtained, effectively handling sensor noise and uncertainty.Furthermore, based on rules or fuzzy logic, the combination of force information can be determined through a pre-defined rule base or fuzzy inference system, according to the current working conditions (such as object category, grasping stage, and slip detection signal) and the characteristics of each force estimate (such as trend and consistency). Additionally, machine learning-based fusion models can train a fusion network (such as a small multilayer perceptron) with inputs of Fvision, Fcurrent, Fsensor, and possible auxiliary features (such as object category, grasping speed, etc.), and output as Ffused.

[0038] Continue reading Figure 1 The specific hardware structure and circuit implementation of the force-controlled gripper system described above are just an example. Figure 1The control method for a robot gripper shown in the first embodiment can be implemented using the force-controlled gripper system described above or hardware with a similar structure. In some embodiments, a VR remote control operation mode can be implemented by combining a VR interaction unit. Specifically, the control method of the force-controlled gripper system based on vision recognition, after identifying the category of the target object, queries the material property database to obtain preset force control parameters corresponding to the object category, and controls the force control actuator to perform a closed-loop force control gripping action on the target object based on the preset force control parameters (and optional user instructions from the remote VR interaction unit) and the final estimated contact force (or the force value of the direct force sensing unit); the final estimated contact force (and optional gripper status information, object recognition results, etc.) is transmitted to the remote VR interaction unit to drive the force feedback input device to provide force feedback related to the final estimated contact force to the user, and the relevant information is presented on the VR display device; and a user interface is presented on the VR display device of the remote VR interaction unit, which includes innovative interactive elements for the user to intuitively set or dynamically adjust the target gripping force. In some embodiments, the system can be in an autonomous operation mode, that is, the system itself determines how to control the force control actuator to perform a closed-loop force control grasping action on the target object. In some embodiments, the VR remote control operation mode and the autonomous operation mode can be integrated together, and the user can flexibly choose to switch between modes. Both the VR remote control operation mode and the autonomous operation mode require the final estimated contact force value. Therefore, the control method that integrates the VR remote control operation mode and the autonomous operation mode includes the following specific steps: (1) the image acquisition unit acquires the image information of the target object and monitors the visual deformation information of the target object during the clamping process; (2) the digital processing unit receives the image information and executes the object recognition algorithm to identify the category of the target object; (3) the digital processing unit runs the contact force estimation module, which specifically includes estimating the first contact force value based on the visual deformation information, estimating the second contact force value based on the motor current information acquired from the force control actuator, and, according to a preset fusion algorithm, fusing the first contact force value and the second contact force value (and optional direct force sensing data) to generate the final estimated contact force. Then, when the system is in autonomous operation mode, the digital processing unit queries the material property database to obtain the corresponding preset force control parameters according to the identified target object category, and controls the force control actuator to perform a closed-loop force control grasping action on the target object based on the preset force control parameters and the final estimated contact force.When the system is in VR remote control operation mode, the final estimated contact force, the video stream of the gripper's working angle, and other relevant status information are transmitted to the remote VR interaction unit through the communication link. A user interface containing the target object, gripper status, and force adjustment interactive elements is presented on the VR display device of the remote VR interaction unit, and force feedback related to the final estimated contact force is provided to the user through the force feedback input device. The system receives target gripping force setting or adjustment commands input by the user through the force adjustment interactive elements from the remote VR interaction unit. Based on the received user commands and the final estimated contact force, the digital processing unit controls the force control actuator to perform a closed-loop force control gripping action on the target object.

[0039] Continue reading Figure 1 , Figure 1 The control method for the robot gripper shown is designed by balancing multiple monitoring methods to create a dynamic force control scheme, optimizing the accuracy, anti-interference, and user-customizable aspects of gripping force monitoring. Specifically, Figure 1The control method for robot grippers shown has the following improvements: (1) High integration and low latency: By realizing visual recognition, multimodal force estimation, data fusion and most control decisions in the digital processing unit at the gripper end, the data transmission and processing links are significantly reduced, and the latency from recognition to execution is compressed to less than 50 milliseconds, improving the real-time response capability of the system. (2) Strong adaptability and high success rate: By combining visual object classification and material attribute database, suitable initial gripping parameters (such as initial contact, safety force threshold) and PID control parameters can be dynamically generated for objects with different characteristics, and dynamic adjustments can be made by combining slip detection, so that the gripping success rate in complex scenarios is increased from 82% to 98.6%. (3) Accurate and robust force perception: By fusing force estimation based on visual object deformation and motor current (and optional direct force sensor data), the complementarity of multimodal information is utilized to overcome the limitations of a single indirect force estimation method, improve the accuracy of contact force perception and robustness in complex working conditions, and may reduce the dependence on expensive direct force sensors. (4) Intuitive remote force control and enhanced interactive experience: Combining VR technology, it provides remote operators with an immersive visual experience and real-time force feedback based on estimated force. Through an innovative VR force adjustment interface, users can perceive and control the interaction force between the gripper and the object more naturally and accurately, improving the efficiency and safety of remote operation, especially suitable for remote operation of delicate, fragile or unknown objects. (5) Cost and space optimization: The integrated design and the reduction of reliance on high-end force sensors through force estimation help reduce the overall hardware cost of the system (approximately 25%) and installation space (approximately 30%). (6) Good scalability: The system supports online learning of new object grasping strategies (e.g., updating the material database and control parameters through demonstration learning or reinforcement learning), and has good application prospects and expansion potential.

[0040] Continue reading Figure 1An image acquisition unit is deployed on the gripper body to acquire image information of the target object. This image information can be used to identify the object's category and match it with material properties and preset force control parameters in a database. It is also used to monitor visual deformation information during the gripping process. Additionally, a force control execution unit is deployed on the gripper body. This unit can monitor the drive current information corresponding to two or more relatively movable gripping components on the gripper, thus indirectly calculating the contact force, i.e., the gripping force, by monitoring the drive current information. In some embodiments, a direct force sensor can be deployed on the gripper body to directly monitor the gripping force. In some embodiments, a position sensor is also deployed on the gripper body to detect the actual position of the gripper in real time. Considering the limitations of various monitoring methods—for example, indirect calculation methods based on motor current are susceptible to interference from friction, inertia, and the nonlinear characteristics of the motor; visual deformation information obtained from visual information such as image acquisition devices may be affected by changes in lighting, occlusion, and the optical properties of the object itself; and direct force sensors are expensive, structurally fragile, and large, and their reliability may decrease under certain extreme environments (such as strong electromagnetic interference and high temperatures). To this end, a digital processing unit deployed outside the gripper body first uses a visual force estimation submodule to estimate the contact force based on visual information (object deformation information) and a motor current force estimation submodule to estimate the contact force based on motor current information. Then, the estimated contact force values ​​are fused, optionally combining position information from a position sensor and the gripping force value from a direct force sensor, to obtain the final estimated contact force through a data fusion submodule. In some embodiments, the preset fusion algorithm can employ a dynamic weighted average algorithm, where each weighting parameter can be dynamically adjusted according to the real-time confidence level of each sensor source. For example, a higher confidence level may be set for visual information when visual conditions are good, or a higher confidence level may be set for motor current information when the motor is running smoothly. The resulting final estimated contact force represents the dynamic monitoring result of the gripping force on the robot gripper, taking into account the object's shape characteristics (deformation information). Then, using a material property database and a pre-trained model, suitable initial gripping parameters and force control parameters (which can employ PID control algorithms or other automatic control algorithms) can be dynamically generated for different objects. To further enhance the reliability of contact force judgment results based on visual recognition and force feedback, a slip monitoring unit can be incorporated to monitor high-frequency vibrations of the contact surface to determine whether minute slippage of the clamped object has occurred. A virtual reality operation mode is also provided for the user (or an autonomous system operation mode if no user intervention is required). In some embodiments, the visual force estimation submodule estimates the first contact force value by analyzing changes in object contour, size, or displacement of specific marker points in the visual deformation information, and using a preset force-deformation model or a machine learning-based regression model.In some embodiments, the motor current force estimation submodule estimates the second contact force value from the motor current information by establishing a motor model of the drive source and a transmission system model of the force control actuator, and compensating for disturbances such as friction and inertia. In some embodiments, the preset fusion algorithm used by the data fusion submodule is a dynamic weighted average algorithm, a Kalman filter algorithm, a rule-based inference algorithm, or a machine learning-based fusion model.

[0041] Continue reading Figure 1 The system utilizes an image acquisition unit to obtain visual perception-based monitoring data, i.e., image information of the target object. It also uses a motor driver (or a more macroscopic force control actuator) to obtain monitoring data based on indirect force estimation, i.e., motor current information. Then, a pre-defined fusion algorithm is used to fuse multi-source, multi-modal inputs to obtain the final estimated contact force value. This achieves dynamic force control and precise force feedback, and possesses advantages such as strong anti-interference and scalability. In summary, Figure 1 The control method for a robot gripper shown realizes an intelligent gripper system and control method integrating visual recognition, dynamic force control, multimodal force estimation, and virtual reality remote control force feedback. It is particularly suitable for industrial automation, medical assistance, and service robot scenarios that require adaptive gripping of diverse workpieces, precise force control operations, or remote operation in complex environments. It also effectively integrates multi-source, multimodal sensor information to obtain more accurate and reliable contact force estimation. In some embodiments, the digital processing unit is further configured to: control the force control actuator to perform a closed-loop force control gripping action on the target object based on the identified object category, the preset force control parameters obtained from the material property database, optional user instructions from the remote VR interaction unit, and the final estimated contact force; transmit the final estimated contact force to the remote VR interaction unit to drive the force feedback input device to provide force feedback related to the final estimated contact force to the user; and present a user interface on the VR display device of the remote VR interaction unit, which includes interactive elements for the user to set or adjust the target gripping force.

[0042] See Figure 1In one possible implementation, elastic elements are respectively deployed on one or more of the at least two grippers. When the target object is gripped by the at least two grippers, the elastic elements of each of the one or more grippers deform due to contact with the target object. The control method further includes: acquiring the deformation of the elastic elements of each of the one or more grippers through the image acquisition unit, thereby acquiring elastic element deformation information associated with the target object. The elastic element deformation information associated with the target object is used to calibrate the calculation of the first contact force value. As mentioned above, by learning the deformation information of different object types, that is, by using image data to determine the upper limit of the deformation degree corresponding to a certain object type, the force of the gripper is limited. Here, by introducing the design of elastic elements, the deformation of elastic elements (e.g., rubber, sponge, foam) on the grippers (e.g., fingers of a humanoid hand) can be learned, and the deformation of the elastic elements can be captured by an image acquisition unit (e.g., a camera) on the grippers. Here, one or more grippers can be all or part of at least two grippers. The specific location of the image acquisition unit and its relative spatial relationship with one or more gripping components can be flexibly set, as long as it ensures that the image acquisition unit can accurately acquire the deformation of the elastic elements of each of the one or more gripping components. For example, assuming the gripper body is a humanoid hand with five gripping components corresponding to the five fingers of the humanoid hand, the image acquisition unit can be deployed at the web of the humanoid hand. This allows the camera to capture images of the inner side of the gripper body from the web, as well as the five contact surfaces of the object gripped by the five fingers of the humanoid hand. This allows for accurate acquisition of the deformation of the elastic elements on these five contact surfaces due to contact with the object. In some embodiments, to better observe the deformation of the elastic elements, patterns or markings are set on the elastic elements, similar to the wavy lines on a knife or a water depth gauge. This allows the image acquisition unit to accurately determine the deformation of the elastic elements, for example, by whether a specific pattern or marking is obscured. In this way, by learning and optimizing the above two types of deformation variables, object deformation variables and elastic component deformation variables together through artificial intelligence algorithms, the robot gripper control system is given reasoning ability. It can determine the object type and the upper limit of the corresponding object deformation variable and the upper limit of the elastic component deformation variable based on image data, which helps to improve the visual force estimation results.

[0043] In some embodiments, the digital processing unit calculates the first contact force value using an artificial intelligence model. The learning and optimization of this model are based on the visual deformation information of the clamped object acquired by the image acquisition unit and the deformation information of the elastic element associated with the clamped object. Thus, by incorporating the design of the elastic element and acquiring its deformation information through the image acquisition unit, the calculation of the first contact force value is calibrated, thereby improving the visual force estimation results. By learning and optimizing both types of deformation variables—object deformation variable and elastic element deformation variable—using an artificial intelligence algorithm, the robot gripper control system gains reasoning capabilities, enabling it to determine the object type and the corresponding upper limits of object deformation variable and elastic element deformation variable based on image data.

[0044] In some embodiments, the elastic element of each of the one or more clamping members includes patterns or markings for measuring the deformation of the elastic element of each of the one or more clamping members. Thus, by incorporating the design of the elastic element and acquiring the deformation information of the elastic element through an image acquisition unit, it helps to calibrate the calculation of the first contact force value, i.e., improve the visual force estimation results. By setting patterns or markings on the elastic element, similar to the wavy lines on a cutting tool or a water depth gauge, the deformation of the elastic element can be accurately determined by the image acquisition unit, for example, by whether a specific pattern or a specific marking is obscured.

[0045] Figure 2 This is a flowchart illustrating a second embodiment of a control method for a robot gripper, provided in this application. Figure 2 As shown, the flow of the control method for a robot gripper in the second embodiment includes the following steps.

[0046] Step S201: Obtain image information of the target object.

[0047] Step S210: Obtain visual deformation information of the target object.

[0048] Step S212: Calculate the first contact force value based on the visual deformation information of the target object, and calculate the second contact force value based on the motor current information.

[0049] Step S214: Based on the preset fusion algorithm, fuse the first contact force value and the second contact force value to obtain the final estimated contact force value.

[0050] Step S220: Obtain the overall image data of the target object.

[0051] Step S222: Use an object recognition algorithm to determine the object category, position, and pose of the target object.

[0052] Step S224: Based on the object category of the target object, determine the force control parameters of the target object by accessing the material property database.

[0053] Step S226: Based on the object category, position, and pose of the target object, determine the initial grasping parameters of the target object by accessing the material property database.

[0054] Step S230: Using the force control parameters of the target object and the final estimated contact force value, perform a force-controlled grasping action on the target object.

[0055] See Figure 1 and Figure 2 In one possible implementation, the image information of the target object further includes overall image data of the target object. The control method further includes: using the digital processing unit, based on the overall image data of the target object, determining the object category, position, and orientation of the target object using an object recognition algorithm; using the digital processing unit, based on the object category of the target object, determining force control parameters of the target object for clamping force control associated with the target object by accessing a material property database, wherein the material property database includes load characteristics and force control parameters corresponding to different object categories based on prior knowledge; using the digital processing unit, based on the object category, position, and orientation of the target object, determining initial gripping parameters of the target object by accessing the material property database, wherein the initial gripping parameters of the target object include initial contact force and a safety force threshold. Here, the image acquisition unit can be, for example, an embedded industrial camera integrated into the central axis of the gripper or the end of the gripper finger, such as a CMOS camera with a global shutter and high dynamic range. The image acquisition unit is used not only to capture overall image information of the target object for identification and positioning, but also to monitor the visual deformation information of the contact area between the target object and the gripper in real time during the gripping process. The digital processing unit is the core control and computing unit, and its physical implementation can be a highly integrated circuit board, such as integrating a high-performance microcontroller and / or a dedicated deep learning acceleration chip on the camera motherboard. (Reference) Figure 2The control method for a robot gripper shown in the second embodiment involves a digital processing unit performing a series of processing and calculations based on the image information of the target object, specifically the visual deformation information of the target object and the overall image data of the target object: (1) running an object recognition algorithm (e.g., lightweight YOLOv5, MobileNet-SSD, etc.) to analyze the image acquired by the image acquisition unit in real time and output information such as object category, position, and posture; (2) accessing the storage material attribute database, object recognition model, force estimation model, control algorithm parameters, etc.; (3) running the algorithms of each sub-module of the contact force estimation module; (4) executing a closed-loop force control algorithm (e.g., PID control) and dynamically adjusting the control parameters according to the object category, estimated force, slip signal, etc. Optionally, when the system is in VR control mode, the digital processing unit also handles communication with the remote VR interaction unit, including sending video streams, estimated forces, status information, and receiving user control commands. The digital processing unit (DMU) is responsible for the efficient coordination of camera data and force control execution commands (and force estimation) at the embedded level. If a hardware-level shared clock, direct memory access, or high-speed serial interface (such as MIPI-CSI / DSI directly connected to the processing core) exists, it can be considered a form of "hardware-level synchronization," thereby minimizing internal data transmission latency. Additionally, the material property database's core function is to provide prior grasping knowledge for different types of objects. The database can be a structured table recording various information, such as "Object_Class_ID, Object_Name, Estimated_Mass_Range (g), Brittleness_Coefficient (0-1), Surface_Friction_Est (0-1), k1_coeff (for F0), k2_coeff (for Fmax), Recommended_PID_Profile_ID, Vision_Force_Model_ID, Current_Force_Model_Params_ID, Max_Grasp_Speed ​​(mm / s)." Exemplary objects could be glass test tubes, metal blocks, etc. Database entries can be populated and updated through experimental calibration, simulation analysis, or machine learning methods. When encountering a new object that does not precisely match the database, a "fuzzy matching strategy" can be employed. For example, based on the similarity of the new object to known categories in the database in terms of visual features (e.g., extracted by models such as YOLOv5), the closest set of grasping parameters can be interpolated or selected. Furthermore, force control parameters and initial grasping parameters are used to control the force control actuator (or force control unit). The force control actuator includes a clamping component, a drive source (such as a servo motor, whose driver can provide real-time feedback on motor current, speed, position, etc.), and a transmission mechanism (such as a lead screw or connecting rod).Optionally, a direct force sensing unit (such as a strain gauge force sensor or a piezoresistive force sensor) can be installed at the end of the gripper finger or at a critical force point. Its readings can be used as additional input to the force estimation module or for calibrating / verifying the estimation results.

[0056] See Figure 2 The second implementation of the control method for a robot gripper, based on visual deformation information of the target object and overall image data of the target object, overcomes the limitations of a single indirect force estimation method by fusing force estimation based on visual object deformation and motor current. This leverages the complementarity of multimodal information, improving the accuracy of contact force sensing and robustness under complex working conditions, while potentially reducing reliance on expensive direct force sensors. By combining visual object classification and material property databases, suitable initial gripping parameters (such as initial contact and safety force threshold) and force control parameters can be dynamically generated for objects with different characteristics. Visual recognition, multimodal force estimation, data fusion, and most control decisions are implemented in a digital processing unit at the gripper end, significantly reducing data transmission and processing steps and improving the system's real-time response capability. (Reference) Figure 2The control method for a robot gripper shown in the second embodiment includes the following steps as an example of a system control method and its workflow: (1) Initialization and object recognition: After the system starts, the digital processing unit initializes each module. When the gripper approaches the target object, the image acquisition unit captures an image of the object. The digital processing unit runs object recognition algorithms such as YOLOv5 to identify the object category (e.g., "glass bottle", "metal part"), position, and orientation. (2) Parameter acquisition and force estimation preparation: Based on the identified object category, the digital processing unit queries the material property database to obtain the corresponding preset force control parameters. Exemplary preset force control parameters include coefficients for calculating the initial contact force, coefficients for calculating the object mass and safety force threshold, a recommended set of PID control parameters, and a relevant model ID or parameter set for the force estimation module. Here, the "object mass" can be estimated from the database or estimated online by combining visual volume estimation with density assumptions. (3) Contact and force estimation: The gripper begins to move toward the target object and make contact. Throughout the gripping and operation process: the image acquisition unit continuously monitors the visual deformation of the contact area between the object and the gripper. The motor driver in the force control actuator provides real-time feedback of motor current information. (4) The contact force estimation module on the digital processing unit operates in parallel: the visual force estimation submodule outputs based on deformation information. The motor current force estimation submodule outputs based on current information and compensation model. The data fusion submodule fuses the data into the final estimated contact force. Then, in autonomous mode, (5) closed-loop force control and slip compensation: in autonomous mode, the digital processing unit uses the final estimated contact force (or direct force feedback) as feedback based on the initial target force (or position control during the approach process, switching to force control after contact) and with a safety force threshold as the upper limit, and controls the servo motor through PID (or other advanced control algorithms whose parameters can be dynamically tuned from the database according to the object category) to drive the clamping part to apply a precise clamping force to the object. Optionally, slip detection and compensation can be introduced, and the piezoelectric film sensor monitors the contact vibration in real time. If slippage is detected (e.g., by analyzing the spectrum of vibration signals to extract and estimate slippage velocity), a gradient force compensation algorithm or a general PD-type compensation is triggered to dynamically increase the clamping force to suppress slippage until it disappears or a safety force threshold is reached. In VR control mode, data transmission and presentation are performed: the digital processing unit sends real-time video streams, the final estimated contact force, object recognition results, and gripper status to a remote VR interaction unit via a communication link. The VR display device can present an immersive user interface. The force feedback input device provides the user with a force sensation based on the final estimated contact force. Additionally, user interaction and command issuance are possible: users can observe real-time images and status information using VR controllers and other devices, and set or fine-tune the target clamping force using force adjustment interactive elements (such as virtual sliders) on the VR interface. Users can also issue commands such as gripping and releasing.These instructions are sent back to the digital processing unit via a communication link. Additionally, remote closed-loop force control can be performed: after receiving the target clamping force set by the user, the digital processing unit uses this as the set point and the final estimated contact force (or direct force feedback) as feedback to perform closed-loop force control.

[0057] Continue reading Figure 2 After the grasping task is completed, the system can record the key parameters of this grasping, whether it is successful or not, and the sensor data in the process. These data can be used for offline analysis, or to update the entries in the material property database or optimize the parameters of the control / estimation model through online learning algorithms (such as reinforcement learning), thereby improving the system's ability to adapt to new objects or new tasks. Taking the automotive assembly scenario application of grasping and assembling different types of parts on the automotive assembly line as an example, the following practical applications can be provided. (1) Grasping engine wiring harness (soft PVC material): The image acquisition unit identifies the wiring harness, and the digital processing unit queries the material database for the characteristics of the PVC wiring harness (such as low stiffness, easy deformation, and medium surface friction coefficient), and automatically calls a small initial contact force (e.g., 2.5N ± 0.3N) and a set of "gentle" PID parameters. In VR remote control mode, the operator can finely adjust the clamping force through the VR interface and sense the slight resistance of the wiring harness through the force feedback handle to avoid excessive clamping and damage to the wiring harness. The contact force estimation module may focus more on visual deformation analysis (because the wiring harness deformation is obvious) and motor model under low current at this time. (2) Grasping the metal casing of the Electronic Control Unit (ECU): After recognizing the metal casing, the system retrieves its high rigidity and smooth (potentially slippery) characteristics from the database, automatically enables a larger initial contact force, sets a higher safety force threshold (e.g., peak force 15N), and enables a more sensitive slip detection and compensation strategy. The VR operator can feel the rigidity feedback of the metal part when placing the ECU. At this time, force estimation may rely more on motor current (due to the small metal deformation) and optional direct force sensors. (3) Handling glass fuses: When a fragile glass fuse is detected, the system immediately switches to "ultra-soft contact mode", which may further reduce the target contact force, reduce the PID gain, and increase the force control bandwidth (e.g., to above 100Hz) to achieve faster force response and finer force control. The VR operator will feel very slight force feedback and confirm the safety of the operation through high-fidelity vision. Visual deformation (if small deformation can be observed or enhanced by special lighting) and a high-precision motor model under extremely low current are crucial for force estimation. Thus, through the integrated design of "perception-decision (local intelligence + remote VR command)-execution (precise force control)" and combined with multimodal force estimation and fusion, it is possible to efficiently, safely and adaptively complete diverse grasping and assembly tasks.

[0058] In some embodiments, the control method further includes: searching historical data for the object category, position, and orientation of objects previously gripped by the robot gripper; when it is detected that the object category of the gripped object in the historical data is the same as the object category of the target object and the initial gripping parameters of the gripped object are similar to the initial gripping parameters of the target object, the force control parameters of the gripped object are used as the force control parameters of the target object. In practical applications, sensor malfunctions may occur, such as direct force sensors being easily damaged by external forces. Therefore, historical data can be used to provide fuzzy judgment when perception is hindered. For example, if the current object category is identified as being the same as the object category in the historical data, the initial gripping parameters and force control parameters at that time can be retrieved. Here, the initial gripping parameters can be set based on the object position and gripper pose information captured from image information. Then, when the initial gripping parameters are sufficiently similar and the object categories are the same, past force control parameters can be referenced, for example, the gripping parameters and clamping forces of previously gripped objects of the same type can be used as a reference.

[0059] In some embodiments, multidimensional sensors are deployed on each of the at least two grippers, and the multidimensional sensors on each of the at least two grippers provide direct force data. The control method further includes fusing the final estimated contact force value and the direct force data to update the final estimated contact force value. Thus, fusing the final estimated contact force and the direct force feedback, combined with a force feedback mechanism based on image information and force control information, provides a basis for finely controlling the gripping force and the grasping operation of the grippers.

[0060] In some embodiments, when the target object is a vulnerable object type, the lower of the final estimated contact force value and the direct force data is used to update the final estimated contact force value; or, when the target object is a non-vulnerable object type, the weighted average of the final estimated contact force value and the direct force data, or the higher of the final estimated contact force value and the direct force data, is used to update the final estimated contact force value. The digital processing unit for calculating the final estimated contact force value may include a contact force estimation module, which logically runs on the digital processing unit. The contact force estimation module may include a visual force estimation submodule. The visual force estimation submodule is used for image preprocessing: performing denoising, enhancement, and other processing on the real-time images acquired by the image acquisition unit. The visual force estimation submodule is also used for deformation feature extraction: during the object clamping process, the object contour is obtained through image segmentation (using YOLOv5's segmentation output). Then, features of the object's deformation are extracted and quantified through contour analysis, template matching, or keypoint tracking, such as the compression of the object in the direction of force, the rate of change of area in a specific region, and the relative displacement of preset marker points (if any). Here, the mapping from deformation to force can be model-based: for objects with specific geometric shapes and known elastic moduli (the model parameters pointed to by the model number corresponding to the object category can be retrieved from the material database), a simplified elasticity model or a pre-calculated lookup table can be used to map the deformation to the first contact force value. Alternatively, the mapping from deformation to force can be learning-based. A lightweight machine learning model (such as Support Vector Regression (SVR) or a small neural network) can be trained using a large number of offline-collected data pairs of "object deformation visual features relative to real contact force" and deployed on a digital processing unit. By inputting the deformation features extracted in real time, the final estimated contact force value is output. The contact force estimation module can also include a motor current force estimation submodule. The motor current force estimation submodule is used for data acquisition: real-time acquisition of the motor phase current fed back by the servo motor driver (or direct acquisition of the estimated motor torque). The motor current force estimation submodule is also used for motor and transmission modeling: establishing a mathematical model from the motor current to the output force at the fingertips. This model needs to consider the motor torque constant, reducer transmission ratio, the force transmission Jacobian matrix of the five-bar linkage, and various nonlinear factors. The material database can store model numbers for different object categories (corresponding to different load characteristics), pointing to the applicable model parameter set. The motor current force estimation submodule is also used for disturbance compensation: based on the motor's real-time speed, acceleration (which can be obtained by differentiating encoder data), and pre-identified friction model parameters (such as Coulomb friction, viscous friction coefficient), and system inertia parameters, it compensates for the torque components in the motor current caused by friction and inertia, thereby more accurately separating the second contact force value generated by actual contact. Furthermore, the contact force estimation module can also include a data fusion submodule.The data fusion submodule is responsible for fusing the final estimated contact force value and the direct force data to update the final estimated contact force value. Specifically, it fuses the first contact force value based on the visual deformation information of the target object, the second contact force value based on the motor current information, and the direct force data from the multi-dimensional sensors to obtain the updated final estimated contact force value. The fusion method adopted by the data fusion submodule can employ a dynamic weighted average algorithm, where each weight can be dynamically adjusted based on the real-time confidence level of each sensor source. For example, when visual conditions are good (sufficient lighting, no obstruction, significant object deformation), the weighting parameter of the first contact force value is increased. When the motor runs smoothly and the model matching degree is high, the weighting parameter of the second contact force value is increased. When the direct force sensor is working properly, the weighting parameter used for the direct force data can have a higher baseline value. Confidence assessment can be based on signal quality, noise level, model residuals, etc. Kalman filter: Treats the contact force as a system state variable and the force data from the multimodal input as observations. A state transition equation (describing the dynamic changes of force) and an observation equation (describing the relationship between observed values ​​and the true force, as well as noise characteristics) are established. Through Kalman filtering prediction and iterative updates, the optimal estimate of the contact force is obtained. This effectively handles sensor noise and uncertainty. Furthermore, the data fusion submodule can employ rule-based or fuzzy logic-based fusion methods. Based on the current working conditions (e.g., object type, grasping stage, slip detection signal) and the characteristics of each force estimate (e.g., changing trend, consistency), a preset rule base or fuzzy inference system determines how to combine these force information. Alternatively, the data fusion submodule can employ a machine learning-based fusion method, training a fusion network (e.g., a small multilayer perceptron) with multimodal force data and possible auxiliary features (e.g., object type, grasping speed) as input, and the final estimated contact force value as output. Further, taking the fingers of a humanoid hand gripper (e.g., 3 fingers, 5 fingers) as an example, multidimensional sensors on it are used for macroscopic adjustments. For example, when the visual data feedback indicates an easily damaged object such as an egg or a glass, the lower force data from the multi-dimensional sensor and the estimated force is used. When the object is a less easily damaged object such as metal, a weighted average result or a larger force data can be used. This further improves the accuracy of the force estimation results.

[0061] In some embodiments, the multidimensional sensors of each of the at least two grippers include a position sensor for detecting the actual position of the robot gripper and a force sensor for detecting the magnitude of the gripping force. Thus, by combining multimodal force estimation and fusion, diverse gripping and assembly tasks can be completed efficiently, safely, and adaptively.

[0062] See Figure 1 and Figure 2In one possible implementation, the digital processing unit includes a visual force estimation submodule, a motor current force estimation submodule, and a data fusion submodule. The visual force estimation submodule calculates the first contact force value based on the visual deformation information of the target object; the motor current force estimation submodule calculates the second contact force value based on the motor current information; and the data fusion submodule fuses the first and second contact force values ​​to obtain the final estimated contact force value. Thus, the image acquisition unit obtains visually perceptual monitoring data, i.e., image information of the target object; the motor driver (or a more macroscopic force control execution unit, force control execution mechanism) obtains monitoring data based on indirect force estimation, i.e., motor current information; and then, using a preset fusion algorithm, the multi-source, multi-modal inputs are fused to obtain the final estimated contact force value. This achieves dynamic force control and precise force feedback, and has the advantages of strong anti-interference and scalability. In summary, Figure 1 The control method for robot grippers shown realizes an intelligent gripper system and control method that integrates visual recognition, dynamic force control, multimodal force estimation and virtual reality remote control force feedback. It is especially suitable for industrial automation, medical assistance, service robots and other scenarios that require adaptive gripping of diverse workpieces, precision force control operations or remote operation in complex environments. It also achieves effective fusion of multi-source and multimodal sensor information to obtain more accurate and reliable contact force estimation.

[0063] In one possible implementation, the final estimated contact force value is used for clamping force control associated with the target object, including: based on the final estimated contact force value, controlling the at least two motor drivers through a closed-loop force control algorithm to drive the at least two clamping members to move relative to each other in order to apply a clamping force to the target object. Thus, by fusing the final estimated contact force and direct force feedback, and combining a force feedback mechanism based on image information and force control information, a basis is provided for fine-tuning the clamping force and the gripping operation of the jaws.

[0064] Figure 3 This is a schematic diagram of a gripper body showing the position of an image acquisition unit, provided as an embodiment of this application. Figure 3 As shown, the gripper body 310 has the shape of a human hand, and the image acquisition unit A320 is located at the tiger's mouth position of the human hand shape so as to capture the inner side of the gripper body 310. Figure 3The diagram also schematically illustrates clamping elements A312 and B314. By learning the deformation information of different object types—that is, by using image data to determine the upper limit of the deformation degree corresponding to a certain object type—the force of the grippers is limited. Here, by introducing the design of elastic elements, one or more clamping elements can be all or part of at least two clamping elements. The specific location of the image acquisition unit A320 and its relative spatial relationship with one or more clamping elements can be flexibly set, as long as it ensures that the image acquisition unit A320 can accurately acquire the deformation of the elastic elements of each of the one or more clamping elements. Figure 3 For example, the image acquisition unit A320 is located at the web of the gripper body 310 of the humanoid hand. This allows the image acquisition unit A320 to capture images of the inner side of the gripper body 310 from the web of the hand, and also to observe the five contact surfaces of the object held by the five fingers of the humanoid hand. This allows for accurate acquisition of the deformation of the elastic elements on these five contact surfaces due to contact with the object. For example, the contact surfaces of grippers A312 and B314 with the object can be observed. In some embodiments, to better observe the deformation of the elastic elements, patterns or markings are provided on the elastic elements, similar to the wavy lines on a knife or a water depth gauge. This allows the image acquisition unit A320 to accurately determine the deformation of the elastic elements, for example, by whether a specific pattern or marking is obscured. In this way, by learning and optimizing the above two types of deformation variables, object deformation variables and elastic component deformation variables together through artificial intelligence algorithms, the robot gripper control system is given reasoning ability. It can determine the object type and the upper limit of the corresponding object deformation variable and the upper limit of the elastic component deformation variable based on image data, which helps to improve the visual force estimation results.

[0065] In one possible implementation, the preset fusion algorithm is a dynamic weighted average algorithm, a Kalman filter algorithm, or a machine learning-based fusion model. This achieves effective fusion of multi-source, multi-modal sensor information to obtain a more accurate and reliable contact force estimate.

[0066] In one possible implementation, the gripper body includes a parallel two-finger gripping structure, a multi-finger gripping structure, or a gripping structure of a specific shape, wherein the multi-finger gripping structure includes a three-finger or five-finger gripping structure. Thus, the gripper body can adopt a parallel two-finger, multi-finger (e.g., three-finger, five-finger), or specific-shaped gripping structure, on which other functional modules are integrated or mounted. In some embodiments, the force control actuator includes a five-bar linkage driven by a planetary gear servo motor via a harmonic reducer and a lead screw with optimized lead (e.g., 0.5 mm) to achieve high-precision, high-response force / position control. This achieves structural and application adaptability to various gripper requirements.

[0067] In one possible implementation, the gripper body includes a force-controlled actuator, with at least two motor drivers belonging to the force-controlled actuator. The force-controlled actuator includes a transmission mechanism composed of multiple links. One or more of these links have a slip detection unit embedded within them. This slip detection unit detects high-frequency vibrations of the contact surface caused by minute slippage of the gripped object, thus determining whether slippage has occurred. When the slip detection unit detects slippage, the digital processing unit executes a gradient force compensation algorithm to suppress the slippage. The slip detection unit may be a piezoelectric thin-film sensor embedded inside or on the surface of one or more links of the gripper. When the gripped object experiences minute slippage, the contact surface generates high-frequency vibrations, which are transmitted to the piezoelectric thin film, causing it to generate a corresponding electrical signal. The digital processing unit filters, amplifies, and extracts features from this signal (e.g., using FFT analysis of energy changes in a specific frequency band) to determine whether slippage has occurred. This achieves slippage detection and improves system stability. In some embodiments, the slip detection unit includes, for example, a piezoelectric thin film sensor disposed on the connecting rod of the gripper body; the digital processing unit is further configured to: determine whether slippage exists based on the signal from the slip detection unit, and when slippage is determined to exist, dynamically adjust the control of the force control actuator to increase the clamping force.

[0068] In one possible implementation, the visual deformation information of the target object is obtained through a deformation feature extraction algorithm. This algorithm includes obtaining the object contour of the target object through image segmentation, and then extracting quantified object deformation features of the target object through contour analysis, template matching, or key point tracking. The quantified object deformation features include compression in the force direction, the rate of change of area in a specific region, or the relative displacement of preset marker points. Thus, by fusing force estimation based on visual object deformation and motor current, and utilizing the complementarity of multimodal information, the limitations of single indirect force estimation methods are overcome, improving the accuracy of contact force sensing and robustness under complex working conditions, while potentially reducing reliance on expensive direct force sensors. Combining visual object classification with a material property database, suitable initial grasping parameters (such as initial contact and safety force threshold) and force control parameters can be dynamically generated for objects with different characteristics. By implementing visual recognition, multimodal force estimation, data fusion, and most control decisions in the digital processing unit at the gripper end, data transmission and processing steps are significantly reduced, improving the system's real-time response capability.

[0069] In one possible implementation, the preset fusion algorithm includes a dynamic weighted average between the first contact force value and the second contact force value. The weighting parameter of the first contact force value has a high confidence level when visual conditions are good, and the weighting parameter of the second contact force value has a high confidence level when the motor is running smoothly. This achieves an intelligent gripper system and control method integrating visual recognition, dynamic force control, multimodal force estimation, and virtual reality remote control force feedback. It is particularly suitable for industrial automation, medical assistance, and service robot scenarios requiring adaptive gripping of diverse workpieces, precise force control operations, or remote operation in complex environments. Furthermore, it effectively fuses multi-source, multimodal sensor information to obtain more accurate and reliable contact force estimation.

[0070] Figure 4 This is a schematic diagram of a control system for a robot gripper provided in an embodiment of this application. Figure 4 As shown, the control system includes: an image acquisition unit B401, at least two motor drivers 403, and a digital processing unit 405. The image acquisition unit B401, deployed on the gripper body, is used to acquire image information of the target object. The gripper body includes at least two clamping members and at least two motor drivers 403 corresponding to each of the at least two clamping members. The at least two motor drivers 403 are respectively used to drive the at least two clamping members to move relative to each other in order to clamp the target object. The image information of the target object includes at least the visual deformation information of the target object when it is clamped by the at least two clamping members. The at least two motor drivers 403 are used to acquire motor current information of their respective motors. The digital processing unit 405 is used to calculate a first contact force value based on the visual deformation information of the target object, and to calculate a second contact force value based on the motor current information. Then, based on a preset fusion algorithm, the first contact force value and the second contact force value are fused to obtain a final estimated contact force value. The final estimated contact force value is used for clamping force control associated with the target object.

[0071] Figure 4 The control system for a robot gripper shown realizes an intelligent gripper system and control method that integrates visual recognition, dynamic force control, multimodal force estimation and virtual reality remote control force feedback. It is particularly suitable for industrial automation, medical assistance, service robots and other scenarios that require adaptive gripping of diverse workpieces, precision force control operations or remote operation in complex environments. It also achieves effective fusion of multi-source and multimodal sensor information to obtain more accurate and reliable contact force estimation.

[0072] See Figure 1 , Figure 2 , Figure 3 as well as Figure 4 In one possible implementation, a remote VR interaction unit can be used to enhance the interaction between the user and the system, as well as the user's control experience. The remote VR interaction unit may include a VR display device, such as a mainstream PC VR headset or an all-in-one VR headset, providing the user with a first-person or third-person immersive stereoscopic view of the gripper's working area. The remote VR interaction unit may include a force feedback input device, such as a VR handle or data glove with a vibration motor, variable damping actuator, or more complex force feedback mechanism. This device can receive instructions from a digital processing unit and apply relevant force or vibration feedback to the user's hand. The remote VR interaction unit may include a communication link employing low-latency, high-bandwidth bidirectional communication technologies (such as wired Ethernet, high-speed Wi-Fi such as Wi-Fi 6 / 6E, or a dedicated wireless communication module) to transmit control commands, video streams (which may use efficient encoding such as H.264 / H.265), estimated force values, gripper status, and other data. The remote VR interaction unit may include a VR user interface, overlaying an interactive user interface (UI) onto the virtual scene presented by the VR display device. This UI may include multiple areas. For example, a real-time status display area can show the real-time video stream from the gripper, the currently identified object category, the final estimated contact force value or graphical indication (such as a force bar), the reference force range suggested by the material database, and the gripper opening degree. Another example is a force adjustment interaction element, such as a virtual force adjustment slider or knob, which the user can drag or rotate using the virtual pointer on the VR controller to set the target gripping force. Alternatively, force can be increased or decreased via gesture recognition (if the VR system supports it). Another example is a mode switching button, such as switching between gripping modes (e.g., "gentle mode," "standard mode," "strong mode," corresponding to different PID parameters or force control strategies), switching between autonomous / remote modes, etc. Another example is an operation command button, such as "grip," "release," "reset," etc. When the user adjusts the target gripping force through these interactive elements, the feedback intensity on the force feedback input device can change in real time, giving the user a "pre-sense" or "shaping" experience of the target force. In some embodiments, the interactive elements on the user interface of the remote VR interaction unit include a virtual force scale, a target force adjustment slider, and / or a preset gripping mode selection button.

[0073] See Figure 1 , Figure 2 , Figure 3 as well as Figure 4In one possible implementation, a vision-based force-controlled gripper system and control method are provided, belonging to the field of robot end effector technology. The system includes a gripper body, an image acquisition unit, a digital processing unit, a data storage unit storing a material property database, a force-controlled actuator, a contact force estimation module running on the digital processing unit, and a remote virtual reality (VR) interaction unit. The contact force estimation module estimates a first contact force value based on the object's visual deformation information monitored by the image acquisition unit, estimates a second contact force value based on the motor current information fed back by the force-controlled actuator, and generates a final estimated contact force through a fusion algorithm. The digital processing unit performs closed-loop force-controlled gripping based on the object recognition results, material database parameters, the final estimated contact force, and optional VR user commands, and feeds back the estimated contact force to the VR user. The VR interaction unit provides immersive visual and force feedback and includes an innovative force adjustment interface. This invention solves the problems of poor gripper adaptability, inaccurate force perception, and unintuitive remote operation in the prior art by integrating visual recognition, multimodal force estimation and fusion, dynamic force control, and VR remote control force feedback at the gripper data terminal. It achieves intelligent gripping and remote fine force control with low latency, high success rate, and high robustness.

[0074] Figure 5 This is a schematic diagram of a computing device 500 provided in an embodiment of this application. The computing device 500 includes one or more processors 510, a communication interface 520, and a memory 530. The processors 510, communication interface 520, and memory 530 are interconnected via a bus 540. Optionally, the computing device 500 may further include an input / output interface 550, which is connected to input / output devices for receiving user-set parameters, etc. The computing device 500 can be used to implement some or all of the functions of the device embodiment or system embodiment in the above-described embodiments of this application; the processor 510 can also be used to implement some or all of the operation steps of the method embodiment in the above-described embodiments of this application. For example, the specific implementation of various operations performed by the computing device 500 can be referred to the specific details in the above embodiments, such as the processor 510 being used to execute some or all of the steps or operations in the above-described method embodiments. For example, in the embodiments of this application, the computing device 500 can be used to implement some or all of the functions of one or more components in the above-described device embodiments. In addition, the communication interface 520 can be used for communication functions necessary to implement the functions of these devices and components, and the processor 510 can be used for processing functions necessary to implement the functions of these devices and components.

[0075] It should be understood that, Figure 5The computing device 500 may include one or more processors 510, and the multiple processors 510 may collaboratively provide processing power in a parallel connection mode, a serial connection mode, a serial-parallel connection mode, or an arbitrary connection mode; or the multiple processors 510 may form a processor sequence or a processor array; or the multiple processors 510 may be divided into a main processor and an auxiliary processor; or the multiple processors 510 may have different architectures, such as adopting a heterogeneous computing architecture. Furthermore, Figure 5 The structural and functional descriptions of the computing device 500 shown are exemplary and non-limiting. In some exemplary embodiments, the computing device 500 may include... Figure 5 The diagram shows more or fewer components, or combinations of some components, or splitting of some components, or different arrangements of components.

[0076] The processor 510 can have various specific implementations. For example, the processor 510 may include one or more combinations of a central processing unit (CPU), a graphics processing unit (GPU), a neural network processing unit (NPU), a tensor processing unit (TPU), or a data processing unit (DPU), etc. This application embodiment does not impose specific limitations. The processor 510 can also be a single-core processor or a multi-core processor. The processor 510 can be a combination of a CPU and hardware chips. The aforementioned hardware chips can be application-specific integrated circuits (ASICs), programmable logic devices (PLDs), or combinations thereof. The aforementioned PLDs can be complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), generic array logic (GALs), or any combination thereof. The processor 510 can also be implemented solely using logic devices with built-in processing logic, such as FPGAs or digital signal processors (DSPs). The communication interface 520 can be a wired interface or a wireless interface, used to communicate with other modules or devices. The wired interface can be an Ethernet interface, a local interconnect network (LIN), etc., and the wireless interface can be a cellular network interface or a wireless LAN interface, etc.

[0077] Memory 530 may be non-volatile memory, such as read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Memory 530 may also be volatile memory, which may be random access memory (RAM) used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM). The memory 530 can also be used to store program code and data, so that the processor 510 can call the program code stored in the memory 530 to execute some or all of the operation steps in the above method embodiments, or to execute the corresponding functions in the above device embodiments. Furthermore, the computing device 500 may include, compared to... Figure 5 The number of components displayed may be more or less, or there may be different component configurations.

[0078] Bus 540 can be a Peripheral Component Interconnect Express (PCIe) bus, or an Extended Industry Standard Architecture (EISA) bus, a Unified Bus (Ubus or UB), a Compute Express Link (CXL) bus, a Cache Coherent Interconnect for Accelerators (CCIX) bus, etc. Bus 540 can be divided into address bus, data bus, control bus, etc. In addition to the data bus, bus 540 can also include a power bus, control bus, and status signal bus. However, for clarity, Figure 5 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0079] The methods and devices provided in this application are based on the same inventive concept. Since the principles by which the methods and devices solve problems are similar, the embodiments, implementation methods, examples, or methods of implementation of the methods and devices can be referred to each other, and repeated details will not be repeated. This application also provides a system comprising multiple computing devices, the structure of each computing device of which can refer to the structure of the computing devices described above. The functions or operations achievable by this system can refer to the specific implementation steps in the above method embodiments and / or the specific functions described in the above device embodiments, and will not be repeated here.

[0080] This application also provides a computer-readable storage medium storing computer instructions. When these computer instructions are executed on a computer device (such as one or more processors), they can implement the method steps described in the above method embodiments. The specific implementation of the above method steps by the processor of the computer-readable storage medium can refer to the specific operations described in the above method embodiments and / or the specific functions described in the above device embodiments, and will not be repeated here.

[0081] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. This application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Embodiments of this application can be implemented wholly or partially by software, hardware, firmware, or any other combination. When implemented in software, the above embodiments can be implemented wholly or partially as a computer program product. This application can take the form of a computer program product embodied on one or more computer-usable storage media containing computer-usable program code. The computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. Computer-readable storage media can be any available medium that a computer can access, or a data storage device such as a server or data center that contains one or more sets of available media. Available media can be magnetic media (such as floppy disks, hard disks, and magnetic tapes), optical media, or semiconductor media. Semiconductor media can be solid-state drives, random access memory, flash memory, read-only memory, erasable programmable read-only memory, electrically erasable programmable read-only memory, registers, or any other suitable form of storage medium.

[0082] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. Each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0083] In the above embodiments, the descriptions of each embodiment have their own emphasis. Parts not described in detail in a certain embodiment can be referred to in the relevant descriptions of other embodiments. Obviously, those skilled in the art can make various modifications and variations to the embodiments of this application without departing from the spirit and scope of the embodiments of this application. The steps in the methods of the embodiments of this application can be adjusted in order, combined, or deleted according to actual needs; the modules in the systems of the embodiments of this application can be divided, combined, or deleted according to actual needs. If these modifications and variations of the embodiments of this application fall within the scope of the claims of this application and their equivalents, then this application also intends to include these modifications and variations.

Claims

1. A control method for a robot gripper, characterized in that, The control method includes: The image information of the target object is acquired, wherein the gripper body includes at least two gripping members and at least two motor drivers corresponding to the at least two gripping members. The at least two motor drivers are respectively used to drive the at least two gripping members to move relative to each other so as to grip the target object. The image information of the target object includes at least the visual deformation information of the target object when the target object is gripped by the at least two gripping members. The motor current information of each of the at least two motor drivers is obtained through the at least two motor drivers; A first contact force value is calculated based on the visual deformation information of the target object, and a second contact force value is calculated based on the motor current information. Then, the first contact force value and the second contact force value are fused based on a preset fusion algorithm to obtain a final estimated contact force value. The final estimated contact force value is used for clamping force control associated with the target object.

2. The control method according to claim 1, characterized in that, One or more of the at least two clamping members are respectively deployed with elastic elements, and when the target object is clamped by the at least two clamping members, the elastic elements of each of the one or more clamping members deform due to contact with the target object. The control method further includes: obtaining the deformation of the elastic elements of each of the one or more clamping members, thereby obtaining elastic element deformation information associated with the target object, and the elastic element deformation information associated with the target object is used to calibrate the calculation of the first contact force value.

3. The control method according to claim 2, characterized in that, The digital processing unit calculates the first contact force value using an artificial intelligence model. The learning and optimization of the artificial intelligence model are based on the visual deformation information of the clamped object acquired by the image acquisition unit and the deformation information of the elastic element associated with the clamped object.

4. The control method according to claim 1, characterized in that, The image information of the target object also includes the overall image data of the target object, and the control method further includes: Based on the overall image data of the target object, an object recognition algorithm is used to determine the object category, position, and orientation of the target object. Based on the object category of the target object, the force control parameters of the target object are determined by accessing the material property database for the clamping force control associated with the target object. The material property database includes the load characteristics and force control parameters corresponding to different object categories based on prior knowledge. Based on the object category, position, and orientation of the target object, the initial grasping parameters of the target object are determined by accessing the material property database. The initial grasping parameters of the target object include initial contact force and safety force threshold.

5. The control method according to claim 4, characterized in that, The control method further includes: Search the historical data to record the object category, position, and orientation of objects that have been gripped by the robot gripper. When it is detected that the object category of the gripped object in the historical data is the same as the object category of the target object and the initial gripping parameters of the gripped object are similar to the initial gripping parameters of the target object, the force control parameters of the gripped object are used as the force control parameters of the target object.

6. The control method according to any one of claims 1 to 5, characterized in that, The gripper body includes a force control actuator, and the at least two motor drivers belong to the force control actuator. The force control actuator includes a transmission mechanism composed of multiple links. One or more of the multiple links have a slip detection unit embedded inside. The slip detection unit is used to detect high-frequency vibration of the contact surface caused by the slight slippage of the gripped object, thereby determining whether slippage has occurred. When the slip detection unit detects slippage, the digital processing unit executes a gradient force compensation algorithm to suppress slippage.

7. The control method according to any one of claims 1 to 5, characterized in that, The preset fusion algorithm includes a dynamic weighted average between the first contact force value and the second contact force value, wherein the weighting parameter of the first contact force value has a high confidence level when visual conditions are good, and the weighting parameter of the second contact force value has a high confidence level when the motor is running smoothly.

8. An electronic device, characterized in that, The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method according to any one of claims 1 to 7.

9. A computer-readable storage medium or computer program product, characterized in that, The computer-readable storage medium stores computer instructions that, when executed on a computer device, cause the computer device to perform the method according to any one of claims 1 to 7; or, When the computer instructions contained in the computer program product are run on a computer device, the computer device causes the computer device to perform the method according to any one of claims 1 to 7.

10. A control system for a robot gripper, characterized in that, The control system includes: An image acquisition unit deployed on the gripper body is used to acquire image information of a target object. The gripper body includes at least two gripping members and at least two motor drivers corresponding to the at least two gripping members. The at least two motor drivers are used to drive the at least two gripping members to move relative to each other in order to grip the target object. The image information of the target object includes at least the visual deformation information of the target object when it is gripped by the at least two gripping members. The at least two motor drivers are used to acquire motor current information of each of the at least two motor drivers; A digital processing unit is used to calculate a first contact force value based on the visual deformation information of the target object, and to calculate a second contact force value based on the motor current information. Then, based on a preset fusion algorithm, the first contact force value and the second contact force value are fused to obtain a final estimated contact force value. The final estimated contact force value is used for clamping force control associated with the target object.

Images