Grabbing control method and device based on collaborative grabbing characteristics, equipment and medium

Abstract

The invention relates to the technical field of artificial intelligence, can be applied to service scenes such as pension service, financial science and technology and medical health, and discloses a grabbing control method, device and equipment based on collaborative grabbing characteristics and a medium. Generating collaborative grabbing features and feature weight vectors, constructing a grabbing model in combination with historical grabbing data, and determining an initial grabbing strategy; and in combination with an initial grabbing strategy and real-time tactile feedback data, a grabbing control sequence and a cooperative adaptation coefficient are generated through constraint optimization, grabbing operation is executed, real-time feedback is monitored, the grabbing force and posture are dynamically adjusted according to deviation and the cooperative adaptation coefficient, and accurate control is achieved. According to the method, the problem that in the prior art, grabbing response is not flexible is solved by constructing the collaborative grabbing features and the grabbing model and fusing real-time feedback and control strategies, dynamic self-adaption and fine adjustment in the grabbing process of different objects are achieved, and the damage risk is reduced.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a grasping control method, apparatus, device, and storage medium based on collaborative grasping features. Background Technology

[0002] In various application scenarios such as elderly care services, medical and health assistance, and fintech terminal interaction, the ability of robots to accurately and safely grasp different types of objects is receiving increasing attention. However, existing technologies still have significant shortcomings in tactile and mechanical perception during grasping, making it difficult to adapt to the complex and diverse physical characteristics of objects, which may lead to grasping failure or object damage, affecting user experience and task stability.

[0003] In elderly care service scenarios, robots often need to handle a variety of objects with different structures and materials, such as glass cups, ceramic bowls, plush toys, medicine boxes, and paper books. These objects vary significantly in terms of hardness, surface texture, and elasticity. If operation is based solely on statically configured grasping parameters, the objects are easily dropped due to insufficient force or crushed due to excessive force. However, most current elderly care robots only support force control strategies based on experience-preset parameters, lacking dynamic recognition and personalized adjustment capabilities, and thus unable to adapt to the ever-changing grasping needs in real-world environments.

[0004] In the healthcare field, assisted grasping is commonly used in tasks such as patient care, medication dispensing, or medical device handling, requiring extremely high levels of compliance and safety in the grasping motion. Especially when dealing with objects made of soft materials (such as gauze, surgical gloves, and plastic test tubes), traditional grasping mechanisms lack sufficient tactile resolution accuracy, failing to effectively identify the object's state and easily causing equipment deformation or accidental injury. Furthermore, current tactile sensors generally suffer from low resolution and high response latency in practical deployments, resulting in a significant time lag between data acquisition and grasping execution, hindering the achievement of high real-time perception and response.

[0005] In the fintech sector, service robots are widely used for operations such as smart terminal interaction, document retrieval, and contract transfer. These tasks typically involve the secure retrieval of important documents or flexible media (such as bills, bank cards, and paper contracts). However, existing technologies often fail to effectively quantify and identify the surface physical characteristics of target objects, leading to problems such as curling, creases, or detachment when grasping flexible or fragile objects, potentially causing user disputes or service interruptions. Furthermore, the lack of joint analysis and dynamic coupling capabilities between mechanical feedback mechanisms and tactile feedback results in a lack of flexible adjustment basis for the grasping strategy and insufficient overall system adaptability. Summary of the Invention

[0006] The main objective of this invention is to provide a grasping control method, device, equipment, and storage medium based on collaborative grasping features, aiming to solve the technical problem in the prior art where tactile perception and mechanical modeling are independent and lack a collaborative processing mechanism, resulting in the robot's inability to dynamically adjust the grasping force and posture based on the actual tactile and mechanical properties of the object.

[0007] To achieve the above objectives, the present invention provides a crawling control method based on collaborative crawling features, comprising:

[0008] Acquire surface tactile data, interactive mechanical data, and physical parameters of the target object;

[0009] Based on the surface tactile data, interactive mechanical data, and physical parameters, collaborative grasping features and feature weight vectors are generated.

[0010] Based on the collaborative grasping features, feature weight vectors and historical grasping data, a grasping model is constructed, and an initial grasping strategy including optimal grasping force, maximum allowable grasping force, contact point distribution coordinates and finger posture parameters is determined based on the grasping model.

[0011] Combining the initial grasping strategy with real-time tactile feedback data during the grasping process, a grasping control sequence and cooperative adaptation coefficients are generated through constraint optimization.

[0012] The grasping operation is executed according to the grasping control sequence, and the real-time tactile feedback data and real-time mechanical feedback data are monitored during the execution process.

[0013] Based on the deviation between the real-time mechanical feedback data and the optimal gripping force, and the deviation between the real-time tactile feedback data and the reference tactile features, the gripping force and posture are dynamically adjusted in conjunction with the cooperative adaptation coefficient.

[0014] Furthermore, to achieve the above objectives, the present invention provides a grasping control device based on collaborative grasping features, comprising:

[0015] The multi-source sensing and acquisition module is used to acquire surface tactile data, interactive mechanical data, and physical parameters of the target object;

[0016] The feature fusion calculation module is used to generate collaborative grasping features and feature weight vectors based on the surface tactile data, interactive mechanical data and physical parameters.

[0017] The initial strategy generation module is used to construct a grasping model based on the collaborative grasping features, feature weight vectors and historical grasping data, and to determine an initial grasping strategy based on the grasping model, which includes the optimal grasping force, the maximum allowable grasping force, the contact point distribution coordinates and the finger posture parameters.

[0018] The collaborative optimization control module is used to combine the initial grasping strategy with real-time tactile feedback data during the grasping process, and generate a grasping control sequence and collaborative adaptation coefficients through constraint optimization solution.

[0019] The execution monitoring module is used to execute the grasping operation according to the grasping control sequence and monitor the real-time tactile feedback data and real-time mechanical feedback data during the execution process.

[0020] The dynamic adjustment decision module is used to dynamically adjust the gripping force and posture based on the deviation between the real-time mechanical feedback data and the optimal gripping force, the deviation between the real-time tactile feedback data and the reference tactile features, and in conjunction with the cooperative adaptation coefficient.

[0021] Furthermore, to achieve the above objectives, the present invention also provides a computer device, the computer device including a memory, a processor, and a crawling control program based on cooperative crawling features stored in the memory and executable on the processor, wherein when the crawling control program based on cooperative crawling features is executed by the processor, it implements the steps of the crawling control method based on cooperative crawling features as described above.

[0022] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a crawling control program based on cooperative crawling features, wherein when the crawling control program based on cooperative crawling features is executed by a processor, it implements the steps of the crawling control method based on cooperative crawling features as described above.

[0023] Beneficial Effects: This invention relates to the field of artificial intelligence technology and can be applied to business scenarios such as elderly care services, fintech, and healthcare. It discloses a grasping control method, device, equipment, and medium based on collaborative grasping features, including: acquiring surface tactile data, interactive mechanical data, and physical parameters of a target object; generating collaborative grasping features and feature weight vectors based on the above data; constructing a grasping model by combining historical grasping data; generating an initial grasping strategy; combining the initial grasping strategy with real-time tactile feedback data during the grasping process; using constraint optimization to generate a grasping control sequence and collaborative adaptation coefficients; executing the grasping operation and monitoring real-time feedback data during execution; and dynamically adjusting the grasping force and posture based on feedback deviation and collaborative adaptation coefficients to achieve precise grasping control of different objects. This invention solves the problem of inflexible grasping response in existing technologies by constructing collaborative grasping features and a grasping model, integrating real-time feedback and control strategies, achieving dynamic adaptation and fine adjustment during the grasping process of different objects, and reducing the risk of damage. Attached Figure Description

[0024] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0025] Figure 1 This is a schematic diagram of an application environment for a crawling control method based on collaborative crawling features according to an embodiment of the present invention;

[0026] Figure 2 This is a flowchart illustrating an embodiment of the grasping control method based on collaborative grasping features of the present invention.

[0027] Figure 3 This is a schematic diagram of the functional modules of a preferred embodiment of the grasping control device based on collaborative grasping features of the present invention;

[0028] Figure 4 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention;

[0029] Figure 5 This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0030] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0031] The crawling control method based on collaborative crawling features provided in this invention can be applied to, for example... Figure 1 In this application environment, the user terminal communicates with the server via a network. The server can obtain surface tactile data, interactive mechanical data, and physical parameters of the target object from the user terminal. Based on this data, it generates collaborative grasping features and feature weight vectors, and constructs a grasping model by combining historical grasping data, generating an initial grasping strategy. Combining the initial grasping strategy with real-time tactile feedback data during the grasping process, it uses constraint optimization to generate a grasping control sequence and collaborative adaptation coefficients, executes the grasping operation, and monitors real-time feedback data during execution. Based on the feedback deviation and collaborative adaptation coefficients, it dynamically adjusts the grasping force and posture to achieve precise grasping control of different objects. This invention solves the problem of inflexible grasping response in existing technologies by constructing collaborative grasping features and grasping models and integrating real-time feedback and control strategies, achieving dynamic adaptation and fine adjustment during the grasping process of different objects, and reducing the risk of damage. The user terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster composed of multiple servers. The invention will be described in detail below through specific embodiments.

[0032] Please see Figure 2 , Figure 2 This is a flowchart illustrating an embodiment of the collaborative grasping feature-based grasping control method provided by the present invention. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0033] like Figure 2 As shown, the crawling control method based on collaborative crawling features proposed in this invention includes the following steps:

[0034] S10, acquire the surface tactile data, interactive mechanical data and physical parameters of the target object;

[0035] In this embodiment, the acquisition of surface tactile information of the target object can be accomplished through a tactile sensor array. This sensor array should possess spatial resolution of pressure distribution and accurate measurement of contact area. Tactile data includes parameters such as normal pressure value per unit area, boundary contact conditions, and contact point density. This information is a prerequisite for obtaining the surface morphology, stiffness, and local response characteristics of the object. In specific operation, a high-density tactile array based on capacitance, resistance, or piezoelectric effects can be selected. Through a two-dimensional or three-dimensional spatial coordinate grid, it can capture the subtle deformations and load responses of the target surface during initial contact and loading.

[0036] Interactive mechanics data acquisition requires a multi-dimensional force sensor system to record in real time the mechanical quantities generated along different directions during contact. These forces include normal and tangential forces applied to the target object, and can also be extended to six-dimensional force / torque information. For complex interactive behaviors in grasping scenarios, a high-frequency sampling device should be used to ensure sensitive capture of minute force changes, along with vibration isolation and signal anti-interference processing modules to improve data validity. This type of mechanical data will be used to construct a force-tactile behavior model in a real contact environment, assisting in subsequent control reasoning and judgment.

[0037] The acquisition of physical parameters encompasses the static properties of the target object, including size, mass, material type, and surface structure characteristics. Dimensional information can be precisely measured using depth cameras, structured light systems, or laser scanners, and its complete outline can be reconstructed using visual 3D reconstruction methods. Mass information can be estimated using weighing modules, pressure sensing platforms, or inertial measurement units. Material type can be identified through sensor fusion, such as combining near-infrared spectroscopy analysis with surface texture recognition algorithms to distinguish common materials like glass, plastic, and fabric.

[0038] The acquisition of these tactile, mechanical, and physical parameters should be completed on hardware with closed-loop execution capabilities in the robotic arm and gripping tools, ensuring that all data streams have consistent timestamps for subsequent fusion analysis. A unified data bus protocol, such as CAN, EtherCAT, or TSN, needs to be established between different types of data acquisition channels to ensure the stability and real-time performance of communication between sensor inputs and data processing modules.

[0039] In one implementation, the tactile sensor uses a capacitive array-based structure with a node spacing of less than 2 mm, laid on the surface of a flexible substrate inside the gripper, and connected to a low-power analog-to-digital converter. The tactile acquisition frequency is set to 200 Hz. Mechanical data acquisition uses six-axis force / torque sensors embedded in the gripper joints, with each joint corresponding to an independent acquisition channel. The sampling frequency is higher than 1 kHz, and real-time filtering and calibration are implemented in the embedded system.

[0040] The identification of physical parameters relies on a structured light module integrated on the side of the gripper to construct a 3D point cloud model for estimating the object's length, width, and height. Weight information is obtained through the loading platform, and combined with force sensors to read the initial load change at the first contact, the object's center of gravity and its inertial distribution are calculated using the least squares method.

[0041] In another approach, interactive mechanical data is acquired jointly by a MEMS inertial unit and a tension meter, and multi-source data calibration is achieved through a dynamic fusion algorithm based on Kalman filtering. Meanwhile, surface tactile data is obtained using a piezoresistive thin-film array, combined with a convolutional neural network to classify and regress contact patterns, thereby extracting surface elasticity distribution and roughness features.

[0042] Example: In the healthcare field, service robots handle grasping targets such as syringes, medical tissues, and stethoscopes. These items require high sensitivity and precision in tactile feedback and force control. By acquiring accurate tactile and mechanical data, the system can determine the stiffness of the syringe's plastic shell, preventing damage to its seal due to excessive gripping force, and can also identify changes in surface tension when handling paper items.

[0043] In the financial sector, intelligent service robots may perform tasks such as ticket collection, document distribution, and moving display items. For example, when moving a customer's folder, the system can identify the edge hardness and coefficient of friction through real-time tactile data, and combine this with quality information to determine whether it is an empty folder or a heavy folder, and adjust the grasping strategy accordingly to prevent the file from slipping or being damaged.

[0044] In elderly care settings, intelligent robots often need to grasp lightweight, fragile, or soft items such as glass cups, ceramic tableware, medicine boxes commonly used by the elderly, and plush toys. Because these objects exhibit significant differences in surface texture, uneven elasticity distribution, and varying structural strength, the system can utilize collected surface tactile data and real-time mechanical feedback to identify their physical characteristics. Taking a glass cup as an example, the robot senses changes in surface rigidity and gripping resistance, adjusting the gripper's force range to reduce the risk of breakage. When grasping a plush toy, it adjusts the contact area and finger angle based on the surface compression deformation distribution, ensuring grip stability and comfort. These capabilities effectively enhance the adaptability and safety of elderly care assistant robots with a variety of everyday items.

[0045] This embodiment achieves global mechanical and tactile perception of a target object before it is grasped through synchronous acquisition and dynamic integration of data from multiple sensing devices. It accurately characterizes the object's elasticity, hardness, roughness, and weight at the data level. This comprehensive perception capability can serve as input for subsequent grasping strategy optimization and grasping control inference. Compared to single-sensor dimensions or offline inference methods, this process offers advantages such as high real-time performance, high resolution, and wide adaptability, effectively solving the problem in traditional systems where grasping strategies cannot be adjusted for different object characteristics.

[0046] S20, Based on the surface tactile data, interactive mechanical data and physical parameters, generate collaborative grasping features and feature weight vectors;

[0047] In this embodiment, the collected surface tactile data, interactive mechanical data, and physical parameters are used as input data. Data parsing and preprocessing are required to ensure the temporal consistency and numerical comparability of the input information. Surface tactile data includes parameters such as normal pressure distribution, pressure gradient of the contact area, and instantaneous stress change rate, characterizing the response characteristics of the object's surface material under contact. Interactive mechanical data covers indicators such as six-dimensional force / torque components, force change rate per unit time, and shear force ratio, used to describe the dynamic interaction between the robot and the object. Physical parameters include the object's size, mass, material type, and elastic modulus, which are important dimensions for characterizing static structural information.

[0048] Fusing these three types of information requires constructing a unified data representation structure. Through vector concatenation, tensor merging, or multi-scale encoding mechanisms, data from different sources are mapped to a unified feature space. Within this feature space, collaborative grasping features reflecting grasping security, stability, and adaptability are extracted, including but not limited to pressure-contact area coupling features, elasticity-mass ratio features, and force-tactile response differences under dynamic loading. These features can be extracted from the fused data using methods such as statistical regression, principal component analysis, and graph structure modeling, or obtained through training neural network models to achieve deep feature representations with nonlinear expressive capabilities.

[0049] After generating collaborative crawling features, it is necessary to assign corresponding feature weight vectors to each feature to quantify the influence of each dimension of the feature in subsequent strategy decisions. The weight vectors can be dynamically adjusted based on multiple factors such as historical crawling success rate, error propagation sensitivity, and task category suitability. This weight generation process can be trained using a supervised learning model or constructed using heuristic algorithms, such as weighted information gain, gradient contribution analysis, or Markov chain mobility learning mechanisms, to ensure that the feature distribution has good generalization performance and crawling robustness across different scenarios.

[0050] Throughout the processing, input data must be timestamped and anomalies removed to prevent mismatch interference. Furthermore, the feature generation module must be able to update online to adapt to rapid model reconstruction when the target object changes.

[0051] In one implementation, a multi-input neural network structure is used, where tactile data is input to a convolutional network branch, mechanical data to a recurrent network branch, and physical parameters to a fully connected layer branch. The outputs of the intermediate layers of each branch are concatenated by a fusion layer and then fed into a two-layer Transformer structure to extract collaborative grasping features. Finally, a set of 64-dimensional vector representations is output, and a feature weight vector is generated by softmax weighting.

[0052] In another approach, a graph neural network is used to model each sensing point as a graph node. Node features include local tactile response, neighborhood force changes, and material feature embedding vectors, while edge features are defined as load transmission relationships on the contact boundaries. An importance score for each node is calculated through an iterative propagation mechanism and mapped to a global feature vector. Simultaneously, a self-attention mechanism is used to redistribute the weights of dependencies between nodes, thereby constructing a feature-weight structure oriented towards multi-task grasping.

[0053] Unsupervised clustering methods (such as spectral clustering or Gaussian hybrid models) can also be used to cluster the fused high-dimensional data on multiple object samples, extract representative features from each cluster center, and generate weight coefficients by evaluating the consistency of different features in multiple scenarios, so as to improve the model's ability to generalize the strategy for grasping unknown objects.

[0054] Example: In the healthcare field, when a nursing robot grasps a glass syringe or medical vial, it needs to identify the object's rigidity distribution and weight characteristics. By fusing data on the sliding tactile sensation of the glass surface, dynamic force information under the loaded vial conditions, and torque changes caused by the shift in the center of mass, the robot can extract a combination of features suitable for delicate manipulation and assign higher weights to elastic distribution and shear force variations, thereby ensuring grasping stability and vial integrity.

[0055] In the financial sector, front-desk intelligent robots need to pick up paper vouchers, metallic cards, or folders with flexible covers. By combining surface roughness and resistance to deformation, and considering the trends in force uniformity and local friction coefficients, a grasping feature can be constructed that distinguishes between card edge states and document stacking states. This feature, coupled with adaptive weights, enables flexible, responsive strategy configuration for different materials, improving data processing efficiency and preserving the integrity of items.

[0056] In elderly care scenarios, when the robot grasps objects such as ceramic bowls, plush toys, and plastic medicine boxes, it acquires data on surface tactile distribution, contact force evolution, and object mass and material. This data generates a collaborative grasping feature set, including pressure response characteristics, elastic rebound characteristics, and material compressive strength. Subsequently, a weight vector is generated based on the performance of these features in existing historical grasping cases. The system can then determine whether to increase the deformation tolerance of plush toys or reduce the maximum clamping force on ceramic bowls, ensuring safe and stable grasping and avoiding the risk of dropping or breakage.

[0057] This embodiment, by fusing and analyzing data from different sources in a unified feature space, can extract highly coupled grasping parameters with physical interpretability, effectively bridging the gap between tactile and mechanical information and forming a decision-making basis for cross-modal fusion. The generated feature weight vector provides a refined reference in strategy adjustment, enabling dynamic fine-tuning of the grasping strategy. Compared to the traditional approach of formulating strategies based on a single data source, it improves the ability to distinguish the grasping characteristics of heterogeneous objects, resulting in stronger task adaptability and control accuracy in actual grasping.

[0058] S30, based on the collaborative grasping features, feature weight vector and historical grasping data, a grasping model is constructed, and based on the grasping model, an initial grasping strategy including optimal grasping force, maximum allowable grasping force, contact point distribution coordinates and finger posture parameters is determined;

[0059] In this embodiment, collaborative grasping features, feature weight vectors, and historical grasping data are jointly input into the model training process. First, it must be ensured that the input vectors have a unified format and semantic correspondence. Collaborative grasping features are a structured set of multidimensional features, typically including local pressure distribution, force-tactile interaction coupling response, contact area elastic field, and strain tensor configuration. The feature weight vector defines the proportion of influence of different feature dimensions in the grasping decision. Historical grasping data includes execution parameters, result feedback, grasping success markers, and object state records for different grasping objects in various contact scenarios from previous tasks.

[0060] When building a grasping model, the first step is to establish a function mapping relationship that maps the input co-features and grasping context information to the grasping policy parameters output. The model structure can be a neural network with non-linear fitting capabilities, an ensemble regressor, or a graph-based multi-pass sensor encoder. During training, historical grasping data is introduced as supervision labels, and backpropagation optimization is performed to enable the model to gradually learn to generate accurate and safe policy parameter sets when facing new objects.

[0061] The parameters output by the model include: optimal gripping force, representing the force scalar required to achieve stable gripping under the current tactile and mechanical properties of the object; maximum allowable gripping force, defining the maximum non-damaging threshold that the robot can withstand during gripping, learned based on dimensions such as material strength and compressibility limit; contact point distribution coordinates, representing the spatial distribution strategy of the gripping mechanism's contact surface, usually represented in three-dimensional coordinates or geometric topology, used to ensure torque balance and contact stability during gripping; and finger posture parameters, referring to the angle configuration, gripping opening, and pose sequence of each joint of the control terminal, used to adapt to the gripping operation requirements of different object contours.

[0062] The entire decision-making process needs to consider the model's sensitivity to the novelty of the target object, changes in feature weights, and risk indicators, to avoid a mismatch between the model's output strategy and the actual object's attributes. Simultaneously, the model should support an incremental learning mechanism, dynamically updating parameters after introducing new sample data to maintain its adaptability to environmental changes.

[0063] In one implementation, an end-to-end deep network architecture comprising an input encoder, a policy generator, and a parameter decoder is employed. The input encoder receives collaboratively grasping features and their weight vectors, which are then subjected to dimensionality mapping and feature cross-fusion via multi-layer residual modules. Historical grasping data is used to generate contextual representations through embedding layers and attention mechanisms. The policy generator employs stacked Transformer modules to resolve global dependencies between input data, and the parameter decoder ultimately outputs four policy parameters: optimal force, maximum force, contact point coordinates, and pose configuration.

[0064] In another approach, a graph-based model representing the grasping space is constructed, where each node represents a candidate contact region. Node attributes are the combined values ​​of tactile and mechanical features at that location, and edge weights represent the dynamic coupling degree between adjacent regions. The optimal contact subgraph is obtained through graph neural network propagation calculations, and the optimal set of grasping points and corresponding force boundaries are output based on a graph energy minimization strategy. Finger posture is automatically solved by an inverse kinematics algorithm based on the contact point positions.

[0065] Alternatively, a strategy model framework based on Bayesian optimization can be used to fit the relationship between the success probability of capturing and the input features for the same object type using Gaussian process regression. Then, the strategy parameter set can be selected through confidence intervals to ensure that the strategy has risk control capabilities and interpretability.

[0066] Example Description: In healthcare scenarios, nursing robots need to grasp different types of medicine containers, such as flexible IV bags, glass ampoules, and aluminum foil blister packs. Based on the deformation patterns of flexible containers, the brittleness of glass bottles, and the tactile characteristics of aluminum foil edges, the system combines historical grasping pressure distribution and success rates to output a force configuration with a dynamic buffering strategy, an inclined contact point distribution, and a rotatable fingertip posture. This ensures that the robot neither punctures the medicine nor slips due to insufficient grip.

[0067] In financial service scenarios, front-desk service robots need to pick up objects such as bank cards, contracts, and binding materials. By accumulating records of the tensile strength and edge thickness of different materials during historical processing, and combining the changes in the friction coefficient of contact response and tactile micro-vibration modes, the system generates the optimal gripping force and maximum clamping tolerance force difference, the contact point structure based on edge and corner distribution, and low error margin posture parameters, thereby ensuring the stability and neatness of financial data transmission.

[0068] In elderly care robot applications, when grasping heterogeneous objects such as ceramic bowls, plush toys, and plastic eyeglass cases, the model generates a set of adaptive features and corresponding strategy parameters based on the tactile compliance, pressure resistance limit, center of mass, and boundary deformation rate of each object type. For example, the grasping strategy for ceramic bowls outputs a lower maximum allowable force and semi-enclosed contact point coordinates, and configures a low gripping speed and anti-slip grip posture; for plush toys, a strategy with a higher optimal force but higher posture tolerance is generated; while for plastic boxes, a multi-point contact distribution and moderate gripping force are preferred to suit their rigid and lightweight physical characteristics. These strategies effectively reduce the grasping error rate and damage probability in actual operation, improving the safety and practicality of robot grasping in elderly care environments.

[0069] This embodiment introduces collaborative features and weighted structure into the grasping model and integrates historical data for supervised training. The system can automatically learn the mapping relationship between different types of objects and successful grasping strategies, significantly improving the accuracy and responsiveness of strategy generation. The model can not only output the optimal force and maximum force range, but also generate structurally stable contact point configurations and motion-accessible finger posture combinations, realizing a stable, safe, and controllable initial grasping strategy. This overcomes the shortcomings of traditional fixed strategy libraries in adapting to new objects and has the ability to generalize and continuously optimize strategies in the face of changing scenarios.

[0070] S40, combining the initial grasping strategy with real-time tactile feedback data during the grasping process, a grasping control sequence and cooperative adaptation coefficients are generated through constraint optimization solution;

[0071] In this embodiment, the initial grasping strategy provides a baseline including optimal grasping force, maximum allowable grasping force, contact point distribution coordinates, and finger posture parameters, possessing a reasonable initial structural configuration oriented towards the target object. Real-time tactile feedback data during the grasping process is collected by a high-precision flexible array sensor installed on the actuator. Data types include pressure change distribution, pressure fluctuation rate per unit area, contact point deformation curvature sequence, and sliding edge amplitude response, reflecting the actual interaction state between the object and the gripping interface. The initial strategy parameters and real-time tactile feedback data are fused to form the variable input set for the optimization problem.

[0072] The constraint optimization solution model uses optimal grasping safety and control smoothness as objective functions, integrating a set of multidimensional linear and nonlinear constraints. These constraints include ensuring the displacement of the contact point does not exceed the geometric boundary, the grasping force does not exceed the maximum allowable value, the tactile feedback stability is within a threshold range, and the finger posture satisfies motion accessibility conditions. In this solution process, the control variables are the grasping force, contact area, and joint posture change rate at each time step. Gradient projection, Lagrange multiplier method with penalty terms, or dynamic programming within the feasible region are used for continuous solution to generate the grasping control sequence.

[0073] The grasping control sequence refers to the flow of instructions applied over time during grasping execution, including progressive force adjustments, contact surface expansion strategies, joint adjustment sequences, and micro-amplitude suppression control. The sequence generation process considers the response delay between tactile fluctuations and strategy matching, achieving smooth adaptation to the object's state and fine-tuning feedback. The co-adaptation coefficient is used to dynamically adjust the convergence speed and amplitude of the control sequence under feedback influence, representing the degree of interactive coupling, response sensitivity, and compliance adaptability. This coefficient is determined by analyzing the degree of co-operation between the current feedback and the initial strategy, combined with the distribution of co-operation response patterns in the historical model, and then normalized. Finally, it is used to adjust the adjustment speed term and gain matrix structure in the control strategy.

[0074] This process not only establishes a connection between the static strategy and the dynamic feedback, but also effectively improves the adaptive robustness of the system under nonlinear feedback by continuously updating the control sequence parameters and the adaptation structure. It solves the problem of dynamic deviation between the initial strategy and the actual contact state, and provides a complete parameter basis for subsequent stability determination and state analysis.

[0075] In one implementation, a reinforcement learning-driven constraint optimization module receives initial policy parameters and real-time haptic feedback data. During the training phase, it constructs a state-action-reward triplet, where the state includes the current grasping force, contact point pressure distribution, and feedback perturbation spectrum; the action is the spatial control policy increment; and the reward function integrates a haptic stability index and a policy matching score. During the execution phase, the optimization module outputs a grasping control sequence and co-fit coefficients to perform round-by-round corrections to the control policy.

[0076] In another implementation, a constraint solver is constructed based on a multi-objective optimization mechanism, where one optimization objective is to maximize contact stability and the other is to minimize grasping energy consumption. After real-time haptic feedback is input as a high-dimensional data stream, it first undergoes principal component analysis and noise reduction, and then a genetic algorithm performs discrete sequence iterations on the control variables, outputting the optimal grasping control sequence and corresponding adjustment coefficients. This approach is suitable for operating environments with severe feedback fluctuations or low sensor accuracy.

[0077] Another approach is to use physical modeling combined with feedback reconstruction. By establishing a mapping function between the tactile feedback field and the strategy response field, the deformation distribution under different strategy responses during the clamping process can be solved using the finite element method. The evolution direction of each parameter in the control sequence can be derived in reverse. The compression adjustment weight of the cooperative adaptation coefficient can be determined by combining the tactile perturbation suppression function. This approach is suitable for flexible grasping systems with high requirements for control accuracy and feedback delay control.

[0078] Example Description: In healthcare scenarios, service robots need to pick up medicine packaging with different shapes and textures, such as tablet plates, blister packs, and tube-shaped ointments. Initial strategies may not be sufficient for a perfect fit due to material differences. The control sequence optimization module, combined with real-time tactile feedback, such as signs of fingertip slippage or uneven pressure fluctuations, dynamically adjusts the gripping force and contact posture. A cooperative adaptation coefficient is introduced to mitigate feedback overshoot during the adjustment process, preventing items from breaking or falling.

[0079] In the financial sector, business robots may need to grasp bank cards, folders, or deliver envelopes, which exhibit significant differences in edge structure, bending stiffness, and surface friction characteristics. When executing the initial strategy, the system quickly adjusts the control sequence by detecting anomalies such as minute slip signals and deformation / displacement of the gripping contact surface in real-time feedback. Furthermore, it suppresses oscillating feedback responses through a cooperative adaptation coefficient, ensuring that file grasping does not result in curling, stuttering, or file tearing, thus guaranteeing the complete transmission of financial data.

[0080] In elderly care environments, robots need to grasp various items such as plush toys, eyeglass cases, and metal kettles. While initial strategies provide basic parameters, uneven force field distribution often leads to gripping deviations when grasping soft objects. During execution, the system collects the deformation gradient trends and local pressure differences from tactile feedback and generates control sequences in real time to adjust the gripping surface shape. Simultaneously, it uses a cooperative adaptation coefficient to suppress instability caused by excessively rapid adjustments, ensuring that frequently used items are safely picked up without damage.

[0081] This embodiment introduces a constraint optimization solution mechanism to dynamically fuse the initial strategy structure with real-time haptic feedback data. The system can continuously iterate and generate new grasping control sequences during the grasping process, and introduces a cooperative adaptation coefficient between these sequences as an adjustment buffer, significantly enhancing the compliance, stability, and real-time feedback response capability of the grasping strategy. This process effectively avoids strategy bias caused by feedback disturbances, improves adaptability to heterogeneous object grasping scenarios, and achieves an efficient balance between control accuracy and execution safety.

[0082] S50, perform a grasping operation according to the grasping control sequence, and monitor real-time tactile feedback data and real-time mechanical feedback data during the execution process;

[0083] In this embodiment, the grasping control sequence is a set of control commands generated over time, covering execution parameters such as grasping force adjustment, finger joint posture adjustment, gripping speed limitation, and contact area expansion. This sequence was generated in the previous step based on the fusion and optimization of the initial strategy and feedback data, and possesses dynamic adjustment and real-time adaptability. In actual execution, the control sequence is sent item by item to the actuator array, including the multi-joint mechanical finger, flexible gripping unit, and grasping path tracking system, triggering each subsystem to perform the corresponding grasping operation.

[0084] During the grasping process, real-time tactile feedback data and real-time mechanical feedback data are collected synchronously and continuously input to the control module for analysis. The real-time tactile feedback data is collected through a distributed high-resolution capacitive sensor array, a piezoresistive flexible sensor network, or a fiber optic micro-displacement sensor array. The data includes pressure distribution matrices, contact stress fluctuation maps, slip characteristic scalars, and surface texture mapping features, which are used to reflect the contact uniformity, structural stability, and surface condition change trends of the gripping area.

[0085] Real-time mechanical feedback data is collected by torque sensors, strain gauges, and inertial measurement units in the robot's joints and connecting structures, covering physical indicators such as the rate of change of clamping force, the displacement velocity of the contact point, the response lag time of the actuator, and the accuracy of finger posture reconstruction. This type of data is used to capture the overall mechanical response caused by object inertia, structural rigidity, or posture deviation during the clamping process.

[0086] The real-time nature of both types of feedback data is ensured by a high-frequency data path and a synchronous timing control mechanism. Data is transmitted between the acquisition module and the execution module via a local bus communication protocol, triggering interrupt feedback. This allows the control system to promptly determine whether there are any abnormal risks such as clamping offset, contact surface slippage, or structural deformation based on the current execution status. Furthermore, the acquired feedback information is also cached in real-time at local edge nodes for subsequent strategy updates and dynamic modeling.

[0087] A closed-loop response structure is established between the grasping control sequence and feedback monitoring, which can immediately terminate the current control command or execute a rollback command when feedback anomalies occur, ensuring the steady-state controllability of the grasping process and the fault tolerance of the system. This mechanism is particularly suitable for grasping tasks with significant differences in the stiffness of the objects being grasped or with irregular structures, improving control accuracy and grasping success rate.

[0088] In one implementation, the grasping control sequence is controlled by a real-time scheduling module and sent to the drive execution end in millisecond-level time steps. Within each time step, the tactile and mechanical feedback states are updated by a high-speed acquisition module. Once the slippage index in the feedback state is found to exceed the safety threshold, a control backoff command is triggered to reduce the grasping force to the previous stable value and adjust the finger posture structure to prevent slippage or overload damage.

[0089] In another implementation, real-time mechanical feedback continuously collects the clamping force vector and torque response through a six-dimensional force sensor located at each joint axis, and compares it with the clamping path predicted by the model. If the path deviation exceeds the tolerance range, the finger movement rate and clamping angle sequence in the control sequence are dynamically adjusted to achieve real-time compensation control of the grasping path.

[0090] Furthermore, by constructing a tactile-mechanical joint feedback mapping map, the offset trend between the two types of feedback data can be detected during execution, and the trend can be used to predict the upcoming clamping stability risks, thereby adjusting the control sequence content of the next stage in advance and enhancing the early warning response capability in high-risk object grasping tasks.

[0091] Example: In healthcare scenarios, robots may need to grasp fragile medicine bottles or irregularly shaped medicine bags when performing tasks. By executing grasping control sequences and collecting tactile feedback data such as pressure overload and micro-deformation of sliding edges in real time, as well as joint torque mutation information in mechanical feedback, the system can quickly determine whether the grasp is stable and automatically reduce the grasping force or adjust the posture to prevent the medicine from breaking or falling, ensuring the safety of the medication retrieval process.

[0092] In the financial sector, service robots require extremely high precision and protection when handling users' bank cards, paper documents, and contracts. By implementing control sequences and monitoring real-time feedback on uneven stress distribution or abnormal tactile fluctuations during the gripping process, they can automatically identify problems such as card edge deformation and document misalignment, and adjust strategies to prevent document tearing or card damage, thus ensuring the integrity of document data.

[0093] In elderly care scenarios, when robots assist seniors in grasping plush toys, fragile items, or irregular daily necessities, the system can sense minute tactile disturbances and unstable posture signals during the grasping process and immediately adjust the gripping structure or reduce the contact intensity to reduce the risk of accidental injury and improve the gentleness and safety of the grasping experience.

[0094] This embodiment links the grasping control sequence with real-time feedback monitoring, enabling the system to not only perform various grasping operations based on predetermined parameters but also dynamically sense minute deviations and stress anomalies during execution. The combined monitoring of tactile and mechanical feedback overcomes the problems of distortion and high latency associated with single feedback channels, effectively improving response accuracy, flexibility, and safety control during the grasping process. It is particularly suitable for grasping objects with complex shapes, significant material differences, or sensitive surfaces.

[0095] S60, based on the deviation between the real-time mechanical feedback data and the optimal gripping force, and the deviation between the real-time tactile feedback data and the reference tactile features, the gripping force and posture are dynamically adjusted in conjunction with the cooperative adaptation coefficient.

[0096] In this embodiment, during the grasping process, the deviation between the real-time mechanical feedback data and the optimal grasping force is used to characterize the degree of deviation between the current grasping behavior and the expected optimal state at the mechanical level, constituting an important measure of grasping execution accuracy. The optimal grasping force can be generated by a previously constructed grasping model, which is modeled based on multiple factors such as the object's texture, mass, and contact surface structure. It is usually a multi-dimensional vector describing the ideal clamping force at each contact point. The real-time mechanical feedback data is continuously collected through a six-dimensional force sensor, strain gauge array, or inertial unit integrated at the end of the hand or knuckles, forming a high-resolution mechanical time-series stream within the control cycle to express the actual force dynamics during the grasping process.

[0097] Deviation can be calculated using methods such as sequential interpolation, Euclidean distance measurement, and cosine similarity analysis to ensure that it can characterize both the total error and reflect structural shifts in the mechanical distribution. This deviation is used to determine whether there is local overload or insufficient clamping force, and forms a quantitative input for subsequent attitude adjustment or force redistribution.

[0098] Meanwhile, the deviation between real-time haptic feedback data and reference haptic features is used to describe the stability and matching of the grasping process in the haptic perception dimension. Reference haptic features can be extracted from high-quality contact segments in historical grasping processes and typically include structured representations such as pressure distribution maps, stress transmission trajectories, and contact surface texture response curves. Real-time haptic data comes from distributed flexible haptic arrays, fiber optic interferometer arrays, or piezoelectric response layers. This data can be encoded as a graph or a set of high-dimensional vectors, and the deviation is calculated using algorithms based on graph matching, tensor transformation, or convolutional similarity analysis.

[0099] The two deviations mentioned above are simultaneously input into the dynamic adjustment module, which further combines the co-adaptation coefficient to generate a joint adjustment strategy for gripping force and posture. The co-adaptation coefficient is used to characterize the degree of adaptive coupling between tactile information and mechanical behavior. Its value can be dynamically adjusted according to the feature category of the target object, the stability of the gripping environment, and the historical performance of the system, supporting a balance between stability and responsiveness in complex contact tasks.

[0100] The gripping force adjustment is based on a weighted mapping of the current mechanical deviation and tactile deviation. A new target gripping force value is generated through linear gain, adaptive gain, or fuzzy control, and then gradually transitioned to this target value using PID, robust control, or model predictive control methods. Posture adjustment dynamically adjusts finger joint angles, gripping surface normal vectors, and contact envelope shape based on the directional characteristics of the tactile deviation, ensuring stable gripping while adapting to changes in different object surfaces and preventing slippage, twisting, or force field shift.

[0101] The above adjustment process operates in a closed loop with periodic feedback. Each control cycle updates the output based on the latest deviation value and the coordination coefficient, thus forming a high-frequency, low-latency dynamic adjustment control logic.

[0102] In one implementation, the system calculates the vector difference between the currently acquired mechanical feedback and the optimal gripping force in each control cycle. The calculation result generates a gripping force correction amount through a gain adjustment function and is applied to each execution joint through a distributed drive system to achieve synchronous adjustment of the clamping force field.

[0103] In another implementation, tactile feedback is analyzed in the form of a two-dimensional pressure map. Its spatial features are extracted by a graph neural network and compared with a reference map. The difference between the local slip risk area and the average contact stability is calculated. This difference is used to guide the control system to adjust the normal direction of the gripping surface and further optimize the gripping posture.

[0104] It can also construct a tactile-mechanical deviation joint mapping map, mapping the two deviations to the direction and intensity of control vectors in a three-dimensional adjustment space, and then adjust the adjustment range according to the current cooperative adaptation coefficient, supporting adaptive grasping control of flexible bodies, high friction coefficients or locally uneven objects.

[0105] Example: In a healthcare scenario, when a nursing robot grasps a patient's plastic oxygen tube or silicone device, it analyzes the deviation between the grasping force and the actual feedback in real time. If the tactile map shows signs of uneven surface compression or slippage, the system can reduce the local clamping force and adjust the clamping angle based on the co-adaptation coefficient to prevent indentation or deformation of the device surface.

[0106] In financial services scenarios, when service robots automatically distribute bank cards or process paper checks, if real-time mechanical feedback shows that the clamping force exceeds the preset safety range and tactile distribution shows that the pressure is concentrated at the edge of the object, the system will adjust the clamping strength and finger tilt angle together based on the deviation value and adaptation coefficient to ensure that the documents are not damaged and are accurately positioned.

[0107] In elderly care scenarios, when assistive robots grasp ceramic cups or plush items, slight deviations between tactile and mechanical feedback can indicate uneven grasping force or posture deviation risks. By adjusting the grasping posture angle and the force application strategy of each actuating finger, the system can maintain safe gripping while preserving the structural integrity and comfort of the grasped object, thus improving the humanization and safety of the grasping experience.

[0108] This embodiment combines mechanical and tactile feedback deviation signals for joint analysis, and dynamically adjusts the grasping force and posture using a cooperative adaptation coefficient. This effectively bridges the system response gap between modeling and execution errors, improving the accuracy, compliance, and stability of grasping control. This mechanism is particularly suitable for dynamic response scenarios involving complex tactile objects, enabling adaptive control of the grasping process without relying on prior manual parameter tuning, significantly reducing object breakage rates and the probability of gripping failure.

[0109] This invention relates to the field of artificial intelligence technology and can be applied to business scenarios such as elderly care services, fintech, and healthcare. It discloses a grasping control method, device, equipment, and medium based on collaborative grasping features, including: acquiring surface tactile data, interactive mechanical data, and physical parameters of a target object; generating collaborative grasping features and feature weight vectors based on the above data; constructing a grasping model by combining historical grasping data; generating an initial grasping strategy; combining the initial grasping strategy with real-time tactile feedback data during the grasping process; generating a grasping control sequence and collaborative adaptation coefficients through constraint optimization; executing the grasping operation and monitoring real-time feedback data during execution; and dynamically adjusting the grasping force and posture based on feedback deviation and collaborative adaptation coefficients to achieve precise grasping control of different objects. This invention solves the problem of inflexible grasping response in existing technologies by constructing collaborative grasping features and a grasping model, integrating real-time feedback and control strategies, achieving dynamic adaptation and fine adjustment during the grasping process of different objects, and reducing the risk of damage.

[0110] In one embodiment, step S10 above includes:

[0111] S101 collects surface tactile data, including pressure distribution data and contact area data of the target object, through an array of tactile sensors.

[0112] S102 collects interactive mechanical data, including the normal force and tangential force applied to the target object during the grasping process, through a force sensor;

[0113] S103, Obtain physical parameters including the size and weight information of the target object;

[0114] S104, Gaussian filtering is performed on the pressure distribution data in the surface tactile data to generate filtered pressure distribution data;

[0115] S105, perform zero-drift error calibration on the normal force and tangential force in the interactive mechanical data to generate calibrated normal force and calibrated tangential force;

[0116] S106, Determine the surface roughness of the object based on the filtered pressure distribution data;

[0117] S107, Determine the object hardness coefficient based on the calibrated normal force and contact area;

[0118] S108, Generate a pressure-deformation relationship curve based on the calibrated normal force and contact deformation, and determine the elastic modulus of the object based on the pressure-deformation relationship curve;

[0119] S109, Determine the object's inertial parameters based on the size and weight information;

[0120] S110, Analyze the spatial frequency characteristics of the surface tactile data, and determine the surface texture complexity parameters of the object based on the spatial frequency characteristics;

[0121] S111, Analyze the real-time rate of change of the interactive mechanical data, and determine the dynamic friction characteristic coefficient of the object surface based on the real-time rate of change;

[0122] S112, Integrate the object surface roughness, object hardness coefficient, object elastic modulus and object surface texture complexity parameters to generate a tactile feature vector;

[0123] S113, integrate the calibrated normal force, calibrated tangential force, contact area, object inertial parameters and dynamic friction coefficient to generate an initial mechanical parameter vector.

[0124] In this embodiment, the acquisition of surface tactile data of the target object relies on an array of tactile sensors. These sensors are deployed at multiple points within the contact area of ​​the clamping mechanism, possessing high spatial resolution and a high temporal sampling frequency. The acquisition of pressure distribution data reflects the force conditions on the object's surface at different contact points, providing crucial foundational data for expressing contact uniformity and surface reaction force distribution. Simultaneously, the spatial arrangement of the array units allows for the measurement of the tactile response area, thereby generating a numerical estimate of the contact area and providing geometric dimensional information for subsequent material hardness analysis and contact modeling.

[0125] The acquisition of interactive mechanical data is accomplished through multi-axis force sensors integrated into the fingers, palm, or connecting joints. Measurements include normal and tangential forces acting on the object. Normal force reflects the degree of clamping and compression, while tangential force reflects the object's slippage tendency, frictional resistance, and shear stress behavior. Together, they constitute the mechanical state parameters describing the interaction between the robot and the object. The raw mechanical data is affected by system baseline drift, temperature drift, and electromagnetic interference, requiring zero-drift error calibration. Calibration methods include static benchmark determination, recursive sliding mean difference compensation, or dynamic drift estimation based on Kalman filtering, outputting calibrated normal and tangential forces for subsequent stability modeling.

[0126] Acquiring physical parameters involves measuring the size and weight of the target object. Size information can be obtained through a 3D vision reconstruction system, a structured light depth camera, or a contact encoder, yielding geometric dimensions such as length, width, height, surface area, and volume. Weight information can be acquired through a weighing module or load sensor integrated under the gripper or base platform. These two types of information together constitute the input to the configuration space, serving as the basis for subsequent dynamic modeling and inertial response estimation.

[0127] Gaussian filtering is applied to the pressure distribution data in surface tactile data to eliminate high-frequency interference caused by local sensor noise, mechanical vibration, or instantaneous non-uniform contact. Gaussian filtering acts on the array data in the form of two-dimensional spatial convolution, smoothing outomas and preserving the true contact trend, thereby improving the stability of subsequent parameter extraction. The filtered pressure distribution data accurately reflects the continuity of the contact response of the object surface, providing a basis for calculating surface roughness. Roughness parameters can be estimated through the rate of change of local pressure gradient, the distribution density of pressure peaks, or the curvature of the contact equipotential profile; the calculation method can be adapted to the resolution of different tactile arrays and the characteristics of the contact medium.

[0128] The determination of the hardness coefficient relies on the joint analysis of normal force and contact area. In actual contact, harder objects exhibit smaller contact areas under the same normal force, while softer objects exhibit the opposite. Based on Hertz contact theory or a nonlinear elastic extension model, a ratio coefficient that quantitatively expresses hardness can be derived to characterize the material's resistance to normal compression. This coefficient can serve as one of the reference factors for subsequent force-controlled gain adjustment.

[0129] The calculation of the elastic modulus is based on the relationship between pressure and deformation. Contact deformation can be obtained through tactile array sensing of depth changes, visual 3D reconstruction, or surface profile change inversion. By plotting the calibrated normal force and deformation as a two-dimensional relationship curve, the slope of the stress-strain relationship of the object can be obtained using linear fitting, segmental function fitting, or exponential curve regression, thereby extracting an approximate elastic modulus parameter that represents the compressibility of the object under external force.

[0130] Inertial parameters are derived from standard physics based on size and weight information. The combination of mass and size can be used to calculate the object's moment of inertia, center of mass position, and inertial tensor in different directions, providing input for inertial compensation and dynamic control during the grasping process.

[0131] The surface texture complexity parameter is derived from the spatial frequency characteristics of surface tactile data. Spatial frequency can be obtained by performing a frequency domain transformation on the pressure distribution map using Fast Fourier Transform (FFT), wavelet analysis, or sparse coding, yielding the ratio distribution of high-frequency to low-frequency components in the texture structure. Objects with high texture complexity exhibit a high proportion of high-frequency components and a dispersed spectral density, while objects with low complexity have a concentrated spectrum and small fluctuations. This parameter is used to determine the differences in object surface structure, providing input for tactile pattern recognition and anti-slip control.

[0132] The dynamic friction coefficient of an object's surface is determined by the real-time rate of change of interactive mechanical data. By analyzing the gradient, peak ratio, and relative phase changes of normal and tangential forces along the time axis, behavioral indicators such as frictional hysteresis and nonlinear dynamic characteristics of the object during sliding can be extracted and normalized into a dynamic friction coefficient. This coefficient reflects the force response characteristics of the object's surface in the initial, middle, and later stages of slippage, and is used to identify potential slippage, vibration response, and material composition properties.

[0133] Ultimately, the tactile feature vector integrates four types of tactile perception parameters—roughness, hardness coefficient, elastic modulus, and texture complexity—to construct a high-dimensional representation space for characterizing the detailed features of an object's contact response. The initial mechanical parameter vector integrates five types of dynamic mechanical data: calibrated normal force, calibrated tangential force, contact area, inertial parameters, and friction coefficient, depicting the interaction structure between the current object and the external clamping force. These two sets of vectors serve as high-quality inputs for subsequent modeling and strategy generation, providing a structured foundation of tactile and mechanical features for the grasping task.

[0134] This embodiment improves the modeling accuracy of the target object's material properties, surface characteristics, and mechanical response behavior by simultaneously fusing surface tactile data, interactive mechanical data, and physical parameters during the data acquisition phase, and by introducing filtering, calibration, analysis, and feature extraction processes during the preprocessing phase. The system no longer relies on static configurations or manually set prior parameters, but instead generates high-dimensional tactile and mechanical feature representations in real time based on each grasping task, thereby constructing a basic data representation capability with dynamic adaptability and structural scalability. Through this multi-source data collaborative analysis and feature generation process, a precise, rich, and generalizable input foundation is provided for subsequent grasping strategy construction, significantly improving the stability, adaptability, and security of the grasping behavior.

[0135] In one embodiment, step S20 above includes:

[0136] S201, Obtain the tactile feature vector and initial mechanical parameter vector generated based on the surface tactile data and interactive mechanical data;

[0137] S202, perform principal component analysis to reduce the dimensionality of the tactile feature vector and the initial mechanical parameter vector to obtain the dimensionality-reduced tactile features and the dimensionality-reduced mechanical parameters, respectively.

[0138] S203, determine the correlation coefficient between the dimension-reduced tactile features and the dimension-reduced mechanical parameters;

[0139] S204, Based on the correlation coefficient, determine the tactile feature weight coefficient and the mechanical parameter weight coefficient by the entropy method;

[0140] S205, adjust the tactile feature weight coefficient and the mechanical parameter weight coefficient based on the physical parameters;

[0141] S206, The dimensionality-reduced tactile features are weighted using the adjusted tactile feature weight coefficients to generate weighted tactile features;

[0142] S207, The reduced mechanical parameters are weighted using the adjusted mechanical parameter weighting coefficients to generate weighted mechanical parameters;

[0143] S208, integrate the weighted tactile features with the weighted mechanical parameters to construct a collaborative grasping feature matrix;

[0144] S209, determine the variance contribution rate of each feature in the collaborative capture feature matrix;

[0145] S210, determine the feature time stability index based on historical crawled data;

[0146] S211, Generate a feature weight vector based on the variance contribution rate and time stability index.

[0147] In this embodiment, obtaining tactile feature vectors and initial mechanical parameter vectors generated based on surface tactile data and interactive mechanical data is a prerequisite for transforming physical perception data into a structured representation. The tactile feature vectors originate from the comprehensive encoding of parameters such as pressure distribution, surface texture, surface roughness, elastic modulus, and hardness coefficient, possessing a clear ability to express spatial texture and material properties. The initial mechanical parameter vectors include dimensions such as normal force, tangential force, dynamic friction characteristics, contact area, and object inertia, reflecting the dynamic interaction state between the robot and the object. These two types of vectors respectively reflect static material properties and dynamic interaction properties, and are necessary inputs for constructing a collaborative representation structure.

[0148] Principal Component Analysis (PCA) is performed on the two vectors to reduce their dimensionality, extracting orthogonal feature components with the highest information content from the original high-dimensional features. The process first calculates the covariance matrix of the input vectors and solves for their eigenvalues ​​and eigenvectors. Principal components with a cumulative contribution rate higher than a preset threshold (e.g., 95%) are retained, resulting in the dimensionality-reduced tactile features and mechanical parameters. This compressed representation reduces the interference of redundant features on subsequent learning or control while preserving the main physical change trends and high-frequency information.

[0149] To establish the intrinsic coupling relationship between tactile and mechanical features, it is necessary to calculate the correlation coefficient matrix between the dimensionality-reduced tactile features and the dimensionality-reduced mechanical parameters. This matrix is ​​calculated using the Pearson correlation coefficients between the dimensional features, with values ​​ranging from -1 to 1, representing the strength and direction of positive and negative correlations. Features with significant correlations play a synergistic regulatory role in the final grasping behavior and should be assigned a higher sensitivity factor in the weighting evaluation.

[0150] Next, the entropy method is used to assign feature weights to the aforementioned correlation coefficients. The entropy method is an objective weighting method based on information entropy measurement, which determines the relative importance of each feature by analyzing its distribution dispersion and information content. Specifically, it involves normalizing the correlation coefficient matrix, calculating the information entropy and redundancy of each feature, and then calculating the weights. The resulting tactile feature weight coefficients and mechanical parameter weight coefficients reflect their respective contributions to the joint expression, forming a unified metric without relying on prior assumptions.

[0151] However, weighting solely based on correlation distribution may overlook the impact of object properties on the importance of grasping. Therefore, physical parameters need to be introduced as adjustment factors to further refine the initially obtained weight coefficients. Larger, heavier, or centroid-shifted objects require higher force control precision and surface stability during operation, necessitating increased weighting for dimensions such as inertia parameters or roughness. Conversely, for structurally stable, uniformly textured objects, the sensitivity of feature dimensions can be appropriately reduced. Adjustment operations can be performed based on physical parameter mapping functions, using linear weighting, piecewise mapping, or dynamic factor adjustment based on cluster analysis.

[0152] Weighted tactile features are obtained by weighting the dimensionality-reduced tactile features using adjusted tactile feature weighting coefficients. This process involves using the weighting coefficients as proportional adjustment factors for each dimension of the feature, multiplying them dimension-by-dimensional with the corresponding reduced feature, and then summing the results to form a tactile vector that highlights key changing dimensions and suppresses noise dimensions. Similarly, weighting the dimensionality-reduced mechanical parameters with adjusted mechanical parameter weighting coefficients yields weighted mechanical parameters, which enhance dynamic interactive attributes more directly related to grasping behavior.

[0153] The fusion of weighted tactile features and weighted mechanical parameters to obtain a collaborative grasping feature matrix is ​​a key step in constructing a unified control mapping space. Each row or column of this matrix can be viewed as a joint encoding of tactile and mechanical states in a specific dimension, or abstracted as a projection of the current grasping target into a multimodal perception space. This matrix structure supports its subsequent use as a high-dimensional input in policy modeling, similarity search, and policy transfer.

[0154] To quantify the role of each feature dimension in the joint representation, the variance contribution rate of each feature in the collaborative crawling feature matrix needs to be calculated. By analyzing the distribution dispersion of this feature in different crawled samples, its contribution to the discrimination and expressive power of the final strategy can be evaluated. The larger the variance, the stronger the feature's ability to distinguish different scenarios, and a higher proportion of dynamic weight should be assigned to it.

[0155] At the same time, the stability of features over time cannot be ignored. Therefore, a feature temporal stability index is introduced to measure the fluctuation range of the feature across multiple time windows, multiple objects, or multiple rounds of interaction. Features with high stability indicate that their physical meaning is stable, their perceptual error is small, and their generalization ability is strong, making them suitable as reliable control references in long-term crawling behavior.

[0156] Finally, a feature weight vector is constructed by integrating the variance contribution rate and the time stability index. This weight vector not only reflects the information distribution among feature dimensions but also introduces a time consistency dimension, making it a composite weighted representation with physical interpretability, statistical stability, and task orientation. This weight vector can be used to guide model structure optimization, control parameter selection, or feedback adjustment strategy generation, providing a stable, effective, and adaptive input foundation for subsequent crawling tasks.

[0157] This embodiment achieves joint modeling of object grasping behavior based on multi-source sensory data by performing dimensionality reduction, correlation analysis, weighted fusion, and feature stability quantification on tactile and mechanical vectors. This process not only solves the problems of high redundancy, strong feature interference, and task mismatch in multimodal data, but also dynamically quantifies the effectiveness and temporal reliability of each feature in actual tasks while constructing collaborative grasping features. Through entropy weight adjustment and physical parameter correction, the final generated feature weight vector possesses both statistical rationality and reflects the task constraints of specific object attributes, thus providing a more accurate and stable input foundation for the subsequent construction of grasping models and improving the grasping success rate and operational safety in complex scenarios.

[0158] In one embodiment, step S30 above includes:

[0159] S301, based on the collaborative grasping feature matrix and feature weight vector, constructs a grasping strength regression model;

[0160] S302, Based on the feature weight vector, select high-importance features from the collaboratively captured feature matrix;

[0161] S303, Based on the aforementioned high-importance features and historical crawling data, construct a contact point distribution decision model;

[0162] S304. Based on the aforementioned high-importance features and historical grasping data, a finger posture decision model is constructed, and a finger joint motion continuity constraint is added during the construction process.

[0163] S305, Analyze the spatial stability of the contact point distribution and obtain the spatial stability analysis results;

[0164] S306, Determine the optimal gripping force through the gripping force regression model;

[0165] S307, dynamically adjust the safety factor based on the material type of the object to obtain the adjusted safety factor;

[0166] S308, Multiply the optimal gripping force by the adjusted safety factor to obtain the maximum allowable gripping force;

[0167] S309, Determine the coordinates of the contact point distribution based on the spatial stability analysis results and the contact point distribution decision model;

[0168] S310, Determine finger posture parameters based on the finger joint motion continuity constraint and finger posture decision model;

[0169] S311, use historical grasping data to verify the prediction accuracy of the grasping force regression model, contact point distribution decision model and finger posture decision model, and adjust the model parameters based on the verification results;

[0170] S312, Generate an initial gripping strategy that includes the optimal gripping force, the maximum allowable gripping force, the contact point distribution coordinates, and the finger posture parameters.

[0171] In this embodiment, constructing a grasping force regression model based on a collaborative grasping feature matrix and feature weight vector is a key step in achieving accurate force control prediction. The model's input consists of multi-dimensional joint features, including tactile and mechanical features after physical interpretation enhancement and information redundancy compression, along with corresponding weighting coefficients. By employing methods such as support vector regression, random forest regression, or multi-layer neural networks, the model learns the mapping relationship between input features and required grasping force from existing historical grasping data, effectively generating targeted force control predictions for objects with different materials, structures, and dynamic properties. This model not only outputs numerical results but also possesses a certain degree of uncertainty assessment capability, providing a basis for subsequent safety factor adjustments.

[0172] Selecting highly important features from the collaboratively extracted feature matrix based on feature weight vectors is a necessary step to improve model training efficiency and generalization ability. Highly important features are typically characterized by large weight coefficients, strong temporal stability, and high sensitivity to the extraction results. By setting thresholds or employing selection mechanisms assisted by statistical indicators such as information gain and mutual information, the most representative set of features can be extracted as input for model training and decision-making, thereby reducing the risk of model overfitting and improving the response speed and adaptability of the decision-making system.

[0173] A contact point distribution decision model is constructed based on high-importance features and historical grasping data to predict the spatial layout of contact between the robotic arm and objects. This model determines the contactable area, mechanically stable area, and multi-point cooperative area on the object's surface based on high-dimensional feature input, and outputs a set of optimal contact point combinations. These contact points must not only satisfy mechanical stability but also consider grasping efficiency and control convenience. During the modeling process, graph neural networks can be introduced to model the geometric relationships between contact points, or Bayesian networks can be used to infer and analyze the stability and failure risk of contact points.

[0174] Building a finger posture decision model based on shared high-importance features and historical grasping data is a prerequisite for collaborative motion planning. This model needs to predict the spatial posture of each finger during grasping, including parameters such as relative pose, spread angle, and rotation direction. Since the finger posture sequence must satisfy joint continuity and motion physics constraints, biomechanical constraint rules or a path continuity penalty term based on dynamic programming should be embedded in the model. Specifically, a combination of trajectory prediction networks and autoregressive models can be used to ensure the smoothness and executability of the output posture sequence over time.

[0175] To improve the practical effectiveness of contact point prediction, the spatial stability of the contact point distribution needs to be analyzed. This analysis can statistically analyze the range of contact point variations across multiple object grasping samples or multiple trials, measuring its spatial variance or center of gravity drift to form a quantitative spatial stability index. Higher spatial stability indicates that the contact area is more versatile and provides more stable support in grasping tasks, and can be prioritized as an output area.

[0176] After model construction is completed, the optimal gripping force is predicted using a gripping force regression model. This involves regression prediction aimed at minimizing energy consumption or maximizing contact efficiency while ensuring stable object support. Subsequently, a safety factor is dynamically adjusted based on the object's material type. Different materials are mapped to a set of safety factors using a material mapping table or a data-driven risk model. For example, fragile materials such as glass or ceramics should have a higher safety factor, while wood or plastic materials can tolerate larger fluctuations in gripping force. The safety factor can be determined using a lookup table or dynamically estimated based on perceived data.

[0177] The optimal gripping force is multiplied by the adjusted safety factor to obtain the maximum permissible gripping force. This force limit restricts the maximum gripping force output by the control system, preventing damage to objects caused by excessive gripping force due to modeling errors or sensor noise. This upper limit of force is input to the execution module as a control boundary condition during the execution phase.

[0178] By combining the output of the contact point distribution decision model with the results of spatial stability analysis, the final contact point distribution coordinates are determined. This ensures that the selected contact points can maximize the use of the stable area of ​​the object surface while avoiding problems such as contact failure or eccentric grasping. Similarly, by combining the continuity constraints of finger joint motion with the output of the posture decision model, finger posture parameters that conform to the laws of motion and the coherence of execution are determined, making the entire control action physically executable and energy consumption reasonable.

[0179] To ensure the reliability of the models in real-world scenarios, historical crawling data was used to validate the three models. Validation included prediction accuracy, actual crawling success rate, and posture execution error. Based on the validation results, the model parameters were fine-tuned, including adjustments to the loss function in the regression model, the redistribution of feature weights in the decision model, and the adjustment of the regularization term in the posture model.

[0180] Finally, the above parameters are integrated to generate an initial grasping strategy. This strategy encompasses stability, force control accuracy, spatial coordination, and dynamic continuity. It is a set of parameters oriented towards the execution system and can be directly used by the controller to drive the robot arm to complete the grasping task.

[0181] This embodiment constructs a multi-model collaborative grasping decision structure, comprehensively considering three control dimensions: grasping force prediction, contact point layout, and finger posture, achieving high-precision modeling and motion output of the object grasping process. Multi-source data input combined with a high-importance feature filtering mechanism effectively reduces redundant information interference, improving model training efficiency and output stability. The fusion of dynamic output from the force control model and a material property-driven safety factor adjustment mechanism enables the grasping force to adapt to different object characteristics while possessing flexible protection capabilities. The introduction of joint continuity constraints in posture modeling improves the smoothness and biocompatibility of the robotic arm's movements during execution, while the stability analysis mechanism in the contact point layout effectively reduces the contact failure rate.

[0182] In one embodiment, step S40 above includes:

[0183] S401, Obtain the optimal grasping force, maximum allowable grasping force, contact point distribution coordinates, and finger posture parameters in the initial grasping strategy;

[0184] S402, acquire real-time tactile feedback data during the grasping process;

[0185] S403, determine the characteristics of the reference contact area based on the coordinates of the contact point distribution;

[0186] S404, Construct a gripping force tracking target to minimize the deviation between real-time gripping force and optimal gripping force;

[0187] S405, Construct a tactile feature matching target to minimize the deviation between real-time tactile features and reference contact area features;

[0188] S406, Construct an objective function that includes the target for grasping force tracking and the target for tactile feature matching;

[0189] S407, Set the weight coefficients of the grasping force tracking target and the tactile feature matching target;

[0190] S408, Set a force constraint that the gripping force does not exceed the maximum allowed gripping force;

[0191] S409, Set the posture constraints of the finger posture parameters within a preset joint angle range;

[0192] S410 defines the prediction time domain range;

[0193] S411, within each control cycle in the prediction time domain, an optimization problem is solved based on the objective function, force constraint, and posture constraint to obtain the grasping force control amount and finger posture control amount for the control cycle.

[0194] S412, determine the degree of synergy between mechanical control and tactile control, and generate a synergy adaptation coefficient based on the degree of synergy;

[0195] S413 integrates the grasping force control and finger posture control from all control cycles to generate a grasping control sequence.

[0196] In this embodiment, the optimal gripping force, maximum allowable gripping force, contact point distribution coordinates, and finger posture parameters in the initial gripping strategy are obtained as the initial reference configuration for the control optimization process. This is a key prerequisite for constructing dynamic control based on static modeling results. Specifically, the optimal gripping force represents the ideal output for maintaining gripping stability and energy consumption balance under specific object conditions; the maximum allowable gripping force is the upper limit of physical constraints after mapping the object's material properties; the contact point distribution coordinates reflect the contact geometry between the fingers and the object; and the finger posture parameters provide information on the finger joint positions and angles, limiting the controllable range of the actuator in space. These parameters collectively define the reference trajectory and physical boundaries of the control output during subsequent optimization.

[0197] Acquiring real-time tactile feedback data during the grasping process is the perceptual foundation for achieving closed-loop grasping control. The collected data can include changes in pressure response on the contact surface, dynamic adjustments in local contact area, changes in texture sliding, and trends in minute deformations. By continuously sampling the contact area between the robot and the object using arrayed tactile sensors or flexible sensor patches, a multi-dimensional tactile feature flow evolving over time can be obtained to reflect the degree of difference between the current grasping state and the ideal strategy.

[0198] The purpose of determining the reference contact area features based on the contact point distribution coordinates is to construct a target perception pattern consistent with the execution strategy. This process involves extracting static tactile templates or statistical features from the contact point region, including average pressure gradient, pressure distribution eccentricity, and boundary contact stiffness, to characterize the spatial distribution characteristics of the ideal contact state. These reference contact area features will serve as a perception alignment target during the optimization process, used to determine whether the real-time perception state deviates from the original design intent.

[0199] Constructing a gripping force tracking target to minimize the deviation between the real-time gripping force and the optimal gripping force is a quantitative expression of the dynamic adjustment of force control output by the control system. Its objective function can be defined as the weighted squared error between the current gripping force and the target value, ensuring that the output gripping force always approaches the theoretical optimum and avoiding force control deviations caused by environmental disturbances or attitude deviations.

[0200] Constructing a tactile feature matching objective to minimize the deviation between real-time tactile features and reference contact area features is key to achieving perceptual consistency. By defining the difference between Euclidean distance or cosine similarity in the tactile feature space as a loss metric, the control system can reconstruct the ideal contact perception pattern as much as possible during execution. Especially under multi-point contact and dynamic friction conditions, this objective can significantly improve contact stability and local response balance.

[0201] Constructing an objective function that includes both grasping force tracking and tactile feature matching is a path aggregation method for achieving multi-objective collaborative optimization. The two sub-objective functions are combined using linear weighting or the Pareto front to form the main objective function, which can be expressed using convex combinations and possesses good stability and efficient solution properties. The weight coefficients of this objective function can be dynamically adjusted according to task priority; for example, increasing the weight of the force matching target when dealing with fragile objects, and increasing the proportion of the tactile consistency target when dealing with multi-contact grasping.

[0202] Setting a force constraint that prevents the gripping force from exceeding the maximum permissible gripping force is a key safety mechanism in physical execution systems. This constraint can be achieved by limiting the upper limit of the gripping force control quantity to the maximum value multiplied by a safety factor. This forms a convex constraint boundary when solving optimization problems, preventing the control result from exceeding the object's tolerance range.

[0203] Setting posture constraints for finger posture parameters within a preset joint angle range is a necessary constraint based on the biological structural limitations of the actuator. Upper and lower limits for the angles of each finger joint can be set in the model, while a second-order continuity constraint is introduced in the optimization solution to prevent abrupt posture changes or unacceptable torsional angles. This constraint enhances the practical executability of the control sequence and effectively prevents structural damage to the robotic arm due to excessive rotation during grasping actions.

[0204] Defining the prediction time domain range is a structural transformation operation that converts the control problem into an optimization problem in a finite time domain. This range is typically set to several hundred milliseconds to several seconds, covering the complete grasping action cycle. Decomposing the entire grasping time axis into multiple control cycles allows the optimization problem to be solved independently in each cycle, improving real-time performance and parallelism.

[0205] Within each control cycle, an optimization problem is solved based on the objective function, force constraints, and attitude constraints to obtain the grasping force control and finger attitude control quantities for the current cycle. The solution method can employ model predictive control (MPC) or mixed-integer optimization strategies, combining a forward dynamics model and a controller predictor to estimate future grasping states and generate optimal control commands.

[0206] Determining the synergy between force control and haptic control measures the degree of coordination among multiple channels within a control system. A synergy index is formed by statistically analyzing the consistency of their control output directions, the synchronization rate of control increments, and the consistency of the objective function's descent trend. This index quantifies the inherent coordination performance of the control system in terms of strategy consistency, preventing directional conflicts between force control and haptic control.

[0207] Based on the degree of synergy, a synergy adaptation coefficient is generated and used as an adjustment parameter of the control system to dynamically adjust the objective function weight, prediction time domain length, or execution frequency, thereby achieving flexible coordination and resource optimization of the control strategy.

[0208] By integrating the grasping force control and finger posture control from all control cycles, a complete grasping control sequence is generated. This sequence not only includes mechanical execution instructions but also integrates posture dynamics and perceived targets, achieving unified scheduling of multimodal control information. It is the direct output that ultimately drives the actuator to complete the precise grasping task.

[0209] This embodiment integrates static grasping strategies and dynamic feedback data to construct a multi-objective, constraint-driven optimized control structure, achieving comprehensive optimization of grasping control accuracy, posture coordination, and perception consistency. The introduction of a dual-objective function of force tracking and haptic matching effectively balances force control output with perceptual feature alignment, enhancing adaptability when grasping various object types. Setting force and posture boundary constraints ensures that the control output conforms to the feasibility and safety of the physical system, preventing overload and abnormal actions. Constructing a periodic optimization problem based on the predictive time domain allows for the introduction of forward-looking strategies while maintaining real-time performance. A coordination index enables consistency evaluation of the control path and drives adaptation coefficient updates, enhancing multi-channel collaboration efficiency. The final generated control sequence integrates structural constraints, physical boundaries, and perceptual feedback, exhibiting strong robustness, adjustable accuracy, and continuous action, effectively improving the success rate of grasping tasks in complex environments.

[0210] In one embodiment, step S50 above includes:

[0211] S501, parse the grasping force control quantity and finger posture control quantity in the grasping control sequence;

[0212] S502, the actuator is controlled to output a corresponding gripping force according to the gripping force control amount;

[0213] S503, monitor the actual output force of the actuator to obtain the actual gripping force value;

[0214] S504, the actuator is controlled to adjust the finger posture according to the finger posture control amount;

[0215] S505 detects the deviation between the actual position and the target position of the finger posture and obtains the posture deviation value;

[0216] S506 acquires real-time tactile feedback data through a tactile sensor array;

[0217] S507 acquires real-time mechanical feedback data through a multi-dimensional force sensor;

[0218] S508, perform timestamp alignment processing on the real-time tactile feedback data and the real-time mechanical feedback data to generate aligned tactile feedback data and aligned mechanical feedback data;

[0219] S509, Detect the validity of the aligned tactile feedback data and the aligned mechanical feedback data;

[0220] S510, when the aligned tactile feedback data is invalid or the aligned mechanical feedback data is invalid, the data anomaly handling mechanism is triggered.

[0221] In this embodiment, parsing the grasping force control and finger posture control quantities in the grasping control sequence is a crucial process for the execution module to understand the upper-level optimization results. This sequence is typically expressed as a time-series array. The grasping force control quantity represents the force vector required for each control cycle, including the applied direction, force magnitude, and rate of change. The finger posture control quantity encompasses the angle adjustment range, rotation direction, and target structural parameters of each joint. The parsing operation decodes the control commands through a parser or middleware, breaking them down into motion parameters recognizable by the low-level hardware interface, ensuring consistent units and encoding standards to avoid misparsing or control conflicts.

[0222] The control system controls the output gripping force of the actuator based on the gripping force control quantity, which involves the force adjustment module of the actuator or the power dispatch controller of the piezoelectric / hydraulic output unit. In specific implementation, the control system converts the analyzed force vector into target signals for each drive unit, such as servo drive voltage or pressure valve opening, and performs dynamic compensation in combination with the current load state and the desired output curve to ensure that the output gripping force converges on the target trajectory. Especially in non-rigid structures or multi-point contact environments, its stability control capability becomes critical.

[0223] Monitoring the actual output force of the actuator to obtain the actual grasping force value is the data source for forming the basis of closed-loop control. The acquisition of the actual output force typically relies on embedded force sensor arrays or integrated tactile sensing units. Dynamic tracking of the output force is achieved by periodically reading the mechanical response of the actuator at each contact point. This process must possess high precision and low latency characteristics to prevent control deviations caused by control delays, and should also implement error compensation mechanisms for force value offsets, mechanical hysteresis, or abnormal drift.

[0224] The control device adjusts the finger posture based on the finger posture control parameters, requiring the finger drive structure to have high-resolution angle control capabilities. The control system maps the posture control parameters to angle adjustment signals from the joint motors or servo modules, adjusting the spatial position of the finger structure in real time. In flexible actuators or multi-degree-of-freedom gripper structures, this adjustment process must consider the effects of linkage coupling and posture continuity constraints to ensure smooth finger trajectory, seamless rotation, and to avoid posture oscillations or mechanism interference during grasping.

[0225] Detecting the deviation between the actual and target finger positions is a crucial indicator for evaluating execution accuracy. Using an angle encoder, optical tracking system, or IMU module, the spatial pose of each joint or phalanx is acquired, and the difference between this pose and the target pose is calculated to form a pose deviation value. This deviation value not only reflects control accuracy but also serves as feedback input for adaptive adjustment of subsequent actions, ensuring that the control output maintains a minimum error state during closed-loop adjustments.

[0226] Acquiring real-time tactile feedback data through tactile sensor arrays is the perceptual foundation for recognizing and controlling the grasping state. These arrays are typically deployed on fingertips, joints, or contact surfaces, and can collect multi-dimensional tactile indicators such as contact pressure, surface texture friction, sliding displacement, and local deformation. The sensor array should possess high temporal resolution and spatial sampling accuracy to prevent slippage, misalignment, or contact distortion during the grasping process.

[0227] Real-time mechanical feedback data is acquired through multi-dimensional force sensors to supplement the monitoring of the overall force control state. These sensors can sense parameters such as normal force, tangential force, and torque distribution applied to an object, providing information on the mechanical evolution of the macroscopic contact structure. They are particularly suitable for detecting problems such as overall grip stability, center of gravity shift response, or abnormal friction conditions.

[0228] Timestamp alignment of real-time haptic feedback data and real-time mechanical feedback data is a key technical step to ensure the consistency and synchronization of multi-channel sensing information. Different sensing channels often have problems such as sampling delay, transmission path differences, and inconsistent response speeds. These issues must be corrected through unified clock synchronization, data interpolation compensation, or alignment sliding window algorithms to enable joint analysis of the two types of data within a unified time frame, thereby improving the overall system's data fusion accuracy and timeliness.

[0229] Detecting the validity of aligned tactile and mechanical feedback data is a crucial mechanism to ensure the reliability of subsequent analysis, judgment, and control output. Validity assessment criteria can include multiple dimensions such as data noise level, signal stability, response amplitude range, and sampling integrity. By combining threshold judgment, data redundancy verification, and contextual history feature analysis, abnormal data segments, data loss, or signal drift can be quickly identified, preventing erroneous feedback from causing the accumulation of control errors.

[0230] When the haptic feedback data or mechanical feedback data becomes invalid after alignment, a data anomaly handling mechanism is triggered. This mechanism performs actions such as switching sensor channels, retrieving historical prediction data for state estimation, issuing alarm signals, or entering a safe stop mode to prevent unstable outputs or execution risks in the control system due to data anomalies. This mechanism should possess rapid response capabilities and multi-level response strategies, supporting soft recovery under minor anomalies and hard protection under severe anomalies, thus ensuring both robustness and safety of system operation.

[0231] This embodiment maps the force control and posture control quantities parsed from the grasping control sequence to the execution device, achieving synchronous control of mechanical movements and spatial pose. Real-time tactile and mechanical feedback is acquired through a multi-channel sensing system and precisely aligned, effectively supporting dynamic perception and accurate identification of the grasping state. Feedback on real-time posture deviation and actual output force supplements the control accuracy closed loop, improving the control system's responsiveness in complex grasping scenarios. The introduction of data validity verification and anomaly handling mechanisms significantly enhances the system's robustness against data anomalies, sensor failures, or signal drift, ensuring the stability and safety of the control link during actual execution. Through multi-dimensional fusion control and dynamic monitoring of the grasping execution process, the overall grasping strategy's execution accuracy and environmental adaptability are significantly improved, making it suitable for demanding scenarios such as medical rehabilitation, flexible manufacturing, and precision assembly.

[0232] In one embodiment, step S60 above includes:

[0233] S601, determine the deviation between the real-time mechanical feedback data and the optimal gripping force, and obtain the gripping force deviation value;

[0234] S602, determine the deviation between the real-time tactile feedback data and the reference tactile features, and obtain the tactile feature deviation value;

[0235] S603, based on the gripping force deviation value and the cooperative adaptation coefficient, determine the gripping force adjustment amount;

[0236] S604, Based on the tactile feature deviation value and the cooperative adaptation coefficient, determine the amount of finger posture adjustment;

[0237] S605, the grasping force adjustment amount and the finger posture adjustment amount are executed by a proportional-integral-derivative controller;

[0238] S606 integrates visual sensor data with real-time tactile feedback data to generate multimodal grasping status analysis results;

[0239] S607, Based on the multimodal grasping state analysis results, analyze whether the grasping operation was successfully completed, and obtain a grasping success flag;

[0240] S608, Based on the multimodal grasping state analysis results and material stress model, predict the damage risk level of the object;

[0241] S609, Determine the integrity status of the object based on the object's damage risk level according to the preset risk level and integrity status mapping rule;

[0242] S610, based on the grasping force adjustment amount, finger posture adjustment amount and object integrity state, generate a grasping execution command containing continuous control parameters;

[0243] S611, output the grabbing status analysis result including the grabbing success flag and the object integrity status.

[0244] In this embodiment, a real-time multimodal perception closed-loop system is constructed during the grasping process, and the grasping strategy is dynamically adjusted by the controller to improve execution accuracy and object protection capability. The first stage involves deviation analysis between actual perception and the ideal grasping strategy. The grasping force deviation value is calculated by comparing the actual applied force intensity reflected in the real-time mechanical feedback data with the preset optimal grasping force, taking into account the mean shift, fluctuation amplitude, and loading path consistency within the time window, thereby quantifying the grasping force execution accuracy. The tactile feature deviation value is calculated based on the similarity between the texture, elasticity, pressure distribution, and other features extracted from the real-time tactile feedback data and the features of the reference contact area. Indicators such as cosine similarity, dynamic time warping (DTW), or structured Euclidean distance can be used to accurately characterize the degree of deviation of the tactile feedback shape.

[0245] By combining the generated co-adaptation coefficients, the system performs weighted mapping on the aforementioned deviation values ​​to generate grasping force adjustment and finger posture adjustment. This mapping reflects the interactive weight allocation mechanism between mechanical control and tactile perception. Specifically, the co-adaptation coefficients can reflect the control emphasis in tactile-dominated (e.g., fragile objects) or force-dominated (e.g., rigid objects) scenarios, making control adjustments more adaptive.

[0246] The proportional-integral-derivative (PID) controller is the core module executing this adjustment strategy. Its input is the aforementioned adjustment amount, and its output is the adjusted control signal. The proportional term provides the immediate response to the deviation, the integral term eliminates the steady-state error caused by persistent deviations, and the derivative term predicts the trend of deviation changes and suppresses system oscillations. During dynamic capture, the PID controller should update its parameters in real time, supporting incremental adjustment strategies and anti-saturation mechanisms to ensure stable control under nonlinear mechanical responses.

[0247] Next, a multimodal state analysis system for the grasping process is constructed by fusing visual sensor data with real-time tactile feedback data. Visual information includes the object's position, rotation angle, and surface state before and after grasping. Combined with pressure diffusion, deformation response, and contact stability parameters from the tactile feedback, a graph neural network (GNN) or Transformer structure is used for joint modeling, outputting the grasping state analysis results. This analysis not only determines whether the grasping action is complete but also assesses the stability of the grasp, the object's slippage tendency, or the risk of structural deformation.

[0248] Based on this analysis, the system further determines whether the grasping operation was successfully completed, generating a grasping success flag, which can be a Boolean value (success / failure) or a graded label (high confidence / medium / failure). Simultaneously, combining a historical mechanical behavior database and a material stress model, the multimodal data is mapped to a material safety boundary model to predict whether the object's stress state has exceeded the safe tolerance range, outputting the object's damage risk level. The stress model can include linear elastic constitutive relations, material yield limit data, and fatigue criteria, adapting to the dynamic response characteristics of different material types.

[0249] Based on the risk level, the system determines the integrity status of the current object using preset integrity status mapping rules (e.g., a three-stage system: intact / minor deformation / damage). This status is used not only for feedback control but also as feedback information prompts for robot-human interaction.

[0250] Finally, by combining the gripping force adjustment, finger posture adjustment, and object integrity status, the system generates execution instructions containing continuous control parameters. These parameters can be used for subsequent posture stabilization, contact force balance, or fine placement processes, ensuring that the gripping operation is not only completed but also that subsequent processing is robust. Simultaneously, the system outputs gripping status analysis results, including a successful gripping flag and object integrity status, achieving closed-loop quality monitoring and task-level feedback for the entire gripping process.

[0251] Example Description: In the field of elderly care services, the application of a collaborative adaptive grasping control system based on multimodal tactile and mechanical sensing can significantly improve the accuracy and safety of nursing robots in assisted care processes. The nursing robot first performs a perception initialization task. Using an array of tactile sensors attached to the end of the gripper, the robot contacts the edge surface of a ceramic medicine bowl and collects its pressure distribution map and contact area information, generating surface tactile data. Simultaneously, force sensors record the normal and tangential forces applied by the gripper to the bowl wall during grasping, generating interactive mechanical data. Furthermore, the size of the ceramic medicine bowl is measured using a 3D vision system or an infrared ranging sensor, and its standard weight data is synchronized from an IoT drug management system to obtain physical parameters. The system then performs Gaussian filtering on the pressure distribution data to remove sensor noise and zero-drift calibration on the mechanical data to improve signal quality.

[0252] Next, the system extracts surface roughness based on the filtered pressure data, calculates the surface hardness coefficient using normal force and contact area, constructs a pressure-deformation curve based on contact deformation, and extracts the elastic modulus. Simultaneously, it calculates the bowl's inertial parameters based on size and weight data. Further analysis of the spatial frequency distribution of the pressure spectrum extracts texture complexity; the rate of change of mechanical data is used to infer the frictional characteristics during the force application process on the bowl's surface. All of this information is integrated into a tactile feature vector and an initial mechanical parameter vector, representing the comprehensive perceptual expression of the ceramic bowl.

[0253] The system then performs a parameter modeling task. First, principal component analysis is used to reduce the dimensionality of the tactile and mechanical vectors, extracting key collaborative feature components. The correlation coefficient between tactile and mechanical data is further calculated, and the entropy method is used to determine the importance weight of each feature type. Then, based on the physical properties of ceramics as a brittle material, the weight coefficients are adjusted to enhance attention to structurally sensitive indicators. The weighted tactile and mechanical features are fused into a collaborative grasping feature matrix. Combined with grasping data from different elderly patients' medication-collecting scenarios in historical grasping records, the stability and reliability of each feature over time are evaluated, ultimately generating a feature weight vector for training the grasping model.

[0254] During the model building phase, the system constructs a grasping force regression model based on the aforementioned feature matrix and weight vector, and selects a subset of features that significantly influence grasping success or failure for use in constructing the contact point distribution decision model and the finger posture decision model. Joint continuity constraints are introduced into the finger posture model to prevent the robotic arm from generating high-frequency vibrations or abrupt posture changes during execution. To improve the robustness of contact point selection, the model also evaluates the spatial stability of each candidate contact area and dynamically adjusts the safety factor for different materials in the grasping force model, thereby setting the maximum permissible grasping force. After all models are built, the system uses historical data to verify prediction accuracy and perform parameter calibration. Finally, the output is an initial grasping strategy containing the optimal grasping force, the maximum permissible grasping force, contact point coordinates, and finger posture parameters.

[0255] Subsequently, the robot enters the real-time optimization control phase. The system collects tactile feedback data in real time as the gripper approaches the ceramic bowl, and extracts reference contact area features by combining the contact point coordinates set in the initial strategy. An objective function is constructed that integrates minimizing gripping force deviation with tactile feature matching, setting constraint boundaries for maximum gripping force and finger posture angle, and defining a prediction time window of 10 control cycles. Within each cycle, the system executes an incremental control strategy based on constraint optimization, outputting the gripping force control value and finger posture adjustment value for that cycle. Simultaneously, the synergy between tactile and force control objectives in the output control is evaluated, and a synergy adaptation coefficient is generated to adjust the weights of different objective functions in the optimization. As the control sequence is integrated cycle by cycle, a complete grasping control trajectory is formed.

[0256] The robot then drives its gripper to perform the actual grasping action according to the grasping control sequence. The system analyzes the grasping force and posture commands for each cycle, controls the actuators to output the corresponding grasping force, and simultaneously performs fine-tuning of the joint angles to match the target posture. Embedded sensors continuously collect actual force data and finger spatial trajectories, recording posture deviations. During execution, the tactile sensor array and multi-dimensional force sensors synchronously collect feedback data and align the timestamps to generate a unified temporal feedback stream. When missing or abnormally drifting sensor data is detected, the system immediately triggers a redundant sensing mechanism or issues a task pause command.

[0257] After grasping, the system enters the dynamic correction and quality assessment phase. Based on the deviation between the real-time mechanical data collected in the feedback and the target grasping force, and combined with the co-adaptation coefficient, the fine-tuning range of the grasping force is calculated. Similarly, finger posture adjustments are generated based on tactile feature deviations and weighting factors. These adjustment commands are executed by a PID controller, ensuring good stability and responsiveness in the control output. The system integrates displacement and posture information from visual sensors with tactile feedback to jointly model and analyze whether the object has been grasped stably and intact. Further referencing the material stress curve and loading state of the ceramic bowl, it predicts whether it is in a breakage risk range and outputs the corresponding risk level. Based on the risk level, the integrity status is mapped, such as intact, slightly cracked, or critically broken, and subsequent continuous control commands are adjusted accordingly to ensure the bowl is safely transferred to the elderly person's dining area. The final state of the entire grasping process, including whether the grasp was successful and the bowl's integrity assessment results, is synchronously fed back to the nursing management platform, providing reference and training data for subsequent care tasks.

[0258] In the healthcare field, collaborative adaptive grasping control systems based on multimodal tactile and mechanical sensing can be widely applied in ward service robots, surgical assistance robots, and rehabilitation aids, especially suitable for high-precision, low-damage instrument delivery, sample handling, and patient assistance scenarios. During system initialization, the service robot first contacts the outer wall of a medical glass measuring cup using an array of tactile sensors to collect surface tactile data, including pressure distribution maps and contact area information. Simultaneously, force sensors in the grippers record the normal and tangential forces applied to the glass cup during the grasping action, generating interactive mechanical data. The system also utilizes a visual measurement module to identify the measuring cup's dimensions and structure, and retrieves the weight information of this batch of measuring cups from the electronic ward supplies system, constructing a complete set of physical parameters.

[0259] The system then performs signal processing and physical modeling tasks. Gaussian filtering is applied to the pressure distribution data to remove noise, and zero-drift error calibration is performed on the normal and tangential forces to improve the accuracy of the mechanical data. Further, surface roughness is calculated based on the filtered tactile data, the hardness coefficient is derived from the normal force and contact area, the elastic modulus is estimated based on the pressure-deformation curve, and finally, the inertial characteristics of the measuring cup are calculated by combining size and weight parameters. In addition, the system extracts texture complexity by analyzing the spatial frequency of the tactile data and estimates the surface friction coefficient using the real-time fluctuation rate of the mechanical data. This multidimensional information is constructed into a tactile feature vector and an initial mechanical parameter vector, serving as the basis for subsequent modeling.

[0260] The system then performs collaborative feature construction. The tactile feature vector and the mechanical parameter vector are subjected to dimensionality reduction using principal component analysis to extract a core subset of collaborative features. The weight coefficients of the two types of features are calculated using the entropy method, and the feature weight proportions are adjusted based on the fragile nature of medical glass to enhance the model's responsiveness to structurally sensitive features. A fused collaborative grasping feature matrix is ​​generated through weighted processing, and the stability index of each feature over time is evaluated based on historical grasping data. Finally, a feature weight vector is generated to train the grasping control model.

[0261] During the training phase of the grasping model, the system constructs a grasping force regression model, a contact point distribution model, and a finger posture model based on a collaborative feature matrix and weight vector. The contact point distribution model focuses on the spatial stability of different contact areas, while the finger posture model introduces continuity constraints on finger joints to avoid abrupt posture changes and improve control smoothness. The system adjusts the grasping force safety factor according to the glass material of the measuring cup to generate the maximum permissible grasping force. The initial grasping strategy is constructed by combining the outputs of the above models, including the optimal force, the maximum permissible force, contact point coordinates, and finger posture.

[0262] During the task execution phase, the system receives tactile feedback data in real time and derives reference tactile features based on the contact point coordinates contained in the initial strategy. Subsequently, a joint optimization objective function is constructed to minimize the deviation between the real-time grasping force and the target force, and the deviation between the real-time tactile features and the reference area features, with upper limits on force and feasible attitude regions set as constraints. In the rolling prediction time domain, the system continuously optimizes and solves to generate grasping force control sequences and attitude control sequences, while simultaneously calculating the co-adaptation coefficient between force and tactile control.

[0263] The robot executes the actual grasping task according to the control sequence. The actuator precisely outputs the grasping force and adjusts the finger joint angles based on the control values ​​of each cycle; the system continuously collects actual execution data and monitors posture deviations; simultaneously, the tactile array and multi-dimensional force sensors synchronously collect feedback data and perform time alignment to ensure the timing consistency of the control chain. When abnormal sensing data is detected, the system triggers a data re-acquisition or grasping abort mechanism.

[0264] After the task is completed, the system calculates the fine-tuning range of gripping force and posture based on the deviation between real-time feedback and model reference values, and executes it precisely through a PID controller. Simultaneously, it constructs a multimodal gripping state model by fusing depth maps from visual sensors and tactile data to determine whether the measuring cup has been securely gripped. Combining the material stress curve with the current applied force, it predicts the risk of breakage of the measuring cup and generates a risk level, which is further mapped to an integrity status, such as intact, slightly cracked, or severely damaged. Finally, the system outputs whether the gripping was successful and the object's integrity status, used to update model parameters and provide risk feedback.

[0265] In the healthcare setting, this application is particularly suitable for tasks requiring high instrument protection and precise operation. Through multi-source data perception modeling, control strategy optimization, and real-time risk assessment, it ensures that the robot is both stable and safe when performing high-risk tasks such as delivering medical glassware, sampling bottles, and rehabilitation aids, effectively reducing human intervention and improving the efficiency of intelligent nursing.

[0266] In the fintech sector, particularly in smart banking, intelligent wealth management terminals, self-service equipment (such as smart teller machines), and intelligent logistics vault systems, robots are frequently used to automate the grasping and delivery of high-value, easily damaged objects (such as financial documents, seals, smart cards, chip modules, and small precious metal samples). The system first identifies the location and basic attributes of the target seal, and then collects its surface tactile data and interactive mechanical data using a contact array tactile sensor and multi-dimensional force sensors. This data includes the pressure distribution map and contact area of ​​the seal surface, the normal and tangential forces applied by the grippers during the grasping process, and physical parameters such as the seal's size and weight, obtained through structural recognition and historical data retrieval to ensure that subsequent grasping actions can be modeled based on real physical properties.

[0267] The system performs high-precision processing on the collected raw tactile and mechanical data: the pressure image in the tactile data is first filtered by Gaussian to remove minor noise fluctuations; the normal and tangential forces in the mechanical data undergo zero-drift calibration to eliminate initial deviations. Based on the calibrated data, the system further calculates the seal's surface roughness, material hardness coefficient, elastic modulus, texture complexity parameters, and dynamic friction coefficient; physical parameters are used to calculate inertia and mass distribution characteristics. These indicators are constructed into a multi-dimensional tactile feature vector and an initial mechanical parameter vector for subsequent control modeling.

[0268] Subsequently, the system executes a collaborative grasping feature generation process. Tactile and mechanical features are dimensionality-reduced separately, core principal components are extracted, and their internal coupling structure is revealed by calculating their correlation coefficients. Furthermore, the entropy method is introduced to determine the weight coefficients of each principal component. Considering the high confidentiality and vulnerability of financial media, the system applies an adjustment function based on risk and value levels to the weight coefficients, making the model more biased towards fine mechanical control when facing high-risk objects. After weighted fusion, the system generates a unified collaborative grasping feature matrix and combines it with historical grasping data to generate a feature weight vector, which serves as the input for model training.

[0269] In the grasping model construction phase, the system constructs a grasping force regression model, a contact point decision model, and a finger posture model based on a collaborative feature matrix and feature weight vector, respectively. The contact point decision model generates a stable grasping distribution based on a high-stability region, while the finger posture model introduces motion continuity constraints from the gripper mechanism to ensure that the movements are not abrupt or jumpy. The upper limit safety factor of the grasping force is adjusted by combining the material coding of the stamp material, ultimately generating an initial grasping strategy that includes the optimal grasping force, the maximum allowable grasping force, the contact point distribution coordinates, and the finger posture parameters.

[0270] During the execution phase, the system collects tactile and mechanical data returned during the stamp grasping process in real time and extracts reference tactile features based on the current contact point area. The system constructs a constrained optimization problem with the dual objectives of minimizing force error and minimizing tactile error, while setting upper limits for force and attitude continuity boundaries. It calculates the control quantity for each cycle based on the sliding prediction time domain and generates cooperative adaptation coefficients to evaluate the consistency of tactile-mechanical control.

[0271] The grasping control sequence drives the robot to execute gripper movements, adjusting the gripping force and joint angles each cycle and monitoring execution deviations. Real-time tactile and mechanical data are processed synchronously through a timestamp alignment mechanism, and a data anomaly handling mechanism is included to address issues such as frame drops and distortion. If the tactile or mechanical data is unreliable, the system will pause the task and report the risk.

[0272] During the task feedback phase, the system generates force and posture adjustments based on tactile and mechanical deviations, combined with a co-adaptation coefficient. A PID controller smoothly adjusts the gripper's movements. By integrating visual information and tactile feedback, the system determines whether the grasping was successful and predicts the risk of damage to the stamp using a stress model. Based on the damage risk mapping rules set in the system, it determines whether the stamp remains intact. Finally, the system outputs a successful grasp indicator and the object's integrity status, and updates the model and makes feedback corrections based on the current strategy results.

[0273] This embodiment effectively addresses the shortcomings of traditional systems in adapting to diverse tactile characteristics of objects by dynamically adjusting grasping force and finger posture based on deviation calculation and collaborative adaptation mechanisms through real-time acquisition and analysis of tactile and mechanical feedback during the grasping process. By introducing PID control and multimodal fusion modeling, high-precision control and state perception of the grasping execution process are achieved. Simultaneously, material stress models are used to predict the risk of object damage and output the object's integrity status, providing a clear quality assessment signal for the grasping task. This strategy significantly improves the robot's autonomous adaptability, control stability, and object protection capabilities in complex tactile scenarios, making it particularly suitable for high-security and sensitive scenarios such as medical device handling, assistive retrieval for the elderly, and grasping valuables at financial counters.

[0274] In one embodiment, a grasping control device based on collaborative grasping features is provided, which corresponds one-to-one with the grasping control method based on collaborative grasping features in the above embodiments. (Refer to...) Figure 3 , Figure 3 This is a schematic diagram of the functional modules of a preferred embodiment of the grasping control device based on collaborative grasping features of the present invention. The modules include a multi-source sensing and acquisition module 10, a feature fusion calculation module 20, an initial strategy generation module 30, a collaborative optimization control module 40, an execution monitoring module 50, and a dynamic adjustment decision module 60. Detailed descriptions of each functional module are as follows:

[0275] The multi-source sensing and acquisition module 10 is used to acquire surface tactile data, interactive mechanical data and physical parameters of the target object;

[0276] The feature fusion calculation module 20 is used to generate collaborative grasping features and feature weight vectors based on the surface tactile data, interactive mechanical data and physical parameters.

[0277] The initial strategy generation module 30 is used to construct a grasping model based on the collaborative grasping features, feature weight vectors and historical grasping data, and to determine an initial grasping strategy based on the grasping model, which includes the optimal grasping force, the maximum allowable grasping force, the contact point distribution coordinates and the finger posture parameters.

[0278] The collaborative optimization control module 40 is used to combine the initial grasping strategy with real-time tactile feedback data during the grasping process, and generate a grasping control sequence and collaborative adaptation coefficients through constraint optimization solution.

[0279] The execution monitoring module 50 is used to execute the grasping operation according to the grasping control sequence and monitor the real-time tactile feedback data and real-time mechanical feedback data during the execution process.

[0280] The dynamic adjustment decision module 60 is used to dynamically adjust the gripping force and posture based on the deviation between the real-time mechanical feedback data and the optimal gripping force, the deviation between the real-time tactile feedback data and the reference tactile features, and in combination with the cooperative adaptation coefficient.

[0281] In one embodiment, the multi-source sensing and acquisition module 10 is specifically used for:

[0282] Surface tactile data, including pressure distribution data and contact area data, are collected using an array of tactile sensors.

[0283] The force sensor collects interactive mechanical data, including the normal and tangential forces applied to the target object during the grasping process;

[0284] Obtain physical parameters including the size and weight information of the target object;

[0285] Gaussian filtering is applied to the pressure distribution data in the surface tactile data to generate filtered pressure distribution data.

[0286] Zero-drift error calibration is performed on the normal force and tangential force in the interactive mechanical data to generate calibrated normal force and calibrated tangential force;

[0287] The surface roughness of the object is determined based on the filtered pressure distribution data.

[0288] The object's hardness coefficient is determined based on the calibrated normal force and contact area.

[0289] A pressure-deformation relationship curve is generated based on the calibrated normal force and contact deformation, and the elastic modulus of the object is determined based on the pressure-deformation relationship curve.

[0290] The object's inertial parameters are determined based on the size and weight information.

[0291] Analyze the spatial frequency characteristics of the surface tactile data, and determine the surface texture complexity parameters of the object based on the spatial frequency characteristics;

[0292] Analyze the real-time rate of change of the interactive mechanical data, and determine the dynamic friction characteristic coefficient of the object surface based on the real-time rate of change;

[0293] The object's surface roughness, hardness coefficient, elastic modulus, and surface texture complexity parameters are integrated to generate a tactile feature vector;

[0294] The initial mechanical parameter vector is generated by integrating the calibrated normal force, calibrated tangential force, contact area, object inertial parameters, and dynamic friction coefficient.

[0295] In one embodiment, the feature fusion calculation module 20 is specifically used for:

[0296] Obtain the tactile feature vector and initial mechanical parameter vector generated based on the surface tactile data and interactive mechanical data;

[0297] Principal component analysis is performed on the tactile feature vector and the initial mechanical parameter vector to reduce their dimensionality, thereby obtaining the dimensionality-reduced tactile features and mechanical parameters, respectively.

[0298] Determine the correlation coefficient between the dimensionality-reduced tactile features and the dimensionality-reduced mechanical parameters;

[0299] Based on the correlation coefficient, the tactile feature weight coefficient and the mechanical parameter weight coefficient are determined by the entropy method;

[0300] The weighting coefficients of the tactile features and the mechanical parameters are adjusted based on the physical parameters.

[0301] The dimensionality-reduced tactile features are weighted using the adjusted tactile feature weighting coefficients to generate weighted tactile features;

[0302] The reduced mechanical parameters are weighted using the adjusted mechanical parameter weighting coefficients to generate weighted mechanical parameters;

[0303] By fusing the weighted tactile features with the weighted mechanical parameters, a collaborative grasping feature matrix is ​​constructed.

[0304] Determine the variance contribution rate of each feature in the collaborative crawling feature matrix;

[0305] Determine the time stability index of features based on historical crawled data;

[0306] A feature weight vector is generated based on the variance contribution rate and time stability index.

[0307] In one embodiment, the initial policy generation module 30 is specifically used for:

[0308] Based on the collaborative crawling feature matrix and feature weight vector, a crawling strength regression model is constructed;

[0309] Based on the feature weight vector, highly important features are selected from the collaboratively extracted feature matrix;

[0310] Based on the aforementioned high-importance features and historical crawling data, a decision model for contact point distribution is constructed.

[0311] Based on the aforementioned high-importance features and historical grasping data, a finger posture decision model is constructed, and finger joint motion continuity constraints are added during the construction process.

[0312] The spatial stability of the contact point distribution is analyzed, and the spatial stability analysis results are obtained.

[0313] The optimal gripping force is determined using the aforementioned gripping force regression model;

[0314] The safety factor is dynamically adjusted based on the material type of the object to obtain the adjusted safety factor.

[0315] Multiply the optimal gripping force by the adjusted safety factor to obtain the maximum permissible gripping force;

[0316] The coordinates of the contact point distribution are determined based on the spatial stability analysis results and the contact point distribution decision model.

[0317] The finger posture parameters are determined based on the aforementioned finger joint motion continuity constraints and finger posture decision model.

[0318] The prediction accuracy of the grasping force regression model, contact point distribution decision model, and finger posture decision model was verified using historical grasping data, and the model parameters were adjusted based on the verification results.

[0319] Generate an initial gripping strategy that includes the optimal gripping force, the maximum allowable gripping force, the contact point distribution coordinates, and the finger posture parameters.

[0320] In one embodiment, the collaborative optimization control module 40 is specifically used for:

[0321] Obtain the optimal gripping force, maximum allowable gripping force, contact point distribution coordinates, and finger posture parameters from the initial gripping strategy;

[0322] Acquire real-time tactile feedback data during the grasping process;

[0323] The characteristics of the reference contact area are determined based on the coordinates of the contact point distribution.

[0324] Construct a gripping force tracking target to minimize the deviation between real-time gripping force and optimal gripping force;

[0325] Construct a haptic feature matching target to minimize the deviation between real-time haptic features and reference contact area features;

[0326] Construct an objective function that includes a target for grasping force tracking and a target for tactile feature matching;

[0327] Set the weight coefficients for the grasping force tracking target and the tactile feature matching target;

[0328] Set a force constraint that the grabbing force does not exceed the maximum allowed grabbing force;

[0329] Set the posture constraints of the finger posture parameters within a preset joint angle range;

[0330] Define the prediction time domain range;

[0331] Within each control cycle in the predicted time domain, an optimization problem is solved based on the objective function, force constraints, and posture constraints to obtain the grasping force control amount and finger posture control amount for the control cycle.

[0332] Determine the degree of synergy between mechanical control and tactile control, and generate a synergy adaptation coefficient based on the degree of synergy;

[0333] Integrate the grasping force control and finger posture control from all control cycles to generate a grasping control sequence.

[0334] In one embodiment, the monitoring module 50 is specifically used for:

[0335] Analyze the grasping force control and finger posture control quantities in the grasping control sequence;

[0336] The gripping force control quantity controls the actuator to output a corresponding gripping force;

[0337] Monitor the actual output force of the actuator to obtain the actual gripping force value;

[0338] The actuator adjusts the finger posture according to the finger posture control quantity;

[0339] The deviation between the actual position and the target position of the finger is detected to obtain the posture deviation value;

[0340] Real-time tactile feedback data is acquired through a tactile sensor array;

[0341] Real-time mechanical feedback data is obtained through multi-dimensional force sensors;

[0342] The real-time tactile feedback data and the real-time mechanical feedback data are timestamped to generate aligned tactile feedback data and aligned mechanical feedback data.

[0343] The validity of the aligned tactile feedback data and the aligned mechanical feedback data is detected.

[0344] When the aligned tactile feedback data or the aligned mechanical feedback data is invalid, the data anomaly handling mechanism is triggered.

[0345] In one embodiment, the dynamic adjustment decision module 60 is specifically used for:

[0346] The deviation between the real-time mechanical feedback data and the optimal gripping force is determined to obtain the gripping force deviation value;

[0347] The deviation between the real-time tactile feedback data and the reference tactile features is determined to obtain the tactile feature deviation value;

[0348] Based on the gripping force deviation value and the cooperative adaptation coefficient, the gripping force adjustment amount is determined;

[0349] Based on the tactile feature deviation value and the co-adaptation coefficient, the amount of finger posture adjustment is determined;

[0350] The grasping force adjustment and the finger posture adjustment are executed by a proportional-integral-derivative controller.

[0351] By integrating visual sensor data with real-time tactile feedback data, multimodal grasping status analysis results are generated;

[0352] Based on the results of the multimodal grasping state analysis, the grasping operation is analyzed to determine whether it was successfully completed, and a grasping success flag is obtained.

[0353] Based on the multimodal grasping state analysis results and material stress model, the damage risk level of the object is predicted;

[0354] Based on the preset risk level and integrity status mapping rule, the integrity status of the object is determined according to the object's damage risk level.

[0355] Based on the grasping force adjustment, finger posture adjustment, and object integrity status, a grasping execution command containing continuous control parameters is generated.

[0356] The output includes the grab success flag and the object integrity status as grab status analysis results.

[0357] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides determination and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used for communication with external user terminals via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a server-side grasping control method based on cooperative grasping features.

[0358] In one embodiment, a computer device is provided, which may be a user terminal, and its internal structure diagram may be as follows: Figure 5As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides determination and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a user-side grasping control method based on cooperative grasping features.

[0359] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps:

[0360] Acquire surface tactile data, interactive mechanical data, and physical parameters of the target object;

[0361] Based on the surface tactile data, interactive mechanical data, and physical parameters, collaborative grasping features and feature weight vectors are generated.

[0362] Based on the collaborative grasping features, feature weight vectors and historical grasping data, a grasping model is constructed, and an initial grasping strategy including optimal grasping force, maximum allowable grasping force, contact point distribution coordinates and finger posture parameters is determined based on the grasping model.

[0363] Combining the initial grasping strategy with real-time tactile feedback data during the grasping process, a grasping control sequence and cooperative adaptation coefficients are generated through constraint optimization.

[0364] The grasping operation is executed according to the grasping control sequence, and the real-time tactile feedback data and real-time mechanical feedback data are monitored during the execution process.

[0365] Based on the deviation between the real-time mechanical feedback data and the optimal gripping force, and the deviation between the real-time tactile feedback data and the reference tactile features, the gripping force and posture are dynamically adjusted in conjunction with the cooperative adaptation coefficient.

[0366] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0367] Acquire surface tactile data, interactive mechanical data, and physical parameters of the target object;

[0368] Based on the surface tactile data, interactive mechanical data, and physical parameters, collaborative grasping features and feature weight vectors are generated.

[0369] Based on the collaborative grasping features, feature weight vectors and historical grasping data, a grasping model is constructed, and an initial grasping strategy including optimal grasping force, maximum allowable grasping force, contact point distribution coordinates and finger posture parameters is determined based on the grasping model.

[0370] Combining the initial grasping strategy with real-time tactile feedback data during the grasping process, a grasping control sequence and cooperative adaptation coefficients are generated through constraint optimization.

[0371] The grasping operation is executed according to the grasping control sequence, and the real-time tactile feedback data and real-time mechanical feedback data are monitored during the execution process.

[0372] Based on the deviation between the real-time mechanical feedback data and the optimal gripping force, and the deviation between the real-time tactile feedback data and the reference tactile features, the gripping force and posture are dynamically adjusted in conjunction with the cooperative adaptation coefficient.

[0373] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and user side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0374] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0375] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0376] It should be noted that if any software tools or components not belonging to this company appear in the embodiments of this application, they are merely illustrative examples and do not represent actual use. The embodiments described above are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A grasping control method based on collaborative grasping features, characterized in that, Includes the following steps: Acquire surface tactile data, interactive mechanical data, and physical parameters of the target object; Based on the surface tactile data, interactive mechanical data, and physical parameters, collaborative grasping features and feature weight vectors are generated. Based on the collaborative grasping features, feature weight vectors and historical grasping data, a grasping model is constructed, and an initial grasping strategy including optimal grasping force, maximum allowable grasping force, contact point distribution coordinates and finger posture parameters is determined based on the grasping model. Combining the initial grasping strategy with real-time tactile feedback data during the grasping process, a grasping control sequence and cooperative adaptation coefficients are generated through constraint optimization. The grasping operation is executed according to the grasping control sequence, and the real-time tactile feedback data and real-time mechanical feedback data are monitored during the execution process. Based on the deviation between the real-time mechanical feedback data and the optimal gripping force, and the deviation between the real-time tactile feedback data and the reference tactile features, the gripping force and posture are dynamically adjusted in conjunction with the cooperative adaptation coefficient.

2. The grasping control method based on collaborative grasping features as described in claim 1, characterized in that, Acquire surface tactile data, interactive mechanical data, and physical parameters of the target object, including: Surface tactile data, including pressure distribution data and contact area data, are collected using an array of tactile sensors. The force sensor collects interactive mechanical data, including the normal and tangential forces applied to the target object during the grasping process; Obtain physical parameters including the size and weight information of the target object; Gaussian filtering is applied to the pressure distribution data in the surface tactile data to generate filtered pressure distribution data. Zero-drift error calibration is performed on the normal force and tangential force in the interactive mechanical data to generate calibrated normal force and calibrated tangential force; The surface roughness of the object is determined based on the filtered pressure distribution data. The object's hardness coefficient is determined based on the calibrated normal force and contact area. A pressure-deformation relationship curve is generated based on the calibrated normal force and contact deformation, and the elastic modulus of the object is determined based on the pressure-deformation relationship curve. The object's inertial parameters are determined based on the size and weight information. Analyze the spatial frequency characteristics of the surface tactile data, and determine the surface texture complexity parameters of the object based on the spatial frequency characteristics; Analyze the real-time rate of change of the interactive mechanical data, and determine the dynamic friction characteristic coefficient of the object surface based on the real-time rate of change; The object's surface roughness, hardness coefficient, elastic modulus, and surface texture complexity parameters are integrated to generate a tactile feature vector; The initial mechanical parameter vector is generated by integrating the calibrated normal force, calibrated tangential force, contact area, object inertial parameters, and dynamic friction coefficient.

3. The grasping control method based on collaborative grasping features as described in claim 1, characterized in that, Based on the surface tactile data, interactive mechanical data, and physical parameters, collaborative grasping features are generated. and feature weight vectors, including: Obtain the tactile feature vector and initial mechanical parameter vector generated based on the surface tactile data and interactive mechanical data; Principal component analysis is performed on the tactile feature vector and the initial mechanical parameter vector to reduce their dimensionality, thereby obtaining the dimensionality-reduced tactile features and mechanical parameters, respectively. Determine the correlation coefficient between the dimensionality-reduced tactile features and the dimensionality-reduced mechanical parameters; Based on the correlation coefficient, the tactile feature weight coefficient and the mechanical parameter weight coefficient are determined by the entropy method; The weighting coefficients of the tactile features and the mechanical parameters are adjusted based on the physical parameters. The dimensionality-reduced tactile features are weighted using the adjusted tactile feature weighting coefficients to generate weighted tactile features; The reduced mechanical parameters are weighted using the adjusted mechanical parameter weighting coefficients to generate weighted mechanical parameters; By fusing the weighted tactile features with the weighted mechanical parameters, a collaborative grasping feature matrix is ​​constructed. Determine the variance contribution rate of each feature in the collaborative crawling feature matrix; Determine the time stability index of features based on historical crawled data; A feature weight vector is generated based on the variance contribution rate and time stability index.

4. The grasping control method based on collaborative grasping features as described in claim 1, characterized in that, Based on the collaborative grasping features, feature weight vectors, and historical grasping data, a grasping model is constructed, and based on the grasping model, an initial grasping strategy is determined, including optimal grasping force, maximum allowable grasping force, contact point distribution coordinates, and finger posture parameters, including: Based on the collaborative crawling feature matrix and feature weight vector, a crawling strength regression model is constructed; Based on the feature weight vector, highly important features are selected from the collaboratively extracted feature matrix; Based on the aforementioned high-importance features and historical crawling data, a decision model for contact point distribution is constructed. Based on the aforementioned high-importance features and historical grasping data, a finger posture decision model is constructed, and finger joint motion continuity constraints are added during the construction process. The spatial stability of the contact point distribution is analyzed, and the spatial stability analysis results are obtained. The optimal gripping force is determined using the aforementioned gripping force regression model; The safety factor is dynamically adjusted based on the material type of the object to obtain the adjusted safety factor. Multiply the optimal gripping force by the adjusted safety factor to obtain the maximum permissible gripping force; The coordinates of the contact point distribution are determined based on the spatial stability analysis results and the contact point distribution decision model. The finger posture parameters are determined based on the aforementioned finger joint motion continuity constraints and finger posture decision model. The prediction accuracy of the grasping force regression model, contact point distribution decision model, and finger posture decision model was verified using historical grasping data, and the model parameters were adjusted based on the verification results. Generate an initial gripping strategy that includes the optimal gripping force, the maximum allowable gripping force, the contact point distribution coordinates, and the finger posture parameters.

5. The grasping control method based on collaborative grasping features as described in claim 1, characterized in that, Combining the initial grasping strategy with real-time haptic feedback data during the grasping process, a grasping control sequence and cooperative adaptation coefficients are generated through constraint optimization, including: Obtain the optimal gripping force, maximum allowable gripping force, contact point distribution coordinates, and finger posture parameters from the initial gripping strategy; Acquire real-time tactile feedback data during the grasping process; The characteristics of the reference contact area are determined based on the coordinates of the contact point distribution. Construct a gripping force tracking target to minimize the deviation between real-time gripping force and optimal gripping force; Construct a haptic feature matching target to minimize the deviation between real-time haptic features and reference contact area features; Construct an objective function that includes a target for grasping force tracking and a target for tactile feature matching; Set the weight coefficients for the grasping force tracking target and the tactile feature matching target; Set a force constraint that the grabbing force does not exceed the maximum allowed grabbing force; Set the posture constraints of the finger posture parameters within a preset joint angle range; Define the prediction time domain range; Within each control cycle in the predicted time domain, an optimization problem is solved based on the objective function, force constraints, and posture constraints to obtain the grasping force control amount and finger posture control amount for the control cycle. Determine the degree of synergy between mechanical control and tactile control, and generate a synergy adaptation coefficient based on the degree of synergy; Integrate the grasping force control and finger posture control from all control cycles to generate a grasping control sequence.

6. The grasping control method based on collaborative grasping features as described in claim 1, characterized in that, The grasping operation is executed according to the grasping control sequence, and real-time tactile feedback data and real-time mechanical feedback data are monitored during the execution process, including: Analyze the grasping force control and finger posture control quantities in the grasping control sequence; The gripping force control quantity controls the actuator to output a corresponding gripping force; Monitor the actual output force of the actuator to obtain the actual gripping force value; The actuator adjusts the finger posture according to the finger posture control quantity; The deviation between the actual position and the target position of the finger is detected to obtain the posture deviation value; Real-time tactile feedback data is acquired through a tactile sensor array; Real-time mechanical feedback data is obtained through multi-dimensional force sensors; The real-time tactile feedback data and the real-time mechanical feedback data are timestamped to generate aligned tactile feedback data and aligned mechanical feedback data. The validity of the aligned tactile feedback data and the aligned mechanical feedback data is detected. When the aligned tactile feedback data or the aligned mechanical feedback data is invalid, the data anomaly handling mechanism is triggered.

7. The grasping control method based on collaborative grasping features as described in claim 1, characterized in that, Based on the deviation between the real-time mechanical feedback data and the optimal gripping force, and the deviation between the real-time tactile feedback data and the reference tactile features, the gripping force and posture are dynamically adjusted in conjunction with the cooperative adaptation coefficient, including: The deviation between the real-time mechanical feedback data and the optimal gripping force is determined to obtain the gripping force deviation value; The deviation between the real-time tactile feedback data and the reference tactile features is determined to obtain the tactile feature deviation value; Based on the gripping force deviation value and the cooperative adaptation coefficient, the gripping force adjustment amount is determined; Based on the tactile feature deviation value and the co-adaptation coefficient, the amount of finger posture adjustment is determined; The grasping force adjustment and the finger posture adjustment are executed by a proportional-integral-derivative controller. By integrating visual sensor data with real-time tactile feedback data, multimodal grasping status analysis results are generated; Based on the results of the multimodal grasping state analysis, the grasping operation is analyzed to determine whether it was successfully completed, and a grasping success flag is obtained. Based on the multimodal grasping state analysis results and material stress model, the damage risk level of the object is predicted; Based on the preset risk level and integrity status mapping rule, the integrity status of the object is determined according to the object's damage risk level. Based on the grasping force adjustment, finger posture adjustment, and object integrity status, a grasping execution command containing continuous control parameters is generated. The output includes the grab success flag and the object integrity status as grab status analysis results.

8. A grasping control device based on collaborative grasping features, characterized in that, The grasping control device based on collaborative grasping features includes: The multi-source sensing and acquisition module is used to acquire surface tactile data, interactive mechanical data, and physical parameters of the target object; The feature fusion calculation module is used to generate collaborative grasping features and feature weight vectors based on the surface tactile data, interactive mechanical data and physical parameters. The initial strategy generation module is used to construct a grasping model based on the collaborative grasping features, feature weight vectors and historical grasping data, and to determine an initial grasping strategy based on the grasping model, which includes the optimal grasping force, the maximum allowable grasping force, the contact point distribution coordinates and the finger posture parameters. The collaborative optimization control module is used to combine the initial grasping strategy with real-time tactile feedback data during the grasping process, and generate a grasping control sequence and collaborative adaptation coefficients through constraint optimization solution. The execution monitoring module is used to execute the grasping operation according to the grasping control sequence and monitor the real-time tactile feedback data and real-time mechanical feedback data during the execution process. The dynamic adjustment decision module is used to dynamically adjust the gripping force and posture based on the deviation between the real-time mechanical feedback data and the optimal gripping force, the deviation between the real-time tactile feedback data and the reference tactile features, and in conjunction with the cooperative adaptation coefficient.

9. A computer device, characterized in that, The computer device includes a memory, a processor, and a crawling control program based on collaborative crawling features stored in the memory and executable on the processor. When executed by the processor, the crawling control program based on collaborative crawling features implements the steps of the crawling control method based on collaborative crawling features as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a crawling control program based on collaborative crawling features. When the processor executes the crawling control program based on collaborative crawling features, it implements the steps of the crawling control method based on collaborative crawling features as described in any one of claims 1-7.

Images