Service-type embodied intelligent robot tactile auxiliary positioning and grabbing control method

Abstract

The application discloses a service type embodied intelligent robot tactile auxiliary positioning and grabbing control method, including the following steps: data collected by a tactile sensor is transmitted into a trained convolutional neural network, an error vector of an executor key point and an object key point is output, and actual error amount is confirmed through a closed loop control mechanism; shortest safe paths are planned and dynamic obstacles are avoided in combination with the actual error amount and physical constants, a grabbing preparation area is reached, a gripper position and posture are fine-tuned through tactile feedback, and initial conditions of safe grabbing are met; coordinate transformation is completed according to the pose of the gripper and a target object, wrist arm displacement instructions are converted, joint parameters are adjusted through force-position hybrid control and shape adaptive law, control instructions are output through a closed loop execution, and a mechanical arm is driven to complete stable grabbing. The application significantly improves the positioning and grabbing precision and robustness, enhances dynamic obstacle avoidance and safety in a complex environment, and improves the shape adaptive capacity for special-shaped and unknown material objects.

Description

Technical Field

[0001] This invention relates to the interdisciplinary field of computer vision, robot control and deep learning, and specifically to a tactile-assisted localization and grasping control method for a service-oriented embodied intelligent robot. Background Technology

[0002] Currently, the localization and grasping functions of service-oriented embodied intelligent robots mainly rely on visual modeling technology. However, in practical applications, visual systems are susceptible to factors such as insufficient depth measurement accuracy, surface reflection, or external occlusion, leading to positioning errors and a high grasping failure rate. Traditional solutions to this problem require high-precision visual sensors or complex algorithms, which not only significantly increase equipment costs but also suffer from low computational efficiency. Therefore, there is an urgent need for a low-cost, non-visual assistance method to overcome the technical shortcomings of visual systems and improve the accuracy and stability of robot localization and grasping.

[0003] An existing robot system (CN117207190A) that generates grasping commands based on the fusion of visual and tactile data mainly relies on post-contact state feedback and lacks fine-tuning logic for the pre-grasping preparation stage. This makes it difficult to correct posture in time when initial visual positioning deviates, easily leading to collisions or grasping failures. Furthermore, its trajectory generation process lacks real-time spatial verification and dynamic obstacle avoidance mechanisms for potential obstacles in the workspace, making it difficult to ensure the robot's operational safety in complex environments. These shortcomings in fine-tuning posture and ensuring safe operational trajectory are precisely the technical problems that this invention aims to solve. Summary of the Invention

[0004] To address the technical problems existing in the prior art, this invention provides a tactile-assisted positioning and grasping control method for service-oriented embodied intelligent robots. The method constructs a tactile-assisted robot positioning and grasping system through three core modules: tactile information acquisition and analysis, posture correction decision-making, and control scheme calculation. By combining convolutional neural networks trained through simulation and fine-tuned on actual machines, dynamic path planning and obstacle avoidance mechanisms, and force-position hybrid control strategies, the method achieves precise, stable, and efficient grasping operations for the robot.

[0005] This invention is achieved through at least one of the following technical solutions: A tactile-assisted positioning and grasping control method for a service-oriented embodied intelligent robot includes the following steps: S1. Input the data collected by the tactile sensor into the trained convolutional neural network, output the deviation vector between the key points of the actuator and the key points of the object, and then confirm the actual error through the closed-loop control mechanism. S2. Combining actual error and physical constants, plan the shortest safe path and dynamically avoid obstacles. After arriving at the grasping preparation area, fine-tune the position and attitude of the gripper through tactile feedback to meet the initial conditions for safe grasping. S3. Based on the position and orientation of the gripper and the target object, coordinate transformation is completed and converted into wrist arm displacement commands. Joint parameters are adjusted through force-position hybrid control and shape adaptive law, and then output control commands are executed through closed loop to drive the robotic arm to complete stable gripping.

[0006] Furthermore, the training of a convolutional neural network includes the following two stages: During the simulation phase, the dataset required for training the convolutional neural network is generated using real physical data from the simulation environment. The tactile information output by the simulator and the relative positional deviation data between the actuator and the object are used as real labels for the supervised learning process of the convolutional neural network. In the practical stage, actual operating data is collected in the real robotic arm operating environment. The operating data includes tactile input signals recorded by tactile sensors and actual deviation data between the actuator and key points of the object. The pre-trained convolutional neural network trained by simulation is fine-tuned using a few-shot learning method based on the actual operating data.

[0007] Furthermore, the key points of the actuator include the center point of the two-finger gripper and the midpoint between the index finger and thumb on the gripper; the key points of the object need to be defined according to the actual needs of the specific grasping task.

[0008] Furthermore, the actual error amount confirmed by the closed-loop control mechanism includes: The deviation vector is continuously compared with a preset threshold. If the deviation is higher than the threshold, the robotic arm is triggered to recalculate the movement command and make adjustments. If the deviation is lower than the threshold, the current state is maintained. The above loop continues until the deviation returned by the convolutional neural network is stably lower than the preset threshold, ensuring the robotic arm's operational accuracy and task execution reliability. The stable completion of the grasping task is achieved through iterative feedback.

[0009] Furthermore, the physical constants include the gripper block radius Rfinger, the target object radius Robj, and the safety distance threshold between the gripper and the outer perimeter of the target object, wherein the safety distance threshold is not less than 3 cm.

[0010] Furthermore, in step S2, combining the actual error amount and physical constants, the shortest safe path is planned and obstacles are dynamically avoided to reach the grabbing preparation area, including: Based on the 3D spatial modeling and the 3D occupancy mesh output by the 3D-R2N2 network, the coordinates of the target object and the current coordinates of the gripper are obtained. The shortest straight path between the coordinates of the target object and the current coordinates of the gripper is calculated and generated first. During the path generation process, the coordinate data of environmental obstacles are called simultaneously, and the safe distance of each point on the path is monitored and verified in real time: the radius Rfinger of the gripper block is extended outward from the center of the gripper, and the radius of the obstacle is extended outward from the center of the obstacle. The radius of non-target objects is set according to the preset general value or the size obtained by image recognition, ensuring that the distance between the extended range of the gripper and the obstacle is always not less than 3 cm. If the safety conditions are met throughout the process, the movement of the gripper is controlled. If, during the gripper's movement, real-time coordinate monitoring detects that the distance between the gripper and an obstacle is less than m centimeters, the obstacle avoidance mechanism is immediately triggered. Based on the single obstacle handling process, a virtual obstacle avoidance sphere is constructed with the geometric center of the obstacle as the center and the radius of the obstacle plus the gripper block radius Rfinger + m centimeters as the radius. The virtual obstacle avoidance sphere ensures that the gripper maintains a safe distance from the obstacle while moving along the sphere. The line connecting the gripper's current position and the target object's coordinates is calculated, and the intersection point of this line and the virtual obstacle avoidance sphere is determined. The intersection point is the shortest path node after obstacle avoidance, i.e., the sum of the distances to the gripper's current position and the target object is the shortest path in obstacle avoidance mode. The gripper is controlled to smoothly move along the virtual obstacle avoidance sphere to this intersection point, maintaining the gripper's posture during the movement to avoid increasing path time due to direction adjustments. If a new obstacle is encountered during obstacle avoidance, the judgment criterion is that the distance between the gripper and the new obstacle is less than m centimeters when the gripper moves along the previous virtual obstacle avoidance sphere. In this case, the obstacle avoidance strategy is switched: a new virtual obstacle avoidance sphere is constructed with the geometric center of the new obstacle as the center; the gripper is controlled to move from the current position along the new virtual obstacle avoidance sphere until it leaves the safe distance range of the new obstacle. The criterion for leaving the sphere is moving to a sphere node where the distance between the gripper and the new obstacle is not less than m centimeters; the shortest straight path to the target object is recalculated from this departure node as the new starting point, and the safe distance on the path is checked again; if there is no safety hazard on the path, the gripper continues to move along the straight line; if there is still an obstacle, the obstacle avoidance process is repeated until the gripper enters the target object's grasping preparation area.

[0011] Furthermore, in step S2, after arriving at the grasping preparation area, the position and posture of the gripper are finely adjusted through tactile feedback, including: When the gripper reaches the outer safety circle of the target object along the planned path, it enters the gripping preparation stage, pauses the straight-line movement of the gripper, and switches to the tactile feedback adjustment mode; the outer safety circle is when the distance from the center of the gripper to the center of the target object is equal to the radius of the target object Robj + the radius of the gripper block Rfinger + m centimeters. The haptic feedback modulation mode includes the following processes: During the fine-tuning process, the gripper moves slowly toward the target object in tiny steps. During the movement, the pressure signal collected in real time by the tactile sensor is input into the convolutional neural network. If the convolutional neural network interprets the pressure intensity feature as no contact, the judgment criterion is that no effective pressure is detected, and the movement continues. If a contact signal is detected, the judgment criterion is that the pressure intensity reaches the preset threshold, and the movement stops immediately. At this time, the distance between the gripper and the target object is maintained at about m centimeters, which meets the initial conditions for safe gripping. During the posture fine-tuning stage, when the pressure distribution center of gravity features analyzed by the convolutional neural network have a horizontal or vertical offset compared with the preset optimal gripping reference value, the deviation data is input into the feedforward neural network, which outputs the corresponding posture correction parameters (F / T output parameters). The robotic arm drives the gripper to fine-tune the angle around the center of the target object according to the posture correction parameters. After each adjustment, the tactile signal is re-acquired until the pressure distribution center of gravity features fall within the error allowable range. After the attitude calibration is completed, the system triggers the gripper closing command: the gripper retracts at a uniform speed of 1 cm per second until the tactile sensor detects that the gripping force has reached a safe and stable threshold. The safe and stable threshold is dynamically adjusted according to the radius Robj of the target object. At the same time, the long short-term memory network analyzes the change curve of the gripping force over time in real time. If it predicts that there is a risk of the object slipping, it immediately outputs a force compensation command to fine-tune the gripper closing degree. Once the gripping force is maintained stably for more than n seconds and the deviation output by the convolutional neural network is lower than the preset threshold, the gripping is determined to be complete, and subsequent moving or placing operations can be performed.

[0012] Further, step S3 includes the following steps: S31. First, coordinate transformation is performed, and displacement target information is obtained by combining the data returned by the sensor. Then, the transformation relationship between the target coordinate system and the robot arm base coordinate system is established to provide accurate pose basis for the subsequent generation of wrist arm displacement commands. S32. Based on the pose of the target object in the base coordinate system obtained by coordinate transformation, the robotic arm actuator converts the pose into a wrist arm displacement command, drives the wrist arm to complete the corresponding displacement, and makes the gripper approach the target gripping position. S33. The robotic arm end effector employs a force-position hybrid control mechanism to achieve precise control of the finger and wrist ends: the desired trajectory of the end effector is defined as follows: The actual trajectory is The positional deviation is The speed correction of the end effector is calculated in real time using the impedance control equation, which is:

[0013] in , , These are the inertia, damping, and stiffness matrices, respectively. For the desired contact force, The actual contact force is collected by a force sensor. express The positional deviation of the end effector at any given moment; The first derivative of the positional deviation, i.e., the velocity deviation; The second derivative of the position deviation, i.e., the acceleration deviation; This represents the current time variable.

[0014] To address the uncertainty in the shape of the target object, the joint parameters of the robotic arm are controlled by an adaptive mechanism. The core principle is to construct an adaptive law based on end-effector force feedback and positional deviation.

[0015] in These are estimated values ​​for joint parameters. It is a positive definite adaptive gain matrix. Let Jacobian matrix be the current time step. This is a force estimate based on the current parameters. Through this adaptive law, the robotic arm can dynamically adjust the joint angles during the grasping process, enabling... Ultimately, this enables stable gripping of objects of different shapes and materials; Transforming velocity corrections and joint parameter adjustments into actual control commands: Utilizing the inverse kinematics of the robotic arm, from the end-effector velocity... Solve for joint velocities:

[0016] in This is the pseudo-inverse of the Jacobian matrix. express The angular velocity vectors of each joint of the robotic arm at any given time; express The end effector's linear velocity and angular velocity vector at any given moment; the joint angle command is obtained through integration. :

[0017] in The initial joint angle is set; the joint angle command is sent to each joint motor, driving the motor to execute the action and complete the grasping task.

[0018] A computer device according to the present invention includes a memory and a processor, the memory being electrically connected to the processor, the memory storing a computer program, which, when executed by the processor, causes the processor to implement the method described herein.

[0019] The present invention provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor implements the method described herein.

[0020] Compared with existing technologies, the beneficial effects of the present invention are as follows: 1. Significantly improved positioning and grasping accuracy and robustness: By introducing the Paxini tactile sensor and combining it with a convolutional neural network trained by Sim2Real, a tactile fine-tuning mechanism is added to the grasping preparation stage on the basis of visual positioning, which effectively compensates for the errors caused by the influence of lighting and occlusion on the visual system.

[0021] 2. Enhanced dynamic obstacle avoidance and safety in complex environments: Based on dynamic path planning using a virtual obstacle avoidance sphere, the shortest safe path is generated and verified in real time by combining physical constants when approaching the target object, thus avoiding collisions between the robotic arm and non-target objects in complex work spaces.

[0022] 3. Improved shape adaptation capability for irregularly shaped and unknown-material objects: During the end-effector grasping execution phase, a force-position hybrid control mechanism is adopted, and a shape adaptation law based on end-effector force feedback is constructed. This mechanism can dynamically correct joint angles according to the actual contact force, avoiding crushing damage to the object while ensuring stable grasping. Attached Figure Description

[0023] Figure 1 This is a schematic diagram illustrating the logic for filtering effective data during the training simulation phase of a convolutional neural network, as shown in the example.

[0024] Figure 2 This is a schematic diagram of the closed-loop process for adjusting tactile information feedback, as shown in the embodiment.

[0025] Figure 3 This is a schematic diagram of a tactile-assisted positioning and grasping control system for a service-type embodied intelligent robot, as an example.

[0026] Figure 4 This is a schematic diagram of the overall framework for solving the control scheme in the embodiment. Detailed Implementation

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] like Figure 3 As shown in this embodiment, a tactile-assisted positioning and grasping control system for a service-oriented embodied intelligent robot includes, The tactile information acquisition and analysis module is used to obtain the deviation vector between the actuator and the key points of the object, and then the actual error is determined through a closed-loop control mechanism.

[0029] The attitude correction decision module is used to realize precise path planning and attitude calibration of the robotic arm.

[0030] The control scheme calculation module is used to control the robotic arm to complete the grasping process.

[0031] like Figures 1-4 As shown in the figure, a tactile-assisted positioning and grasping control method for a service-oriented embodied intelligent robot in this embodiment includes the following steps: S1. Data is collected through the Paxini dexterous hand tactile sensor and fed into a convolutional neural network (CNN) that has been trained in simulation and fine-tuned in actual operation. The output is the deviation vector between the actuator and the key points of the object. The actual error is then confirmed by a closed-loop control mechanism.

[0032] Specifically, the Paxini dexterous hand first collects input data using its tactile sensors, then feeds this data into a pre-built convolutional neural network. The core function of the convolutional neural network is to output the deviation vector between the actuator keypoints and the object keypoints, i.e., the actual error. Actuator keypoints include the center point of the two-finger gripper, the midpoint between the index finger and thumb on the gripper, etc. Object keypoints need to be pre-defined in the system according to the specific grasping task requirements (e.g., for a mug with a handle, the object keypoint can be defined as the center position of the handle; for a cylindrical water bottle, it can be defined as the geometric center of the middle section of the bottle). The convolutional neural network extracts and processes features from the tactile input data, calculates and outputs the spatial deviation between these two types of keypoints, providing a precise error basis for subsequent robotic arm control.

[0033] The convolutional neural network (CNN) employs a simulation-to-real-world training framework, encompassing two key steps: simulation data generation and real-world transfer learning. This ensures the network model's generalization ability in real-world application scenarios. During the simulation phase, the dataset required for CNN training is generated using real physical data from the simulation environment. The simulator outputs tactile information, relative positional deviations between the actuator and the object, and other data. All simulator output data serves as real labels for the supervised learning process of the CNN. The data generation process follows this logic: The target object and its key points are pre-determined in the simulation environment, and the actuator's structural parameters are configured. The object's size and the spatial relative position between the object and the actuator are randomly set using the system randomization module. The generated data is then filtered to determine whether the actuator's tactile patch is in contact with the object, filtering out non-contact data points and retaining only valid contact data. The filtered valid data is then input into the CNN for training. The tactile information output by the simulator is used as the network input, and the actual deviation position is used as the true value. By comparing the deviation position output by the CNN with the true value, the gradient descent algorithm is used to iteratively optimize the network parameters and minimize the prediction error.

[0034] In the practical testing phase, a small amount of actual operational data was collected using a real robotic arm operating environment. This data included tactile input signals recorded by tactile sensors and actual deviation data between the actuator and key points of the object. Using this practical data, a few-shot learning method was employed to fine-tune the pre-trained convolutional neural network trained in simulation. This allowed the model to adapt to noise interference and variations in physical parameters in the real environment, ensuring its robustness and accuracy in error calculation during practical applications.

[0035] After the robotic arm completes the calculation and movement operation based on the deviation vector output by the convolutional neural network, the convolutional neural network is integrated into the feedback adjustment loop to construct a closed-loop control mechanism. The convolutional neural network continuously receives the updated input data from the tactile sensor and outputs the current deviation in real time. The system continuously compares this deviation with a preset threshold. If the deviation is higher than the threshold, the robotic arm is triggered to recalculate the movement command (this movement command is the joint angle command obtained in subsequent step S3 based on the end effector velocity and inverse kinematics). The system then performs adjustments; if the deviation is below a threshold, the current state is maintained. This loop continues until the deviation returned by the convolutional neural network is consistently below a preset threshold, ensuring the robotic arm's operational accuracy and task execution reliability. It achieves stable completion of the grasping task through iterative feedback.

[0036] S2. Combining the actual error amount with the preset physical constant, the shortest safe path is planned and obstacles are dynamically avoided. After arriving at the grasping preparation area, the position and posture of the gripper are finely adjusted through tactile feedback to meet the initial conditions for safe grasping.

[0037] In the robotic arm control system that integrates AI image recognition 3D modeling, tactile fine correction and sensor deviation conversion, the gripper grasps objects with the shortest path priority and obstacle avoidance as the core principle. Combined with preset physical constants and dynamic feedback adjustment mechanisms, it performs precise operations in stages.

[0038] Specifically, the first step is to set core constants. In one embodiment, the preset physical constants include two key physical constants: one is the radius of the gripper block, Rfinger, which is fixed as a built-in parameter of the system according to the specific gripper model and is set to 8 centimeters; the other is the radius of the target object, Robj, which is pre-entered into the system according to the specific grasping task and is set to 5 centimeters. At the same time, the safety interval threshold of the outer range of the two is defined, requiring that the outer range of the target object radius Robj and the gripper block radius Rfinger be not less than 3 centimeters. This interval is the straight-line distance from the outer edge of the gripper to the outer edge of the object, so as to avoid the gripper colliding with the object before grasping.

[0039] Based on the 3D spatial modeling and the 3D occupancy mesh output by the 3D-R2N2 network, the coordinates of the target object and the current coordinates of the gripper are determined. The shortest straight path between the two points is calculated and generated first. During the path generation process, the coordinate data of environmental obstacles are called simultaneously to verify the safe distance of each point on the path in real time: the radius Rfinger of the gripper block is extended outward from the center of the gripper, and the radius of the obstacle itself is extended outward from the center of the obstacle. The radius of non-target objects is set according to the preset general value or the size obtained by image recognition to ensure that the distance between the extended range of the gripper and the obstacle is always not less than 3 cm. If this safety condition is met throughout the process, the gripper movement is controlled according to this straight path.

[0040] If, during path execution, real-time coordinate monitoring detects that the distance between the gripper and an obstacle is less than 3 centimeters, the obstacle avoidance mechanism is immediately triggered. Based on the single obstacle handling process, a virtual obstacle avoidance sphere is constructed with the geometric center of the obstacle as its center and a radius equal to the obstacle's radius plus the gripper's block radius Rfinger + 3 centimeters. This sphere ensures that the gripper maintains a safe distance from the obstacle while moving along it. The line connecting the gripper's current position and the target object's coordinates is calculated, and the intersection of this line and the virtual obstacle avoidance sphere is determined. This intersection point is the shortest path node after obstacle avoidance, characterized by the sum of the distances to the gripper's current position and the target object being the shortest path in obstacle avoidance mode. The gripper is then smoothly moved along the virtual obstacle avoidance sphere to this intersection point, maintaining its posture throughout the movement to avoid increasing path time due to directional adjustments.

[0041] If a new obstacle is encountered during obstacle avoidance, the judgment criterion is that the distance between the gripper and the new obstacle is less than 3 cm when the gripper moves along the previous virtual obstacle avoidance sphere. In this case, the obstacle avoidance strategy is switched. Referring to the multi-obstacle handling process, a new virtual obstacle avoidance sphere is reconstructed with the geometric center of the new obstacle as the center of the sphere, and the radius is calculated as described above. The gripper is controlled to move from its current position along the new virtual obstacle avoidance sphere until it leaves the safe distance range of the new obstacle. The criterion for leaving the sphere is moving to a sphere node where the distance between the gripper and the new obstacle is not less than 3 cm. Using this leaving node as the new starting point, the shortest straight path to the target object is recalculated, and the safe distance on the path is checked again. If there is no safety hazard on the path, the gripper continues to move along the straight line. If there is still an obstacle, the obstacle avoidance process is repeated until the gripper enters the target object's grasping preparation area.

[0042] When the gripper reaches the outer safety circle of the target object along the planned path, the distance from the center of the gripper to the center of the target object is equal to the radius of the target object Robj + the radius of the gripper block Rfinger + 3 cm. At this point, the system determines that it has entered the gripping preparation stage, pauses the linear movement of the gripper, and switches to the tactile feedback adjustment mode.

[0043] During the position fine-tuning process, the gripper slowly moves towards the target object in tiny steps of 0.3 cm per step. During the movement, the pressure signal is collected in real time by the Paxini dexterous hand tactile sensor. If the convolutional neural network (i.e., the convolutional neural network trained by simulation and fine-tuned in actual operation in step S1) analyzes the pressure intensity feature as not contacting, the judgment criterion is that no effective pressure is detected, and the movement continues. If a contact signal is detected, the judgment criterion is that the pressure intensity reaches the preset threshold, and the movement stops immediately. At this time, the distance between the gripper and the target object is maintained at about 3 cm, which meets the initial conditions for safe gripping.

[0044] During the posture fine-tuning stage, when the pressure distribution center of gravity feature analyzed by the convolutional neural network has a horizontal deviation of 2° or a vertical deviation of 1.5° compared with the preset optimal gripping reference value, the deviation data is input into the feedforward neural network, which outputs the corresponding posture correction parameters (referred to as "F / T output parameters" in the attached figure). The robotic arm drives the gripper to fine-tune the angle around the center of the target object according to the parameters. After each adjustment, the tactile signal is re-acquired until the pressure distribution center of gravity feature falls within the error allowable range. In one embodiment, the error allowable range is an offset angle of no more than 0.5°.

[0045] After posture calibration, the system triggers a gripper closure command: the gripper retracts at a uniform speed of 1 cm per second until the Paxini dexterous hand tactile sensor detects that the gripping force has reached a safe and stable threshold. This threshold is dynamically adjusted based on the radius Robj of the target object; for objects with a radius of 5 cm, the gripping force threshold is set to 20 N. Simultaneously, the Long Short-Term Memory (LSTM) network analyzes the gripping force over time in real time. If a risk of object slippage is predicted, the criterion is a decrease in gripping force exceeding 5 N within one second. An immediate force compensation command is then output to fine-tune the gripper closure. Once the gripping force has stabilized for more than 3 seconds, and the deviation calculated from the Paxini dexterous hand tactile sensor data by the convolutional neural network is below a preset threshold (set to be less than 0.2 cm), the system determines that the gripping is complete and can proceed with subsequent movement or placement operations.

[0046] S3. Based on the position and orientation of the gripper and the target object, coordinate transformation is completed and converted into wrist arm displacement commands. Joint parameters are adjusted through force-position hybrid control and shape adaptive law, and then output control commands are executed through closed loop to drive the robotic arm to complete stable gripping.

[0047] The robotic arm achieves precise control by following the logical flow of coordinate transformation, wrist displacement, force adaptation, shape adaptation, and end effector execution.

[0048] First, coordinate transformation is performed. The displacement target information is obtained by combining the data returned by the vision system and the Paxini dexterous hand tactile sensor. Then, the transformation relationship between the target coordinate system and the robot arm base coordinate system is established, which provides accurate pose basis for the subsequent generation of wrist and arm displacement commands.

[0049] Based on the pose of the target object in the base coordinate system obtained by coordinate transformation, the pose is converted into a wrist arm displacement command, which drives the wrist arm to complete the corresponding displacement, so that the gripper approaches the target gripping position.

[0050] Based on the above pose information, the robotic arm completes the displacement from the end effector to the wrist. At this point, the gripper is close to the target gripping position. The next step is to achieve gripping through precise control of the fingers and wrist end effector. To achieve precise gripping and avoid crushing or damaging the object, the robotic arm end effector employs a force-position hybrid control mechanism. The desired trajectory of the end effector is defined as follows: The actual trajectory is The positional deviation is The impedance control equation is:

[0051] in , , These are the inertia, damping, and stiffness matrices, respectively. For the desired contact force, The actual contact force is collected by a force sensor. express The positional deviation of the end effector at any given moment; The first derivative of the positional deviation, i.e., the velocity deviation; The second derivative of the position deviation, i.e., the acceleration deviation; This represents the current time variable. This equation allows for real-time calculation of the end effector's speed correction, ensuring force compliance during the grasping process and preventing excessive contact force from damaging the object.

[0052] To address the uncertainty in the shape of the target object, the joint parameters of the robotic arm are controlled by an adaptive mechanism. The core principle is to construct an adaptive law based on end-effector force feedback and positional deviation.

[0053] in These are estimated values ​​for joint parameters. It is a positive definite adaptive gain matrix. Let Jacobian matrix be the current time step. This is a force estimate based on the current parameters. Through this adaptive law, the robotic arm can dynamically adjust the joint angles during the grasping process, enabling... Ultimately, this enables stable gripping of objects of different shapes and materials.

[0054] In the closed-loop control of the system, the aforementioned speed correction and joint parameter adjustments need to be converted into actual control commands. The specific process is as follows: using the inverse kinematics of the robotic arm, the end-effector speed... Solve for joint velocities:

[0055] in This is the pseudo-inverse of the Jacobian matrix. express The angular velocity vectors of each joint of the robotic arm at any given time; express The end effector's linear velocity and angular velocity vector at any given moment; the joint angle command is obtained through integration. :

[0056] in The initial joint angle is set; the joint angle command is sent to each joint motor, driving the motor to execute the action and complete the grasping task.

[0057] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, enabling those skilled in the art to better understand and utilize the invention.

Claims

1. A tactile-assisted positioning and grasping control method for a service-oriented embodied intelligent robot, characterized in that, Includes the following steps: S1. Input the data collected by the tactile sensor into the trained convolutional neural network, output the deviation vector between the key points of the actuator and the key points of the object, and then confirm the actual error through the closed-loop control mechanism. S2. Combining actual error and physical constants, plan the shortest safe path and dynamically avoid obstacles. After arriving at the grasping preparation area, fine-tune the position and attitude of the gripper through tactile feedback to meet the initial conditions for safe grasping. S3. Based on the position and orientation of the gripper and the target object, coordinate transformation is completed and converted into wrist arm displacement commands. Joint parameters are adjusted through force-position hybrid control and shape adaptive law, and then output control commands are executed through closed loop to drive the robotic arm to complete stable gripping.

2. The tactile-assisted positioning and grasping control method for a service-oriented embodied intelligent robot according to claim 1, characterized in that, Training a convolutional neural network involves the following two stages: During the simulation phase, the dataset required for training the convolutional neural network is generated using real physical data from the simulation environment. The tactile information output by the simulator and the relative positional deviation data between the actuator and the object are used as real labels for the supervised learning process of the convolutional neural network. In the practical stage, actual operating data is collected in the real robotic arm operating environment. The operating data includes tactile input signals recorded by tactile sensors and actual deviation data between the actuator and key points of the object. The pre-trained convolutional neural network trained by simulation is fine-tuned using a few-shot learning method based on the actual operating data.

3. The tactile-assisted positioning and grasping control method for a service-oriented embodied intelligent robot according to claim 1, characterized in that, The key points of the actuator include the center point of the two-finger gripper and the midpoint between the index finger and thumb on the gripper; the key points of the object need to be defined according to the actual needs of the specific grasping task.

4. The tactile-assisted positioning and grasping control method for a service-oriented embodied intelligent robot as described in claim 1, characterized in that, The actual error quantities confirmed by the closed-loop control mechanism include: The deviation vector is continuously compared with a preset threshold. If the deviation is higher than the threshold, the robotic arm is triggered to recalculate the movement command and make adjustments. If the deviation is lower than the threshold, the current state is maintained. The above loop continues until the deviation returned by the convolutional neural network is stably lower than the preset threshold, ensuring the robotic arm's operational accuracy and task execution reliability. The stable completion of the grasping task is achieved through iterative feedback.

5. The tactile-assisted positioning and grasping control method for a service-oriented embodied intelligent robot according to claim 1, characterized in that, The physical constants include the radius of the gripper block Rfinger, the radius of the target object Robj, and the safety distance threshold between the gripper and the outer perimeter of the target object, with the safety distance threshold being no less than 3 centimeters.

6. The tactile-assisted positioning and grasping control method for a service-oriented embodied intelligent robot according to claim 1, characterized in that, In step S2, combining the actual error amount and physical constants, the shortest safe path is planned and obstacles are dynamically avoided to reach the grabbing preparation area, including: Based on the 3D spatial modeling and the 3D occupancy mesh output by the 3D-R2N2 network, the coordinates of the target object and the current coordinates of the gripper are obtained. The shortest straight path between the coordinates of the target object and the current coordinates of the gripper is calculated and generated first. During the path generation process, the coordinate data of environmental obstacles are called simultaneously, and the safe distance of each point on the path is monitored and verified in real time: the radius Rfinger of the gripper block is extended outward from the center of the gripper, and the radius of the obstacle is extended outward from the center of the obstacle. The radius of non-target objects is set according to the preset general value or the size obtained by image recognition, ensuring that the distance between the extended range of the gripper and the obstacle is always not less than 3 cm. If the safety conditions are met throughout the process, the movement of the gripper is controlled. If, during the gripper's movement, real-time coordinate monitoring detects that the distance between the gripper and an obstacle is less than m centimeters, the obstacle avoidance mechanism is immediately triggered. Based on the single obstacle handling process, a virtual obstacle avoidance sphere is constructed with the geometric center of the obstacle as the center and the radius of the obstacle plus the gripper block radius Rfinger + m centimeters as the radius. The virtual obstacle avoidance sphere ensures that the gripper maintains a safe distance from the obstacle while moving along the sphere. The line connecting the gripper's current position and the target object's coordinates is calculated, and the intersection point of this line and the virtual obstacle avoidance sphere is determined. The intersection point is the shortest path node after obstacle avoidance, i.e., the sum of the distances to the gripper's current position and the target object is the shortest path in obstacle avoidance mode. The gripper is controlled to smoothly move along the virtual obstacle avoidance sphere to this intersection point, maintaining the gripper's posture during the movement to avoid increasing path time due to direction adjustments. If a new obstacle is encountered during obstacle avoidance, the judgment criterion is that the distance between the gripper and the new obstacle is less than m centimeters when the gripper moves along the previous virtual obstacle avoidance sphere. In this case, the obstacle avoidance strategy is switched: a new virtual obstacle avoidance sphere is constructed with the geometric center of the new obstacle as the center; the gripper is controlled to move from the current position along the new virtual obstacle avoidance sphere until it leaves the safe distance range of the new obstacle. The criterion for leaving the sphere is moving to a sphere node where the distance between the gripper and the new obstacle is not less than m centimeters; the shortest straight path to the target object is recalculated from this departure node as the new starting point, and the safe distance on the path is checked again; if there is no safety hazard on the path, the gripper continues to move along the straight line; if there is still an obstacle, the obstacle avoidance process is repeated until the gripper enters the target object's grasping preparation area.

7. The tactile-assisted positioning and grasping control method for a service-oriented embodied intelligent robot as described in claim 1, characterized in that, In step S2, after arriving at the grasping preparation area, the position and posture of the gripper are fine-tuned through tactile feedback, including: When the gripper reaches the outer safety circle of the target object along the planned path, it enters the gripping preparation stage, pauses the straight-line movement of the gripper, and switches to the tactile feedback adjustment mode; the outer safety circle is when the distance from the center of the gripper to the center of the target object is equal to the radius of the target object Robj + the radius of the gripper block Rfinger + m centimeters. The haptic feedback modulation mode includes the following processes: During the fine-tuning process, the gripper moves slowly toward the target object in tiny steps. During the movement, the pressure signal collected in real time by the tactile sensor is input into the convolutional neural network. If the convolutional neural network interprets the pressure intensity feature as no contact, the judgment criterion is that no effective pressure is detected, and the movement continues. If a contact signal is detected, the judgment criterion is that the pressure intensity reaches the preset threshold, and the movement stops immediately. At this time, the distance between the gripper and the target object is maintained at about m centimeters, which meets the initial conditions for safe gripping. During the posture fine-tuning stage, when the pressure distribution center of gravity features analyzed by the convolutional neural network have a horizontal or vertical offset compared with the preset optimal gripping reference value, the deviation data is input into the feedforward neural network, which outputs the corresponding posture correction parameters. The robotic arm drives the gripper to fine-tune the angle around the center of the target object according to the posture correction parameters. After each adjustment, the tactile signal is re-acquired until the pressure distribution center of gravity features fall within the error allowable range. After the attitude calibration is completed, the system triggers the gripper closing command: the gripper retracts at a uniform speed of 1 cm per second until the tactile sensor detects that the gripping force has reached a safe and stable threshold. The safe and stable threshold is dynamically adjusted according to the radius Robj of the target object. At the same time, the long short-term memory network analyzes the change curve of the gripping force over time in real time. If it predicts that there is a risk of the object slipping, it immediately outputs a force compensation command to fine-tune the gripper closing degree. Once the gripping force is maintained stably for more than n seconds and the deviation output by the convolutional neural network is lower than the preset threshold, the gripping is determined to be complete, and subsequent moving or placing operations can be performed.

8. The tactile-assisted positioning and grasping control method for a service-oriented embodied intelligent robot as described in claim 1, characterized in that, Step S3 includes the following steps: S31. First, coordinate transformation is performed, and displacement target information is obtained by combining the data returned by the sensor. Then, the transformation relationship between the target coordinate system and the robot arm base coordinate system is established to provide accurate pose basis for the subsequent generation of wrist arm displacement commands. S32. Based on the pose of the target object in the base coordinate system obtained by coordinate transformation, the robotic arm actuator converts the pose into a wrist arm displacement command, drives the wrist arm to complete the corresponding displacement, and makes the gripper approach the target gripping position. S33. The robotic arm end effector employs a force-position hybrid control mechanism to achieve precise control of the finger and wrist ends: the desired trajectory of the end effector is defined as follows: The actual trajectory is The positional deviation is The speed correction of the end effector is calculated in real time using the impedance control equation, which is: in , , These are the inertia, damping, and stiffness matrices, respectively. For the desired contact force, The actual contact force is collected by a force sensor. express The positional deviation of the end effector at any given moment; The first derivative of the positional deviation, i.e., the velocity deviation; The second derivative of the position deviation, i.e., the acceleration deviation; This represents the current time variable; To address the uncertainty in the shape of the target object, the joint parameters of the robotic arm are controlled by an adaptive mechanism. The core principle is to construct an adaptive law based on end-effector force feedback and positional deviation. in These are estimated values ​​for joint parameters. It is a positive definite adaptive gain matrix. Let Jacobian matrix be the current time step. This is a force estimate based on the current parameters; through this adaptive law, the robotic arm can dynamically correct the joint angles during the grasping process, so that... Ultimately, this enables stable gripping of objects of different shapes and materials; Transforming velocity corrections and joint parameter adjustments into actual control commands: Utilizing the inverse kinematics of the robotic arm, from the end-effector velocity... Solve for joint velocities: in This is the pseudo-inverse of the Jacobian matrix. express The angular velocity vectors of each joint of the robotic arm at any given time; express The end effector's linear velocity and angular velocity vector at any given moment; the joint angle command is obtained through integration. : in The initial joint angle is set; the joint angle command is sent to each joint motor, driving the motor to execute the action and complete the grasping task.

9. A computer device comprising a memory and a processor, the memory being electrically connected to the processor, the memory storing a computer program, characterized in that: When the computer program is executed by the processor, it causes the processor to implement the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the processor implements the method as described in any one of claims 1 to 8.

Images