A robot manipulator grasping and docking method, device, medium and product

Abstract

The application discloses a robot manipulator grabbing and docking method, equipment, medium and product, relates to the field of robots and sensor perception, and the method comprises the following steps: constructing a visual-haptic combined data set of a manipulator in the process of grabbing a filter hose; establishing a haptic prior knowledge learning model according to the visual-haptic combined data set; outputting a current grabbing configuration according to the haptic prior knowledge learning model; obtaining motion planning information of the manipulator according to the current grabbing configuration; controlling the manipulator to dock the filter hose according to the motion planning information of the manipulator, and performing mechanical modeling on the contact state of a filter hose connector and a machine tool docking interface to obtain a mechanical control model; and obtaining pose information and contact force information in the docking process of the manipulator by using a DDPG parameter identification algorithm according to the mechanical control model. The application can more efficiently and stably grab and dock the hose of the cutting fluid filtering device.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of robots and sensor perception, and in particular, to a robot manipulator grasping and docking method, device, medium and product. BACKGROUND

[0002] In today's industrial production field, whether it is an industrial robot or a service robot, a manipulator is an indispensable component. In order to complete various tasks, stable grasping and assembly functions are one of the necessary functions of the manipulator. With the continuous development of intelligent robots, increasing the ability of the manipulator to perceive the environment can make the manipulator more like a human being and have a more efficient work efficiency.

[0003] The accuracy and efficiency of the grasping action and assembly process of the robot depend largely on complex vision and tactile feedback. Vision-based grasping methods can be divided into perception-based grasping detection methods and deep learning-based grasping detection methods. The perception-based method identifies the structure or features of the grasped object through vision, generates and sorts candidate grasping poses, while the deep learning-based grasping detection method inputs data into a convolutional neural network, uses a classifier to determine which part of the image has the highest success rate of grasping, and outputs the best grasping detection box. However, robot grasping is a physical contact process closely related to the inherent properties of the object, the characteristics of the manipulator, and the relative friction. In the process of docking the filter hose and the docking port, the hose is easy to bend and sway, and visual alone cannot accurately locate the position of its end, so tactile feedback is needed to adjust the pose of the manipulator.

[0004] In order to better represent the physical contact between the manipulator and the target object, tactile sensing is used to perceive the subtle changes in contact force during the operation process to assess the grasping quality in real time. Therefore, tactile perception plays a very important role in the process of grasping and docking by the manipulator; and how the existing cutting fluid filter device and robot (including the manipulator and two-fingered manipulator) can more efficiently and stably grasp and dock the hose of the cutting fluid filter device based on vision-tactile fusion is a problem that needs to be solved. SUMMARY

[0005] The purpose of the present application is to provide a robot manipulator grasping and docking method, device, medium and product, which can more efficiently and stably grasp and dock the hose of the cutting fluid filter device.

[0006] To achieve the above purpose, the present application provides the following solutions:

[0007] In a first aspect, the application provides a robot manipulator grasping and docking method, the robot comprising: a mechanical arm, a manipulator, and a filter hose of a cutting fluid filtering device; the manipulator comprising: a two-fingered gripper, a depth camera, and a 6-dimensional force sensor; the robot manipulator grasping and docking method comprising:

[0008] constructing a visual-haptic joint dataset of the manipulator in the process of grasping the filter hose; the visual-haptic joint dataset comprising: acquiring a visual image of the filter hose by the depth camera, obtaining pressure value information of grasping the filter hose by the 6-dimensional force sensor, and obtaining a quality evaluation index according to the pressure value information; the quality evaluation index comprising: falling, sliding, stability, and overload;

[0009] establishing a haptic priori knowledge learning model according to the visual-haptic joint dataset; the haptic priori knowledge learning model is used to find a grasping configuration that satisfies the quality evaluation index of stability from the image space based on the learned haptic priori knowledge; the grasping configuration comprising: grasping position, grasping angle, grasping width, and grasping depth;

[0010] outputting a current grasping configuration according to the haptic priori knowledge learning model; and obtaining motion planning information of the manipulator according to the current grasping configuration;

[0011] controlling the manipulator to dock the filter hose according to the motion planning information of the manipulator, and performing mechanical modeling on the contact state of the filter hose docking head and the machine tool docking interface to obtain a mechanical control model; the mechanical control model is used to represent the relationship between the contact force and torque between the filter hose docking head and the machine tool docking interface and the robot end pose adjustment amount;

[0012] obtaining pose information and contact force information in the docking process of the manipulator according to the mechanical control model by using a DDPG parameter identification algorithm.

[0013] Optionally, the constructing a visual-haptic joint dataset of the manipulator in the process of grasping the filter hose specifically comprises:

[0014] acquiring a visual image of the filter hose by the depth camera;

[0015] obtaining pressure value information of grasping the filter hose by the 6-dimensional force sensor;

[0016] converting the pressure value information into an RGB image form to obtain a haptic image;

[0017] obtaining a grasping quality evaluation index according to the haptic image and a convolution kernel; the convolution kernel is a 2x2 convolution kernel, comprising 1 and -1;

[0018] labeling the visual image and the haptic image by the grasping quality evaluation index.

[0019] Optionally, the step of determining the grasping quality evaluation index specifically comprises:

[0020]

[0021] Wherein, t represents the maximum value after the tactile image of the falling object and the convolution kernel convolution operation, the smaller t is, the earlier the falling is, and the worse the grasping stability is; s represents the element sum after the tactile image of the non-falling object and the convolution kernel convolution operation, the smaller s is, the smaller the pressure value change is, the relatively low the sliding intensity is, and the more stable the object is, and e is the retardation logarithm.

[0022] Optionally, the current grasping configuration is output according to the tactile prior knowledge learning model; and the motion planning information of the manipulator is obtained according to the current grasping configuration, specifically comprising:

[0023] The current grasping configuration is converted into the motion planning information g of the manipulator in the world coordinate system by using the formula g = f(I), q(T); R T c f(I), q(T));

[0024] Wherein, is a visual image space, represents a pixel point and a depth value of a target object in the visual image space, is a rotation angle of the target object in the visual image space relative to a camera coordinate system, is a pixel width of the target object in the visual image space, T = (m, n) is a tactile image space, and (m, n) is a pixel point coordinate of the tactile image space, R T c represents a coordinate conversion from the camera coordinate system to the robot coordinate system, is obtained from a camera extrinsic parameter calibration result, f(I) represents a transformation matrix from a 2D image coordinate system to a 3D camera coordinate system, is obtained from a camera intrinsic parameter calibration result, q(T) is the quality evaluation index, u is a position of the target object on an x-axis in the camera coordinate system, v is a position of the target object on a y-axis in the camera coordinate system, (u, v) is pixel point position information of the target object in the camera coordinate system, and d is a position of the target object on a z-axis in the camera coordinate system, that is, depth information of the target object.

[0025] Optionally, the motion planning information of the manipulator is used to control the manipulator to connect the filter hose, and a mechanical model of a contact state between the filter hose connector and the machine tool connector is established to obtain a mechanical control model, specifically comprising:

[0026] The transfer function H(l) of the controller is determined by using the formula wherein, l is a Laplace transform variable of a continuous time system;

[0027] The Tustin transformation formula is used The transfer function H(l) of the controller is discretized to obtain the formula

[0028] The formula is used The parameters A, C and D are determined;

[0029] The formula is used The relationship between the contact force and torque between the filter hose and the joint and the robot end pose adjustment is determined;

[0030] Where, ΔF is the difference between the force applied to the robot end and the expected force, ΔX is the robot pose correction amount, M is the virtual inertia parameter, B is the virtual damping parameter, K is the virtual stiffness parameter, F(k) is the difference between the contact force and the expected force at the kth time, F(k-1) is the difference between the contact force and the expected force at the k-1th time, F(k-2) is the difference between the contact force and the expected force at the k-2th time, X(k) is the pose correction amount of the robot at the kth time, X(k-2) is the pose correction amount of the robot at the k-2th time, X(k-1) is the pose correction amount of the robot at the k-1th time, T is the sampling period, and z is the Z transform variable of the discrete time system.

[0031] Optionally, the DDPG parameter identification algorithm uses an Actor neural network and a Critic network for updating.

[0032] In a second aspect, the application provides a robot manipulator grasping and docking device, the robot comprising: a mechanical arm, a manipulator, and a filter hose of a cutting fluid filtering device; the manipulator comprising: a two-fingered gripper, a depth camera, and a 6-dimensional mechanical sensor; the robot manipulator grasping and docking device comprising:

[0033] A data set construction module for constructing a visual-haptic joint data set of the manipulator in the process of grasping the filter hose; the visual-haptic joint data set comprising: a visual image of the filter hose acquired by the depth camera, pressure value information of grasping the filter hose obtained by the 6-dimensional mechanical sensor, and a quality evaluation index obtained according to the pressure value information; the quality evaluation index comprising: falling, sliding, stability, and overload;

[0034] A prior model construction module for establishing a haptic prior knowledge learning model according to the visual-haptic joint data set; the haptic prior knowledge learning model is used to find a grasping configuration that satisfies the quality evaluation index of stability from the image space based on the learned haptic prior knowledge; the grasping configuration comprising: grasping position, grasping angle, grasping width, and grasping depth;

[0035] The motion planning information determination module is used to output the current grasping configuration based on the learning model of tactile prior knowledge; and to obtain the motion planning information of the robotic arm based on the current grasping configuration.

[0036] The mechanical control model construction module is used to control the robot arm to dock with the filter hose based on the robot arm's motion planning information, and to perform mechanical modeling on the contact state between the filter hose connector and the machine tool interface to obtain the mechanical control model; the mechanical control model is used to characterize the relationship between the contact force and torque between the filter hose connector and the machine tool interface and the robot end-effector pose adjustment amount;

[0037] The strategy determination module is used to obtain the pose and contact force information of the robot arm during the docking process based on the mechanical control model and the DDPG parameter identification algorithm.

[0038] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the robotic arm grasping and docking method described above.

[0039] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the described robotic arm grasping and docking method.

[0040] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the described robotic arm grasping and docking method.

[0041] According to the specific embodiments provided in this application, this application has the following technical effects:

[0042] This application provides a robotic gripper method, device, medium, and product for gripping and docking. During gripping, a tactile prior knowledge learning model is first established based on a combined visual-tactile dataset to learn the gripping pose of the filter device's hose. Then, a gripping strategy is generated based on the tactile prior knowledge and visual images. During docking, a mechanical model of the contact force between the hose and the machine tool interface is first performed, allowing the robot to adjust its axis pose according to the contact force to achieve compliant assembly. Then, based on the mechanical control model, the DDPG parameter identification algorithm is used to process the input information during docking, generating a virtual damping parameter adjustment strategy, enabling the robot to efficiently complete the hose docking. The DDPG parameter identification algorithm addresses the problem that both state and control are continuous variables during assembly, learning the optimal adjustment strategy through interaction with the environment. This application enables more efficient and stable gripping and docking of cutting fluid filter device hoses. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a schematic diagram of a robot arm grasping and docking method in one embodiment of this application;

[0045] Figure 2 This is a schematic diagram illustrating the principle of a robotic arm grasping and docking method in one embodiment of this application;

[0046] Figure 3 This is a flowchart of the capture process provided in one embodiment of this application;

[0047] Figure 4 This is a flowchart of the docking process in one embodiment of this application;

[0048] Figure 5 This is a schematic diagram of the approaching stage;

[0049] Figure 6 This is a schematic diagram showing the contact state of the hose and the interface provided in one embodiment of this application; Detailed Implementation

[0050] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0051] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] In one exemplary embodiment, such as Figure 1 and Figure 2 As shown, a method for gripping and docking with a robotic arm is provided. The robot includes: a robotic arm, a robotic hand, and a filter hose for a cutting fluid filtration device; the robotic hand includes: a two-finger gripper, a depth camera, and a 6-dimensional force sensor; the method includes the following steps S101 to S105. Wherein:

[0053] S101, construct a visual-haptic joint data set in the process of the manipulator grabbing the filter hose; the visual-haptic joint data set comprises: collecting a visual image of the filter hose by a depth camera, obtaining pressure value information of grabbing the filter hose by a 6-dimensional mechanical sensor, and obtaining a quality evaluation index according to the pressure value information; the quality evaluation index comprises: falling, sliding, stability and overload; wherein the falling is manifested as a large change in the pressure value of one contact point; the sliding is manifested as the pressure values of adjacent contact points changing in opposite directions; the stability is manifested as a state with little sliding; and the overload is manifested as the pressure values of two contact points continuously showing high values. The quality evaluation index represents the stability of grabbing by a continuous value of 0-1.

[0054] The determination step of the grabbing quality evaluation index specifically comprises:

[0055]

[0056] Wherein t represents the maximum value after the haptic image of the falling object and the convolution kernel convolution operation, the smaller t is, the earlier the falling is, and the worse the grabbing stability is; s represents the element sum after the haptic image of the non-falling object and the convolution kernel convolution operation, the smaller s is, the smaller the pressure value change is, the relatively low the sliding intensity is, and the more stable the object is, and e is the fire retardant logarithm.

[0057] S101 specifically comprises:

[0058] S11, collecting a visual image of the filter hose by a depth camera;

[0059] S12, obtaining pressure value information of grabbing the filter hose by a 6-dimensional mechanical sensor;

[0060] S13, converting the pressure value information into the form of an RGB image to obtain a haptic image;

[0061] S14, obtaining a grabbing quality evaluation index according to the convolution of the haptic image and the convolution kernel; the convolution kernel is a 2x2 convolution kernel comprising 1 and -1; the over convolution operation can calculate the pressure value change of adjacent contact points.

[0062] S15, marking the visual image and the haptic image by the grabbing quality evaluation index.

[0063] S102, establishing a haptic priori knowledge learning model according to the visual-haptic joint data set, and as shown in Figure 3 The haptic priori knowledge learning model is used to find a stable grabbing configuration that meets the quality evaluation index from the image space based on the learned haptic priori knowledge; the grabbing configuration comprises: a grabbing position, a grabbing angle, a grabbing width and a grabbing depth;

[0064] Specifically, based on the combined visual and tactile dataset, the ResNet-50 model was used as the feature extractor, the Adam optimization method was used for network training, the mean squared error (MSE loss) was used as the loss function, and the PyTorch framework was used for training to establish a tactile prior knowledge learning model.

[0065] As a specific implementation, the tactile prior knowledge learning model represents the mapping relationship between visual images and grasping quality. The process of determining the mapping relationship is as follows:

[0066] S1 takes the images in the dataset as input to the network and contains three channels: R, G, and B.

[0067] S2 generates a feature map through the feature extractor.

[0068] S3, pooling and linear regression yield a single regression value, namely the grasping quality score corresponding to the tactile image.

[0069] S103, output the current grasping configuration based on the tactile prior knowledge learning model; and obtain the motion planning information of the robotic arm based on the current grasping configuration;

[0070] Since the mechanical analysis process of space docking is similar to that of planar docking, a planar docking model is used for ease of analysis. The equations for planar gripping can be expressed as follows:

[0071] g = (p, θ, q);

[0072] Where p = (x, y, z) represents the position of the center point of the two-finger hand, θ represents the angle of rotation of the two-finger hand around the z-axis, and q represents the grasping mass. x and y are the coordinates of the center point of the two-finger hand on the x-axis and y-axis, respectively.

[0073] Using the formula g = ( R T c f(I),q(T)) transforms the current grasping configuration into motion planning information g for the manipulator in the world coordinate system;

[0074] in, For visual image space, Represents the pixel points and depth values ​​of the target object within the visual image space. Let be the rotation angle of the target object in the visual image space relative to the camera coordinate system. Let T = (m,n) be the pixel width of the target object in the visual image space, and let (m,n) be the tactile image space, where (m,n) are the pixel coordinates in the tactile image space. R T cThe coordinate transformation from the camera coordinate system to the robot coordinate system is obtained from the camera extrinsic calibration result. f(I) represents the transformation matrix from the 2D image coordinate system to the 3D camera coordinate system, which is obtained from the camera intrinsic calibration result. q(T) is the quality evaluation index. u is the position of the target object on the x-axis in the camera coordinate system. v is the position of the target object on the y-axis in the camera coordinate system. (u,v) is the pixel position information of the target object in the camera coordinate system. d is the position of the target object on the z-axis in the camera coordinate system, which is the depth information of the target object.

[0075] S104, based on the motion planning information of the robot arm, control the robot arm to dock with the filter hose, and perform mechanical modeling on the contact state between the filter hose connector and the machine tool interface to obtain a mechanical control model; the mechanical control model is used to characterize the relationship between the contact force and torque between the filter hose connector and the machine tool interface and the robot end-effector pose adjustment amount;

[0076] To analyze the relationship between contact state and force value during the docking process, a mechanical model of the contact force at the interface between the hose and the machine tool is performed. For ease of analysis, the docking is modeled as a planar process, and the docking process can be approximated as a shaft-hole assembly problem.

[0077] The entire docking process can be divided into three stages based on the relative positions of the hose and the mating interface: the approach stage, the single-point contact stage, and the two-point contact stage. Only the single-point contact and two-point contact stages are subject to force.

[0078] A schematic diagram of the approach phase is shown below. Figure 5 As shown. Figure 5 In the diagram, u0 represents the distance between the vertical line connecting the center of the robotic arm and the vertical line connecting the center of the hose gripping point during the approach phase, serving as an initial relative position; r represents the radius of the hose; R represents the radius of the interface; l g θ0 indicates the length of the hose to be connected; θ0 indicates the angle between the centerline of the hose and the centerline of the interface during the approach phase.

[0079] The force analysis process for single-point contact is as follows:

[0080] S41, when the shaft and the hole are in single-point contact, the state is as follows: Figure 6 As shown in part (a). Geometric analysis of the contact state yields:

[0081] u=(l g -l)·sinθ+Rr (1)

[0082] Where u represents the distance between the vertical line where the center of the robot arm is located and the vertical line where the center of the hose gripping point is located during the single-point contact phase; and represents the length of the portion below the contact point between the hose and the interface.

[0083] S42, because θ is small, so sinθ≈θ, formula (1) can be converted to:

[0084]

[0085] S43, the force balance equation of the contact point is as follows:

[0086]

[0087] In the formula, F1, F2 and M are the force and torque exerted by the gripper on the hose, FN is the reaction force of the interface on the hose at the contact point, and f is the friction force of the interface on the hose at the contact point.

[0088] S44, let the lateral elastic coefficient be k x , and the torsional elastic coefficient be k θ , then formula (3) can be converted to:

[0089]

[0090] Where μ is the friction factor, and θ is the angle between the center line of the hose and the center line of the interface in the single-point contact stage.

[0091] The force analysis process of two-point contact is as follows:

[0092] S51, when the shaft and the hole are in two-point contact, the state is shown in part (b) of Figure 6 The geometric analysis of the contact state is as follows:

[0093]

[0094] S52, the force balance equation is:

[0095]

[0096] Where f1 is the friction force of the interface on the hose at the left contact point; f2 is the friction force of the interface on the hose at the right contact point.

[0097] S53, by combining formula (5) and formula (6), we get:

[0098]

[0099] In formula (6):

[0100]

[0101] The transfer function of the controller is defined as:

[0102] According to the Tustin transformation formula After discretization

[0103] The formula is used to determine the parameters A, C and D.

[0104] The formula is used to determine the pose correction amount X(k) of the manipulator at the kth time.

[0105] Where ΔF is the difference between the force applied to the robot end and the expected force, ΔX is the robot pose correction amount, M is the virtual inertia parameter, B is the virtual damping parameter, K is the virtual stiffness parameter, F(k) is the difference between the contact force and the expected force at the kth time, F(k-1) is the difference between the contact force and the expected force at the k-1th time, F(k-2) is the difference between the contact force and the expected force at the k-2th time, X(k-2) is the pose correction amount of the manipulator at the k-2th time, X(k-1) is the pose correction amount of the manipulator at the k-1th time, T is the sampling period, and z is the Z transform variable of the discrete-time system.

[0106] S105, according to the mechanical control model, using DDPG parameter identification algorithm, get the pose information and contact force information of the manipulator during the docking process, and as shown in Figure 4 .

[0107] The purpose of the DDPG parameter identification algorithm is to learn the adjustment strategy of the damping parameter B. The adjustment process of the parameter B can be described as a Markov decision process. For an agent, the long-term reward R t is maximized to improve the learning strategy, as follows:

[0108]

[0109] In the formula, γ is the discount rate of the reward, γ ∈ (0, 1), and the reward value r t depends on the good or bad of the action a t performed at state s t .

[0110] The strategy π is a mapping from state to action, and the action selection strategy π is defined as follows:

[0111] a = π (s | θ π ).

[0112] In the formula, the strategy π is represented using an Actor neural network. θ π is the weight of the Actor network.

[0113] During the docking process, the effects of force, torque, axis attitude and insertion depth need to be considered. Therefore, the state of the environment consists of 8 elements, which are defined as follows: ​​

[0114] s t = [F x , F y , F z , M x , M y , θ x , θ y , h];

[0115] where s t is the environment state at time t, F and M are the force and torque measured by the 6D force sensor, θ x and θ y are the rotation angles of the end of the robot arm around the x-axis and y-axis of the end coordinate system, and h is the insertion depth.

[0116] The parameters θ π of the strategy π are updated by , where β is the learning rate, γ is the discount rate of the reward, ▽ represents the gradient, and t represents the time;

[0117] The parameters θ q of the strategy π are updated by , where y = r t + γq[s t , π(s t | θ π )].

[0118] where q(s t , a | θ q ) is the action value function, represented using a Critic network, and θ q is the weight of the Critic network, and α and β are the learning rates of the network.

[0119] In the process of grabbing, the robot hand needs to correctly identify the position of the hose and grab it with the appropriate pose. Since the hose is prone to bending and deformation, the position of the grab and the size of the force applied need to be controlled. During the process of connecting the hose and the machine tool interface, the position of the interface needs to be identified first, and then the pose of the robot hand is adjusted in real time according to the force feedback information of the 6D force sensor, so that the connection task can be completed efficiently and filtered. In order to achieve the effect of filtering, the filter has a water inlet pipe and a water outlet pipe that need to be connected. After successful connection, the filter starts to extract cutting fluid waste, and the waste is sent back to the waste tank through a filtering device composed of a semiconductor inorganic membrane, and then it is extracted and filtered again. During the filtering process, the sensor in the filter detects the concentration of cutting fluid and other parameters in real time until the cutting fluid meets the requirements for continued use. Further, the application can effectively solve the problem of filtering cutting fluid waste generated during the machining process of the machine tool, not only reducing the processing cost of waste liquid, but also being beneficial to environmental protection.

[0120] Based on the same inventive concept, the embodiments of the present application also provide a robot manipulator grasping and docking device for implementing the robot manipulator grasping and docking method described above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more robot manipulator grasping and docking device embodiments provided below can refer to the limitations of the robot manipulator grasping and docking method described above, and will not be repeated here.

[0121] In an exemplary embodiment, a robot manipulator grasping and docking device is provided, comprising:

[0122] A data set construction module is configured to construct a visual-haptic joint data set of the manipulator in the process of grasping the filter hose; the visual-haptic joint data set comprises: a visual image of the filter hose collected by a depth camera, pressure value information of grasping the filter hose obtained by a 6-dimensional mechanical sensor, and a quality evaluation index obtained according to the pressure value information; the quality evaluation index comprises: falling, sliding, stability, and overload;

[0123] A prior model construction module is configured to establish a haptic prior knowledge learning model according to the visual-haptic joint data set; the haptic prior knowledge learning model is used to find a grasping configuration that satisfies the quality evaluation index of stability from the image space based on the learned haptic prior knowledge; the grasping configuration comprises: grasping position, grasping angle, grasping width, and grasping depth;

[0124] A motion planning information determination module is configured to output a current grasping configuration according to the haptic prior knowledge learning model; and obtain motion planning information of the manipulator according to the current grasping configuration;

[0125] A mechanical control model construction module is configured to control the manipulator to dock the filter hose according to the motion planning information of the manipulator, and to model the contact state between the filter hose docking head and the machine tool docking interface to obtain a mechanical control model; the mechanical control model is used to represent the relationship between the contact force and torque between the filter hose docking head and the machine tool docking interface and the adjustment amount of the robot end position and orientation;

[0126] A strategy determination module is configured to obtain the position and orientation information and contact force information in the process of the manipulator docking according to the mechanical control model by using a DDPG parameter identification algorithm.

[0127] In an example embodiment, a computer device is provided, which can be a server or a terminal. The computer device includes a processor, a memory, an input / output interface (I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to communicate with external terminals through a network connection. The computer program is executed by the processor to implement a robot manipulator grasping and docking method.

[0128] In an example embodiment, a computer readable storage medium is provided, which stores a computer program. The computer program is executed by a processor to implement the steps in the above method embodiments.

[0129] In an example embodiment, a computer program product is provided, which includes a computer program. The computer program is executed by a processor to implement the steps in the above method embodiments.

[0130] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of the related data need to comply with relevant regulations.

[0131] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc.

[0132] The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a blockchain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0133] In the present application, all actions of obtaining signals, information or data are performed in accordance with the corresponding data protection regulations and policies of the country where the device is located, and with the authorization of the owner of the corresponding device.

[0134] The technical features of the above embodiments can be combined arbitrarily. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, but as long as the combination of the technical features does not exist, it should be considered as the scope of the present application.

[0135] The principles and implementation manners of the present application are described herein by using specific examples, and the above examples are only used to help understand the method of the present application and its core idea; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manners and application ranges will have changes. In conclusion, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A robot manipulator grasping and docking method, characterized by, Robots include: The robotic arm, robotic hand, and filter hose of the cutting fluid filtration device; the robotic hand includes: a two-finger gripper, a depth camera, and a 6-dimensional force sensor; the robotic hand's grasping and docking method includes: A combined visual-tactile dataset is constructed for the process of a robotic arm grasping a filter hose. The combined visual-tactile dataset includes: visual images of the filter hose acquired by a depth camera, pressure value information of grasping the filter hose obtained by a 6D mechanical sensor, and quality evaluation indicators obtained based on the pressure value information. The quality evaluation indicators include: drop, slippage, stability, and overload. A tactile prior knowledge learning model is established based on a vision-tactile joint dataset; the tactile prior knowledge learning model is used to find a stable grasping configuration that meets the quality evaluation index from the image space based on the learned tactile prior knowledge; the grasping configuration includes: grasping position, grasping angle, grasping width, and grasping depth; The current grasping configuration is output based on the tactile prior knowledge learning model; and the motion planning information of the robotic arm is obtained based on the current grasping configuration. Based on the motion planning information of the robot arm, the robot arm is controlled to dock with the filter hose, and the contact state between the filter hose connector and the machine tool interface is mechanically modeled to obtain a mechanical control model; the mechanical control model is used to characterize the relationship between the contact force and torque between the filter hose connector and the machine tool interface and the robot end-effector pose adjustment amount; Based on the mechanical control model, the DDPG parameter identification algorithm is used to obtain the pose and contact force information of the robot during the docking process.

2. The robotic manipulator grasping and docking method of claim 1, wherein, The dataset for constructing the combined visual and tactile feedback of the robotic arm during the process of grasping the filter hose specifically includes: Visual images of the filter hose are acquired using a depth camera; The pressure value information for gripping the filter hose is obtained through a 6-dimensional mechanical sensor; The pressure value information is converted into an RGB image to obtain a tactile image; The grasping quality evaluation index is obtained by convolving the tactile image with a convolution kernel; the convolution kernel is a 2×2 convolution kernel, including 1 and -1; Visual and tactile images are labeled by capturing quality evaluation indicators.

3. The robotic manipulator grasping and docking method of claim 1 or claim 2, wherein, The specific steps for determining the quality evaluation indicators include: Where t represents the maximum value after convolving the tactile image of the fallen object with the convolution kernel. The smaller t is, the earlier the object fell, indicating poorer grasping stability. s represents the sum of elements after convolving the tactile image of the non-fallen object with the convolution kernel. The smaller s is, the smaller the pressure value change, the relatively low sliding intensity, and the more stable the object. e is the flame retardant logarithm.

4. The robotic gripper grasping and docking method according to claim 1, characterized in that, The current grasping configuration is output based on the tactile prior knowledge learning model; and the motion planning information of the robotic arm is obtained based on the current grasping configuration, specifically including: Using the formula g = ( R T c f(I), q(T)) to convert the current grasp configuration into the motion planning information g of the manipulator in the world coordinate system; in, For visual image space, Represents the pixel points and depth values ​​of the target object within the visual image space. Let be the rotation angle of the target object in the visual image space relative to the camera coordinate system. Let T = (m,n) be the pixel width of the target object in the visual image space, and let (m,n) be the tactile image space, where (m,n) are the pixel coordinates in the tactile image space. R T c The coordinate transformation from the camera coordinate system to the robot coordinate system is obtained from the camera extrinsic calibration result. f(I) represents the transformation matrix from the 2D image coordinate system to the 3D camera coordinate system, which is obtained from the camera intrinsic calibration result. q(T) is the quality evaluation index. u is the position of the target object on the x-axis in the camera coordinate system. v is the position of the target object on the y-axis in the camera coordinate system. (u,v) is the pixel position information of the target object in the camera coordinate system. d is the position of the target object on the z-axis in the camera coordinate system, which is the depth information of the target object.

5. The robotic gripper grasping and docking method according to claim 1, characterized in that, The process involves controlling the robot arm to dock with the filter hose based on the robot arm's motion planning information, and performing mechanical modeling on the contact state between the filter hose connector and the machine tool interface to obtain a mechanical control model, specifically including: Using formula Determine the transfer function H(l) of the controller, where l is the Laplace transform variable of the continuous-time system; Using Tustin Transform Discretize the controller's transfer function H(l) to obtain the formula. Using formula Determine parameters A, C, and D; Using formula Determine the relationship between the contact force and torque between the filter hose connector and the machine tool interface and the robot end effector pose adjustment amount; Where ΔF is the difference between the force applied to the robot end effector and the desired force, ΔX is the robot pose correction, M is the virtual inertial parameter, B is the virtual damping parameter, K is the virtual stiffness parameter, F(k) is the difference between the contact force and the desired force at time k, F(k-1) is the difference between the contact force and the desired force at time k-1, F(k-2) is the difference between the contact force and the desired force at time k-2, X(k) is the pose correction of the manipulator at time k, X(k-2) is the pose correction of the manipulator at time k-2, X(k-1) is the pose correction of the manipulator at time k-1, T is the sampling period, and z is the Z-transform variable of the discrete-time system.

6. The robotic gripper grasping and docking method according to claim 1, characterized in that, The DDPG parameter identification algorithm is updated using an Actor neural network and a Critic network.

7. A robotic gripper for grasping and docking, characterized in that, Robots include: The robotic arm, robotic hand, and filter hose of the cutting fluid filtration device; the robotic hand includes: a two-finger gripper, a depth camera, and a 6D force sensor; the robotic hand's grasping and docking equipment includes: A dataset construction module is used to construct a combined visual-tactile dataset of the robotic arm during the process of grasping the filter hose. The combined visual-tactile dataset includes: visual images of the filter hose acquired by a depth camera, pressure value information of grasping the filter hose obtained by a 6D mechanical sensor, and quality evaluation indicators obtained based on the pressure value information. The quality evaluation indicators include: drop, slippage, stability, and overload. The prior model construction module is used to build a tactile prior knowledge learning model based on the visual-tactile joint dataset; the tactile prior knowledge learning model is used to find a stable grasping configuration that meets the quality evaluation index from the image space based on the learned tactile prior knowledge; the grasping configuration includes: grasping position, grasping angle, grasping width and grasping depth; The motion planning information determination module is used to output the current grasping configuration based on the learning model of tactile prior knowledge; and to obtain the motion planning information of the robotic arm based on the current grasping configuration. The mechanical control model construction module is used to control the robot arm to dock with the filter hose based on the robot arm's motion planning information, and to perform mechanical modeling on the contact state between the filter hose connector and the machine tool interface to obtain the mechanical control model; the mechanical control model is used to characterize the relationship between the contact force and torque between the filter hose connector and the machine tool interface and the robot end-effector pose adjustment amount; The strategy determination module is used to obtain the pose and contact force information of the robot arm during the docking process based on the mechanical control model and the DDPG parameter identification algorithm.

8. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the robotic gripper grasping and docking method according to any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the robotic gripper grasping and docking method as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the robotic gripper grasping and docking method as described in any one of claims 1-6.

Images