Robot manipulator grabbing and butt joint method and equipment, medium and product

Abstract

The invention discloses a robot manipulator grabbing and docking method and device, a medium and a product, and relates to the field of robot and sensor perception.The method comprises the steps that a visual-tactile joint data set of a manipulator in the process of grabbing a filter hose is constructed; establishing a tactile priori knowledge learning model according to the visual-tactile combined data set; outputting a current grabbing configuration according to the tactile priori knowledge learning model; motion planning information of the manipulator is obtained according to the current grabbing configuration; the manipulator is controlled to be in butt joint with the filter hose according to the motion planning information of the manipulator, mechanical modeling is conducted on the contact state of a butt joint of the filter hose and a butt joint of a machine tool, and a mechanical control model is obtained; and according to the mechanical control model, a DDPG parameter identification algorithm is adopted, and pose information and contact force information in the mechanical arm butt joint process are obtained. According to the flexible pipe grabbing and butting device, grabbing and butting of the flexible pipe of the cutting fluid filtering device can be more efficiently and stably carried out.

Description

Technical Field

[0001] The present application relates to the field of robots and sensor perception, and in particular to a robot manipulator grasping and docking method, equipment, medium and product. Background Art

[0002] In today's industrial production field, whether it is an industrial robot or a service robot, the manipulator is an indispensable component. In order to complete various tasks, stable grasping and assembly functions are one of the necessary functions of the manipulator. With the continuous development of intelligent robots, adding the ability to perceive the environment to the manipulator can make the manipulator more like a human and have more efficient work efficiency.

[0003] The accuracy and efficiency of the robot's grasping action and assembly process rely heavily on complex visual and tactile feedback. Vision-based grasping methods can be divided into perception-based grasping detection methods and deep learning-based grasping detection methods. Perception-based methods visually identify the structure or features of the grasped object to generate and sort candidate grasping postures, while deep learning-based grasping detection methods input data into a convolutional neural network, use a classifier to determine which part of the image has the highest grasping success rate, and output the best grasping detection frame. However, robot grasping is a physical contact process that is closely related to the inherent properties of the object, the characteristics of the manipulator, and relative friction. In the process of docking the filter hose and the docking port, because the hose is easy to bend and shake, the position of its end cannot be accurately located by vision alone, so tactile feedback is more needed to adjust the posture of the manipulator.

[0004] In order to better characterize the physical contact between the manipulator and the target object, touch is used to perceive the subtle changes in contact force during the operation to evaluate the grasping quality in real time. Therefore, tactile perception plays a very important role in the grasping and docking process of the manipulator; and how to grasp and dock the hose of the cutting fluid filter device more efficiently and stably based on visual-tactile fusion for existing cutting fluid filter devices and robots (including manipulators and two-finger manipulators) is now an urgent problem to be solved. Summary of the invention

[0005] The purpose of the present application is to provide a robot manipulator grasping and docking method, equipment, medium and product, which can grasp and dock the hose of the cutting fluid filter device more efficiently and stably.

[0006] To achieve the above objectives, this application provides the following solutions:

[0007] In the first aspect, the present application provides a robot manipulator grasping and docking method, the robot comprises: a robot arm, a manipulator, and a filter hose of a cutting fluid filter device; the manipulator comprises: a two-finger gripper, a depth camera, and a 6-dimensional mechanical sensor; the robot manipulator grasping and docking method comprises:

[0008] Constructing a visual-tactile combined dataset of the manipulator in the process of grasping the filter hose; the visual-tactile combined dataset includes: collecting a visual image of the filter hose through a depth camera, obtaining pressure value information of grasping the filter hose through a 6-dimensional mechanical sensor, and obtaining a quality evaluation index based on the pressure value information; the quality evaluation index includes: falling, sliding, stability and overload;

[0009] A tactile prior knowledge learning model is established according to a visual-tactile joint data set; the tactile prior knowledge learning model is used to find a stable grasping configuration that satisfies a quality evaluation index from an image space based on the learned tactile prior knowledge; the grasping configuration includes: grasping position, grasping angle, grasping width and grasping depth;

[0010] Output the current grasping configuration according to the tactile prior knowledge learning model; and obtain the motion planning information of the manipulator according to the current grasping configuration;

[0011] According to the motion planning information of the manipulator, the manipulator is controlled to dock the filter hose, and the contact state of the filter hose docking joint and the machine tool docking interface is mechanically modeled to obtain a mechanical control model; the mechanical control model is used to characterize the relationship between the contact force and torque between the filter hose docking joint and the machine tool docking interface and the position adjustment amount of the robot end;

[0012] According to the mechanical control model, the DDPG parameter identification algorithm is used to obtain the posture information and contact force information of the manipulator during docking.

[0013] Optionally, the step of constructing a visual-tactile combined dataset of the manipulator in the process of grasping the filter hose specifically includes:

[0014] Acquire visual images of the filter hose through a depth camera;

[0015] The pressure value information of the grab filter hose is obtained through a 6-dimensional mechanical sensor;

[0016] The pressure value information is converted into an RGB image to obtain a tactile image;

[0017] The grasping quality evaluation index is obtained by convolving the tactile image with the convolution kernel; the convolution kernel is a 2×2 convolution kernel, including 1 and -1;

[0018] Labeling visual and tactile images with grasp quality evaluation metrics.

[0019] Optionally, the steps of determining the crawling quality evaluation index specifically include:

[0020]

[0021] Among them, t represents the maximum value of the tactile image of the falling object after the convolution operation with the convolution kernel. The smaller t is, the earlier it falls, which means the worse the grasping stability is. s represents the element sum of the tactile image of the non-falling object after the convolution operation with the convolution kernel. The smaller s is, the smaller the change in pressure value is, the lower the sliding intensity is, and the more stable the object is. e is the flame retardant logarithm.

[0022] Optionally, the current grasping configuration is outputted according to the tactile prior knowledge learning model; and the motion planning information of the manipulator is obtained according to the current grasping configuration, specifically including:

[0023] Using the formula g = ( R T c f(I),q(T)) converts the current grasping configuration into the motion planning information g of the manipulator in the world coordinate system;

[0024] in, is the visual image space, Represents the pixel points and depth values ​​of the target object in the visual image space, is the rotation angle of the target object in the visual image space relative to the camera coordinate system, is the pixel width of the target object in the visual image space, T = (m, n) is the tactile image space, (m, n) is the pixel coordinates of the tactile image space, R T c represents the coordinate transformation from the camera coordinate system to the robot coordinate system, which is obtained by the camera extrinsic calibration result. f(I) represents the transformation matrix from the 2D image coordinate system to the 3D camera coordinate system, which is obtained by the camera intrinsic calibration result. q(T) is the quality evaluation index. u is the position of the target object on the x-axis in the camera coordinate system. v is the position of the target object on the y-axis in the camera coordinate system. (u, v) is the pixel position information of the target object in the camera coordinate system. d is the position of the target object on the z-axis in the camera coordinate system, that is, the depth information of the target object.

[0025] Optionally, controlling the manipulator to dock the filter hose according to the motion planning information of the manipulator, and mechanically modeling the contact state between the filter hose docking joint and the machine tool docking interface to obtain a mechanical control model specifically includes:

[0026] Using the formula Determine the controller transfer function H(l), where l is the Laplace transform variable of the continuous-time system;

[0027] Using the Tustin Transformation Discretize the controller transfer function H(l) and get the formula

[0028] Using the formula Determine parameter A, parameter C, and parameter D;

[0029] Using the formula Determine the relationship between the contact force and torque between the filter hose connector and the machine tool connector and the robot end position adjustment amount;

[0030] Among them, ΔF is the difference between the force applied to the end of the robot and the expected force, ΔX is the robot posture correction, M is the virtual inertia parameter, B is the virtual damping parameter, K is the virtual stiffness parameter, F(k) is the difference between the contact force and the expected force at the kth moment, F(k-1) is the difference between the contact force and the expected force at the k-1th moment, F(k-2) is the difference between the contact force and the expected force at the k-2th moment, X(k) is the posture correction of the manipulator at the kth moment, X(k-2) is the posture correction of the manipulator at the k-2th moment, X(k-1) is the posture correction of the manipulator at the k-1th moment, T is the sampling period, and z is the Z transformation variable of the discrete time system.

[0031] Optionally, the DDPG parameter identification algorithm is updated using an Actor neural network and a Critic network.

[0032] In the second aspect, the present application provides a robot manipulator grasping and docking device, the robot comprises: a robot arm, a manipulator, and a filter hose of a cutting fluid filter device; the manipulator comprises: a two-finger gripper, a depth camera, and a 6-dimensional mechanical sensor; the robot manipulator grasping and docking device comprises:

[0033] A data set construction module is used to construct a visual-tactile combined data set of the manipulator in the process of grasping the filter hose; the visual-tactile combined data set includes: collecting a visual image of the filter hose through a depth camera, obtaining pressure value information of grasping the filter hose through a 6-dimensional mechanical sensor, and obtaining a quality evaluation index based on the pressure value information; the quality evaluation index includes: falling, sliding, stability and overload;

[0034] A prior model building module is used to establish a tactile prior knowledge learning model based on a visual-tactile joint data set; the tactile prior knowledge learning model is used to find a stable grasping configuration that meets the quality evaluation index from the image space based on the learned tactile prior knowledge; the grasping configuration includes: grasping position, grasping angle, grasping width and grasping depth;

[0035] The motion planning information determination module is used to output the current grasping configuration according to the tactile prior knowledge learning model; and obtain the motion planning information of the manipulator according to the current grasping configuration;

[0036] A mechanical control model building module is used to control the manipulator to dock the filter hose according to the motion planning information of the manipulator, and to perform mechanical modeling on the contact state between the filter hose docking joint and the machine tool docking interface to obtain a mechanical control model; the mechanical control model is used to characterize the relationship between the contact force and torque between the filter hose docking joint and the machine tool docking interface and the position adjustment amount of the robot end;

[0037] The strategy determination module is used to obtain the posture information and contact force information of the manipulator during docking according to the mechanical control model and the DDPG parameter identification algorithm.

[0038] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the robot manipulator grasping and docking method.

[0039] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the robot manipulator grasping and docking method.

[0040] In a fifth aspect, the present application provides a computer program product, including a computer program, which implements the robot manipulator grasping and docking method when executed by a processor.

[0041] According to the specific embodiments provided in this application, this application has the following technical effects:

[0042] The present application provides a method, device, medium and product for grasping and docking of a robot manipulator. During the grasping process, a tactile prior knowledge learning model is first established based on a visual-tactile joint data set to learn the grasping posture of the hose of the filter device, and then a grasping strategy is generated based on the tactile prior knowledge and the visual image. During the docking process, the contact force between the hose and the machine tool interface is first mechanically modeled so that the robot can adjust the posture of the axis according to the contact force to complete the smooth assembly. Then, based on the mechanical control model, the DDPG parameter identification algorithm is used to process the input information during the docking process, and an adjustment strategy for the virtual damping parameters is generated, so that the robot can efficiently complete the docking of the hose. The DDPG parameter identification algorithm is used to solve the problem that both the state and control in the assembly process are continuous variables, and the optimal adjustment strategy is learned through interaction with the environment. The present application can grasp and dock the hose of the cutting fluid filter device more efficiently and stably. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 This is a schematic diagram of the process of the robot manipulator grasping and docking method in one embodiment of the present application;

[0045] Figure 2 This is a schematic diagram of the principle of a robot manipulator grasping and docking method in one embodiment of the present application;

[0046] Figure 3 A flowchart of a capture process provided in an embodiment of the present application;

[0047] Figure 4 This is a flowchart of the docking process in one embodiment of the present application;

[0048] Figure 5 It is a schematic diagram of the approach stage;

[0049] Figure 6 A schematic diagram of the contact state of a hose and a docking port provided in an embodiment of the present application; DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0051] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0052] In an exemplary embodiment, Figure 1 and Figure 2 As shown, a robot manipulator grasping and docking method is provided, the robot comprises: a robot arm, a manipulator, and a filter hose of a cutting fluid filter device; the manipulator comprises: a two-finger gripper, a depth camera, and a 6-dimensional mechanical sensor; the method comprises the following S101 to S105. Among them:

[0053] S101, constructing a visual-tactile combined dataset of the manipulator in the process of grasping the filter hose; the visual-tactile combined dataset includes: collecting the visual image of the filter hose through the depth camera, obtaining the pressure value information of grasping the filter hose through the 6-dimensional mechanical sensor, and obtaining the quality evaluation index according to the pressure value information; the quality evaluation index includes: falling, sliding, stability and overload; wherein, the manifestation of falling is that the pressure value of one of the contact points changes greatly; the manifestation of sliding is that the pressure values ​​of adjacent contact points will change in the opposite direction; the manifestation of stability is a state with a small degree of sliding; the manifestation of overload is that the pressure values ​​of the two contact points continue to be high. The quality evaluation index characterizes the stability of grasping through a continuous value of 0 to 1.

[0054] The steps for determining crawl quality evaluation indicators include:

[0055]

[0056] Among them, t represents the maximum value of the tactile image of the falling object after the convolution operation with the convolution kernel. The smaller t is, the earlier it falls, which means the worse the grasping stability is. s represents the element sum of the tactile image of the non-falling object after the convolution operation with the convolution kernel. The smaller s is, the smaller the change in pressure value is, the lower the sliding intensity is, and the more stable the object is. e is the flame retardant logarithm.

[0057] S101 specifically includes:

[0058] S11, collecting a visual image of the filter hose through a depth camera;

[0059] S12, obtaining pressure value information of the grabbing filter hose through a 6-dimensional mechanical sensor;

[0060] S13, converting the pressure value information into an RGB image to obtain a tactile image;

[0061] S14, obtaining a grasping quality evaluation index according to the convolution of the tactile image and the convolution kernel; the convolution kernel is a 2×2 convolution kernel, including 1 and -1; the pressure value change of adjacent contact points can be calculated through the convolution operation.

[0062] S15, labeling the visual image and the tactile image by grasping quality evaluation indicators.

[0063] S102, establishing a tactile prior knowledge learning model based on the visual-tactile combined data set, and Figure 3 As shown; the tactile prior knowledge learning model is used to find a stable grasping configuration that satisfies the quality evaluation index from the image space based on the learned tactile prior knowledge; the grasping configuration includes: grasping position, grasping angle, grasping width and grasping depth;

[0064] Specifically, based on the visual-tactile joint dataset, the ResNet-50 model was used as the feature extractor, the Adam optimization method was used for network training, the mean square error MSEloss was used as the loss function, and the PyTorch framework was used for training to establish a tactile prior knowledge learning model.

[0065] As a specific embodiment, the tactile prior knowledge learning model represents the mapping relationship between the visual image and the grasping quality, and the mapping relationship is determined as follows:

[0066] S1, takes the image in the dataset as the input of the network, which contains three channels: R, G, and B.

[0067] S2, generates a feature map through a feature extractor.

[0068] S3, pooling and linear regression obtain a single regressor, which is the grasping quality score corresponding to the tactile image.

[0069] S103, outputting the current grasping configuration according to the tactile prior knowledge learning model; and obtaining the motion planning information of the manipulator according to the current grasping configuration;

[0070] Since the mechanical analysis process of space docking is similar to that of plane docking, in order to facilitate analysis, the plane docking is modeled. The equation for plane grabbing can be expressed as follows:

[0071] g=(p,θ,q);

[0072] Where p = (x, y, z) represents the position of the center point of the two-finger hand, θ represents the rotation angle of the two-finger hand around the z-axis, and q represents the grasping quality. x and y are the coordinates of the center point of the two-finger hand on the x-axis and y-axis respectively.

[0073] Using the formula g = ( R T c f(I),q(T)) converts the current grasping configuration into the motion planning information g of the manipulator in the world coordinate system;

[0074] in, is the visual image space, Represents the pixel points and depth values ​​of the target object in the visual image space, is the rotation angle of the target object in the visual image space relative to the camera coordinate system, is the pixel width of the target object in the visual image space, T = (m, n) is the tactile image space, (m, n) is the pixel coordinates of the tactile image space, R T crepresents the coordinate transformation from the camera coordinate system to the robot coordinate system, which is obtained by the camera extrinsic calibration result. f(I) represents the transformation matrix from the 2D image coordinate system to the 3D camera coordinate system, which is obtained by the camera intrinsic calibration result. q(T) is the quality evaluation index. u is the position of the target object on the x-axis in the camera coordinate system. v is the position of the target object on the y-axis in the camera coordinate system. (u, v) is the pixel position information of the target object in the camera coordinate system. d is the position of the target object on the z-axis in the camera coordinate system, that is, the depth information of the target object.

[0075] S104, controlling the manipulator to dock the filter hose according to the motion planning information of the manipulator, and mechanically modeling the contact state between the filter hose docking joint and the machine tool docking interface to obtain a mechanical control model; the mechanical control model is used to characterize the relationship between the contact force and torque between the filter hose docking joint and the machine tool docking interface and the position adjustment amount of the robot end;

[0076] In order to analyze the relationship between the contact state and the force value during the docking process, the contact force between the hose and the machine tool interface is mechanically modeled. In order to facilitate analysis, the docking of the plane is modeled, and the docking process can be approximately converted into an assembly problem of the shaft hole.

[0077] The entire docking process can be divided into three stages according to the relative position of the hose and the docking port: approach stage, single-point contact stage and two-point contact stage. Among them, only the single-point contact and two-point contact stages are subject to force.

[0078] The schematic diagram of the approach phase is as follows Figure 5 shown. Figure 5 in, u 0 represents the distance between the vertical line where the manipulator center is located and the vertical line where the hose gripping point center is located during the approach phase, as an initial relative position; r represents the radius of the hose; R represents the radius of the docking port; l g Indicates the length of the hose that needs to be connected; θ 0 Represents the angle between the centerline of the hose and the centerline of the docking port during the approach phase.

[0079] The single point contact force analysis process is as follows:

[0080] S41, when the shaft and the hole are in single-point contact, the state is as follows Figure 6 As shown in part (a) of . The geometric analysis of the contact state shows:

[0081] u=(l g -l)·sinθ+Rr (1)

[0082] Among them, u represents the distance between the vertical line where the center of the manipulator is located and the vertical line where the center of the hose grasping point is located during the single-point contact stage; it represents the length of the part below the contact point between the hose and the docking interface.

[0083] S42, because θ is very small, sinθ≈θ, and equation (1) can be transformed into:

[0084]

[0085] S43, the force balance equation at the contact point is as follows:

[0086]

[0087] In the formula, F 1 、F 2 and M are the force and torque applied by the gripper to the hose, FN is the reaction force of the interface on the hose at the contact point, and f is the friction force of the interface on the hose at the contact point.

[0088] S44, assuming the lateral elastic coefficient is k x , the torsional elastic coefficient is k θ , then formula (3) can be transformed into:

[0089]

[0090] Where μ is the friction factor and θ is the angle between the centerline of the hose and the centerline of the docking port during the single point contact phase.

[0091] The two-point contact force analysis process is as follows:

[0092] S51, when the shaft and the hole are in two-point contact, the state is as follows Figure 6 As shown in part (b) of . The geometric analysis of the contact state shows:

[0093]

[0094] S52, the force balance equation is:

[0095]

[0096] Among them, f 1 is the friction force of the interface on the hose at the left contact point; f 2 It is the friction force of the interface on the hose at the right contact point.

[0097] S53, combining equation (5) and equation (6), we get:

[0098]

[0099] In formula (6):

[0100]

[0101] Define the controller's transfer function:

[0102] According to Tustin transformation After discretization, we get

[0103] Using the formula Determine parameter A, parameter C, and parameter D;

[0104] Using the formula Determine the position correction value X(k) of the manipulator at the kth moment;

[0105] Among them, ΔF is the difference between the force applied to the end of the robot and the expected force, ΔX is the robot posture correction, M is the virtual inertia parameter, B is the virtual damping parameter, K is the virtual stiffness parameter, F(k) is the difference between the contact force and the expected force at the kth moment, F(k-1) is the difference between the contact force and the expected force at the k-1th moment, F(k-2) is the difference between the contact force and the expected force at the k-2th moment, X(k-2) is the posture correction of the manipulator at the k-2th moment, X(k-1) is the posture correction of the manipulator at the k-1th moment, T is the sampling period, and z is the Z transformation variable of the discrete time system.

[0106] S105, according to the mechanical control model, the DDPG parameter identification algorithm is used to obtain the posture information and contact force information of the manipulator during docking, and Figure 4 shown.

[0107] The purpose of the DDPG parameter identification algorithm is to learn the adjustment strategy of the damping parameter B. The adjustment process of parameter B can be expressed as a Markov decision process. For an intelligent agent, the long-term benefit R t Maximization is used as the goal to improve the learning strategy as follows:

[0108]

[0109] In the formula, γ is the discount rate of the reward, γ∈(0,1), and the reward value r t Depends on the state s t Next, perform action a t the good or bad.

[0110] Strategy π is a mapping from state to action. The action selection strategy π is defined as follows:

[0111] a=π(s|θ π );

[0112] In the formula, the strategy π is represented by the Actor neural network. π is the weight of the Actor network.

[0113] During the docking process, the effects of force, torque, axis posture and insertion depth need to be considered. Therefore, the state of the environment consists of 8 elements, defined as follows:

[0114] s t =[F x ,F y ,F z ,M x ,M y ,θ x ,θ y ,h];

[0115] Among them, s t is the environmental state at time t, F and M are the force and torque measured by the 6-dimensional force sensor, θ x and θ y is the rotation angle of the end of the robot arm around the x-axis and y-axis of the end coordinate system, and h is the insertion depth.

[0116] The parameters θ of the policy π π pass Update, where β is the learning rate, γ is the discount rate of the reward, ▽ represents the gradient, and t represents the time;

[0117] The parameters θ of the policy π q pass Update, where y = r t +γq[s t ,π(s t |θ π )];

[0118] Among them, q(s t ,a|θ q ) is the action value function, represented by the Critic network, θ q is the weight of the Critic network, and α and β are the learning rates of the network.

[0119] In the process of grasping, the manipulator needs to correctly identify the position of the hose and grasp it in a suitable posture. Since the hose is prone to bending and deformation, the grasping position and the magnitude of the applied force need to be controlled during grasping. In the process of docking between the hose and the machine tool interface, the position of the interface must be identified first, and then the posture of the manipulator is adjusted in real time according to the force feedback information of the 6-dimensional force sensor, so that the docking task can be completed efficiently and filtering can be performed. In order to achieve the effect of circulating filtration, the filter has an inlet pipe and an outlet pipe that need to be docked. After the docking is successful, the filter begins to extract the waste cutting fluid, and the waste liquid is sent back to the waste liquid tank through a filtering device composed of a semiconductor inorganic membrane, and then extracted again and filtered. During the filtering process, the sensor in the filter will detect the parameters such as the impurity concentration of the cutting fluid in real time, until the cutting fluid meets the requirements for continued use and stops filtering. Therefore, the present application can effectively solve the problem of filtering the waste cutting fluid generated by the machine tool during the processing process, which not only reduces the treatment cost of the waste liquid, but also is more conducive to environmental protection.

[0120] Based on the same inventive concept, the embodiment of the present application also provides a robot manipulator grasping and docking device for implementing the robot manipulator grasping and docking method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more robot manipulator grasping and docking device embodiments provided below can refer to the limitations of the robot manipulator grasping and docking method above, and will not be repeated here.

[0121] In an exemplary embodiment, a robot manipulator grasping and docking device is provided, comprising:

[0122] A data set construction module is used to construct a visual-tactile combined data set of the manipulator in the process of grasping the filter hose; the visual-tactile combined data set includes: collecting a visual image of the filter hose through a depth camera, obtaining pressure value information of grasping the filter hose through a 6-dimensional mechanical sensor, and obtaining a quality evaluation index based on the pressure value information; the quality evaluation index includes: falling, sliding, stability and overload;

[0123] A prior model building module is used to establish a tactile prior knowledge learning model based on a visual-tactile joint data set; the tactile prior knowledge learning model is used to find a stable grasping configuration that meets the quality evaluation index from the image space based on the learned tactile prior knowledge; the grasping configuration includes: grasping position, grasping angle, grasping width and grasping depth;

[0124] The motion planning information determination module is used to output the current grasping configuration according to the tactile prior knowledge learning model; and obtain the motion planning information of the manipulator according to the current grasping configuration;

[0125] A mechanical control model building module is used to control the manipulator to dock the filter hose according to the motion planning information of the manipulator, and to perform mechanical modeling on the contact state between the filter hose docking joint and the machine tool docking interface to obtain a mechanical control model; the mechanical control model is used to characterize the relationship between the contact force and torque between the filter hose docking joint and the machine tool docking interface and the position adjustment amount of the robot end;

[0126] The strategy determination module is used to obtain the posture information and contact force information of the manipulator during docking according to the mechanical control model and the DDPG parameter identification algorithm.

[0127] In an exemplary embodiment, a computer device is provided, which may be a server or a terminal. The computer device includes a processor, a memory, an input / output interface (I / O for short) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a robot manipulator grasping and docking method is implemented.

[0128] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0129] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0130] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0131] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0132] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0133] In this application, all actions to obtain signals, information or data are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0134] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0135] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A robot manipulator grasping and docking method, characterized in that: The robots include: A robot arm, a robot hand, and a filter hose of a cutting fluid filter device; the robot hand includes: a two-finger gripper, a depth camera, and a 6-dimensional mechanical sensor; the robot hand grasping and docking method includes: Constructing a visual-tactile combined dataset of the manipulator in the process of grasping the filter hose; the visual-tactile combined dataset includes: collecting a visual image of the filter hose through a depth camera, obtaining pressure value information of grasping the filter hose through a 6-dimensional mechanical sensor, and obtaining a quality evaluation index based on the pressure value information; the quality evaluation index includes: falling, sliding, stability and overload; A tactile prior knowledge learning model is established according to a visual-tactile joint data set; the tactile prior knowledge learning model is used to find a stable grasping configuration that satisfies a quality evaluation index from an image space based on the learned tactile prior knowledge; the grasping configuration includes: grasping position, grasping angle, grasping width and grasping depth; Output the current grasping configuration according to the tactile prior knowledge learning model; and obtain the motion planning information of the manipulator according to the current grasping configuration; According to the motion planning information of the manipulator, the manipulator is controlled to dock the filter hose, and the contact state of the filter hose docking joint and the machine tool docking interface is mechanically modeled to obtain a mechanical control model; the mechanical control model is used to characterize the relationship between the contact force and torque between the filter hose docking joint and the machine tool docking interface and the position adjustment amount of the robot end; According to the mechanical control model, the DDPG parameter identification algorithm is used to obtain the posture information and contact force information of the manipulator during docking.

2. The robot manipulator grasping and docking method according to claim 1, characterized in that: The data set for constructing the visual-tactile combination of the manipulator in the process of grasping the filter hose specifically includes: Acquire visual images of the filter hose through a depth camera; The pressure value information of the grab filter hose is obtained through a 6-dimensional mechanical sensor; The pressure value information is converted into an RGB image to obtain a tactile image; The grasping quality evaluation index is obtained by convolving the tactile image with the convolution kernel; the convolution kernel is a 2×2 convolution kernel, including 1 and -1; Labeling visual and tactile images with grasp quality evaluation metrics.

3. The robot manipulator grasping and docking method according to claim 1 or claim 2, characterized in that: The steps for determining crawl quality evaluation indicators include: Among them, t represents the maximum value of the tactile image of the falling object after the convolution operation with the convolution kernel. The smaller t is, the earlier it falls, which means the worse the grasping stability is. s represents the element sum of the tactile image of the non-falling object after the convolution operation with the convolution kernel. The smaller s is, the smaller the change in pressure value is, the lower the sliding intensity is, and the more stable the object is. e is the flame retardant logarithm.

4. The robot manipulator grasping and docking method according to claim 1, characterized in that: The current grasping configuration is outputted based on the tactile prior knowledge learning model; and the motion planning information of the manipulator is obtained based on the current grasping configuration, including: Using the formula g = ( R T c f(I),q(T)) converts the current grasping configuration into the motion planning information g of the manipulator in the world coordinate system; in, is the visual image space, Represents the pixel points and depth values ​​of the target object in the visual image space, is the rotation angle of the target object in the visual image space relative to the camera coordinate system, is the pixel width of the target object in the visual image space, T = (m, n) is the tactile image space, (m, n) is the pixel coordinates in the tactile image space, R T c represents the coordinate transformation from the camera coordinate system to the robot coordinate system, which is obtained by the camera extrinsic calibration result. f(I) represents the transformation matrix from the 2D image coordinate system to the 3D camera coordinate system, which is obtained by the camera intrinsic calibration result. q(T) is the quality evaluation index. u is the position of the target object on the x-axis in the camera coordinate system. v is the position of the target object on the y-axis in the camera coordinate system. (u, v) is the pixel position information of the target object in the camera coordinate system. d is the position of the target object on the z-axis in the camera coordinate system, that is, the depth information of the target object.

5. The robot manipulator grasping and docking method according to claim 1, characterized in that: The method of controlling the manipulator to dock the filter hose according to the motion planning information of the manipulator and mechanically modeling the contact state between the filter hose docking joint and the machine tool docking interface to obtain a mechanical control model specifically includes: Using the formula Determine the controller transfer function H(l), where l is the Laplace transform variable of the continuous-time system; Using the Tustin Transformation Discretize the controller transfer function H(l) and get the formula Using the formula Determine parameter A, parameter C and parameter D; Using the formula Determine the relationship between the contact force and torque between the filter hose connector and the machine tool connector and the robot end position adjustment amount; Among them, ΔF is the difference between the force applied to the end of the robot and the expected force, ΔX is the robot posture correction, M is the virtual inertia parameter, B is the virtual damping parameter, K is the virtual stiffness parameter, F(k) is the difference between the contact force and the expected force at the kth moment, F(k-1) is the difference between the contact force and the expected force at the k-1th moment, F(k-2) is the difference between the contact force and the expected force at the k-2th moment, X(k) is the posture correction of the manipulator at the kth moment, X(k-2) is the posture correction of the manipulator at the k-2th moment, X(k-1) is the posture correction of the manipulator at the k-1th moment, T is the sampling period, and z is the Z transformation variable of the discrete time system.

6. The robot manipulator grasping and docking method according to claim 1, characterized in that: The DDPG parameter identification algorithm uses Actor neural network and Critic network for update.

7. A robot manipulator grasping and docking device, characterized in that: The robots include: A robot arm, a robot hand, and a filter hose of a cutting fluid filter device; the robot hand includes: a two-finger gripper, a depth camera, and a 6-dimensional mechanical sensor; the robot manipulator grasping and docking equipment includes: A data set construction module is used to construct a visual-tactile combined data set of the manipulator in the process of grasping the filter hose; the visual-tactile combined data set includes: collecting a visual image of the filter hose through a depth camera, obtaining pressure value information of grasping the filter hose through a 6-dimensional mechanical sensor, and obtaining a quality evaluation index based on the pressure value information; the quality evaluation index includes: falling, sliding, stability and overload; A prior model building module is used to establish a tactile prior knowledge learning model based on a visual-tactile joint data set; the tactile prior knowledge learning model is used to find a stable grasping configuration that meets the quality evaluation index from the image space based on the learned tactile prior knowledge; the grasping configuration includes: grasping position, grasping angle, grasping width and grasping depth; The motion planning information determination module is used to output the current grasping configuration according to the tactile prior knowledge learning model; and obtain the motion planning information of the manipulator according to the current grasping configuration; A mechanical control model building module is used to control the manipulator to dock the filter hose according to the motion planning information of the manipulator, and to perform mechanical modeling on the contact state between the filter hose docking joint and the machine tool docking interface to obtain a mechanical control model; the mechanical control model is used to characterize the relationship between the contact force and torque between the filter hose docking joint and the machine tool docking interface and the position adjustment amount of the robot end; The strategy determination module is used to obtain the posture information and contact force information of the manipulator during docking according to the mechanical control model and the DDPG parameter identification algorithm.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the robot manipulator grasping and docking method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the robot manipulator grasping and docking method described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the robot manipulator grasping and docking method described in any one of claims 1 to 6 is implemented.

Images