Six-axis distributed force tactile perception method, system and terminal based on binocular tactile vision

Abstract

The application discloses a six-axis distributed force tactile perception method, system and terminal based on binocular visual tactile, the method comprises the following steps: obtaining virtual binocular visual tactile, thereby collecting the curved surface result of a target object, and calculating the rotation axis of the contact area, six-axis distributed force after contacting the target object, and iteratively updating a plurality of parameters of a spring-damping network model based on the same, so as to further collect the geometric space characteristics of the target object when being touched. The application is based on a binocular reflection binocular visual tactile platform, constructs virtual binocular visual tactile, reconstructs the three-dimensional geometric deformation of the contact surface, estimates the six-axis force of the fingertip by using a spring-damping network model, so as to improve the range, precision and calculation efficiency of the model method in force distribution reconstruction.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot tactile perception analysis, and particularly relates to a six-axis distributed force tactile perception method, system, terminal and computer readable storage medium based on binocular visual tactile. BACKGROUND

[0002] Visual tactile sensor has attracted wide attention in the field of robots due to its high spatial resolution sensing. According to the implementation principle, the visual tactile sensor can be divided into reflective film type and marker point type, which converts the contact deformation information into visual images, and the images can be used to further extract tactile features from three-dimensional contact geometry to multi-axis force; the reflective film type uses photometric method to realize pixel-level spatial resolution, and the marker point type visual tactile sensor acquires the geometric deformation of the contact area according to discrete marker points.

[0003] However, the reflective film type has high requirements for reflective film materials and lighting conditions, and has limitations for further force field perception, while the marker point type has higher selectivity for structure and material, and due to the difference in marker point type and three-dimensional geometric deformation measurement method, as well as the high nonlinearity of elastic materials, there is a problem of low calculation efficiency in using the visual tactile sensor to estimate multi-axis force and torque.

[0004] Therefore, the prior art still needs to be improved and developed. SUMMARY

[0005] The main purpose of the present application is to provide a six-axis distributed force tactile perception method, system, terminal and computer readable storage medium based on binocular visual tactile, which aims to solve the problems of low calculation efficiency caused by directly using finite element calculation method in the prior art of visual tactile sensing research, and limited generalization ability caused by relying on a large amount of data in the learning method.

[0006] To achieve the above purpose, the present application provides a six-axis distributed force tactile perception method based on binocular visual tactile, which comprises the following steps:

[0007] A plurality of virtual cameras of an original camera are obtained, a pixel coordinate system of a plurality of virtual cameras is constructed, a plurality of marker point coordinates of a target object are obtained, all the marker point coordinates are mapped into the pixel coordinate system to obtain corresponding virtual point coordinates, and a surface result of the target object is obtained according to all the virtual point coordinates;

[0008] Offset results corresponding to all the virtual point coordinates are obtained, a marker point set and an offset point set of a contact area are obtained according to all the offset results and the surface result, and a contact rotation axis of the target object is calculated according to the marker point set and the offset point set.

[0009] Calculating an offset vector of the contact area according to the set of marking points and the set of offset points, and constructing a displacement field of the set of offset points according to the offset vector;

[0010] Constructing a spring-damper network model, inputting the displacement field into the spring-damper network model, outputting force information of each marked point, and calculating the six-axis distributed force in the contact area based on each force information and the contact rotation axis;

[0011] An objective function is defined according to the total torque, the spring-damper network model is updated using the objective function, and the updated spring-damper network model is used to represent the force and torque of the target object.

[0012] Optionally, the six-axis distributed force tactile perception method based on binocular tactile perception, wherein the steps of obtaining multiple virtual cameras of the original camera, constructing pixel coordinate systems of the multiple virtual cameras, obtaining multiple marker point coordinates of the target object, mapping all the marker point coordinates to the pixel coordinate system, obtaining corresponding virtual point coordinates, and obtaining the surface result of the target object based on all the virtual point coordinates, specifically include:

[0013] Obtain multiple fixed points of the mirror surface, and obtain multiple virtual cameras of the original camera through each of the fixed points, construct the pixel coordinate systems of the multiple virtual cameras, and obtain the plane equations of the corresponding mirror surface in the pixel coordinate system:

[0014] q1=(x q1 ,y q1 , z q1 ), q2=(x q2 ,y q2 , z q2 );

[0015] A1x+B1y+C1z+D1=0, A2x+B2y+C2z+D2=0;

[0016] n1={x m1 ,y m1 , z m1}, n2={x m2 ,y m2 , z m2};

[0017] Among them, q1 and q2 represent fixed points, x q1 、y q1 and z q1 They represent the horizontal, vertical and vertical coordinates of the fixed point q1, respectively. q2 、y q2 and z q2They represent the horizontal coordinate, vertical coordinate and vertical coordinate of the fixed point q2 respectively, n1 represents the first normal vector of the first mirror corresponding to the fixed point q1, n2 represents the second normal vector of the second mirror corresponding to the fixed point q2, x m1 、y m1 and z m1 Represents the horizontal coordinate, vertical coordinate and vertical coordinate of the first normal vector, x m2 、y m2 and z m2 Represents the horizontal coordinate, vertical coordinate and vertical coordinate of the second normal vector;

[0018] D1=-(A1x q1 +B1y q1 +C1z q1 ), A1=x m1 , B1=y m1 , C1=z m1 ;

[0019] D2=-(A2x q2 +B2y q2 +C2z q2 ), A2=x m2 , B2=y m2 , C2=z m2 ;

[0020] Wherein, D1 represents the constant term in the plane equation of the first mirror surface, D2 represents the constant term in the plane equation of the second mirror surface, A1x+B1y+C1z+D1=0 and A2x+B2y+C2z+D2=0 represent the plane equation of the first mirror surface and the plane equation of the second mirror surface, respectively;

[0021] By mirror reflection, the virtual matrices of the multiple virtual cameras in the pixel coordinate system are calculated:

[0022]

[0023] in, and They represent the horizontal coordinate basis vector, vertical coordinate basis vector and vertical coordinate basis vector of the first virtual camera in the first virtual camera coordinate system respectively, and They represent the coordinate representation of the abscissa basis vector of the first virtual camera, and They represent the coordinate representation of the vertical coordinate basis vector of the first virtual camera, and They represent the coordinate representation of the vertical coordinate basis vector of the first virtual camera, and They represent the horizontal coordinate basis vector, vertical coordinate basis vector and vertical coordinate basis vector of the first virtual camera in the second virtual camera coordinate system respectively, and They represent the coordinate representation of the abscissa basis vector of the second virtual camera, and They represent the coordinate representation of the ordinate basis vector of the second virtual camera, and Respectively represent the coordinate representation of the vertical coordinate basis vector of the second virtual camera;

[0024]

[0025] Where x1, y1, and z1 represent the horizontal coordinate, vertical coordinate, and vertical coordinate of the origin of the first virtual coordinate system, respectively; x2, y2, and z2 represent the horizontal coordinate, vertical coordinate, and vertical coordinate of the origin of the second virtual coordinate system, respectively; x0, y0, and z0 represent the horizontal coordinate, vertical coordinate, and vertical coordinate of the origin v0 of the camera coordinate system, respectively;

[0026]

[0027] in, represents the virtual matrix of the first virtual camera, and Represents the rotation matrix and translation matrix between the first virtual coordinate system and the original camera coordinate system, 0 1×3 represents a 1×3 matrix with all elements equal to 0, Matrix representation of the origin of the camera coordinate system;

[0028]

[0029] in, represents the virtual matrix of the second virtual camera, and Respectively represent the rotation matrix and translation matrix between the second virtual coordinate system and the pixel coordinate system;

[0030] Obtain the coordinates of multiple marker points of the target object in the world coordinate system, and map all the marker point coordinates to the pixel coordinate system to obtain the corresponding virtual point coordinates:

[0031]

[0032]

[0033] A·x=B;

[0034] Among them, w0 and a0 represent the pixel coordinates of the center of the original camera image, f x and f yRepresents the focal length of the original camera in the horizontal and vertical directions, and Represents the coordinates of the marker points in the world coordinate system, P 1 and P 2 Both represent the virtual coordinates of the marked points at the first and second moments, P (t) represents the virtual coordinates of the marker point at time t, P i (t) represents the virtual coordinates of the i-th marker at the t-th moment, A and B represent the transformation matrix for transforming the marker coordinates, T represents transpose, and x represents the virtual point coordinates;

[0035] All the virtual point coordinates are represented by surface fitting to obtain the surface result of the target object.

[0036] Optionally, the six-axis distributed force tactile perception method based on binocular tactile perception, wherein the virtual matrices of the plurality of virtual cameras in the pixel coordinate system are calculated by mirror reflection, further comprises:

[0037] Acquire a single camera image after being reflected by multiple mirror surfaces, and filter out regions of the first mirror surface and the second mirror surface in the single camera image;

[0038] Segmenting the regions of the first mirror surface and the second mirror surface in the single camera image to obtain two groups of images;

[0039] Based on the two sets of images, the internal parameters of the original camera, the rotation matrix, and the translation matrix are calibrated using a stereo camera.

[0040] Optionally, the six-axis distributed force tactile perception method based on binocular tactile perception, wherein the obtaining of the offset results corresponding to all the virtual point coordinates, obtaining a set of marked points and a set of offset points of the contact area based on all the offset results and the surface results, and calculating the contact rotation axis of the target object based on the set of marked points and the set of offset points, specifically includes:

[0041] Obtaining offset results corresponding to all virtual point coordinates, wherein the offset results represent offsets generated after the target object is touched;

[0042] The change of the surface height is obtained through the offset result and the surface result. The contact area is obtained according to the change, and the marker point set and offset point set of the contact area are constructed:

[0043]

[0044] in, a set of marker points representing a contact area, an Nth marker point of the set of marker points, c a set of offset points representing a contact area at a tth time point, an Nth offset point of the set of offset points, c

[0045] a centroid of the set of marker points and a centroid of the set of offset points are calculated respectively, and the set of marker points and the set of offset points are subjected to centroid-removing processing to obtain updated marker coordinates and updated offset coordinates:

[0046]

[0047] wherein, the centroid of the set of marker points is represented by, the centroid of the set of offset points is represented by;

[0048]

[0049] wherein, the updated marker coordinates are represented by, the updated offset coordinates are represented by;

[0050] a covariance matrix H is constructed by using the updated marker coordinates and the updated offset coordinates:

[0051]

[0052] or

[0053] after singular value decomposition of the covariance matrix, a best rotation matrix and an anti-symmetric matrix of the best rotation matrix are calculated, and a direction of a contact rotation axis is extracted according to the anti-symmetric matrix:

[0054] H = U∑V T ;

[0055] R = VU T ;

[0056]

[0057] wherein, U represents a basis vector of the set of offset points, ∑ represents a 3×3 diagonal matrix, diagonal elements are singular values, V represents a 3×3 orthogonal matrix, T represents transposition, R represents the best rotation matrix, R antisym represents the anti-symmetric matrix of the best rotation matrix, represents the direction of the contact rotation axis, R antisym (3, 2), R antisym (1, 3) and R antisym ​​(2, 1) both represent matrix indices of the contact rotation axis.

[0058] Optionally, the six-axis distributed force tactile perception method based on binocular tactile perception, wherein the step of calculating the offset vector of the contact area based on the set of marker points and the set of offset points, and constructing the displacement field of the set of offset points based on the offset vector, specifically includes:

[0059] Calculate the offset vector of the contact area based on the set of marker points and the set of offset points:

[0060]

[0061] in, Represents the offset vector of the i-th marker point in the contact area at time t, and They represent the horizontal, vertical and vertical coordinates of the i-th mark point in the contact area at the t-th moment, respectively. and They represent the horizontal coordinate, vertical coordinate and vertical coordinate of the i-th marker point in the contact area before contact, and They represent the horizontal axis offset vector, vertical axis offset vector, and vertical axis offset vector of the i-th marker point in the contact area at time t, respectively;

[0062] Construct the displacement field of the offset point set through the offset vector:

[0063]

[0064] Where D represents the displacement field and T represents the transpose.

[0065] Optionally, the six-axis distributed force tactile perception method based on binocular tactile perception, wherein the step of constructing a spring-damper network model, inputting the displacement field into the spring-damper network model, outputting force information of each marker point, and calculating the six-axis force distribution within the contact area based on each force information and the contact rotation axis, specifically includes:

[0066] Construct a reconstruction model, input the displacement field into the reconstruction model, and output the force information of each marked point:

[0067] F R =g R (D);

[0068] F R =(f 1，x , f 1，y , f 1，z ,…,f N，x , f N，y , fN，z ) T ;

[0069] Among them, F R represents the reconstruction force, f i，x Indicates the force on the i-th mark point in the horizontal direction, g R Represents the reconstructed model;

[0070] According to the reconstructed force, each of the marking points is connected to a preset number of surrounding marking points through a spring-damper simulation to obtain the force information of each marking point:

[0071]

[0072] Among them, F i Indicates the force on the i-th mark point, F i，j represents the spring connection between the i-th marked point and its j-th neighboring point, k i，j represents the elastic coefficient between the adjacent i-th marker point and the j-th neighbor point, Λ(i) represents the set of domain marker points of the i-th marker point, F i，b Indicates that the i-th marked point is connected in parallel with the spring-damper on the bottom surface, Represents the offset vector between the i-th marker point in the contact area at time t and the j-th neighboring point around it, represents the offset vector between the i-th marker point in the contact area at time t and the origin of the pixel coordinate system;

[0073]

[0074] Among them, k i，b represents the elastic coefficient of the spring-damper between the i-th mark point and the bottom surface, c i，b represents the damping of the spring-damper between the i-th mark point and the bottom surface;

[0075]

[0076] Among them, F total represents the six-axis force, K represents the stiffness matrix composed of all elastic coefficients in the contact area, and C represents the damping matrix composed of all damping in the contact area;

[0077] According to each of the force information and the contact rotation axis, the rotation axis torque of all the marked points in the contact area is calculated, and according to all the rotation axis torques, the six-axis distributed force in the contact area is calculated:

[0078] M i =(M i，x , M i，y , M i，z );

[0079] M i =d i，⊥ ×F i ;

[0080]

[0081] Among them, M i Represents the rotation axis torque of the i-th mark point, M i，x 、M i，y and M i，z Represents the components of the rotation axis torque of the i-th mark point on the horizontal axis, vertical axis and vertical axis respectively, d i，⊥ represents the component perpendicular to the rotation axis, d i Indicates the distance from the i-th marker point to the center of the region, represents the set of offset points in the contact area at time t, represents the damping of the i-th marker at the t-th moment, represents the unit vector of the contact rotation axis, Indicates the direction of contact with the rotation axis, N c Indicates the number of markers in the contact area, M total represents the total moment in the contact area.

[0082] Optionally, the six-axis distributed force tactile perception method based on binocular tactile perception, wherein the objective function is defined according to the total torque, the spring-damper network model is updated using the objective function, and the updated spring-damper network model is used to represent the force and torque of the target object to obtain the six-axis distributed force of the target object, specifically includes:

[0083] The six-axis force of the target area is calibrated by a six-axis force sensor, and the objective function is defined based on the six-axis force:

[0084]

[0085] in, represents the objective function, and It represents the total force and total torque of the target area measured by the six-axis force sensor. and represents the force and moment estimated by the spring-damper model in the contact area, and λ represents the weight parameter for balancing the force error and moment error;

[0086] Initialize multiple parameters of the spring-damper network model, and calculate the partial derivatives of the objective function with respect to the multiple initialization parameters multiple times to obtain a gradient vector:

[0087]

[0088] in, represents the gradient vector;

[0089] Using the gradient vector, iteratively update the plurality of parameters until the change in the objective function is less than a first preset threshold or gradient The norm of is less than the second preset threshold:

[0090]

[0091] in, and denote the elastic coefficients between the i-th marker point and the j-th neighboring point in the (n+1)th iteration and the nth iteration, respectively. and represents the elastic coefficient of the spring-damper between the i-th marker point and the bottom surface in the (n+1)-th iteration and the n-th iteration, and represents the damping of the spring-damper between the i-th marker point and the bottom surface in the (n+1)-th iteration and the n-th iteration, and η represents the learning rate of the spring-damper model;

[0092] The updated parameters are used to update the spring-damper model, and the updated spring-damper network model is used to represent the force and torque of the target object to obtain the six-axis distributed force of the target object in the contact area.

[0093] In addition, to achieve the above-mentioned purpose, the present invention further provides a six-axis distributed force tactile perception system based on binocular tactile perception, wherein the six-axis distributed force tactile perception system based on binocular tactile perception comprises:

[0094] A surface representation module is used to obtain multiple virtual cameras of the original camera, construct pixel coordinate systems of the multiple virtual cameras, obtain multiple marker point coordinates of the target object, map all the marker point coordinates into the pixel coordinate system, obtain corresponding virtual point coordinates, and obtain the surface result of the target object based on all the virtual point coordinates;

[0095] a rotation axis calculation module, configured to obtain offset results corresponding to all virtual point coordinates, obtain a set of marked points and a set of offset points of the contact area based on all offset results and the curved surface results, and calculate the contact rotation axis of the target object based on the set of marked points and the set of offset points;

[0096] a displacement field construction module, configured to calculate an offset vector of the contact area based on the set of marking points and the set of offset points, and to construct a displacement field of the set of offset points based on the offset vector;

[0097] a torque calculation module, configured to construct a spring-damper network model, input the displacement field into the spring-damper network model, output force information of each marked point, and calculate the six-axis distributed force in the contact area based on each force information and the contact rotation axis;

[0098] A model updating module is used to define an objective function based on the total torque, update the spring-damper network model using the objective function, and use the updated spring-damper network model to represent the force and torque of the target object to obtain the six-axis distributed force of the target object.

[0099] In addition, to achieve the above-mentioned purpose, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a six-axis distributed force tactile perception program based on binocular tactiles stored on the memory and runnable on the processor, and when the six-axis distributed force tactile perception program based on binocular tactiles is executed by the processor, the steps of the six-axis distributed force tactile perception method based on binocular tactiles as described above are implemented.

[0100] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a six-axis distributed force tactile perception program based on binocular tactile perception, and when the six-axis distributed force tactile perception program based on binocular tactile perception is executed by a processor, the steps of the six-axis distributed force tactile perception method based on binocular tactile perception as described above are implemented.

[0101] In the present invention, multiple virtual cameras of the original camera are obtained, pixel coordinate systems of the multiple virtual cameras are constructed, and multiple marker point coordinates of the target object are obtained, all the marker point coordinates are mapped into the pixel coordinate system to obtain corresponding virtual point coordinates, and the surface result of the target object is obtained based on all the virtual point coordinates; the offset results corresponding to all the virtual point coordinates are obtained, and the marker point set and the offset point set of the contact area are obtained based on all the offset results and the surface results, and the contact rotation axis of the target object is calculated based on the marker point set and the offset point set; the offset vector of the contact area is calculated based on the marker point set and the offset point set, and the displacement field of the offset point set is constructed based on the offset vector; a spring-damper network model is constructed, the displacement field is input into the spring-damper network model, the force information of each marker point is output, and the six-axis distributed force in the contact area is calculated based on each force information and the contact rotation axis; the objective function is defined based on the total torque, the spring-damper network model is updated using the objective function, and the updated spring-damper network model is used to represent the force and torque of the target object. The present invention is based on a binocular tactile platform with dual-mirror reflection, constructs virtual binocular tactile, reconstructs the three-dimensional geometric deformation of the contact surface, and adopts a spring-damper network model to estimate the six-axis force of the fingertip to improve the range, accuracy and computational efficiency of the model method in force distribution reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0102] Figure 1 It is a flow chart of a preferred embodiment of the six-axis distributed force tactile perception method based on binocular tactile perception of the present invention;

[0103] Figure 2 This is a flowchart of the overall calculation of six-axis distributed force in a preferred embodiment of the six-axis distributed force tactile perception method based on binocular tactile perception of the present invention;

[0104] Figure 3 Schematic diagram of binocular vision and touch sensing based on dual-mirror reflection in a preferred embodiment of the six-axis distributed force tactile perception method of the present invention;

[0105] Figure 4 2 is a schematic diagram of a virtual binocular tactile camera model of a preferred embodiment of the six-axis distributed force tactile perception method based on binocular tactile perception of the present invention;

[0106] Figure 5 This is a schematic diagram of the unit connection between the force of a single marked point on the contact surface and the spring-damper model in a preferred embodiment of the six-axis distributed force tactile perception method based on binocular tactile perception of the present invention;

[0107] Figure 61 is a structural diagram of a preferred embodiment of a six-axis distributed force tactile perception system based on binocular tactile perception of the present invention;

[0108] Figure 7 Schematic diagram of the operating environment of a preferred embodiment of the terminal of the present invention. DETAILED DESCRIPTION

[0109] In order to make the purpose, technical solutions and advantages of the present invention more clear and distinct, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0110] The six-axis distributed force tactile perception method based on binocular tactile perception described in the preferred embodiment of the present invention is as follows: Figure 1 As shown, the six-axis distributed force tactile perception method based on binocular tactile perception includes the following steps:

[0111] First, if Figure 2 As shown, the present invention is based on a binocular tactile platform with dual mirror reflection, constructs virtual binocular tactile, reconstructs the three-dimensional geometric deformation of the contact surface, and estimates the six-axis force of the fingertip after parameter estimation and optimization using a spring-damper network model to improve the range, accuracy and computational efficiency of the model method in force distribution reconstruction.

[0112] Step S10: obtain multiple virtual cameras of the original camera, construct pixel coordinate systems of the multiple virtual cameras, and obtain multiple marker point coordinates of the target object, map all the marker point coordinates to the pixel coordinate system, obtain corresponding virtual point coordinates, and obtain the surface result of the target object based on all the virtual point coordinates.

[0113] Among them, such as Figure 3As shown, the main components of the tactile sensing assembly 1 include a camera 8 for capturing images of target objects in the imaging area, a light source module 9 for providing lighting conditions, a fixed support 5 to ensure that the relative positions between the components remain unchanged during the experiment, a silicone-based elastomer 4 for supporting the spherical marker point array 3, an outer silicone-based protective layer 2, and two plane mirrors 6 and 7 placed at an angle to the camera. The placement of the mirrors is designed to achieve multi-view observation based on a single camera, simulating multiple virtual cameras through mirror reflection; the camera faces the two mirrors, which reflect the image of the silicone-based elastomer back to the camera. Through the reflection effect of the mirrors, the viewing angle of the original single camera is expanded to two virtual camera viewing angles, corresponding to the two reflection areas of the mirrors, respectively. The camera and the mirror are on the same side, and both fields of view need to be reflected, thereby unifying the focal length and focusing clearly. When the silicone-based elastomer contacts different objects causing the marker points to shift, the camera obtains images of two different viewing angles through the mirrors, forming virtual binocular tactile vision, and using the stable material contact characteristics between the spherical marker points and the transparent elastic layer to improve the stability of force field perception.

[0114] Specifically, as shown in Figure 4 , a plurality of fixed points of the mirrors are obtained, and through each of the fixed points, a plurality of virtual cameras of the original camera are obtained, a pixel coordinate system of the plurality of virtual cameras is constructed, and a plane equation of the corresponding mirror in the pixel coordinate system is obtained:

[0115] q1=(x q1 , y q1 , z q1 ), q2=(x q2 , y q2 , z q2 );

[0116] A1x+B1y+C1z+D1=0, A2x+B2y+C2z+D2=0;

[0117] n1={x m1 , y m1 , z m1}, n2={x m2 , y m2 , z m2};

[0118] wherein q1 and q2 represent fixed points, x q1 , y q1 , and z q1 represent the horizontal coordinate, vertical coordinate, and vertical coordinate of the fixed point q1, respectively, x q2 , y q2 , and z q2They represent the horizontal coordinate, vertical coordinate and vertical coordinate of the fixed point q2 respectively, n1 represents the first normal vector of the first mirror corresponding to the fixed point q1, n2 represents the second normal vector of the second mirror corresponding to the fixed point q2, x m1 、y m1 and z m1 Represents the horizontal coordinate, vertical coordinate and vertical coordinate of the first normal vector, x m2 、y m2 and z m2 Represents the horizontal coordinate, vertical coordinate and vertical coordinate of the second normal vector;

[0119] D1=-(A1x s1 +B1y q1 +C1z q1 ), A1=x m1 , B1=y m1 , C1=z m1 ;

[0120] D2=-(A2x q2 +B2y q2 +C2z q2 ), A2=x m2 , B2=y m2 , C2=z m2 ;

[0121] Wherein, D1 represents the constant term in the plane equation of the first mirror surface, D2 represents the constant term in the plane equation of the second mirror surface, A1x+B1y+C1z+D1 and A2x+B2y+C2z+D2 represent the plane equation of the first mirror surface and the plane equation of the second mirror surface, respectively;

[0122] By mirror reflection, the virtual matrices of the multiple virtual cameras in the pixel coordinate system are calculated:

[0123]

[0124] in, and They represent the horizontal coordinate basis vector, vertical coordinate basis vector and vertical coordinate basis vector of the first virtual camera in the first virtual camera coordinate system respectively, and They represent the coordinate representation of the abscissa basis vector of the first virtual camera, and They represent the coordinate representation of the vertical coordinate basis vector of the first virtual camera, and They represent the coordinate representation of the vertical coordinate basis vector of the first virtual camera, and They represent the horizontal coordinate basis vector, vertical coordinate basis vector and vertical coordinate basis vector of the first virtual camera in the second virtual camera coordinate system respectively, and They represent the coordinate representation of the abscissa basis vector of the second virtual camera, and They represent the coordinate representation of the ordinate basis vector of the second virtual camera, and Respectively represent the coordinate representation of the vertical coordinate basis vector of the second virtual camera;

[0125]

[0126] Where x1, y1, and z1 represent the horizontal coordinate, vertical coordinate, and vertical coordinate of the origin of the first virtual coordinate system, respectively; x2, y2, and z2 represent the horizontal coordinate, vertical coordinate, and vertical coordinate of the origin of the second virtual coordinate system, respectively; x0, y0, and z0 represent the horizontal coordinate, vertical coordinate, and vertical coordinate of the origin v0 of the camera coordinate system, respectively;

[0127]

[0128] in, represents the virtual matrix of the first virtual camera, and Represents the rotation matrix and translation matrix between the first virtual coordinate system and the pixel coordinate system, 0 1×3 represents a 1×3 matrix with all elements equal to 0, Matrix representation of the origin of the camera coordinate system;

[0129]

[0130] in, represents the virtual matrix of the second virtual camera, and Respectively represent the rotation matrix and translation matrix between the second virtual coordinate system and the pixel coordinate system;

[0131] Obtain the coordinates of multiple marker points of the target object in the world coordinate system, and map all the marker point coordinates to the pixel coordinate system to obtain the corresponding virtual point coordinates:

[0132]

[0133] A·x=B;

[0134] Among them, w0 and a0 represent the origins of the pixel coordinate systems of the two virtual cameras, respectively, and f x and f y Represents the focal length of the original camera in the horizontal and vertical directions, and Represents the coordinates of the marker points in the world coordinate system, P 1 and P 2 Both represent the virtual coordinates of the marked points at the first and second moments, P (t) represents the virtual coordinates of the marker point at time t, P i (t) represents the virtual coordinates of the i-th marker at the t-th moment, A and B represent the transformation matrix for transforming the marker coordinates, T represents transpose, and x represents the virtual point coordinates;

[0135] All the virtual point coordinates are represented by surface fitting to obtain the surface result of the target object. Among them, the calculation formulas of x1, y1 and z1 (i.e. the calculation formula of the symmetrical points about the plane) can be obtained. and The calculation process:

[0136]

[0137]

[0138] Furthermore, the virtual coordinates of the marker point at the first moment and the second moment can be expressed by the following formula:

[0139]

[0140] Furthermore, a single camera image after multiple mirror reflections is obtained, and the areas of the first mirror and the second mirror in the single camera image are screened out; the areas of the first mirror and the second mirror in the single camera image are segmented to obtain two groups of images; based on the two groups of images, the internal parameters of the original camera, the rotation matrix, and the translation matrix are calibrated using a stereo camera.

[0141] Binocular calibration (also known as stereo calibration) is a key step in a stereo vision system. It involves determining the geometric relationship between the two cameras and performing calibration using a checkerboard pattern. Considering the limited shared field of view caused by binocular vision and mirror reflections, an asymmetric calibration plate is used to improve calibration accuracy and applicability. The process involves the following steps: 1. Acquire a set of single-camera images after mirror reflection; 2. Segment the images to produce two sets of virtual camera images for the left and right mirrors. Segmentation is primarily based on the area occupied by the mirrors in the original camera, retaining the areas occupied by mirrors 1 and 2, and leaving all other values ​​blank, resulting in two sets of images; 3. Stereo camera calibration is used to obtain the internal parameters of the original camera (intrinsic parameter matrix, distortion parameters) and the relative pose parameters of the two mirrors (rotation matrix and translation vector). After binocular calibration, the three-dimensional coordinates of the markers in the world coordinate system are calculated based on the pixel coordinates of the markers in the two virtual cameras, achieving coordinate transformation.

[0142] Step S20: Obtain the offset results corresponding to all the virtual point coordinates, obtain the marking point set and the offset point set of the contact area based on all the offset results and the surface results, and calculate the contact rotation axis of the target object based on the marking point set and the offset point set.

[0143] Among them, after the coordinate transformation is realized, the three-dimensional coordinates of each discrete mark point on the contact surface can be obtained. After the object surface contacts, the contact surface will deform. The discrete point offset vector of the entire surface can be obtained through binocular vision and touch. Through surface fitting, the representation of the three-dimensional geometric surface is obtained, thereby characterizing the geometric space of the contact surface.

[0144] Specifically, the offset results corresponding to all the virtual point coordinates are obtained, wherein the offset results represent the offset generated after the target object is touched; the change in the surface height is obtained through the offset results and the surface result, and the contact area is obtained based on the change, and a set of marker points and a set of offset points of the contact area are constructed:

[0145]

[0146] in, A set of marker points representing the contact area, Indicates contact area N c Marking points, represents the set of offset points in the contact area at time t, Indicates contact area N c offset points;

[0147] Calculate the centroids of the marker point set and the offset point set respectively, remove the centroids of the marker point set and the offset point set, and obtain updated marker coordinates and updated offset coordinates:

[0148]

[0149] in, represents the centroid of the set of marked points, represents the centroid of the offset point set;

[0150]

[0151] in, Update marker coordinates, Indicates updating the offset coordinates; this process can effectively eliminate the translation effect of the overall marker points, so that the center of mass of the point set is located at the origin, thereby improving the accuracy of the characterization results.

[0152] Using the updated marker coordinates and the updated offset coordinates, a covariance matrix H is constructed:

[0153]

[0154] or

[0155] Furthermore, when calculating the rotation axis, it is also necessary to calculate the eigenvalues ​​and eigenvectors of the rotation matrix to determine whether there is a rotation axis. When there is an eigenvalue λ close to 1, such that |λ-1|<ε, where ε is a small tolerance, then it is considered that there is a rotation axis, otherwise it is considered that there is no rotation axis. When there is a rotation axis, the direction of the rotation axis is obtained by continuing to use the antisymmetric matrix.

[0156] After performing singular value decomposition on the covariance matrix, the optimal rotation matrix and the antisymmetric matrix of the optimal rotation matrix are calculated, and the direction of the contact rotation axis is extracted according to the antisymmetric matrix:

[0157] H=U∑V T ;

[0158] R=VU T ;

[0159]

[0160] Among them, U represents the basis vector of the offset point set, ∑ represents a 3×3 diagonal matrix, the diagonal elements are singular values, V represents a 3×3 orthogonal matrix, T represents transpose, R represents the optimal rotation matrix, R antisym represents the antisymmetric matrix of the optimal rotation matrix, Indicates the direction of contact with the rotation axis, R antisym (3, 2), Rantisym (1, 3) and R antisym (2, 1) both represent matrix indices of the contact rotation axis.

[0161] Among them, in the contact process of the object surface, in addition to translational motion along the straight line direction, torsional motion around a certain straight line direction will also occur. Therefore, when torsional motion occurs, it is also necessary to calculate the rotation axis during the torsional process to improve the accuracy and authenticity of the characterization of the object contact surface.

[0162] Step S30: Calculate the offset vector of the contact area according to the set of marking points and the set of offset points, and construct a displacement field of the set of offset points according to the offset vector.

[0163] Specifically, the offset vector of the contact area is calculated based on the set of marking points and the set of offset points:

[0164]

[0165] in, Represents the offset vector of the i-th marker point in the contact area at time t, and They represent the horizontal, vertical and vertical coordinates of the i-th mark point in the contact area at the t-th moment, respectively. and They represent the horizontal coordinate, vertical coordinate and vertical coordinate of the i-th marker point in the contact area before contact, and They represent the horizontal axis offset vector, vertical axis offset vector, and vertical axis offset vector of the i-th marker point in the contact area at time t, respectively;

[0166] Construct the displacement field of the offset point set through the offset vector:

[0167]

[0168] Where D represents the displacement field and T represents the transpose.

[0169] The displacement field is composed of the offset vectors of all marker points, which are composed of the displacement components of all markers in the horizontal, vertical and vertical directions.

[0170] Step S40: construct a spring-damper network model, input the displacement field into the spring-damper network model, output the force information of each marking point, and calculate the six-axis distributed force in the contact area based on each force information and the contact rotation axis.

[0171] Among them, by constructing a spring-damper model, the mapping relationship between displacement and force can be characterized; Figure 5As shown in the figure, a spring-damper model is constructed based on the K-neighbor method. Each marker point is simulated with multiple surrounding marker points using a spring connection, and each marker point is simulated with a spring-damper parallel connection model with the deep elastic body.

[0172] Specifically, a reconstruction model is constructed, the displacement field is input into the reconstruction model, and the force information of each marked point is output:

[0173] F R =g R (D);

[0174] F R =(f 1，x , f 1，y , f 1，z ,…,f N，x , f N，y , f N，z ) T ;

[0175] Among them, F R represents the reconstruction force, f i，x Indicates the force on the i-th mark point in the horizontal direction, g R () indicates the reconstructed model;

[0176] According to the reconstructed force, each of the marking points is connected to a preset number of surrounding marking points through a spring-damper simulation to obtain the force information of each marking point:

[0177]

[0178] Among them, F i Indicates the force on the i-th mark point, F i，j represents the spring connection between the i-th marked point and its j-th neighboring point, k i，j represents the elastic coefficient between the adjacent i-th marker point and the j-th neighbor point, Λ(i) represents the set of domain marker points of the i-th marker point, F i，b Indicates that the i-th marked point is connected in parallel with the spring-damper on the bottom surface, Represents the offset vector between the i-th marker point and the j-th neighboring point in the contact area at the t-th moment, Represents the offset vector between the i-th marker point in the contact area at time t and the origin of the pixel coordinate system;

[0179]

[0180] Among them, k i，b represents the elastic coefficient of the spring-damper between the i-th mark point and the bottom surface, c i，b represents the damping of the spring-damper between the i-th mark point and the bottom surface;

[0181]

[0182] Among them, F total represents the six-axis force, K represents the stiffness matrix composed of all elastic coefficients in the contact area, and C represents the damping matrix composed of all damping in the contact area;

[0183] According to each of the force information and the contact rotation axis, the rotation axis torque of all the marked points in the contact area is calculated, and according to all the rotation axis torques, the six-axis distributed force in the contact area is calculated:

[0184] M i =(M i，x , M i，y , M i，z );

[0185] M i =d i，⊥ ×F i ;

[0186]

[0187] Among them, M i Represents the rotation axis torque of the i-th mark point, M i，x 、M i，y and M i，z Represents the components of the rotation axis torque of the i-th mark point on the horizontal axis, vertical axis and vertical axis respectively, d i，⊥ represents the component perpendicular to the rotation axis, d i Indicates the distance from the i-th marker point to the center of the region, represents the set of offset points in the contact area at time t, represents the damping of the i-th marker at the t-th moment, represents the unit vector of the contact rotation axis, Indicates the direction of contact with the rotation axis, N c Indicates the number of markers in the contact area, M total represents the total moment in the contact area.

[0188] Among them, combining the three-dimensional geometric deformation of the contact surface and calculating the three-axis force and three-axis torque field based on the spring-damper model method can accurately calculate the geometric representation of the contact surface based on the force and torque calibration method of regular shapes, thereby improving the scope, accuracy and computational efficiency of the model method in force distribution reconstruction.

[0189] Step S50: defining an objective function according to the total torque, updating the spring-damper network model using the objective function, and representing the force and torque of the target object using the updated spring-damper network model to obtain the six-axis distributed force of the target object.

[0190] Among them, through the above-mentioned geometric representation of the contact surface and the force and torque representation of the spring-damper model, the following force calibration experiment can be used to estimate the parameters of the model, thereby improving the accuracy of the model calculation.

[0191] Specifically, the six-axis force of the target area is calibrated by a six-axis force sensor, and the objective function is defined according to the six-axis force:

[0192]

[0193] in, represents the objective function, and It represents the total force and total torque of the target area measured by the six-axis force sensor. and represents the force and moment estimated by the spring-damper model in the contact area, and λ represents the weight parameter for balancing the force error and moment error;

[0194] Initialize multiple parameters of the spring-damper network model, and calculate the partial derivatives of the objective function with respect to the multiple initialization parameters multiple times to obtain a gradient vector:

[0195]

[0196] in, represents the gradient vector;

[0197] Using the gradient vector, iteratively update the plurality of parameters until the change in the objective function is less than a first preset threshold or gradient The norm of is less than the second preset threshold:

[0198]

[0199] in, and denote the elastic coefficients between the i-th marker point and the j-th neighboring point in the (n+1)th iteration and the nth iteration, respectively. and represents the elastic coefficient of the spring-damper between the i-th marker point and the bottom surface in the (n+1)-th iteration and the n-th iteration, and represents the damping of the spring-damper between the i-th marking point and the bottom surface in the (n+1)-th iteration and the n-th iteration, and η represents the learning rate of the spring-damper model; using the updated parameters, the spring-damper model is updated, and the updated spring-damper network model is used to represent the force and torque of the target object to obtain the six-axis distributed force of the target object in the contact area.

[0200] A standard six-axis force sensor is used, with a rigid object of a regular shape mounted on the end. These objects are placed in contact with the visual-tactile surface according to different contact patterns. Regular shapes like spheres and cylinders are used here, and the resulting contact force (i.e., the six-axis distributed force) is obtained under a fixed contact pattern. This six-axis distributed force includes forces and moments in three directions.

[0201] Furthermore, the gradient descent method is used to estimate the model parameters. Each parameter is updated according to the learning rate along the negative gradient direction of the objective function. In each iteration, the partial derivative of the objective function with respect to each parameter is calculated to provide a direction for the gradient descent, thereby achieving parameter adjustment. The iteration is stopped until the objective function is less than the set threshold, or the parameter update step is very small and the extracted norm is less than the set threshold, and the estimation of the model parameters of force and torque is completed.

[0202] The present invention is based on a binocular tactile platform with dual-mirror reflection, constructs virtual binocular tactile, reconstructs the three-dimensional geometric deformation of the contact surface, and adopts a spring-damper network model to estimate the six-axis force of the fingertip to improve the range, accuracy and computational efficiency of the model method in force distribution reconstruction.

[0203] Furthermore, if Figure 6 As shown, based on the above-mentioned six-axis distributed force tactile perception method based on binocular tactile perception, the present invention also provides a six-axis distributed force tactile perception system based on binocular tactile perception, wherein the six-axis distributed force tactile perception system based on binocular tactile perception includes:

[0204] The surface representation module 51 is used to obtain multiple virtual cameras of the original camera, construct pixel coordinate systems of the multiple virtual cameras, obtain coordinates of multiple marker points of the target object, map all the marker point coordinates to the pixel coordinate system, obtain corresponding virtual point coordinates, and obtain the surface result of the target object based on all the virtual point coordinates;

[0205] a rotation axis calculation module 52 for obtaining offset results corresponding to all the virtual point coordinates, obtaining a set of marked points and a set of offset points of the contact area based on all the offset results and the surface result, and calculating the contact rotation axis of the target object based on the set of marked points and the set of offset points;

[0206] a displacement field construction module 53, configured to calculate an offset vector of the contact area based on the set of marking points and the set of offset points, and to construct a displacement field of the set of offset points based on the offset vector;

[0207] a torque calculation module 54 for constructing a spring-damper network model, inputting the displacement field into the spring-damper network model, outputting force information of each marked point, and calculating the six-axis distributed force in the contact area based on each force information and the contact rotation axis;

[0208] The model updating module 55 is used to define an objective function based on the total torque, update the spring-damper network model using the objective function, and use the updated spring-damper network model to represent the force and torque of the target object to obtain the six-axis distributed force of the target object.

[0209] Furthermore, if Figure 7 As shown, based on the above-mentioned six-axis distributed force tactile perception method and system based on binocular tactile perception, the present invention also provides a terminal, which includes a processor 10, a memory 20 and a display 30. Figure 7 Only some of the components of the terminal are shown, but it should be understood that implementation of all of the shown components is not required, and more or fewer components may be implemented instead.

[0210] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal. In other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the terminal. Furthermore, the memory 20 may also include both an internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software and various types of data installed on the terminal, such as the program code of the installation terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, a six-axis distributed force tactile perception program 40 based on binocular tactile perception is stored on the memory 20, and the six-axis distributed force tactile perception program 40 based on binocular tactile perception can be executed by the processor 10, thereby realizing the six-axis distributed force tactile perception method based on binocular tactile perception in the present application.

[0211] The processor 10 can be a Central Processing Unit (CPU), microprocessor or other data processing chip in some embodiments, for running program codes stored in the memory 20 or processing data, such as executing the binocular vision haptics based six-axis distributed force haptics perception method, etc.

[0212] The display 30 can be an LED display, liquid crystal display, touch liquid crystal display, OLED (Organic Light-Emitting Diode) touch, etc. in some embodiments. The display 30 is used to display information of the terminal and to display visualized user interface. The components of the terminal communicate with each other through a system bus.

[0213] In an embodiment, the steps as described above are implemented when the processor 10 executes the binocular vision haptics based six-axis distributed force haptics perception program 40 in the memory 20.

[0214] The application also provides a computer readable storage medium, wherein the computer readable storage medium stores a binocular vision haptics based six-axis distributed force haptics perception program, and the binocular vision haptics based six-axis distributed force haptics perception program, when executed by a processor, implements the steps of the binocular vision haptics based six-axis distributed force haptics perception method as described above.

[0215] In summary, the present invention provides a six-axis distributed force tactile perception method based on binocular tactile perception and related equipment, the method comprising: obtaining multiple virtual cameras of the original camera, constructing a pixel coordinate system of the multiple virtual cameras, and obtaining multiple marker point coordinates of the target object, mapping all the marker point coordinates to the pixel coordinate system to obtain corresponding virtual point coordinates, and obtaining a surface result of the target object based on all the virtual point coordinates; obtaining offset results corresponding to all the virtual point coordinates, obtaining a marker point set and an offset point set of the contact area based on all the offset results and the surface result, and calculating the target object based on the marker point set and the offset point set. The invention relates to a method for determining the contact rotation axis of an object; calculating an offset vector of the contact area based on the set of marker points and the set of offset points, and constructing a displacement field of the offset point set based on the offset vector; constructing a spring-damper network model, inputting the displacement field into the spring-damper network model, outputting force information of each marker point, and calculating the six-axis distributed force in the contact area based on each force information and the contact rotation axis; defining an objective function based on the total torque, updating the spring-damper network model using the objective function, and using the updated spring-damper network model to represent the force and torque of the target object to obtain the six-axis distributed force of the target object. The invention is based on a binocular tactile platform with dual mirror reflection, constructs a virtual binocular tactile platform, reconstructs the three-dimensional geometric deformation of the contact surface, and uses a spring-damper network model to estimate the six-axis force of the fingertip to improve the range, accuracy and computational efficiency of the model method in force distribution reconstruction.

[0216] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or terminal comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or terminal comprising the element.

[0217] Of course, those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware (such as a processor, controller, etc.) through a computer program. The program can be stored in a computer-readable storage medium that can be read by a computer. When the program is executed, it can include the processes in the above-described method embodiments. The computer-readable storage medium can be a memory, a magnetic disk, an optical disk, etc.

[0218] It should be understood that the application of the present invention is not limited to the above examples. For those skilled in the art, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A six-axis distributed force tactile perception method based on binocular tactile perception, characterized in that: The six-axis distributed force tactile perception method based on binocular tactile perception includes: Acquire multiple virtual cameras of the original camera, construct pixel coordinate systems of the multiple virtual cameras, and obtain multiple marker point coordinates of the target object, map all the marker point coordinates to the pixel coordinate system to obtain corresponding virtual point coordinates, and obtain the surface result of the target object based on all the virtual point coordinates; Obtaining offset results corresponding to all virtual point coordinates, obtaining a set of marked points and a set of offset points of the contact area based on all the offset results and the surface results, and calculating the contact rotation axis of the target object based on the set of marked points and the set of offset points; Calculating an offset vector of the contact area according to the set of marking points and the set of offset points, and constructing a displacement field of the set of offset points according to the offset vector; Constructing a spring-damper network model, inputting the displacement field into the spring-damper network model, outputting force information of each marked point, and calculating the six-axis distributed force in the contact area based on each force information and the contact rotation axis; An objective function is defined according to the total torque, the spring-damper network model is updated using the objective function, and the updated spring-damper network model is used to represent the force and torque of the target object to obtain the six-axis distributed force of the target object.

2. The six-axis distributed force tactile perception method based on binocular tactile perception according to claim 1, characterized in that: The method of obtaining multiple virtual cameras of the original camera, constructing pixel coordinate systems of the multiple virtual cameras, obtaining coordinates of multiple marker points of the target object, mapping all the marker point coordinates to the pixel coordinate system, obtaining corresponding virtual point coordinates, and obtaining a surface result of the target object based on all the virtual point coordinates specifically includes: Obtain multiple fixed points of the mirror surface, and obtain multiple virtual cameras of the original camera through each of the fixed points, construct the pixel coordinate systems of the multiple virtual cameras, and obtain the plane equations of the corresponding mirror surface in the pixel coordinate system: q1=(x q1 ,y q1 ,z q1 ),q2=(x q2 ,y q2 ,z q2 ); A1x+B1y+C1z+D1=0, A2x+B2y+C2z+D2=0; n1={x m1 ,y m1 ,z m1 },n2={x m2 ,y m2 ,z m2 }; Among them, q1 and q2 represent fixed points, x q1 、y q1 and z q1 They represent the horizontal, vertical and vertical coordinates of the fixed point q1, respectively. q2 、y q2 and z q2 They represent the horizontal coordinate, vertical coordinate and vertical coordinate of the fixed point q2 respectively, n1 represents the first normal vector of the first mirror corresponding to the fixed point q1, n2 represents the second normal vector of the second mirror corresponding to the fixed point q2, x m1 、y m1 and z m1 Represents the horizontal coordinate, vertical coordinate and vertical coordinate of the first normal vector, x m2 、y m2 and z m2 Represents the horizontal coordinate, vertical coordinate and vertical coordinate of the second normal vector; D1=-(A1x q1 +B1y q1 +C1z q1 ),A1=x m1 ,B1=y m1 ,C1=z m1 ; D2=-(A2x q2 +B2y q2 +C2z q2 ),A2=x m2 ,B2=y m2 ,C2=z m2 ; Wherein, D1 represents the constant term in the plane equation of the first mirror surface, D2 represents the constant term in the plane equation of the second mirror surface, A1x+B1y+C1z+D1=0 and A2x+B2y+C2z+D2=0 represent the plane equation of the first mirror surface and the plane equation of the second mirror surface, respectively; By mirror reflection, the virtual matrices of the multiple virtual cameras in the pixel coordinate system are calculated: in, and They represent the horizontal coordinate basis vector, vertical coordinate basis vector and vertical coordinate basis vector of the first virtual camera in the first virtual camera coordinate system respectively, and They represent the coordinate representation of the abscissa basis vector of the first virtual camera, and They represent the coordinate representation of the vertical coordinate basis vector of the first virtual camera, and They represent the coordinate representation of the vertical coordinate basis vector of the first virtual camera, and They represent the horizontal coordinate basis vector, vertical coordinate basis vector and vertical coordinate basis vector of the first virtual camera in the second virtual camera coordinate system respectively, and They represent the coordinate representation of the abscissa basis vector of the second virtual camera, and They represent the coordinate representation of the ordinate basis vector of the second virtual camera, and Respectively represent the coordinate representation of the vertical coordinate basis vector of the second virtual camera; Where x1, y1, and z1 represent the horizontal coordinate, vertical coordinate, and vertical coordinate of the origin of the first virtual coordinate system, respectively; x2, y2, and z2 represent the horizontal coordinate, vertical coordinate, and vertical coordinate of the origin of the second virtual coordinate system, respectively; x0, y0, and z0 represent the horizontal coordinate, vertical coordinate, and vertical coordinate of the origin v0 of the camera coordinate system, respectively; in, represents the virtual matrix of the first virtual camera, and Represents the rotation matrix and translation matrix between the first virtual coordinate system and the original camera coordinate system, 0 1×3 represents a 1×3 matrix with all elements equal to 0, Matrix representation of the origin of the camera coordinate system; in, represents the virtual matrix of the second virtual camera, and Respectively represent the rotation matrix and translation matrix between the second virtual coordinate system and the pixel coordinate system; Obtain the coordinates of multiple marker points of the target object in the world coordinate system, and map all the marker point coordinates to the pixel coordinate system to obtain the corresponding virtual point coordinates: A·x=B; Among them, w0 and a0 represent the pixel coordinates of the center of the original camera image, f x and f y Represents the focal length of the original camera in the horizontal and vertical directions, and Represents the coordinates of the marker points in the world coordinate system, P 1 and P 2 Both represent the virtual coordinates of the marked points at the first and second moments, P (t) represents the virtual coordinates of the marker point at time t, P i (t) represents the virtual coordinates of the i-th marker at the t-th moment, A and B represent the transformation matrix for transforming the marker coordinates, T represents transpose, and x represents the virtual point coordinates; All the virtual point coordinates are represented by surface fitting to obtain the surface result of the target object.

3. The six-axis distributed force tactile perception method based on binocular tactile perception according to claim 2, characterized in that: The step of calculating virtual matrices of the plurality of virtual cameras in the pixel coordinate system through mirror reflection further includes: Acquire a single camera image after being reflected by multiple mirror surfaces, and filter out regions of the first mirror surface and the second mirror surface in the single camera image; Segmenting the regions of the first mirror surface and the second mirror surface in the single camera image to obtain two groups of images; Based on the two sets of images, the internal parameters of the original camera, the rotation matrix, and the translation matrix are calibrated using a stereo camera.

4. The six-axis distributed force tactile perception method based on binocular tactile perception according to claim 1, characterized in that: The step of obtaining the offset results corresponding to all the virtual point coordinates, obtaining a set of marked points and a set of offset points of the contact area according to all the offset results and the surface results, and calculating the contact rotation axis of the target object according to the set of marked points and the set of offset points, specifically includes: Obtaining offset results corresponding to all virtual point coordinates, wherein the offset results represent offsets generated after the target object is touched; The change of the surface height is obtained through the offset result and the surface result. The contact area is obtained according to the change, and the marker point set and offset point set of the contact area are constructed: in, A set of marker points representing the contact area, Indicates contact area N c Marking points, represents the set of offset points in the contact area at time t, Indicates contact area N c offset points; Calculate the centroids of the marker point set and the offset point set respectively, remove the centroids of the marker point set and the offset point set, and obtain updated marker coordinates and updated offset coordinates: in, represents the centroid of the set of marked points, represents the centroid of the offset point set; in, Update marker coordinates, Indicates updating offset coordinates; Using the updated marker coordinates and the updated offset coordinates, a covariance matrix H is constructed: or After performing singular value decomposition on the covariance matrix, the optimal rotation matrix and the antisymmetric matrix of the optimal rotation matrix are calculated, and the direction of the contact rotation axis is extracted according to the antisymmetric matrix: H=U∑V T ; R=VU T ; Among them, U represents the basis vector of the offset point set, ∑ represents a 3×3 diagonal matrix, the diagonal elements are singular values, V represents a 3×3 orthogonal matrix, T represents transpose, R represents the optimal rotation matrix, R antisym represents the antisymmetric matrix of the optimal rotation matrix, Indicates the direction of contact with the rotation axis, R antisym (3,2), R antisym (1,3) and R antisym (2,1) both represent the matrix index of the contact rotation axis.

5. The six-axis distributed force tactile perception method based on binocular tactile perception according to claim 1, characterized in that: Calculating the offset vector of the contact area according to the marker point set and the offset point set, and constructing the displacement field of the offset point set according to the offset vector, specifically includes: Calculate the offset vector of the contact area based on the set of marker points and the set of offset points: in, Represents the offset vector of the i-th marker point in the contact area at time t, and They represent the horizontal, vertical and vertical coordinates of the i-th mark point in the contact area at the t-th moment, respectively. and They represent the horizontal coordinate, vertical coordinate and vertical coordinate of the i-th mark point in the contact area before it is released, and They represent the horizontal axis offset vector, vertical axis offset vector, and vertical axis offset vector of the i-th marker point in the contact area at time t, respectively; Construct the displacement field of the offset point set through the offset vector: Where D represents the displacement field and T represents the transpose.

6. The six-axis distributed force tactile perception method based on binocular tactile perception according to claim 5, characterized in that: The step of constructing a spring-damper network model, inputting the displacement field into the spring-damper network model, outputting force information of each marker point, and calculating the six-axis distributed force in the contact area based on each force information and the contact rotation axis specifically includes: Construct a reconstruction model, input the displacement field into the reconstruction model, and output the force information of each marked point: F R =g R (D); F R =(f 1,x ,f 1,y ,f 1,z ,…,f N,x ,f N,y ,f N,z ) T ; Among them, F R represents the reconstruction force, f i,x Indicates the force on the i-th mark point in the horizontal direction, g R () indicates the reconstructed model; According to the reconstructed force, each of the marking points is connected to a preset number of surrounding marking points through a spring-damper simulation to obtain the force information of each marking point: Among them, F i Indicates the force on the i-th mark point, F i,j represents the spring connection between the i-th marked point and its j-th neighboring point, k i,j represents the elastic coefficient between the adjacent i-th marker point and the j-th neighbor point, Λ(i) represents the set of domain marker points of the i-th marker point, F i,b Indicates that the i-th marked point is connected in parallel with the spring-damper on the bottom surface, Represents the offset vector between the i-th marker point in the contact area at time t and the j-th neighboring point around it, Represents the offset vector between the i-th marker point in the contact area at time t and the origin of the pixel coordinate system; Among them, k i,b represents the elastic coefficient of the spring-damper between the i-th mark point and the bottom surface, c i,b represents the damping of the spring-damper between the i-th mark point and the bottom surface; Among them, F total represents the six-axis force, K represents the stiffness matrix composed of all elastic coefficients in the contact area, and C represents the damping matrix composed of all damping in the contact area; According to each of the force information and the contact rotation axis, the rotation axis torque of all the marked points in the contact area is calculated, and according to all the rotation axis torques, the six-axis distributed force in the contact area is calculated: M i =(M i,x ,M i,y ,M i,z ); M i =d i,⊥ ×F i ; Among them, M i Represents the rotation axis torque of the i-th mark point, M i,x 、M i,y and M i,z Represents the components of the rotation axis torque of the i-th mark point on the horizontal axis, vertical axis and vertical axis respectively, d i,⊥ represents the component perpendicular to the rotation axis, d i Indicates the distance from the i-th marker point to the center of the region, represents the set of offset points in the contact area at time t, represents the damping of the i-th marker at the t-th moment, represents the unit vector of the contact rotation axis, Indicates the direction of contact with the rotation axis, N c Indicates the number of markers in the contact area, M total represents the total moment in the contact area.

7. The six-axis distributed force tactile perception method based on binocular tactile perception according to claim 6, characterized in that: The step of defining an objective function according to the total torque, updating the spring-damper network model using the objective function, and representing the force and torque of the target object using the updated spring-damper network model to obtain the six-axis distributed force of the target object specifically includes: The six-axis force of the target area is calibrated by a six-axis force sensor, and the objective function is defined based on the six-axis force: in, represents the objective function, and It represents the total force and total torque of the target area measured by the six-axis force sensor. and represents the force and moment estimated by the spring-damper model in the contact area, and λ represents the weight parameter for balancing the force error and moment error; Initialize multiple parameters of the spring-damper network model, and calculate the partial derivatives of the objective function with respect to the multiple initialization parameters multiple times to obtain a gradient vector: in, represents the gradient vector; Using the gradient vector, iteratively update the plurality of parameters until the change in the objective function is less than a first preset threshold or gradient The norm of is less than the second preset threshold: in, and denote the elastic coefficients between the i-th marker point and the j-th neighboring point in the (n+1)th iteration and the nth iteration, respectively. and represents the elastic coefficient of the spring-damper between the i-th marker point and the bottom surface in the (n+1)-th iteration and the n-th iteration, and represents the damping of the spring-damper between the i-th marker point and the bottom surface in the (n+1)-th iteration and the n-th iteration, and η represents the learning rate of the spring-damper model; The updated parameters are used to update the spring-damper model, and the updated spring-damper network model is used to represent the force and torque of the target object to obtain the six-axis distributed force of the target object in the contact area.

8. A six-axis distributed force tactile perception system based on binocular tactile perception, characterized in that: The six-axis distributed force tactile perception system based on binocular tactile perception includes: A surface representation module is used to obtain multiple virtual cameras of the original camera, construct pixel coordinate systems of the multiple virtual cameras, obtain multiple marker point coordinates of the target object, map all the marker point coordinates into the pixel coordinate system, obtain corresponding virtual point coordinates, and obtain the surface result of the target object based on all the virtual point coordinates; a rotation axis calculation module, configured to obtain offset results corresponding to all virtual point coordinates, obtain a set of marked points and a set of offset points of the contact area based on all offset results and the curved surface results, and calculate the contact rotation axis of the target object based on the set of marked points and the set of offset points; a displacement field construction module, configured to calculate an offset vector of the contact area based on the set of marking points and the set of offset points, and to construct a displacement field of the set of offset points based on the offset vector; a torque calculation module, configured to construct a spring-damper network model, input the displacement field into the spring-damper network model, output force information of each marked point, and calculate the six-axis distributed force in the contact area based on each force information and the contact rotation axis; A model updating module is used to define an objective function based on the total torque, update the spring-damper network model using the objective function, and use the updated spring-damper network model to represent the force and torque of the target object to obtain the six-axis distributed force of the target object.

9. A terminal, characterized in that: The terminal includes: a memory, a processor, and a six-axis distributed force tactile perception program based on binocular tactiles stored in the memory and runnable on the processor. When the six-axis distributed force tactile perception program based on binocular tactiles is executed by the processor, the steps of the six-axis distributed force tactile perception method based on binocular tactiles are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a six-axis distributed force tactile perception program based on binocular tactile perception. When the six-axis distributed force tactile perception program based on binocular tactile perception is executed by a processor, the steps of the six-axis distributed force tactile perception method based on binocular tactile perception are implemented as described in any one of claims 1 to 7.

Images