Six-axis distributed force touch sensing method, system and terminal based on binocular visual touch

Abstract

The invention discloses a six-axis distributed force tactile sensing method, system and terminal based on binocular tactile sense, and the method comprises the steps: obtaining virtual binocular tactile sense, collecting a curved surface result of a target object, and calculating a rotation axis and six-axis distributed force of a contact area after the target object is contacted, and iteratively updating a plurality of parameters of the spring-damping network model so as to further collect geometric space characteristics of the target object when the target object is touched. According to the method, on the basis of a double-mirror reflection double-vision tactile platform, virtual double-vision tactile sense is constructed, three-dimensional geometric deformation of a contact surface is reconstructed, a spring-damping network model is adopted, and the six-axis force of the fingertip is estimated, so that the range, precision and calculation efficiency of the model method in force distribution reconstruction are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of robot tactile perception analysis, and in particular to a six-axis distributed force tactile perception method, system, terminal and computer-readable storage medium based on binocular vision and touch. Background Art

[0002] Visual tactile sensors have attracted wide attention in the robotics community for their high spatial resolution sensing advantages. According to the implementation principle, visual sensors can be divided into reflective film type and marker point type, which convert contact deformation information into visual images, which can be used to further extract tactile features, from three-dimensional contact geometry to multi-axis force; reflective film type uses photometric methods to achieve pixel-level spatial resolution, and marker point type visual tactile sensor instruments obtain the geometric deformation of the contact area based on discrete marker points.

[0003] However, the reflective film type has high requirements on the reflective film material and lighting conditions, and has limitations for further force field perception. The marker point type is highly selective in structure and material. In addition, due to the differences in marker point types and three-dimensional geometric deformation measurement methods, as well as the high nonlinearity of elastic materials, the use of visual-tactile sensors for multi-axis force and torque estimation has the problem of low computational efficiency.

[0004] Therefore, the prior art still needs to be improved and developed. Summary of the invention

[0005] The main purpose of the present invention is to provide a six-axis distributed force tactile perception method, system, terminal and computer-readable storage medium based on binocular vision and touch, aiming to solve the problems in the prior art of conducting vision and touch sensing research, that is, the direct use of finite element calculation methods leads to low computational efficiency, and the use of learning methods relies on a large amount of data, resulting in limited generalization ability.

[0006] To achieve the above object, the present invention provides a six-axis distributed force tactile perception method based on binocular vision and touch, and the six-axis distributed force tactile perception method based on binocular vision and touch comprises the following steps:

[0007] Acquire multiple virtual cameras of the original camera, construct pixel coordinate systems of the multiple virtual cameras, and acquire multiple marker point coordinates of the target object, map all the marker point coordinates to the pixel coordinate system, obtain corresponding virtual point coordinates, and obtain the surface result of the target object according to all the virtual point coordinates;

[0008] Obtaining the offset results corresponding to all the virtual point coordinates, obtaining a set of marked points and a set of offset points of the contact area according to all the offset results and the surface results, and calculating the contact rotation axis of the target object according to the set of marked points and the set of offset points;

[0009] Calculating an offset vector of the contact area according to the set of marking points and the set of offset points, and constructing a displacement field of the set of offset points according to the offset vector;

[0010] Constructing a spring-damper network model, inputting the displacement field into the spring-damper network model, outputting force information of each marking point, and calculating the six-axis distributed force in the contact area according to each force information and the contact rotation axis;

[0011] An objective function is defined according to the total torque, the spring-damper network model is updated using the objective function, and the updated spring-damper network model is used to represent the force and torque of the target object.

[0012] Optionally, the six-axis distributed force tactile perception method based on binocular tactile perception, wherein the step of acquiring multiple virtual cameras of the original camera, constructing pixel coordinate systems of the multiple virtual cameras, acquiring multiple marker point coordinates of the target object, mapping all the marker point coordinates to the pixel coordinate system, obtaining corresponding virtual point coordinates, and obtaining the surface result of the target object according to all the virtual point coordinates, specifically includes:

[0013] Obtain multiple fixed points of the mirror surface, and obtain multiple virtual cameras of the original camera through each of the fixed points, construct pixel coordinate systems of the multiple virtual cameras, and obtain the plane equation of the corresponding mirror surface in the pixel coordinate system:

[0014] q1=(x q1 ,y q1 , z q1 ), q2=(x q2 ,y q2 , z q2 );

[0015] A1x+B1y+C1z+D1=0, A2x+B2y+C2z+D2=0;

[0016] n1={x m1 ,y m1 , z m1}, n2={x m2 ,y m2 , z m2};

[0017] Among them, q1 and q2 represent fixed points, x q1 ,y q1 and z q1 They represent the horizontal, vertical and vertical coordinates of the fixed point q1, respectively. q2 ,y q2 and z q2represents the horizontal coordinate, vertical coordinate and vertical coordinate of the fixed point q2 respectively, n1 represents the first normal vector of the first mirror corresponding to the fixed point q1, n2 represents the second normal vector of the second mirror corresponding to the fixed point q2, x m1 ,y m1 and z m1 represents the horizontal coordinate, vertical coordinate and vertical coordinate of the first normal vector, x m2 ,y m2 and z m2 Represents the abscissa, ordinate and ordinate of the second normal vector;

[0018] D1=-(A1x q1 +B1y q1 +C1z q1 ), A1=x m1 , B1=y m1 , C1=z m1 ;

[0019] D2=-(A2x q2 +B2y q2 +C2z q2 ), A2=x m2 , B2=y m2 , C2=z m2 ;

[0020] Wherein, D1 represents the constant term in the plane equation of the first mirror surface, D2 represents the constant term in the plane equation of the second mirror surface, A1x+B1y+C1z+D1=0 and A2x+B2y+C2z+D2=0 represent the plane equation of the first mirror surface and the plane equation of the second mirror surface respectively;

[0021] By mirror reflection, the virtual matrices of the multiple virtual cameras in the pixel coordinate system are calculated:

[0022]

[0023] in, and represent the horizontal coordinate basis vector, the vertical coordinate basis vector and the vertical coordinate basis vector of the first virtual camera in the first virtual camera coordinate system respectively, and They respectively represent the coordinate representation of the abscissa basis vector of the first virtual camera, and They respectively represent the coordinate representation of the ordinate basis vector of the first virtual camera, and They respectively represent the coordinate representation of the vertical coordinate basis vector of the first virtual camera, and represent the horizontal coordinate basis vector, the vertical coordinate basis vector and the vertical coordinate basis vector of the first virtual camera in the second virtual camera coordinate system respectively, and They respectively represent the coordinate representation of the horizontal coordinate basis vector of the second virtual camera, and They respectively represent the coordinate representation of the ordinate basis vector of the second virtual camera, and Respectively represent the coordinate representation of the vertical coordinate basis vector of the second virtual camera;

[0024]

[0025] Wherein, x1, y1 and z1 represent the abscissa, ordinate and ordinate of the origin of the first virtual coordinate system, respectively; x2, y2 and z2 represent the abscissa, ordinate and ordinate of the origin of the second virtual coordinate system, respectively; x0, y0 and z0 represent the abscissa, ordinate and ordinate of the origin v0 of the camera coordinate system, respectively;

[0026]

[0027] in, represents the virtual matrix of the first virtual camera, and Respectively represent the rotation matrix and translation matrix between the first virtual coordinate system and the original camera coordinate system, 0 1×3 represents a 1×3 matrix with all elements equal to 0, Matrix representation of the origin of the camera coordinate system;

[0028]

[0029] in, represents the virtual matrix of the second virtual camera, and Respectively represent the rotation matrix and translation matrix between the second virtual coordinate system and the pixel coordinate system;

[0030] Obtain the coordinates of multiple marking points of the target object in the world coordinate system, and map all the marking point coordinates to the pixel coordinate system to obtain the corresponding virtual point coordinates:

[0031]

[0032]

[0033] A·x=B;

[0034] Among them, w0 and a0 represent the pixel coordinates of the center of the original camera image, f x and f yRespectively represent the focal length of the original camera in the horizontal and vertical directions, and Represents the coordinates of the marker points in the world coordinate system, P 1 and P 2 Both represent the virtual coordinates of the marked points at the first and second moments, P (t) represents the virtual coordinates of the marked point at the tth moment, P i (t) represents the virtual coordinates of the i-th marker point at the t-th time, A and B both represent the transformation matrix for transforming the marker point coordinates, T represents transpose, and x represents the virtual point coordinates;

[0035] All the virtual point coordinates are represented by surface fitting to obtain the surface result of the target object.

[0036] Optionally, the six-axis distributed force tactile perception method based on binocular tactile perception, wherein the virtual matrix of the plurality of virtual cameras in the pixel coordinate system is calculated by mirror reflection, further comprising:

[0037] Acquire a single camera image after multiple mirror reflections, and filter out regions of the first mirror and the second mirror in the single camera image;

[0038] Segmenting the regions of the first mirror surface and the second mirror surface in the single camera image to obtain two groups of images;

[0039] According to the two groups of images, the internal parameters of the original camera, the rotation matrix and the translation matrix are calibrated using a stereo camera.

[0040] Optionally, the six-axis distributed force tactile perception method based on binocular tactile perception, wherein the obtaining of the offset results corresponding to all the virtual point coordinates, obtaining a set of marked points and a set of offset points of the contact area according to all the offset results and the surface results, and calculating the contact rotation axis of the target object according to the set of marked points and the set of offset points, specifically includes:

[0041] Obtaining offset results corresponding to all the virtual point coordinates, wherein the offset results represent offsets generated after the target object is touched;

[0042] The change of the surface height is obtained through the offset result and the surface result. According to the change, the contact area is obtained, and the marking point set and the offset point set of the contact area are constructed:

[0043]

[0044] in, A set of marker points representing the contact area, Indicates the contact area N c Marking points, represents the set of offset points in the contact area at time t, Indicates the contact area N c offset points;

[0045] The centroids of the marking point set and the offset point set are calculated respectively, and the marking point set and the offset point set are removed from the centroid to obtain updated marking coordinates and updated offset coordinates:

[0046]

[0047] in, represents the centroid of the set of marked points, represents the centroid of the offset point set;

[0048]

[0049] in, Update marker coordinates, Indicates updating the offset coordinates;

[0050] Using the updated marker coordinates and the updated offset coordinates, a covariance matrix H is constructed:

[0051]

[0052] or

[0053] After performing singular value decomposition on the covariance matrix, the optimal rotation matrix and the antisymmetric matrix of the optimal rotation matrix are calculated, and the direction of the contact rotation axis is extracted according to the antisymmetric matrix:

[0054] H=U∑V T ;

[0055] R=VU T ;

[0056]

[0057] Among them, U represents the basis vector of the offset point set, ∑ represents a 3×3 diagonal matrix, the diagonal elements are singular values, V represents a 3×3 orthogonal matrix, T represents transpose, R represents the optimal rotation matrix, and R antisym represents the antisymmetric matrix of the optimal rotation matrix, Indicates the direction of contact rotation axis, R antisym (3, 2), R antisym (1, 3) and R antisym(2, 1) both represent the matrix index of the contact rotation axis.

[0058] Optionally, the six-axis distributed force tactile perception method based on binocular tactile perception, wherein the step of calculating the offset vector of the contact area according to the set of marked points and the set of offset points, and constructing the displacement field of the set of offset points according to the offset vector, specifically includes:

[0059] According to the set of marking points and the set of offset points, the offset vector of the contact area is calculated:

[0060]

[0061] in, represents the offset vector of the i-th marker point in the contact area at time t, and They represent the horizontal, vertical and vertical coordinates of the i-th mark point in the contact area at the t-th moment, respectively. and They represent the horizontal, vertical and vertical coordinates of the i-th marked point in the contact area before contact, and They represent the horizontal axis offset vector, the vertical axis offset vector and the vertical axis offset vector of the i-th marking point in the contact area at the t-th moment respectively;

[0062] Construct the displacement field of the offset point set through the offset vector:

[0063]

[0064] Where D represents the displacement field and T represents the transpose.

[0065] Optionally, the six-axis distributed force tactile perception method based on binocular tactile perception, wherein the spring-damper network model is constructed, the displacement field is input into the spring-damper network model, the force information of each marking point is output, and the six-axis force distribution force in the contact area is calculated according to each of the force information and the contact rotation axis, specifically including:

[0066] Construct a reconstruction model, input the displacement field into the reconstruction model, and output the force information of each marked point:

[0067] F R =g R (D);

[0068] F R =(f 1，x , f 1，y , f 1，z , …, f N，x , f N，y , fN，z ) T ;

[0069] Among them, F R represents the reconstruction force, f i，x Indicates the force on the i-th mark point in the horizontal direction, g R Represents the reconstructed model;

[0070] According to the reconstructed force, each of the marking points is connected to a preset number of surrounding marking points through a spring-damping simulation to obtain the force information of each of the marking points:

[0071]

[0072] Among them, F i represents the force on the i-th marked point, F i，j represents the spring connection between the i-th marked point and its j-th neighboring point, k i，j represents the elastic coefficient between the i-th marker point and the j-th neighbor point, Λ(i) represents the set of marker points in the area of ​​the i-th marker point, and F i，b Indicates that the i-th marked point is connected in parallel with the spring-damper on the bottom surface, Represents the offset vector between the i-th marker point in the contact area at time t and the j-th neighboring point around it, represents the offset vector between the i-th marker point in the contact area at the t-th moment and the origin of the pixel coordinate system;

[0073]

[0074] Among them, k i，b represents the elastic coefficient of the spring-damper between the ith mark point and the bottom surface, c i，b represents the damping of the spring-damper between the i-th marked point and the bottom surface;

[0075]

[0076] Among them, F total represents the six-axis force, K represents the stiffness matrix composed of all elastic coefficients in the contact area, and C represents the damping matrix composed of all damping in the contact area;

[0077] According to each of the force information and the contact rotation axis, the rotation axis torque of all the marked points in the contact area is calculated, and according to all the rotation axis torques, the six-axis distributed force in the contact area is calculated:

[0078] M i =(M i，x , M i，y , M i，z );

[0079] M i =d i，⊥ ×F i ;

[0080]

[0081] Among them, M i represents the rotation axis torque of the i-th marker point, M i，x 、M i，y and M i，z Respectively represent the components of the rotation axis torque of the i-th mark point on the horizontal axis, vertical axis and vertical axis, d i，⊥ represents the component perpendicular to the rotation axis, d i Indicates the distance from the i-th marker point to the center of the region, represents the set of offset points in the contact area at time t, represents the damping of the i-th marker at the t-th time, represents the unit vector about the contact rotation axis, Indicates the direction of contact rotation axis, N c Indicates the number of markers in the contact area, M total Represents the total moment in the contact area.

[0082] Optionally, the six-axis distributed force tactile perception method based on binocular tactile perception, wherein the objective function is defined according to the total torque, the spring-damper network model is updated using the objective function, and the updated spring-damper network model is used to represent the force and torque of the target object to obtain the six-axis distributed force of the target object, specifically includes:

[0083] The six-axis force of the target area is calibrated by a six-axis force sensor, and the objective function is defined according to the six-axis force:

[0084]

[0085] in, represents the objective function, and represents the total force and total moment of the target area measured by the six-axis force sensor, and represents the force and moment of the contact area estimated by the spring-damper model, and λ represents the weight parameter for balancing the force error and moment error;

[0086] Initialize multiple parameters of the spring-damper network model, and calculate the partial derivatives of the objective function with respect to the multiple initialization parameters multiple times to obtain a gradient vector:

[0087]

[0088] in, represents the gradient vector;

[0089] Using the gradient vector, iteratively update the multiple parameters until the change of the objective function is less than the first preset threshold or the gradient The norm of is less than the second preset threshold:

[0090]

[0091] in, and denote the elastic coefficients between the i-th marked point and the j-th neighboring point in the (n+1)th iteration and the nth iteration, respectively. and represents the elastic coefficient of the spring-damper between the i-th marking point and the bottom surface in the (n+1)th iteration and the nth iteration, and represents the damping of the spring-damper between the i-th marker point and the bottom surface in the (n+1)th iteration and the nth iteration, and η represents the learning rate of the spring-damper model;

[0092] The spring-damper model is updated using the updated parameters, and the updated spring-damper network model is used to represent the force and moment of the target object to obtain the six-axis distributed force of the target object in the contact area.

[0093] In addition, to achieve the above-mentioned purpose, the present invention further provides a six-axis distributed force tactile perception system based on binocular vision and tactile perception, wherein the six-axis distributed force tactile perception system based on binocular vision and tactile perception comprises:

[0094] A surface representation module is used to obtain multiple virtual cameras of the original camera, construct pixel coordinate systems of the multiple virtual cameras, and obtain multiple marker point coordinates of the target object, map all the marker point coordinates to the pixel coordinate system, obtain corresponding virtual point coordinates, and obtain the surface result of the target object according to all the virtual point coordinates;

[0095] A rotation axis calculation module is used to obtain the offset results corresponding to all the virtual point coordinates, obtain a set of marked points and a set of offset points of the contact area according to all the offset results and the surface results, and calculate the contact rotation axis of the target object according to the set of marked points and the set of offset points;

[0096] A displacement field construction module, used to calculate the offset vector of the contact area according to the marking point set and the offset point set, and to construct the displacement field of the offset point set according to the offset vector;

[0097] A torque calculation module, used for constructing a spring-damper network model, inputting the displacement field into the spring-damper network model, outputting force information of each marking point, and calculating the six-axis distributed force in the contact area according to each force information and the contact rotation axis;

[0098] A model updating module is used to define an objective function according to the total torque, update the spring-damper network model using the objective function, and use the updated spring-damper network model to represent the force and torque of the target object to obtain the six-axis distributed force of the target object.

[0099] In addition, to achieve the above-mentioned purpose, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a six-axis distributed force tactile perception program based on binocular vision and tactile sense stored in the memory and runnable on the processor, and when the six-axis distributed force tactile perception program based on binocular vision and tactile sense is executed by the processor, the steps of the six-axis distributed force tactile perception method based on binocular vision and tactile sense as described above are implemented.

[0100] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a six-axis distributed force tactile perception program based on binocular vision and tactile perception, and when the six-axis distributed force tactile perception program based on binocular vision and tactile perception is executed by a processor, the steps of the six-axis distributed force tactile perception method based on binocular vision and tactile perception as described above are implemented.

[0101] In the present invention, multiple virtual cameras of the original camera are obtained, pixel coordinate systems of the multiple virtual cameras are constructed, and multiple marker point coordinates of the target object are obtained, all the marker point coordinates are mapped into the pixel coordinate system to obtain corresponding virtual point coordinates, and the surface result of the target object is obtained according to all the virtual point coordinates; the offset results corresponding to all the virtual point coordinates are obtained, and the marker point set and the offset point set of the contact area are obtained according to all the offset results and the surface results, and the contact rotation axis of the target object is calculated according to the marker point set and the offset point set; the offset vector of the contact area is calculated according to the marker point set and the offset point set, and the displacement field of the offset point set is constructed according to the offset vector; a spring-damper network model is constructed, the displacement field is input into the spring-damper network model, the force information of each marker point is output, and the six-axis distributed force in the contact area is calculated according to each force information and the contact rotation axis; the objective function is defined according to the total torque, the spring-damper network model is updated by using the objective function, and the updated spring-damper network model is used to represent the force and torque of the target object. The present invention constructs a virtual binocular tactile platform based on dual-mirror reflection, reconstructs the three-dimensional geometric deformation of the contact surface, and adopts a spring-damper network model to estimate the six-axis force of the fingertip to improve the range, accuracy and computational efficiency of the model method in force distribution reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0102] Figure 1 It is a flow chart of a preferred embodiment of the six-axis distributed force tactile perception method based on binocular vision and tactile perception of the present invention;

[0103] Figure 2 It is a flowchart of the overall calculation of six-axis distributed force of a preferred embodiment of the six-axis distributed force tactile perception method based on binocular tactile perception of the present invention;

[0104] Figure 3 It is a schematic diagram of visual-tactile sensing of dual-mirror reflection in a preferred embodiment of the six-axis distributed force tactile sensing method based on binocular visual-tactile of the present invention;

[0105] Figure 4 Schematic diagram of a virtual binocular tactile camera model of a preferred embodiment of a six-axis distributed force tactile perception method based on binocular tactile perception of the present invention;

[0106] Figure 5 It is a schematic diagram of the unit connection between the force of a single marked point on the contact surface and the spring-damping model in a preferred embodiment of the six-axis distributed force tactile perception method based on binocular tactile perception of the present invention;

[0107] Figure 6It is a structural diagram of a preferred embodiment of a six-axis distributed force tactile perception system based on binocular vision and tactile perception of the present invention;

[0108] Figure 7 Schematic diagram of the operating environment of a preferred embodiment of the terminal of the present invention. DETAILED DESCRIPTION

[0109] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0110] The six-axis distributed force tactile perception method based on binocular vision and touch described in the preferred embodiment of the present invention is as follows: Figure 1 As shown, the six-axis distributed force tactile perception method based on binocular tactile perception includes the following steps:

[0111] First, if Figure 2 As shown, the present invention is based on a binocular tactile platform with dual mirror reflection, constructs virtual binocular tactile, reconstructs the three-dimensional geometric deformation of the contact surface, and estimates the six-axis force of the fingertip after parameter estimation and optimization using a spring-damper network model to improve the range, accuracy and computational efficiency of the model method in force distribution reconstruction.

[0112] Step S10, obtain multiple virtual cameras of the original camera, construct pixel coordinate systems of the multiple virtual cameras, and obtain multiple marker point coordinates of the target object, map all the marker point coordinates to the pixel coordinate system, obtain corresponding virtual point coordinates, and obtain the surface result of the target object according to all the virtual point coordinates.

[0113] Among them, Figure 3As shown, the visual tactile sensing component 1 mainly includes a camera 8 for capturing the image of the target object in the imaging area; a light source module 9 for providing lighting conditions; a fixed bracket 5 to ensure that the relative positions of the components remain unchanged during the experiment; a silicone-based elastomer 4 for supporting the spherical marker array 3; an outer silicone-based protective layer 2; and two plane mirrors 6 and 7 placed at a certain angle to the camera. The placement of the mirrors is designed to achieve multi-perspective observation based on a single camera, simulating multiple virtual cameras through mirror reflection; the camera faces two mirrors, and the two mirrors reflect the image of the silicone-based elastomer back to the camera. Through the reflection effect of the mirror, the original single camera's perspective is expanded into two virtual camera perspectives, corresponding to the two reflection areas of the mirror respectively. The camera and the mirror are on the same side, and both fields of view need to be reflected, so as to unify the focal length and focus clearly. When the silicone-based elastomer contacts different objects and causes the marker point to shift, the camera obtains images of two different perspectives through the mirror, forming a virtual binocular visual tactile sensation, and utilizes the stable material contact characteristics between the spherical marker point and the transparent elastic layer to improve the stability of force field perception.

[0114] Specifically, Figure 4 As shown, multiple fixed points of the mirrors are obtained, and through each of the fixed points, multiple virtual cameras of the original camera are obtained, pixel coordinate systems of the multiple virtual cameras are constructed, and the plane equations of the corresponding mirrors in the pixel coordinate system are obtained:

[0115] q1=(x q1 ,y q1 , z q1 ), q2=(x q2 ,y q2 , z q2 );

[0116] A1x+B1y+C1z+D1=0, A2x+B2y+C2z+D2=0;

[0117] n1={x m1 ,y m1 , z m1}, n2={x m2 ,y m2 , z m2};

[0118] Among them, q1 and q2 represent fixed points, x q1 ,y q1 and z q1 They represent the horizontal, vertical and vertical coordinates of the fixed point q1, respectively. q2 ,y q2 and z q2represents the horizontal coordinate, vertical coordinate and vertical coordinate of the fixed point q2 respectively, n1 represents the first normal vector of the first mirror corresponding to the fixed point q1, n2 represents the second normal vector of the second mirror corresponding to the fixed point q2, x m1 ,y m1 and z m1 represents the horizontal coordinate, vertical coordinate and vertical coordinate of the first normal vector, x m2 ,y m2 and z m2 Represents the abscissa, ordinate and ordinate of the second normal vector;

[0119] D1=-(A1x s1 +B1y q1 +C1z q1 ), A1=x m1 , B1=y m1 , C1=z m1 ;

[0120] D2=-(A2x q2 +B2y q2 +C2z q2 ), A2=x m2 , B2=y m2 , C2=z m2 ;

[0121] Wherein, D1 represents the constant term in the plane equation of the first mirror surface, D2 represents the constant term in the plane equation of the second mirror surface, A1x+B1y+C1z+D1 and A2x+B2y+C2z+D2 represent the plane equation of the first mirror surface and the plane equation of the second mirror surface respectively;

[0122] By mirror reflection, the virtual matrices of the multiple virtual cameras in the pixel coordinate system are calculated:

[0123]

[0124] in, and represent the horizontal coordinate basis vector, the vertical coordinate basis vector and the vertical coordinate basis vector of the first virtual camera in the first virtual camera coordinate system respectively, and They respectively represent the coordinate representation of the abscissa basis vector of the first virtual camera, and They respectively represent the coordinate representation of the ordinate basis vector of the first virtual camera, and They respectively represent the coordinate representation of the vertical coordinate basis vector of the first virtual camera, and represent the horizontal coordinate basis vector, the vertical coordinate basis vector and the vertical coordinate basis vector of the first virtual camera in the second virtual camera coordinate system respectively, and They respectively represent the coordinate representation of the horizontal coordinate basis vector of the second virtual camera, and They respectively represent the coordinate representation of the ordinate basis vector of the second virtual camera, and Respectively represent the coordinate representation of the vertical coordinate basis vector of the second virtual camera;

[0125]

[0126] Wherein, x1, y1 and z1 represent the abscissa, ordinate and ordinate of the origin of the first virtual coordinate system, respectively; x2, y2 and z2 represent the abscissa, ordinate and ordinate of the origin of the second virtual coordinate system, respectively; x0, y0 and z0 represent the abscissa, ordinate and ordinate of the origin v0 of the camera coordinate system, respectively;

[0127]

[0128] in, represents the virtual matrix of the first virtual camera, and Respectively represent the rotation matrix and translation matrix between the first virtual coordinate system and the pixel coordinate system, 0 1×3 represents a 1×3 matrix with all elements equal to 0, Matrix representation of the origin of the camera coordinate system;

[0129]

[0130] in, represents the virtual matrix of the second virtual camera, and Respectively represent the rotation matrix and translation matrix between the second virtual coordinate system and the pixel coordinate system;

[0131] Obtain the coordinates of multiple marking points of the target object in the world coordinate system, and map all the marking point coordinates to the pixel coordinate system to obtain the corresponding virtual point coordinates:

[0132]

[0133] A·x=B;

[0134] Among them, w0 and a0 represent the origins of the pixel coordinate systems of the two virtual cameras, respectively, and f x and f y Respectively represent the focal length of the original camera in the horizontal and vertical directions, and Represents the coordinates of the marker points in the world coordinate system, P 1 and P 2 Both represent the virtual coordinates of the marked points at the first and second moments, P (t) represents the virtual coordinates of the marked point at the tth moment, P i (t) represents the virtual coordinates of the i-th marker point at the t-th time, A and B both represent the transformation matrix for transforming the marker point coordinates, T represents transpose, and x represents the virtual point coordinates;

[0135] The coordinates of all virtual points are represented by surface fitting to obtain the surface result of the target object. Among them, the calculation formulas of x1, y1 and z1 (i.e. the calculation formula of the symmetrical points about the plane) can be obtained: and The calculation process:

[0136]

[0137]

[0138] Furthermore, the virtual coordinates of the marking point at the first moment and the second moment can be expressed by the following formula:

[0139]

[0140] Furthermore, a single camera image after multiple mirror reflections is obtained, and the areas of the first mirror and the second mirror in the single camera image are screened out; the areas of the first mirror and the second mirror in the single camera image are segmented to obtain two groups of images; and based on the two groups of images, the internal parameters of the original camera, the rotation matrix and the translation matrix are calibrated using a stereo camera.

[0141] Among them, binocular calibration (also known as stereo calibration) is a key step in the stereo vision system, which involves determining the geometric relationship between the two cameras and calibrating them in a chessboard manner. Considering the problem of limited common field of view caused by binocular vision and mirror reflection, an asymmetric calibration plate is selected to improve the accuracy and applicability of calibration. It is mainly divided into the following steps: 1. Collect a set of single camera images after mirror reflection; 2. Segment the image to obtain two sets of virtual camera images of the left and right mirrors. The segmentation method is mainly based on the area occupied by the mirror in the original camera. The areas occupied by mirror 1 and mirror 2 are retained respectively, and all other values ​​are blank, so as to obtain two sets of images; 3. Obtain the internal parameters of the original camera (intrinsic parameter matrix, distortion parameters) and the relative posture parameters of the two mirrors (rotation matrix and translation vector) through stereo camera calibration. After binocular calibration, the three-dimensional coordinates of the marker points in the world coordinate system can be obtained according to the pixel coordinates of the marker points in the two virtual cameras in the mirror to realize the conversion of coordinates.

[0142] Step S20, obtaining the offset results corresponding to all the virtual point coordinates, obtaining a marking point set and an offset point set of the contact area according to all the offset results and the surface results, and calculating the contact rotation axis of the target object according to the marking point set and the offset point set.

[0143] Among them, after the coordinate transformation is realized, the three-dimensional coordinates of each discrete mark point on the contact surface can be obtained. After the object surface contacts, the contact surface will deform. The discrete point offset vector of the entire surface can be obtained through binocular vision and touch. Through surface fitting, the representation of the three-dimensional geometric surface can be obtained, thereby characterizing the geometric space of the contact surface.

[0144] Specifically, the offset results corresponding to all the virtual point coordinates are obtained, wherein the offset result indicates the offset generated after the target object is touched; the change of the surface height is obtained through the offset result and the surface result, and the contact area is obtained according to the change, and the mark point set and the offset point set of the contact area are constructed:

[0145]

[0146] in, A set of marker points representing the contact area, Indicates the contact area N c Marking points, represents the set of offset points in the contact area at time t, Indicates the contact area N c offset points;

[0147] The centroids of the marking point set and the offset point set are calculated respectively, and the marking point set and the offset point set are removed from the centroid to obtain updated marking coordinates and updated offset coordinates:

[0148]

[0149] in, represents the centroid of the set of marked points, represents the centroid of the offset point set;

[0150]

[0151] in, Update marker coordinates, Indicates updating the offset coordinates; this process can effectively eliminate the translation effect of the overall marker points, so that the center of mass of the point set is located at the origin, thereby improving the accuracy of the characterization results.

[0152] Using the updated marker coordinates and the updated offset coordinates, a covariance matrix H is constructed:

[0153]

[0154] or

[0155] Furthermore, when calculating the rotation axis, it is also necessary to calculate the eigenvalues ​​and eigenvectors of the rotation matrix to determine whether there is a rotation axis; when there is an eigenvalue λ close to 1, such that |λ-1|<ε, ε is a small tolerance, then it is considered that there is a rotation axis, otherwise it is considered that there is no rotation axis; when there is a rotation axis, the direction of the rotation axis is obtained through the antisymmetric matrix.

[0156] After performing singular value decomposition on the covariance matrix, the optimal rotation matrix and the antisymmetric matrix of the optimal rotation matrix are calculated, and the direction of the contact rotation axis is extracted according to the antisymmetric matrix:

[0157] H=U∑V T ;

[0158] R=VU T ;

[0159]

[0160] Among them, U represents the basis vector of the offset point set, ∑ represents a 3×3 diagonal matrix, the diagonal elements are singular values, V represents a 3×3 orthogonal matrix, T represents transpose, R represents the optimal rotation matrix, and R antisym represents the antisymmetric matrix of the optimal rotation matrix, Indicates the direction of contact rotation axis, R antisym (3, 2), Rantisym (1, 3) and R antisym (2, 1) both represent the matrix index of the contact rotation axis.

[0161] In the contact process of the object surface, in addition to the translational motion along the straight line direction, there will also be torsional motion around a certain straight line direction. Therefore, when torsional motion occurs, it is also necessary to calculate the rotation axis during the torsional process to improve the accuracy and authenticity of the characterization of the object contact surface.

[0162] Step S30: Calculate the offset vector of the contact area according to the marking point set and the offset point set, and construct the displacement field of the offset point set according to the offset vector.

[0163] Specifically, the offset vector of the contact area is calculated according to the set of marking points and the set of offset points:

[0164]

[0165] in, represents the offset vector of the i-th marker point in the contact area at time t, and They represent the horizontal, vertical and vertical coordinates of the i-th mark point in the contact area at the t-th moment, respectively. and They represent the horizontal, vertical and vertical coordinates of the i-th marked point in the contact area before contact, and They represent the horizontal axis offset vector, the vertical axis offset vector and the vertical axis offset vector of the i-th marking point in the contact area at the t-th moment respectively;

[0166] Construct the displacement field of the offset point set through the offset vector:

[0167]

[0168] Where D represents the displacement field and T represents the transpose.

[0169] The displacement field is composed of the offset vectors of all marker points, which are composed of the displacement components of all markers in the horizontal, vertical and vertical directions.

[0170] Step S40, constructing a spring-damper network model, inputting the displacement field into the spring-damper network model, outputting force information of each marking point, and calculating the six-axis distributed force in the contact area according to each force information and the contact rotation axis.

[0171] Among them, by constructing a spring-damper model, the mapping relationship from displacement to force can be characterized; Figure 5As shown, a spring-damper model is constructed based on the K-neighbor method. Each marker point is simulated with multiple surrounding marker points using a spring connection, and each marker point is simulated with a spring-damper parallel connection model with a deep elastic body.

[0172] Specifically, a reconstruction model is constructed, the displacement field is input into the reconstruction model, and the force information of each marked point is output:

[0173] F R =g R (D);

[0174] F R =(f 1，x , f 1，y , f 1，z , …, f N，x , f N，y , f N，z ) T ;

[0175] Among them, F R represents the reconstruction force, f i，x Indicates the force on the i-th mark point in the horizontal direction, g R () indicates the reconstructed model;

[0176] According to the reconstructed force, each of the marking points is connected to a preset number of surrounding marking points through a spring-damping simulation to obtain the force information of each of the marking points:

[0177]

[0178] Among them, F i represents the force on the i-th marked point, F i，j represents the spring connection between the i-th marked point and its j-th neighboring point, k i，j represents the elastic coefficient between the i-th marker point and the j-th neighbor point, Λ(i) represents the set of marker points in the area of ​​the i-th marker point, and F i，b Indicates that the i-th marked point is connected in parallel with the spring-damper on the bottom surface, Represents the offset vector between the i-th marker point in the contact area at the t-th time and the j-th neighboring point around it, represents the offset vector between the i-th marker point in the contact area at the t-th moment and the origin of the pixel coordinate system;

[0179]

[0180] Among them, k i，b represents the elastic coefficient of the spring-damper between the ith mark point and the bottom surface, c i，b represents the damping of the spring-damper between the i-th marked point and the bottom surface;

[0181]

[0182] Among them, F total represents the six-axis force, K represents the stiffness matrix composed of all elastic coefficients in the contact area, and C represents the damping matrix composed of all damping in the contact area;

[0183] According to each of the force information and the contact rotation axis, the rotation axis torque of all the marked points in the contact area is calculated, and according to all the rotation axis torques, the six-axis distributed force in the contact area is calculated:

[0184] M i =(M i，x , M i，y , M i，z );

[0185] M i =d i，⊥ ×F i ;

[0186]

[0187] Among them, M i represents the rotation axis torque of the i-th marker point, M i，x 、M i，y and M i，z Respectively represent the components of the rotation axis torque of the i-th mark point on the horizontal axis, vertical axis and vertical axis, d i，⊥ represents the component perpendicular to the rotation axis, d i Indicates the distance from the i-th marker point to the center of the region, represents the set of offset points in the contact area at time t, represents the damping of the i-th marker at the t-th time, represents the unit vector about the contact rotation axis, Indicates the direction of contact rotation axis, N c Indicates the number of markers in the contact area, M total Represents the total moment in the contact area.

[0188] Among them, combining the three-dimensional geometric deformation of the contact surface and calculating the three-axis force and three-axis torque field based on the spring-damper model method can accurately calculate the geometric representation of the contact surface based on the force and torque calibration method of regular shapes, thereby improving the scope, accuracy and computational efficiency of the model method in force distribution reconstruction.

[0189] Step S50, defining an objective function according to the total torque, using the objective function to update the spring-damper network model, and using the updated spring-damper network model to represent the force and torque of the target object to obtain the six-axis distributed force of the target object.

[0190] Among them, through the above-mentioned geometric representation of the contact surface and the force and torque representation of the spring-damper model, the parameters of the model can be estimated through the following force calibration experiment, thereby improving the accuracy of the model calculation.

[0191] Specifically, the six-axis force of the target area is calibrated by a six-axis force sensor, and the objective function is defined according to the six-axis force:

[0192]

[0193] in, represents the objective function, and represents the total force and total moment of the target area measured by the six-axis force sensor, and represents the force and moment of the contact area estimated by the spring-damper model, and λ represents the weight parameter for balancing the force error and moment error;

[0194] Initialize multiple parameters of the spring-damper network model, and calculate the partial derivatives of the objective function with respect to the multiple initialization parameters multiple times to obtain a gradient vector:

[0195]

[0196] in, represents the gradient vector;

[0197] Using the gradient vector, iteratively update the multiple parameters until the change of the objective function is less than the first preset threshold or the gradient The norm of is less than the second preset threshold:

[0198]

[0199] in, and denote the elastic coefficients between the i-th marked point and the j-th neighboring point in the (n+1)th iteration and the nth iteration, respectively. and represents the elastic coefficient of the spring-damper between the i-th marking point and the bottom surface in the (n+1)th iteration and the nth iteration, and represents the damping of the spring-damper between the i-th marking point and the bottom surface in the (n+1)-th iteration and the n-th iteration, and η represents the learning rate of the spring-damper model; using the updated parameters, the spring-damper model is updated, and the updated spring-damper network model is used to represent the force and torque of the target object to obtain the six-axis distributed force of the target object in the contact area.

[0200] Among them, a standard six-axis force sensor is used, and a rigid object of regular shape is installed at the end to contact the visual tactile surface according to different contact modes. The regular shapes here include spheres, cylinders, etc., thereby obtaining the contact force (i.e. six-axis distributed force) under a fixed contact shape. The six-axis distributed force obtained at this time includes forces and moments in three directions.

[0201] Furthermore, the gradient descent method is used to estimate the model parameters. Each parameter is updated according to the learning rate along the negative gradient direction of the objective function. In each iteration, the partial derivative of the objective function with respect to each parameter is calculated to provide a direction for the gradient descent, thereby achieving parameter adjustment. The iteration is stopped until the objective function is less than the set threshold, or the parameter update step is very small and the extracted norm is less than the set threshold. The estimation of the model parameters of force and torque is completed.

[0202] The present invention constructs a virtual binocular tactile platform based on dual-mirror reflection, reconstructs the three-dimensional geometric deformation of the contact surface, and adopts a spring-damper network model to estimate the six-axis force of the fingertip to improve the range, accuracy and computational efficiency of the model method in force distribution reconstruction.

[0203] Furthermore, if Figure 6 As shown, based on the above-mentioned six-axis distributed force tactile perception method based on binocular vision and touch, the present invention also provides a six-axis distributed force tactile perception system based on binocular vision and touch, wherein the six-axis distributed force tactile perception system based on binocular vision and touch includes:

[0204] The surface representation module 51 is used to obtain multiple virtual cameras of the original camera, construct pixel coordinate systems of the multiple virtual cameras, and obtain multiple marker point coordinates of the target object, map all the marker point coordinates to the pixel coordinate system, obtain corresponding virtual point coordinates, and obtain the surface result of the target object according to all the virtual point coordinates;

[0205] A rotation axis calculation module 52 is used to obtain the offset results corresponding to all the virtual point coordinates, obtain a set of marked points and a set of offset points of the contact area according to all the offset results and the surface results, and calculate the contact rotation axis of the target object according to the set of marked points and the set of offset points;

[0206] A displacement field construction module 53, used to calculate the offset vector of the contact area according to the marking point set and the offset point set, and to construct the displacement field of the offset point set according to the offset vector;

[0207] A torque calculation module 54 is used to construct a spring-damper network model, input the displacement field into the spring-damper network model, output the force information of each marking point, and calculate the six-axis distributed force in the contact area according to each force information and the contact rotation axis;

[0208] The model updating module 55 is used to define an objective function according to the total torque, update the spring-damper network model using the objective function, and use the updated spring-damper network model to represent the force and torque of the target object to obtain the six-axis distributed force of the target object.

[0209] Furthermore, if Figure 7 As shown, based on the above-mentioned six-axis distributed force tactile perception method and system based on binocular tactile perception, the present invention also provides a terminal accordingly, and the terminal includes a processor 10, a memory 20 and a display 30. Figure 7 Only some components of the terminal are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0210] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal. In other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the terminal. Further, the memory 20 may also include both an internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software and various types of data installed on the terminal, such as the program code of the installation terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, a six-axis distributed force tactile perception program 40 based on binocular tactile perception is stored on the memory 20, and the six-axis distributed force tactile perception program 40 based on binocular tactile perception can be executed by the processor 10, thereby realizing the six-axis distributed force tactile perception method based on binocular tactile perception in the present application.

[0211] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run the program code or process data stored in the memory 20, such as executing the six-axis distributed force tactile perception method based on binocular vision and touch.

[0212] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 30 is used to display information on the terminal and to display a visual user interface. The components of the terminal communicate with each other via a system bus.

[0213] In one embodiment, the above steps are implemented when the processor 10 executes the six-axis distributed force tactile perception program 40 based on binocular tactile perception in the memory 20 .

[0214] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a six-axis distributed force tactile perception program based on binocular vision and tactile perception, and when the six-axis distributed force tactile perception program based on binocular vision and tactile perception is executed by a processor, the steps of the six-axis distributed force tactile perception method based on binocular vision and tactile perception as described above are implemented.

[0215] In summary, the present invention provides a six-axis distributed force tactile perception method based on binocular vision and tactile perception and related equipment, the method comprising: obtaining multiple virtual cameras of the original camera, constructing a pixel coordinate system of the multiple virtual cameras, and obtaining multiple marker point coordinates of the target object, mapping all the marker point coordinates to the pixel coordinate system, obtaining corresponding virtual point coordinates, and obtaining a surface result of the target object according to all the virtual point coordinates; obtaining the offset results corresponding to all the virtual point coordinates, obtaining a marker point set and an offset point set of the contact area according to all the offset results and the surface results, and calculating the target according to the marker point set and the offset point set. The contact rotation axis of the object; according to the set of marking points and the set of offset points, the offset vector of the contact area is calculated, and the displacement field of the offset point set is constructed according to the offset vector; a spring-damper network model is constructed, the displacement field is input into the spring-damper network model, the force information of each marking point is output, and the six-axis distributed force in the contact area is calculated according to each of the force information and the contact rotation axis; the objective function is defined according to the total torque, the spring-damper network model is updated using the objective function, and the updated spring-damper network model is used to represent the force and torque of the target object to obtain the six-axis distributed force of the target object. The present invention is based on a binocular tactile platform with dual-mirror reflection, constructs a virtual binocular tactile, reconstructs the three-dimensional geometric deformation of the contact surface, and uses a spring-damper network model to estimate the six-axis force of the fingertip to improve the range, accuracy and computational efficiency of the model method in force distribution reconstruction.

[0216] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or terminal including the element.

[0217] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the program can be stored in a computer-readable storage medium that can be read by a computer, and the program can include the processes of the above-mentioned method embodiments when executed. The computer-readable storage medium can be a memory, a disk, an optical disk, etc.

[0218] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A six-axis distributed force tactile perception method based on binocular vision and touch, characterized in that: The six-axis distributed force tactile perception method based on binocular tactile perception includes: Acquire multiple virtual cameras of the original camera, construct pixel coordinate systems of the multiple virtual cameras, and acquire multiple marker point coordinates of the target object, map all the marker point coordinates to the pixel coordinate system, obtain corresponding virtual point coordinates, and obtain the surface result of the target object according to all the virtual point coordinates; Obtaining the offset results corresponding to all the virtual point coordinates, obtaining a set of marked points and a set of offset points of the contact area according to all the offset results and the surface results, and calculating the contact rotation axis of the target object according to the set of marked points and the set of offset points; Calculating an offset vector of the contact area according to the set of marking points and the set of offset points, and constructing a displacement field of the set of offset points according to the offset vector; Constructing a spring-damper network model, inputting the displacement field into the spring-damper network model, outputting force information of each marking point, and calculating the six-axis distributed force in the contact area according to each force information and the contact rotation axis; An objective function is defined according to the total moment, the spring-damper network model is updated using the objective function, and the updated spring-damper network model is used to represent the force and moment of the target object to obtain the six-axis distributed force of the target object.

2. The six-axis distributed force tactile perception method based on binocular vision and tactile perception according to claim 1 is characterized in that: The step of acquiring multiple virtual cameras of the original camera, constructing pixel coordinate systems of the multiple virtual cameras, acquiring multiple marker point coordinates of the target object, mapping all the marker point coordinates to the pixel coordinate system, obtaining corresponding virtual point coordinates, and obtaining the surface result of the target object according to all the virtual point coordinates, specifically includes: Obtain multiple fixed points of the mirror surface, and obtain multiple virtual cameras of the original camera through each of the fixed points, construct pixel coordinate systems of the multiple virtual cameras, and obtain the plane equation of the corresponding mirror surface in the pixel coordinate system: q1=(x q1 ,y q1 ,z q1 ),q2=(x q2 ,y q2 ,z q2 ); A1x+B1y+C1z+D1=0, A2x+B2y+C2z+D2=0; n1={x m1 ,y m1 ,z m1 },n2={x m2 ,y m2 ,z m2 }; Among them, q1 and q2 represent fixed points, x q1 ,y q1 and z q1 They represent the horizontal, vertical and vertical coordinates of the fixed point q1, respectively. q2 ,y q2 and z q2 represents the horizontal coordinate, vertical coordinate and vertical coordinate of the fixed point q2 respectively, n1 represents the first normal vector of the first mirror corresponding to the fixed point q1, n2 represents the second normal vector of the second mirror corresponding to the fixed point q2, x m1 ,y m1 and z m1 represents the horizontal coordinate, vertical coordinate and vertical coordinate of the first normal vector, x m2 ,y m2 and z m2 Represents the abscissa, ordinate and ordinate of the second normal vector; D1=-(A1x q1 +B1y q1 +C1z q1 ),A1=x m1 ,B1=y m1 ,C1=z m1 ; <h2 style=";text-align:left;direction:ltr">D2=-(A2x<h2 style=";text-align:left;direction:ltr"> q2 <h2 style=";text-align:left;direction:ltr"> +B2y<h2 style=";text-align:left;direction:ltr"> q2 <h2 style=";text-align:left;direction:ltr"> +C2z<h2 style=";text-align:left;direction:ltr"> q2 <h2 style=";text-align:left;direction:ltr"> A2=x<h2 style=";text-align:left;direction:ltr"> m2 <h2 style=";text-align:left;direction:ltr"> B2=y<h2 style=";text-align:left;direction:ltr"> m2 <h2 style=";text-align:left;direction:ltr"> C2 = z<h2 style=";text-align:left;direction:ltr"> m2 <h2 style=";text-align:left;direction:ltr"> ; Wherein, D1 represents the constant term in the plane equation of the first mirror surface, D2 represents the constant term in the plane equation of the second mirror surface, A1x+B1y+C1z+D1=0 and A2x+B2y+C2z+D2=0 represent the plane equation of the first mirror surface and the plane equation of the second mirror surface respectively; By mirror reflection, the virtual matrices of the multiple virtual cameras in the pixel coordinate system are calculated: in, and represent the horizontal coordinate basis vector, the vertical coordinate basis vector and the vertical coordinate basis vector of the first virtual camera in the first virtual camera coordinate system respectively, and They respectively represent the coordinate representation of the abscissa basis vector of the first virtual camera, and They respectively represent the coordinate representation of the ordinate basis vector of the first virtual camera, and They respectively represent the coordinate representation of the vertical coordinate basis vector of the first virtual camera, and represent the horizontal coordinate basis vector, the vertical coordinate basis vector and the vertical coordinate basis vector of the first virtual camera in the second virtual camera coordinate system respectively, and They respectively represent the coordinate representation of the horizontal coordinate basis vector of the second virtual camera, and They respectively represent the coordinate representation of the ordinate basis vector of the second virtual camera, and Respectively represent the coordinate representation of the vertical coordinate basis vector of the second virtual camera; Wherein, x1, y1 and z1 represent the abscissa, ordinate and ordinate of the origin of the first virtual coordinate system, respectively; x2, y2 and z2 represent the abscissa, ordinate and ordinate of the origin of the second virtual coordinate system, respectively; x0, y0 and z0 represent the abscissa, ordinate and ordinate of the origin v0 of the camera coordinate system, respectively; in, represents the virtual matrix of the first virtual camera, and Respectively represent the rotation matrix and translation matrix between the first virtual coordinate system and the original camera coordinate system, 0 1×3 represents a 1×3 matrix with all elements equal to 0, Matrix representation of the origin of the camera coordinate system; in, represents the virtual matrix of the second virtual camera, and Respectively represent the rotation matrix and translation matrix between the second virtual coordinate system and the pixel coordinate system; Obtain the coordinates of multiple marking points of the target object in the world coordinate system, and map all the marking point coordinates to the pixel coordinate system to obtain the corresponding virtual point coordinates: A·x=B; Among them, w0 and a0 represent the pixel coordinates of the center of the original camera image, f x and f y Respectively represent the focal length of the original camera in the horizontal and vertical directions, and Represents the coordinates of the marker points in the world coordinate system, P 1 and P 2 Both represent the virtual coordinates of the marked points at the first and second moments, P (t) represents the virtual coordinates of the marked point at the tth moment, P i (t) represents the virtual coordinates of the i-th marker point at the t-th time, A and B both represent the transformation matrix for transforming the marker point coordinates, T represents transpose, and x represents the virtual point coordinates; All the virtual point coordinates are represented by surface fitting to obtain the surface result of the target object.

3. The six-axis distributed force tactile perception method based on binocular tactile perception according to claim 2 is characterized in that: The step of calculating virtual matrices of the plurality of virtual cameras in the pixel coordinate system through mirror reflection further comprises: Acquire a single camera image after multiple mirror reflections, and filter out regions of the first mirror and the second mirror in the single camera image; Segmenting the regions of the first mirror surface and the second mirror surface in the single camera image to obtain two groups of images; According to the two groups of images, the internal parameters of the original camera, the rotation matrix and the translation matrix are calibrated using a stereo camera.

4. The six-axis distributed force tactile perception method based on binocular vision and tactile perception according to claim 1, characterized in that: The step of obtaining the offset results corresponding to all the virtual point coordinates, obtaining a set of marked points and a set of offset points of the contact area according to all the offset results and the surface results, and calculating the contact rotation axis of the target object according to the set of marked points and the set of offset points, specifically includes: Obtaining offset results corresponding to all the virtual point coordinates, wherein the offset results represent offsets generated after the target object is touched; The change of the surface height is obtained through the offset result and the surface result. According to the change, the contact area is obtained, and the marking point set and the offset point set of the contact area are constructed: in, A set of marker points representing the contact area, Indicates the contact area N c Marking points, represents the set of offset points in the contact area at time t, Indicates the contact area N c offset points; The centroids of the marking point set and the offset point set are calculated respectively, and the marking point set and the offset point set are removed from the centroid to obtain updated marking coordinates and updated offset coordinates: in, represents the centroid of the set of marked points, represents the centroid of the offset point set; in, Update marker coordinates, Indicates updating the offset coordinates; Using the updated marker coordinates and the updated offset coordinates, a covariance matrix H is constructed: or After performing singular value decomposition on the covariance matrix, the optimal rotation matrix and the antisymmetric matrix of the optimal rotation matrix are calculated, and the direction of the contact rotation axis is extracted according to the antisymmetric matrix: H=U∑V T ; R=VU T ; Among them, U represents the basis vector of the offset point set, ∑ represents a 3×3 diagonal matrix, the diagonal elements are singular values, V represents a 3×3 orthogonal matrix, T represents transpose, R represents the optimal rotation matrix, and R antisym represents the antisymmetric matrix of the optimal rotation matrix, Indicates the direction of contact rotation axis, R antisym (3,2), R antisym (1,3) and R antisym (2,1) both represent the matrix index of the contact rotation axis.

5. The six-axis distributed force tactile perception method based on binocular tactile perception according to claim 1, characterized in that: The step of calculating the offset vector of the contact area according to the set of marking points and the set of offset points, and constructing the displacement field of the set of offset points according to the offset vector, specifically includes: According to the set of marking points and the set of offset points, the offset vector of the contact area is calculated: in, represents the offset vector of the i-th marker point in the contact area at time t, and They represent the horizontal, vertical and vertical coordinates of the i-th mark point in the contact area at the t-th moment, respectively. and They represent the horizontal coordinate, vertical coordinate and vertical coordinate of the i-th marking point in the contact area before it is released. and They represent the horizontal axis offset vector, the vertical axis offset vector and the vertical axis offset vector of the i-th marking point in the contact area at the t-th moment respectively; Construct the displacement field of the offset point set through the offset vector: Where D represents the displacement field and T represents the transpose.

6. The six-axis distributed force tactile perception method based on binocular vision and tactile perception according to claim 5, characterized in that: The step of constructing a spring-damper network model, inputting the displacement field into the spring-damper network model, outputting force information of each marking point, and calculating the six-axis distributed force in the contact area according to each force information and the contact rotation axis, specifically includes: Construct a reconstruction model, input the displacement field into the reconstruction model, and output the force information of each marked point: F R =g R (D); F R =(f 1,x ,f 1,y ,f 1,z ,…,f N,x ,f N,y ,f N,z ) T ; Among them, F R represents the reconstruction force, f i,x Indicates the force on the i-th mark point in the horizontal direction, g R () indicates the reconstructed model; According to the reconstructed force, each of the marking points is connected to a preset number of surrounding marking points through a spring-damping simulation to obtain the force information of each of the marking points: Among them, F i represents the force on the i-th marked point, F i,j represents the spring connection between the i-th marked point and its j-th neighboring point, k i,j represents the elastic coefficient between the i-th marker point and the j-th neighbor point, Λ(i) represents the set of marker points in the area of ​​the i-th marker point, and F i,b Indicates that the i-th marked point is connected in parallel with the spring-damper on the bottom surface, Represents the offset vector between the i-th marker point in the contact area at time t and the j-th neighboring point around it, represents the offset vector between the i-th marker point in the contact area at the t-th moment and the origin of the pixel coordinate system; Among them, k i,b represents the elastic coefficient of the spring-damper between the ith mark point and the bottom surface, c i,b represents the damping of the spring-damper between the i-th marked point and the bottom surface; Among them, F total represents the six-axis force, K represents the stiffness matrix composed of all elastic coefficients in the contact area, and C represents the damping matrix composed of all damping in the contact area; According to each of the force information and the contact rotation axis, the rotation axis torque of all the marked points in the contact area is calculated, and according to all the rotation axis torques, the six-axis distributed force in the contact area is calculated: M i =(M i,x ,M i,y ,M i,z ); M i =d i,⊥ ×F i ; Among them, M i represents the rotation axis torque of the i-th marker point, M i,x 、M i,y and M i,z Respectively represent the components of the rotation axis torque of the i-th mark point on the horizontal axis, vertical axis and vertical axis, d i,⊥ represents the component perpendicular to the rotation axis, d i Indicates the distance from the i-th marker point to the center of the region, represents the set of offset points in the contact area at time t, represents the damping of the i-th marker at the t-th time, represents the unit vector about the contact rotation axis, Indicates the direction of contact rotation axis, N c Indicates the number of markers in the contact area, M total Represents the total moment in the contact area.

7. The six-axis distributed force tactile perception method based on binocular vision and tactile perception according to claim 6, characterized in that: The step of defining an objective function according to the total moment, updating the spring-damper network model using the objective function, and representing the force and moment of the target object using the updated spring-damper network model to obtain the six-axis distributed force of the target object specifically includes: The six-axis force of the target area is calibrated by a six-axis force sensor, and the objective function is defined according to the six-axis force: in, represents the objective function, and represents the total force and total moment of the target area measured by the six-axis force sensor, and represents the force and moment of the contact area estimated by the spring-damper model, and λ represents the weight parameter for balancing the force error and moment error; Initialize multiple parameters of the spring-damper network model, and calculate the partial derivatives of the objective function with respect to the multiple initialization parameters multiple times to obtain a gradient vector: in, represents the gradient vector; Using the gradient vector, iteratively update the multiple parameters until the change of the objective function is less than the first preset threshold or the gradient The norm of is less than the second preset threshold: in, and denote the elastic coefficients between the i-th marked point and the j-th neighboring point in the (n+1)th iteration and the nth iteration, respectively. and represents the elastic coefficient of the spring-damper between the i-th marking point and the bottom surface in the (n+1)th iteration and the nth iteration, and represents the damping of the spring-damper between the i-th marker point and the bottom surface in the (n+1)th iteration and the nth iteration, and η represents the learning rate of the spring-damper model; The spring-damper model is updated using the updated parameters, and the updated spring-damper network model is used to represent the force and moment of the target object to obtain the six-axis distributed force of the target object in the contact area.

8. A six-axis distributed force tactile perception system based on binocular vision and touch, characterized in that: The six-axis distributed force tactile perception system based on binocular vision and tactile perception includes: A surface representation module is used to obtain multiple virtual cameras of the original camera, construct pixel coordinate systems of the multiple virtual cameras, and obtain multiple marker point coordinates of the target object, map all the marker point coordinates to the pixel coordinate system, obtain corresponding virtual point coordinates, and obtain the surface result of the target object according to all the virtual point coordinates; A rotation axis calculation module is used to obtain the offset results corresponding to all the virtual point coordinates, obtain a set of marked points and a set of offset points of the contact area according to all the offset results and the surface results, and calculate the contact rotation axis of the target object according to the set of marked points and the set of offset points; A displacement field construction module, used to calculate the offset vector of the contact area according to the marking point set and the offset point set, and to construct the displacement field of the offset point set according to the offset vector; A torque calculation module, used for constructing a spring-damper network model, inputting the displacement field into the spring-damper network model, outputting force information of each marking point, and calculating the six-axis distributed force in the contact area according to each force information and the contact rotation axis; A model updating module is used to define an objective function according to the total torque, update the spring-damper network model using the objective function, and use the updated spring-damper network model to represent the force and torque of the target object to obtain the six-axis distributed force of the target object.

9. A terminal, characterized in that: The terminal includes: a memory, a processor, and a six-axis distributed force tactile perception program based on binocular vision and tactile perception stored in the memory and executable on the processor. When the six-axis distributed force tactile perception program based on binocular vision and tactile perception is executed by the processor, the steps of the six-axis distributed force tactile perception method based on binocular vision and tactile perception are implemented as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a six-axis distributed force tactile perception program based on binocular vision and tactile perception. When the six-axis distributed force tactile perception program based on binocular vision and tactile perception is executed by a processor, the steps of the six-axis distributed force tactile perception method based on binocular vision and tactile perception are implemented as described in any one of claims 1 to 7.

Images