Human-machine teaching control method and device for serial robot based on isomer mapping

Abstract

The invention discloses a human-machine teaching control method and device for a serial robot based on isomer mapping, and the method comprises: obtaining teaching information of a teaching action of a teacher; obtaining a key frame sequence of the teaching information by improving a K-means algorithm; converting the three-dimensional coordinates of a human joint point corresponding to each key frame into the three-dimensional coordinates of a human reconstructed joint point by pre-constructed spatial mapping based on the key frame sequence of the teaching information; converting the three-dimensional coordinates of each human reconstructed joint point into the three-dimensional coordinates in the coordinate system of the serial robot by homogeneous coordinate transformation; using the three-dimensional coordinates in each serial robot coordinate system as the target point of the three-dimensional coordinate point of the machine reconstructed joint point of the serial robot; and controlling the serial robot to complete the teaching action by a parallel CCD control method based on each target point. The invention can effectively improve the universality and simplicity of the robot teaching motion, and provides a powerful means for further promoting the popularization and application of robots.

Description

Technical Field

[0001] The invention relates to a human-machine teaching control method and device for a serial robot based on isomer mapping, belonging to the technical field of human-machine teaching. Background Art

[0002] At present, the method of human-machine teaching mainly records the movement information of human limbs, captures the corresponding angles of joints, and the robot imitates the movements by tracking the corresponding joint angles. For example, teaching is achieved through RGB-D images and a teaching pendant device connected to the robot controller. The teaching operation requires a dedicated teaching pendant device to control the robot, and sends instructions to the robot controller to guide the robot to reach the target point at the set posture. For example, the publication number CN110238831A "Robot Teaching System and Method Based on RGB-D Images and Teaching Pendants" realizes that the teaching does not require a complex posture tracking system by combining RGB-D images with teaching pendants, and completes the teaching through augmented reality simulation, but the solution requires a robot with a similar structure to the instructor. Publication No. CN111002292B "Method for Teaching Humanoid Motion of Robotic Arm Based on Similarity Measurement" is a technical solution that obtains the elbow angle of the human arm after bending, and uses similarity measurement to determine the elbow definition that is most similar to the robot arm to achieve the imitation of the movement of the robot arm and the arm. This technical solution is limited to the imitation teaching of the movement between the robot arm and the human arm, and can only solve the teaching problem between the robot arm and the human arm. The advantages of these methods can partially realize human-machine teaching, but the robot needs to have a structure similar to that of the instructor, and it is limited to the teaching of fixed joints. This makes human-machine teaching not generalizable.

[0003] With the development of robot technology, more and more industries have put forward higher requirements for human-machine teaching, especially reducing the need for structural similarity between the instructor and the robot and improving the versatility of the robot. Specifically, using non-humanoid robots to imitate teaching movements has high practical value. Summary of the invention

[0004] The purpose of the present invention is to overcome the deficiencies in the prior art and to provide a human-machine teaching control method and device for a serial robot based on isomer mapping, which can effectively improve the versatility and simplicity of the robot's teaching motion and provide a powerful means for further promoting the popularization and application of robots.

[0005] To achieve the above object, the present invention is implemented by adopting the following technical solutions:

[0006] In a first aspect, the present invention provides a human-machine teaching control method for a serial robot based on isomer mapping, comprising:

[0007] Acquire teaching information of the teaching action of the instructor, wherein the teaching information includes frame data of each frame, and the frame data includes time and three-dimensional coordinates of human joints;

[0008] The key frame sequence of teaching information is obtained by improving the K-means algorithm;

[0009] The key frame sequence based on the teaching information converts the three-dimensional coordinates of the human joint points corresponding to each key frame into the three-dimensional coordinates of the human reconstructed joint points through a pre-built spatial mapping;

[0010] The three-dimensional coordinates of each human body reconstructed joint point are converted into the three-dimensional coordinates of the serial robot coordinate system through homogeneous coordinate transformation;

[0011] The three-dimensional coordinates of each serial robot coordinate system are used as the target points of the three-dimensional coordinates of the machine reconstruction joint points of the serial robot;

[0012] Based on each target point, the serial robot is controlled by a parallel CCD control method to complete the teaching action.

[0013] Optionally, the key frame sequence for obtaining teaching information by improving the K-means algorithm includes:

[0014] The cluster evaluation index elbow method is used to determine the K value;

[0015] Randomly and uniformly select K frames from the teaching information as cluster centroids;

[0016] Cluster each frame of the teaching information according to the cluster centroid to obtain K clusters and corresponding cluster centroids;

[0017] The Euclidean distance is calculated between each frame of the cluster and its cluster centroid according to the vector generated by the three-dimensional coordinates of the human joint points in the frame data, and the frame corresponding to the minimum Euclidean distance is selected as the key frame of the cluster;

[0018] The key frames of each cluster are sorted from front to back according to the time of the frame data to obtain the key frame sequence of the teaching information.

[0019] Optionally, the method of using the clustering evaluation index elbow method to determine the K value includes:

[0020] Let K = 2;

[0021] Randomly and uniformly select K frames from the teaching information as cluster centroids;

[0022] Cluster each frame of the teaching information according to the cluster centroid to obtain K clusters and corresponding cluster centroids;

[0023] The Euclidean distance between each frame of the cluster and its cluster centroid is calculated based on the vector generated by the three-dimensional coordinates of the human joint points of the frame data, and the Euclidean distances are summed;

[0024] Let K=K+1, and repeat the above steps to iterate. If the sum of the current number of iterations is suddenly smaller than the sum of the previous number of iterations, the K value corresponding to the current number of iterations is output as the result.

[0025] Optionally, clustering each frame of the teaching information according to the cluster centroid to obtain K clusters and corresponding cluster centroids includes:

[0026] The Euclidean distance between each frame of the teaching information and the vector generated by the cluster mass center according to the three-dimensional coordinates of the human joint points of the frame data is calculated;

[0027] Assign each frame of the teaching information to the cluster centroid with the smallest Euclidean distance to generate K clusters;

[0028] Update the cluster centroid o of the K clusters k :

[0029]

[0030] In the formula, o k is the kth cluster centroid, k = 1, 2, ..., K; N is the number of frames of teaching information, n = 1, 2, ..., N; t nk Whether the nth frame belongs to the cluster centroid o k , if so, then t nk =1, if not, then t nk =0; frame n A vector generated from the three-dimensional coordinates of the human joint points of the frame data of the nth frame;

[0031] Repeat the above steps until the cluster centroid of the current iteration is equal to the cluster centroid of the previous iteration or the difference is less than the preset threshold, and output the final K clusters and the corresponding cluster centroids.

[0032] Optionally, the construction of the spatial mapping includes:

[0033] Initialize the reconstructed joint point matrix, and reconstruct the degrees of freedom of the instructor and the serial robot based on the reconstructed joint point matrix to obtain the reconstructed joint point pairs;

[0034] Based on the reconstructed joint point pairs, the similarity test between the instructor and the serial robot is performed to obtain the similarity measure;

[0035] Adjust the reconstructed joint point matrix, and return the adjusted reconstructed joint point matrix to the initialized reconstructed joint point matrix for iteration;

[0036] The reconstructed joint point pair with the largest similarity measure is taken as the spatial mapping.

[0037] Optionally, the initialization reconstructs the joint point matrix for:

[0038]

[0039] In the formula, X is the moving body. If X=B, the moving body is the instructor. If X=I, the moving body is the serial robot. is the reconstructed joint point matrix of the i-th motion joint point of the moving body, Q X is the number of degrees of freedom of the moving body, S X is the number of degrees of freedom of the i-th joint of the moving body,

[0040] Each row of the reconstructed joint point matrix has only one element 1, and the rest are elements 0; and adjusting the reconstructed joint point matrix is ​​to adjust the position of element 1;

[0041] The acquisition of the reconstructed joint point pair for:

[0042]

[0043] In the formula, is the reconstructed joint point of the i-th joint point of the moving body at the t-th moment; S X (t) is the state vector S of the freedom degree of the moving body at the tth moment X (t):

[0044]

[0045] In the formula, is the state of the mth degree of freedom of the moving body at the tth moment, m=1,2,…,Q X , T is the transpose.

[0046] Optionally, the similarity test between the instructor and the serial robot based on the reconstructed joint point pair is performed to obtain a similarity measure including:

[0047] Performing a test action through a moving body and acquiring test information, wherein the test information includes frame data of each frame, and the frame data includes a time and three-dimensional coordinates of a moving joint point;

[0048] Obtain the key frame sequence of test information by improving the K-means algorithm;

[0049] Quantifying the three-dimensional coordinates of the motion joint points of each frame of the test information to obtain the global motion metric, the local motion metric and the joint angle;

[0050] Obtain the global motion metric, local motion metric and joint angle corresponding to each key frame based on the key frame sequence of the test information;

[0051] Based on the global motion metric, local motion metric and joint angle corresponding to each key frame, the corresponding global motion metric distance, local motion metric distance and joint angle metric distance are calculated:

[0052] G_dis k′ =||G_Motion B，k′ -G_Motion I，k′ ||

[0053] L_dis k′ =||L_Motion B，k′ -L_Motion I，k′ ||

[0054] α_dis k′ =|α B，k′ -α I，k′ ||

[0055] In the formula, G_dis k′ is the global motion metric distance of the k′th key frame, G_Motion B，k′ 、G_Motion I，k′ are the global motion metrics of the k′th key frame of the instructor and the serial robot respectively;

[0056] L_dis k′ is the local motion metric distance of the k′th key frame, LMotion B，k′ 、LMotion I，k′ are the local motion metrics of the k′th key frame of the instructor and the serial robot respectively;

[0057] α_dis k′ is the joint angle distance of the k′th key frame, α B，k′ , α I，k′ are the joint angles of the k′th key frame of the instructor and the serial robot respectively;

[0058] Compare the global motion metric distance, local motion metric distance, and joint angle metric distance corresponding to each keyframe with the preset global threshold, local threshold, and joint angle threshold, and obtain the corresponding similarity measure:

[0059]

[0060]

[0061]

[0062]

[0063] Where G%, L%, and α% are the preset similarity measures of the global motion metric, local motion metric, and joint angle metric, respectively; is the total similarity measure of the k′th key frame, The similarity measures of the global motion metric, local motion metric and joint angle metric are respectively used to measure the similarity of the k′th keyframe;

[0064] Obtain the total similarity measure sim of the test information based on the total similarity measure of each key frame total :

[0065]

[0066] Where K′ is the number of key frames of the test information, k′=1, 2, …, K′.

[0067] Optionally, the step of quantifying the three-dimensional coordinates of the motion joint points of the frame data of each frame of the test information to obtain the global motion metric, the local motion metric and the joint angle comprises:

[0068] Based on the frame data of each frame of the test information, the three-dimensional coordinates of the motion joint points corresponding to each frame are converted into the three-dimensional coordinates of the reconstructed joint points by reconstructing the joint point pairs;

[0069] Obtaining global motion metrics includes:

[0070] Use the Kabsch algorithm to quantize adjacent frames:

[0071]

[0072]

[0073]

[0074]

[0075] In the formula, Set α 、Set β Reconstruct the 3D coordinate set of joint points for the previous frame and the next frame, is the three-dimensional coordinate of the reconstructed joint point of the j-th reconstructed joint point in the moving body in the previous frame and the next frame, j = 1, 2, ..., l, l is the number of reconstructed joint points in the moving body, Cent α 、Cent β Set α 、Set β The centroid of

[0076] Calculate the covariance matrix E based on the centroid and the reconstructed three-dimensional coordinates of the joint points:

[0077]

[0078] The covariance matrix is ​​decomposed into three matrices M, N, O by singular value decomposition:

[0079] [M, N, O] = SVD(E)

[0080] R=OM T

[0081] T=Cent β -R×Cent a

[0082] Where R is the rotation matrix, T = [t x , t y , t z ] T is the translation matrix;

[0083]

[0084] (θ x ,θ y ,θ z )=Euler(R)

[0085] In the formula, TF is the α Transform to Set β The transformation matrix, Euler is the Euler function, (θ x ,θ y ,θ z ) is the rotation angle of the rotation matrix R around the coordinate axis;

[0086] The global motion metric of the jth reconstructed joint point in the n′th adjacent frame for:

[0087]

[0088] The global motion metric G_Motion of all reconstructed joints in the n′th adjacent frame n′ for:

[0089]

[0090] The global motion metric G_Motion of the test information is:

[0091] G_Motion=[G_Motion1, G_Motion2,…,G_Motion N′ ]

[0092] Where, N' is the number of test information frames, n'=1, 2, ..., N';

[0093] Obtaining local motion metrics includes:

[0094] The global motion metric of the j-th reconstructed joint point in the n′th adjacent frame Obtain the local motion metric of the jth reconstructed joint point in the n′th adjacent frame by homogeneous coordinate transformation for:

[0095]

[0096]

[0097] The global motion metric L_Motion of all reconstructed joints in the n′th adjacent frame n′ for:

[0098]

[0099] The local motion metric L_Motion of the test information is:

[0100] L_Motion=[L_Motion1, L_Motion2,…,L_Motion N′ ]

[0101] Getting joint angles includes:

[0102] The joint angle of the j-th reconstructed joint point according to the n′th adjacent frame is:

[0103] Reconstruct the joint angle α of all joint points based on the n′th adjacent frame n′ for:

[0104]

[0105] Where num is the number of joint angles;

[0106] The joint angle α of the test information is:

[0107] α=[α1,α2,…,α N′ ]

[0108] Optionally, controlling the serial robot to complete the teaching action by a parallel CCD control method based on each target point includes:

[0109] Construct the control variable φ of the serial robot according to the first target point:

[0110] φ={φ1,φ2,…,φ f}

[0111]

[0112] Where, e and f are the number of machine joints and machine reconstruction joints; φ f Reconstruct the control variables of the joint points for the fth machine;

[0113]

[0114] In the formula, θ g is the rotation angle of the g-th machine joint, d g is the translation distance of the g-th machine joint point;

[0115] Construct the control objective function θ based on the control variable group u :

[0116] θ u =ρ1||φ1|| 2 +ρ2||φ2|| 2 +…+ρ f ||φ f || 2

[0117] In the formula, ρ r Reconstruct the motion gain of the joint point of the rth machine, r = 1, 2, ..., f;

[0118] Adjust the control objective function θ u Let θ u Approaching zero or less than or equal to the threshold θ t , thus completing the first target point;

[0119] Repeat the above steps until each target point is completed to complete the teaching action.

[0120] In a second aspect, the present invention provides a human-machine teaching control device for a serial robot based on isomer mapping, comprising a processor and a storage medium;

[0121] The storage medium is used to store instructions;

[0122] The processor is used to operate according to the instructions to execute the steps according to the above method.

[0123] Compared with the prior art, the present invention has the following beneficial effects:

[0124] The present invention provides a human-machine teaching control method and device for a serial robot based on isomer mapping, which obtains the motion information of the instructor to extract key frames, reconstructs the three-dimensional coordinates of the joint points of the key frames through spatial mapping, and then converts the reconstructed three-dimensional coordinates into three-dimensional coordinates in the coordinate system of the serial robot. The serial robot is controlled in a CCD parallel manner with the three-dimensional coordinates as the target point to complete the teaching. The present invention can effectively improve the versatility and simplicity of the robot's teaching motion, and provides a powerful means for further promoting the popularization and application of robots. BRIEF DESCRIPTION OF THE DRAWINGS

[0125] Figure 1 It is a flow chart of a human-machine teaching control method of a serial robot based on isomer mapping provided in the first embodiment of the present invention;

[0126] Figure 2 It is a flow chart of a method for constructing a spatial mapping in a human-machine teaching control method for a serial robot provided in Embodiment 1 of the present invention;

[0127] Figure 3 Schematic diagram of spatial mapping of joint points between the left arm and the serial robot provided in the second embodiment of the present invention;

[0128] Figure 4 It is a schematic diagram of the three-dimensional coordinates of the reconstructed joints of the left arm provided in the second embodiment of the present invention being converted into the three-dimensional coordinates of the coordinate system of the serial robot through homogeneous transformation;

[0129] Figure 5 is a schematic diagram of using a parallel CCD control method to control a reconstructed joint point to reach a target point provided by the second embodiment of the present invention;

[0130] Figure 6 This is a schematic diagram of the arm-lifting motion of the left-arm teaching serial robot provided in the second embodiment of the present invention;

[0131] Figure 7 is a schematic diagram of human body joint reconstruction provided by Embodiment 3 of the present invention;

[0132] Figure 8 It is a schematic diagram of the left-right swinging action of the human body teaching serial robot provided in the third embodiment of the present invention. DETAILED DESCRIPTION

[0133] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0134] Embodiment 1:

[0135] like Figure 1As shown, the present invention provides a human-machine teaching control method for a serial robot based on isomer mapping, comprising the following steps:

[0136] 1. Obtain the teaching information of the instructor's teaching action, which includes frame data of each frame. The frame data includes time and three-dimensional coordinates of human joints.

[0137] 2. Obtain the key frame sequence of teaching information by improving the K-means algorithm; specifically including the following steps:

[0138] 2.1. Determine the K value using the clustering evaluation index elbow method; specifically, the following steps are included:

[0139] 2.1.1, let K = 2;

[0140] 2.1.2, randomly and uniformly select K frames from the teaching information as cluster centroids;

[0141] 2.1.3. Cluster each frame of the teaching information according to the cluster centroid to obtain K clusters and corresponding cluster centroids;

[0142] 2.1.4. Calculate the Euclidean distance between each frame of the cluster and its cluster centroid according to the vector generated by the three-dimensional coordinates of the human joint points of the frame data, and sum the Euclidean distances;

[0143] 2.1.5. Let K=K+1, and repeat the above steps to iterate. If the sum of the current number of iterations suddenly becomes smaller than the sum of the previous number of iterations, the K value corresponding to the current number of iterations is output as the result.

[0144] 2.2. Randomly and uniformly select K frames from the teaching information as cluster centroids;

[0145] 2.3. Cluster each frame of the teaching information according to the cluster centroid to obtain K clusters and corresponding cluster centroids;

[0146] 2.4. Calculate the Euclidean distance between each frame of the cluster and its cluster centroid according to the vector generated by the three-dimensional coordinates of the human joint points of the frame data, and select the frame with the smallest Euclidean distance as the key frame of the cluster;

[0147] 2.5. Sort the key frames of each cluster from front to back according to the time of the frame data to obtain the key frame sequence of the teaching information.

[0148] In steps 2.1.3 and 2.3, each frame of the teaching information is clustered according to the cluster centroid to obtain K clusters and the corresponding cluster centroids, and the method is:

[0149] (1) Calculate the Euclidean distance between each frame of the teaching information and the vector generated by the three-dimensional coordinates of the human joint points of the cluster mass center according to the frame data;

[0150] (2) Assign each frame of the teaching information to the cluster centroid with the smallest Euclidean distance to generate K clusters;

[0151] (3) Update the cluster centroid o of the K clusters k :

[0152]

[0153] In the formula, o k is the kth cluster centroid, k = 1, 2, ..., K; N is the number of frames of teaching information, n = 1, 2, ..., N; t nk Whether the nth frame belongs to the cluster centroid o k , if so, then t nk =1, if not, then t nk =0; frame n A vector generated from the three-dimensional coordinates of the human joint points of the frame data of the nth frame;

[0154] (4) Repeat the above steps until the cluster centroid of the current iteration is equal to the cluster centroid of the previous iteration or the difference is less than the preset threshold, and output the final K clusters and the corresponding cluster centroids.

[0155] 3. Based on the key frame sequence of the teaching information, the three-dimensional coordinates of the human joint points corresponding to each key frame are converted into the three-dimensional coordinates of the human reconstructed joint points through the pre-built spatial mapping;

[0156] like Figure 2 As shown, the construction of the spatial map includes:

[0157] 3.1. Initialize the reconstructed joint point matrix, and reconstruct each degree of freedom of the instructor and the serial robot based on the reconstructed joint point matrix to obtain the reconstructed joint point pairs;

[0158] 3.1.1. Initialize and reconstruct the joint point matrix for:

[0159]

[0160] In the formula, X is the moving body. If X=B, the moving body is the instructor. If X=I, the moving body is the serial robot. is the reconstructed joint point matrix of the i-th motion joint point of the moving body, Q X is the number of degrees of freedom of the moving body, S X is the number of degrees of freedom of the i-th joint of the moving body,

[0161] 3.1.2. Each row of the reconstructed joint point matrix has only one element 1 and the rest are elements 0; adjust the reconstructed joint point matrix to adjust the position of element 1;

[0162] 3.1.3. Obtaining reconstructed joint point pairs for:

[0163]

[0164] In the formula, is the reconstructed joint point of the i-th joint point of the moving body at the t-th moment; S X (t) is the state vector S of the freedom degree of the moving body at the tth moment X (t):

[0165]

[0166] In the formula, is the state of the mth degree of freedom of the moving body at the tth moment, m=1,2,…,Q X , T is the transpose.

[0167] 3.2. Conduct similarity test between the instructor and the serial robot based on the reconstructed joint points to obtain similarity measure;

[0168] 3.2.1. Perform test actions through the moving body and obtain test information, the test information includes frame data of each frame, and the frame data includes time and three-dimensional coordinates of motion joints;

[0169] 3.2.2. Obtain the key frame sequence of test information by improving the K-means algorithm;

[0170] 3.2.3. Quantify the three-dimensional coordinates of the motion joint points of each frame of the test information to obtain the global motion metric, local motion metric and joint angle;

[0171] 3.2.4. Obtaining the global motion metric, local motion metric and joint angle corresponding to each key frame based on the key frame sequence of the test information; specifically including the following steps:

[0172] 3.2.4.1. Based on the frame data of each frame of the test information, the three-dimensional coordinates of the motion joint points corresponding to each frame are converted into the three-dimensional coordinates of the reconstructed joint points by reconstructing the joint point pairs;

[0173] 3.2.4.2. Obtaining global motion metrics includes:

[0174] The three-dimensional coordinates of each reconstructed joint point of the moving body are based on the parent node as the origin; the jth frame and the j+1th frame are adjacent frames, and a single reconstructed joint point j is quantized between adjacent frames;

[0175] Use the Kabsch algorithm to quantize adjacent frames:

[0176]

[0177]

[0178]

[0179]

[0180] In the formula, Set α 、Set β Reconstruct the 3D coordinate set of joint points for the previous frame and the next frame, is the three-dimensional coordinate of the reconstructed joint point of the j-th reconstructed joint point in the moving body in the previous frame and the next frame, j = 1, 2, ..., l, l is the number of reconstructed joint points in the moving body, Cent α 、Cent β Set α 、Set β The centroid of

[0181] Calculate the covariance matrix E based on the centroid and the reconstructed three-dimensional coordinates of the joint points:

[0182]

[0183] The covariance matrix is ​​decomposed into three matrices M, N, O by singular value decomposition:

[0184] [M, N, O] = SVD(E)

[0185] R=OM T

[0186] T=Cent β -R×Cent α

[0187] Where R is the rotation matrix, T = [t x , t y , t z ] T is the translation matrix;

[0188]

[0189] (θ x ,θ y ,θ z )=Euler(R)

[0190] In the formula, TF is the α Transform to Set β The transformation matrix, Euler is the Euler function, (θx ,θ y ,θ z ) is the rotation angle of the rotation matrix R around the coordinate axis;

[0191] The global motion metric of the jth reconstructed joint point in the n′th adjacent frame for:

[0192]

[0193] The global motion metric G_Motion of all reconstructed joints in the n′th adjacent frame n′ for:

[0194]

[0195] The global motion metric G_Motion of the test information is:

[0196] G_Motion=[G_Motion1, G_Motion2,…,G_Motion N′ ]

[0197] Where, N' is the number of test information frames, n'=1, 2, ..., N';

[0198] 3.2.4.3. Obtaining local motion metrics includes:

[0199] The global motion metric of the j-th reconstructed joint point in the n′th adjacent frame Obtain the local motion metric of the jth reconstructed joint point in the n′th adjacent frame by homogeneous coordinate transformation for:

[0200]

[0201]

[0202] The global motion metric L_Motion of all reconstructed joints in the n′th adjacent frame n′ for:

[0203]

[0204] The local motion metric L_Motion of the test information is:

[0205] L_Motion=[L_Motion1, L_Motion2,…,L_Motion N′ ]

[0206] 3.2.4.4、Getting joint angles includes:

[0207] The joint angle of the j-th reconstructed joint point according to the n′th adjacent frame is:

[0208] Reconstruct the joint angle α of all joint points based on the n′th adjacent frame n′ for:

[0209]

[0210] Where num is the number of joint angles;

[0211] The joint angle α of the test information is:

[0212] α=[α1,α2,…,α N′ ]

[0213] 3.2.5. Based on the global motion metric, local motion metric and joint angle corresponding to each key frame, the corresponding global motion metric distance, local motion metric distance and joint angle metric distance are calculated:

[0214] G_dis k′ =||G_Motion B，k′ -G_Motion I，k′ ||

[0215] L_dis k′ =||LMotion B，k′ -L_Motion I，k′ ||

[0216] α_dis k′ =||α B，k′ -α I，k′ ||

[0217] In the formula, G_dis k′ is the global motion metric distance of the k′th key frame, G_Motion B，k′ 、G_Motion I,k′ are the global motion metrics of the k′th key frame of the instructor and the serial robot respectively;

[0218] L_dis k′ is the local motion metric distance of the k′th key frame, L_Motion B,k′ 、L_Motion I,k′ are the local motion metrics of the k′th key frame of the instructor and the serial robot respectively;

[0219] α_dis k′ is the joint angle distance of the k′th key frame, α B,k′ , α I,k′are the joint angles of the k′th key frame of the instructor and the serial robot respectively;

[0220] 3.2.6. Compare the global motion metric distance, local motion metric distance and joint angle metric distance corresponding to each key frame with the preset global threshold, local threshold and joint angle threshold, and obtain the corresponding similarity measure:

[0221]

[0222]

[0223]

[0224]

[0225] Where G%, L%, and α% are the preset similarity measures of the global motion metric, local motion metric, and joint angle metric, respectively; is the total similarity measure of the k′th key frame, The similarity measures of the global motion metric, local motion metric and joint angle metric are respectively used to measure the similarity of the k′th keyframe;

[0226] 3.2.7. Obtain the total similarity measure sim of the test information based on the total similarity measure of each key frame total :

[0227]

[0228] Where K′ is the number of key frames of the test information, k′=1,2,…,K′.

[0229] 3.3. Adjust the reconstructed joint point matrix (adjust the position of element 1), and return the adjusted reconstructed joint point matrix to the initialized reconstructed joint point matrix for iteration;

[0230] 3.4. Take the reconstructed joint point pair with the largest similarity measure as the spatial mapping.

[0231] 4. Convert the three-dimensional coordinates of each human body reconstructed joint point into the three-dimensional coordinates of the serial robot coordinate system through homogeneous coordinate transformation;

[0232] 5. The three-dimensional coordinates of each serial robot coordinate system are used as the target points of the three-dimensional coordinate points of the machine reconstruction joints of the serial robot;

[0233] 6. Based on each target point, the serial robot is controlled by a parallel CCD control method to complete the teaching action; specifically, the following steps are included:

[0234] 6.1. Construct the control variable φ of the serial robot according to the first target point:

[0235] φ={φ1,φ2,…,φ f}

[0236]

[0237] Where, e and f are the number of machine joints and machine reconstruction joints; φ f Reconstruct the control variables of the joint points for the fth machine;

[0238]

[0239] In the formula, θ g is the rotation angle of the g-th machine joint, d g is the translation distance of the g-th machine joint point;

[0240] 6.2. Constructing the control objective function θ based on the control variable group u :

[0241] θ u =ρ1‖φ1‖ 2 +ρ2‖φ2‖ 2 +…+ρ f ‖φ f ‖ 2

[0242] In the formula, ρ r Reconstruct the motion gain of the joint point of the rth machine, r = 1, 2, ..., f;

[0243] 6.3. Adjusting the control objective function θ u Let θ u Approaching zero or less than or equal to the threshold θ t , thus completing the first target point;

[0244] 6.4. Repeat the above steps until each target point is completed to complete the teaching action.

[0245] Embodiment 2:

[0246] A human-machine teaching control method for a serial robot based on isomer mapping is provided in Example 1. The human left arm is used as the instructor, and the SRS seven-degree-of-freedom serial robot is used as the teaching serial robot. Kinect (other motion capture systems can also be used) is used to obtain the three-dimensional coordinates of the human left arm, mainly collecting the motion information of the three joints of the left arm: the shoulder, elbow, and wrist.

[0247] The process of this embodiment includes:

[0248] S101. Use a Kinect depth camera to collect three-dimensional coordinate information of a motion sequence of a human left arm, including shoulder, elbow, and wrist joints.

[0249] S102, the motion information of the left arm is subjected to a modified K-means algorithm in chronological order to obtain key frames of the motion information of the left arm. Each frame of a 897-frame action sequence contains three-dimensional coordinate information of three reconstructed joint points;

[0250] (1) Using the clustering evaluation index elbow method, we determined that K of 897 frames of data is 20.

[0251] (2) Randomly select 20 frames as cluster centroids, calculate the Euclidean distance between each frame frame and the centroids of the 20 clusters, and then divide it into the group with the nearest cluster centroid.

[0252] (3) Update the cluster centroids in each cluster.

[0253] (4) Repeat steps 2 and 3 until the new and old centroids are balanced or less than the threshold value of 0.05.

[0254] (5) Calculate the Euclidean distance between the centroid of each group and each frame of the group, and select the one with the smallest distance in each group as the representative of the group. A total of 20 frames are selected as key frames. The 20 key frames are sorted according to the time index.

[0255] S103, performing spatial mapping of the motion of the left arm of the human and the serial robot, and ignoring, reordering, and reusing various degrees of freedom of the human-machine by reconstructing the joint point matrix, so that the human-machine forms a set of reconstructed joint point pairs, and realizing motion mapping of the human-machine spatial geometry;

[0256] (1) The time state vectors of the motion of the human left arm and the serial robot are as follows:

[0257]

[0258]

[0259] Note: B is the left arm and I is the serial robot, both have 7 degrees of freedom.

[0260] (2) The initialization action is selected as waving. By changing the reconstructed joint matrix, different spatial mappings are obtained. The second joint, the fourth joint, and the sixth joint of the serial robot are used to reconstruct the elbow joint and obtain different similarity percentages on the initialization action, which are 31.82%, 84.34%, and 40.91%, respectively. Therefore, the fourth joint is selected to mainly reconstruct the elbow joint.

[0261] S104, such as Figure 3 As shown in the figure, the reconstruction matrix is ​​constructed based on the fourth joint to reconstruct the elbow joint. According to the degrees of freedom of the joints in the left arm teaching and the degrees of freedom of the serial robot, by reconstructing 7 degrees of freedom, the reconstructed joints of the human body teaching part can be well mapped to the reconstructed joints of the serial robot. The reconstructed joint matrix is ​​set as:

[0262]

[0263] By reconstructing the joint matrix, the left arm and the serial robot can use the same number of reconstructed joints, and the degrees of freedom in the reconstructed joints can be ignored, reordered, and reused. Each reconstructed joint of the instructor and the corresponding reconstructed joint of the serial robot form a reconstructed joint pair. The reconstructed joints are obtained by the following method.

[0264]

[0265] Where: i=1, 2, 3 are respectively the reconstructed shoulder joint point, the reconstructed elbow joint point, and the reconstructed wrist joint point;

[0266] S105, Figure 4 Teaching data conversion, the three-dimensional coordinates of the reconstructed shoulder, elbow and wrist joints obtained in S104 are converted into three-dimensional coordinates based on the base coordinate system of the serial robot through homogeneous transformation, and the three-dimensional coordinates of the reconstructed joint points in each key frame are used as the three-dimensional coordinates of the target point that the corresponding reconstructed joint of the serial robot needs to reach.

[0267] S106, such as Figure 5 As shown, the three-dimensional coordinates of the arm reconstructed joint points based on the serial robot base coordinate system obtained in S104 are used as the target points of the serial robot reconstructed joints at each moment, and the parallel CCD control method is used to control the reconstructed joint points to reach the target points. First, the control vector angle θ and the projection distance d are constructed as the control targets, and the reconstructed joints of the serial robot are controlled to move in sequence, so that θ→0, and d→0, so that the target reconstructed joint points of the serial robot are close to the target point position, until the distance between the target position and the actual position is less than the set threshold, that is, the reconstructed joint point completes the motion action of the frame. By moving each frame action of each reconstructed joint point of the serial robot according to the above method, it is achieved that the reconstructed joint points of the serial robot can reach the target point. Then the above control is repeated for 20 key frame sequences to achieve the similarity imitation of the serial robot.

[0268] (1) Using CCD control model.

[0269] (2) Construct the target control function:

[0270]

[0271] Let θu =‖φ1‖ 2 +2‖φ2‖ 2 +‖φ3‖ 2 As our control objective function, the control target is θ u =0 or close to zero.

[0272] (3) Complete one frame of action according to the above method, so that the serial robot completes the action of the first frame. Set the step size to 0.5, control the serial robot to step towards the target point so that the constructed objective function gradually approaches zero, until it is less than the set threshold of 0.6, so that the serial robot completes the action of the first frame.

[0273] (4) Repeat the above steps to complete the 20 key frame actions of the entire motion sequence and complete the teaching of each frame action.

[0274] (5) The above is the process of controlling the serial robot to imitate a frame of arm movement. Through the action of 20 key frame action sequences, it moves to the expected position of each frame according to the above one frame movement method. Finally, by connecting 30 key frame actions in series, the serial robot imitates the arm raising action of a human arm.

[0275] (6) Figure 6 As shown in the figure, the human arm teaches the arm-lifting action of the serial robot. From the whole motion sequence diagram, it can be seen that the similarity of the two actions is high, indicating that this method can better teach the serial robot to make the arm teaching action. In order to evaluate the stability of similar teaching, the above steps are repeated 5 times to obtain 5 groups of similar teaching action sequences. Through the isomer similarity evaluation in step S103, 5 groups are obtained as shown in Table 1:

[0276] Table 1: Similarity percentages of human-machine arm lifting

[0277] 1 2 3 4 5 82.81％ 80.78％ 81.84％ 83.13％ 80.54％

[0278] As can be seen from Table 1, the similarity percentage of the arm-lifting action is above 80%, indicating that the similarity of the teaching action is very high and the method is effective on this teaching action.

[0279] Embodiment three:

[0280] Based on a human-machine teaching control method for a serial robot based on isomer mapping provided in Example 1, a human body is used as a teacher, and an SRS seven-degree-of-freedom serial robot is used as a teaching serial robot; Kinect (other motion capture systems can also be used) is used to obtain the three-dimensional coordinates on the human body.

[0281] The specific steps are as follows:

[0282] S201. Use a Kinect depth camera to collect the left-right swaying motion sequence of the human body, including the three-dimensional coordinate information of 15 joints such as the head, neck, shoulder, elbow, wrist, trunk, hip, knee, and foot.

[0283] S202, the human body's left-right swaying motion sequence is subjected to a modified k-means algorithm in chronological order to obtain key frames of the motion sequence. Each frame of a 987-frame motion sequence contains the three-dimensional coordinate information of five reconstructed joint points;

[0284] (1) Use the elbow method to determine the K value to be 22.

[0285] (2) Randomly select 22 frames as cluster centroids, calculate the Euclidean distance between each frame frame and the centroids of the 22 clusters, and then divide it into the group with the nearest cluster centroid.

[0286] (3) Update the cluster centroids in each cluster.

[0287] (4) Repeat steps 2 and 3 until the new and old centroids are balanced or less than the threshold value of 0.05.

[0288] (5) Calculate the Euclidean distance between the centroid of each group and each frame of the group, and select the one with the smallest distance in each group as the representative of the group. A total of 22 frames are selected as key frames. Sort the 22 key frames according to the time index.

[0289] S203, human-machine space mapping, motion mapping of the entire human body and the serial robot. First, the initialization action is set to standing, and the various degrees of freedom of the human-machine are reorganized by reconstructing the joint matrix to achieve different space mappings. After calculating the similarity percentage on the initialization action, a set of similarity percentage data of 23.70%, 41.61%, 50.74%, 35.41%, 61.78%, 81.65%, 70.23%, and 59.73% is obtained. From the above similarity percentages, the space mapping with the largest similarity of 81.65% is selected as the space mapping method of this embodiment.

[0290] S204, such as Figure 7 As shown, the human body reconstruction joint point V1 is the joint j1 of the head joint, V2 is the neck joint j2, V3 is the trunk joint j3, V4 knee center joint is the center point of j6 and j7, V5 foot center joint is the center point of j8 and j9. According to the above, the mutual reconstruction joint matrix of human-machine teaching is constructed as shown below:

[0291]

[0292]

[0293] Note: i=1, 2, 3, 4, 5 respectively represent the reconstructed head joint, reconstructed neck joint, reconstructed trunk joint, reconstructed knee center joint, and reconstructed foot center joint;

[0294] S205. Convert the three-dimensional coordinates of the five reconstructed joints obtained in S204 into three-dimensional coordinates based on the base coordinate system of the serial robot through hand-eye conversion, and use the three-dimensional coordinates of the reconstructed joint points in each key frame as the three-dimensional coordinates of the target point that the corresponding reconstructed joint of the serial robot needs to reach.

[0295] S206, using the three-dimensional coordinates of the human body reconstructed joint points based on the serial robot base coordinate system obtained in S204 as the target points of the serial robot reconstructed joints at each moment, and using the parallel CCD control method to control the reconstructed joint points to reach the target points, first constructing the control vector angle θ and the projection distance d as the control target, and controlling the serial robot reconstructed joints to move in sequence, so that θ→0, and d→0, so that the target reconstructed joint points of the serial robot are close to the target point position, until the distance between the target position and the actual position is less than the set threshold, that is, the reconstructed joint point completes the motion action of the frame. By moving each frame action of each reconstructed joint point of the serial robot according to the above method, it is achieved that the reconstructed joint points of the serial robot can reach the target point. Then the above control is repeated for the 22 key frame sequences to achieve the similarity imitation of the serial robot.

[0296] (1) Using CCD control model.

[0297] (2) Construct the target control function:

[0298]

[0299] Let θ u =‖φ1‖ 2 +2‖φ2‖ 2 +3‖φ3‖ 2 +2‖φ4‖ 2 +3‖φ5‖ 2 As the control objective function, the control target is θ u =0 or close to zero.

[0300] (3) Complete one frame of action according to the above method, so that the serial robot completes the action of the first frame. Set the step size to 0.5, control the serial robot to step towards the target point so that the constructed objective function gradually approaches zero, until it is less than the set threshold of 0.6, so that the serial robot completes the action of the first frame.

[0301] (4) Repeat the above steps to complete the 22 key frame actions of the entire motion sequence and complete the teaching of each frame action.

[0302] (5) The above is the process of controlling the serial robot to imitate the left-right swaying motion of a human body. Through the action of the 22 key frame action sequence, it moves to the expected position of each frame according to the movement method of the above frame. Finally, by connecting the 22 key frame actions in series, the serial robot imitates the left-right swaying motion of the human body.

[0303] (6) Figure 8 As shown in the figure, the action diagram of the whole human body teaching the serial robot to swing left and right is shown. It can be seen from the figure that the similarity of the two action sequences is relatively high, indicating that this method can better realize the human body teaching the serial robot. In order to evaluate the stability of the similarity teaching of the whole human body and the serial robot, the above steps are repeated 5 times to obtain 5 groups of action sequences of similarity teaching. Through the isomer similarity evaluation in step S203, 5 groups are obtained as shown in Table 2:

[0304] Table 2: Similarity percentages of human-machine swaying

[0305] 1 2 3 4 5 78.54％ 76.78％ 77.84％ 78.13％ 79.54％

[0306] It can be seen from Table 2 that the similarity percentage of each group in the left-right swinging action is above 76%, indicating that this method has good stability for human body teaching serial robots.

[0307] Embodiment 4:

[0308] Based on the first embodiment, the present invention also provides a serial robot human-machine teaching control device based on isomer mapping, including a processor and a storage medium;

[0309] The storage medium is used to store instructions;

[0310] The processor is used to operate according to the instructions to execute the steps according to the above method.

[0311] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0312] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0313] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0314] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0315] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A human-machine teaching control method for a serial robot based on isomer mapping, characterized in that: include: Acquire teaching information of the teaching action of the instructor, wherein the teaching information includes frame data of each frame, and the frame data includes time and three-dimensional coordinates of human joints; The key frame sequence of teaching information is obtained by improving the K-means algorithm; The key frame sequence based on the teaching information converts the three-dimensional coordinates of the human joint points corresponding to each key frame into the three-dimensional coordinates of the human reconstructed joint points through a pre-built spatial mapping; The three-dimensional coordinates of each human body reconstructed joint point are converted into the three-dimensional coordinates of the serial robot coordinate system through homogeneous coordinate transformation; The three-dimensional coordinates of each serial robot coordinate system are used as the target points of the three-dimensional coordinates of the machine reconstruction joint points of the serial robot; Based on each target point, the serial robot is controlled by a parallel CCD control method to complete the teaching action; The key frame sequence for obtaining teaching information by improving the K-means algorithm includes: The cluster evaluation index elbow method is used to determine the K value; Randomly and uniformly select K frames from the teaching information as cluster centroids; Cluster each frame of the teaching information according to the cluster centroid to obtain K clusters and corresponding cluster centroids; The Euclidean distance is calculated between each frame of the cluster and its cluster centroid according to the vector generated by the three-dimensional coordinates of the human joint points in the frame data, and the frame corresponding to the minimum Euclidean distance is selected as the key frame of the cluster; Sort the key frames of each cluster from front to back according to the time of the frame data to obtain the key frame sequence of the teaching information; The step of clustering each frame of the teaching information according to the cluster centroid to obtain K clusters and corresponding cluster centroids includes: The Euclidean distance between each frame of the teaching information and the cluster centroid is calculated based on the vector generated by the three-dimensional coordinates of the human joint points of the frame data; Assign each frame of the teaching information to the cluster centroid with the smallest Euclidean distance to generate K clusters; Update the cluster centroid o of the K clusters k : In the formula, o k is the kth cluster centroid, k = 1, 2, ..., K; N is the number of frames of teaching information, n = 1, 2, ..., N; t nk Whether the nth frame belongs to the cluster centroid o k , if so, then t nk =1, if not, then t nk =0; frame n A vector generated from the three-dimensional coordinates of the human joint points of the frame data of the nth frame; Repeat the above steps to iterate until the cluster centroid of the current iteration is equal to the cluster centroid of the previous iteration or the difference is less than the preset threshold, and output the final K clusters and the corresponding cluster centroids; The construction of the spatial mapping includes: Initialize the reconstructed joint point matrix, and reconstruct each degree of freedom of the instructor and the serial robot based on the reconstructed joint point matrix to obtain the reconstructed joint point pairs; Based on the reconstructed joint point pairs, the similarity test between the instructor and the serial robot is performed to obtain the similarity measure; Adjust the reconstructed joint point matrix, and return the adjusted reconstructed joint point matrix to the initialized reconstructed joint point matrix for iteration; The reconstructed joint point pair with the largest similarity measure is taken as the spatial mapping.

2. The human-machine teaching control method of a serial robot based on isomer mapping according to claim 1, characterized in that: Determining the K value using the clustering evaluation index elbow method includes: Let K = 2; Randomly and uniformly select K frames from the teaching information as cluster centroids; Cluster each frame of the teaching information according to the cluster centroid to obtain K clusters and corresponding cluster centroids; Calculate the Euclidean distance between each frame of the cluster and its cluster centroid according to the vector generated by the three-dimensional coordinates of the human joint points of the frame data, and sum the Euclidean distances; Let K=K+1, and repeat the above steps to iterate. If the sum of the current number of iterations is suddenly smaller than the sum of the previous number of iterations, the K value corresponding to the current number of iterations is output as the result.

3. A human-machine teaching control method for a serial robot based on isomer mapping according to claim 1, characterized in that: The initialization reconstructs the joint point matrix for: In the formula, X is the moving body. If X=B, the moving body is the instructor. If X=I, the moving body is the serial robot. is the reconstructed joint point matrix of the i-th motion joint point of the moving body, Q X is the number of degrees of freedom of the moving body, S X is the number of degrees of freedom of the i-th joint of the moving body, Each row of the reconstructed joint point matrix has only one element 1, and the rest are elements 0; and adjusting the reconstructed joint point matrix is ​​to adjust the position of element 1; The acquisition of the reconstructed joint point pair for: In the formula, is the reconstructed joint point of the i-th joint point of the moving body at the t-th moment; S X (t) is the state vector S of the freedom degree of the moving body at the tth moment X (t): In the formula, is the state of the mth degree of freedom of the moving body at the tth moment, m=1,2,…,Q X , T is the transpose.

4. A human-machine teaching control method for a serial robot based on isomer mapping according to claim 1, characterized in that: The similarity test between the instructor and the serial robot based on the reconstructed joint point pair is performed to obtain the similarity measure, including: Performing a test action through a moving body and acquiring test information, wherein the test information includes frame data of each frame, and the frame data includes a time and three-dimensional coordinates of a moving joint point; Obtain the key frame sequence of test information by improving the K-means algorithm; Quantifying the three-dimensional coordinates of the motion joint points of each frame of the test information to obtain the global motion metric, the local motion metric and the joint angle; Obtain the global motion metric, local motion metric and joint angle corresponding to each key frame based on the key frame sequence of the test information; Based on the global motion metric, local motion metric and joint angle corresponding to each key frame, the corresponding global motion metric distance, local motion metric distance and joint angle metric distance are calculated: G_dis k′ =||G_Motion B,k′ -G_Motion I,k′ || L_dis k′ =||L_Motion B,k′ -L_Motion I，k′ || a_dis k′ =||a B，k′ -a I，k′ || In the formula, G_dis k′ is the global motion metric distance of the k′th key frame, G_Motion B，k′ 、G_Motion I,k′ are the global motion metrics of the k′th key frame of the instructor and the serial robot respectively; L_dis k′ is the local motion metric distance of the k′th key frame, L_Motion B，k′ 、L_Motion I，k′ are the local motion metrics of the k′th key frame of the instructor and the serial robot respectively; α_dis k′ is the joint angle distance of the k′th key frame, α B，k′ , α I，k′ are the joint angles of the k′th key frame of the instructor and the serial robot respectively; Compare the global motion metric distance, local motion metric distance, and joint angle metric distance corresponding to each keyframe with the preset global threshold, local threshold, and joint angle threshold, and obtain the corresponding similarity measure: Where G%, L%, and α% are the preset similarity measures of the global motion metric, local motion metric, and joint angle metric, respectively; is the total similarity measure of the k′th key frame, The similarity measures of the global motion metric, local motion metric and joint angle metric are respectively used to measure the similarity of the k′th keyframe; Obtain the total similarity measure sim of the test information based on the total similarity measure of each key frame total : Where K′ is the number of key frames of the test information, k′=1, 2, …, K′.

5. A human-machine teaching control method for a serial robot based on isomer mapping according to claim 4, characterized in that: The step of quantifying the three-dimensional coordinates of the motion joint points of the frame data of each frame of the test information to obtain the global motion metric, the local motion metric and the joint angle comprises: Based on the frame data of each frame of the test information, the three-dimensional coordinates of the motion joint points corresponding to each frame are converted into the three-dimensional coordinates of the reconstructed joint points by reconstructing the joint point pairs; Obtaining global motion metrics includes: Use the Kabsch algorithm to quantize adjacent frames: In the formula, Set α 、Set β Reconstruct the 3D coordinate set of joint points for the previous frame and the next frame, is the three-dimensional coordinate of the reconstructed joint point of the j-th reconstructed joint point in the moving body in the previous frame and the next frame, j = 1, 2, ..., l, l is the number of reconstructed joint points in the moving body, Cent α 、Cent β Set α 、Set β The centroid of Calculate the covariance matrix E based on the centroid and the reconstructed three-dimensional coordinates of the joint points: The covariance matrix is ​​decomposed into three matrices M, N, O by singular value decomposition: [M, N, O] = SVD(E) R=OM T T=Cent β -R×Cent α Where R is the rotation matrix, T = [t x , t y , t z ] T is the translation matrix; (i x ,the y ,the z )=Euler(R) In the formula, TF is the α Transform to Set β The transformation matrix, Euler is the Euler function, (θ x ,θ y ,θ z ) is the rotation angle of the rotation matrix R around the coordinate axis; The global motion metric of the jth reconstructed joint point in the n′th adjacent frame for: The global motion metric G_Motion of all reconstructed joints in the n′th adjacent frame n′ for: The global motion metric G_Motion of the test information is: G_Motion=[G_Motion1,G_MMotion2,…,G_Motion N′ ] Where, N' is the number of test information frames, n'=1, 2, ..., N'; Obtaining local motion metrics includes: The global motion metric of the j-th reconstructed joint point in the n′th adjacent frame Obtain the local motion metric of the jth reconstructed joint point in the n′th adjacent frame by homogeneous coordinate transformation for: The global motion metric L_Motion of all reconstructed joints in the n′th adjacent frame n′ for: The local motion metric L_Motion of the test information is: L-Motion=[L_Motion1,L_MMotion2,…,L_Motion N′ ] Getting joint angles includes: The joint angle of the j-th reconstructed joint point according to the n′th adjacent frame is: Reconstruct the joint angle α of all joint points based on the n′th adjacent frame n′ for: Where num is the number of joint angles; The joint angle α of the test information is: α＝[α1，α2，…，α N′ ]。 6. A human-machine teaching control method for a serial robot based on isomer mapping according to claim 1, characterized in that: The method of controlling the serial robot to complete the teaching action based on each target point by a parallel CCD control method includes: Construct the control variable φ of the serial robot according to the first target point: φ={φ1,φ2,…,φ f } Where, e and f are the number of machine joints and machine reconstruction joints; φ f Reconstruct the control variables of the joint points for the fth machine; In the formula, θ g is the rotation angle of the g-th machine joint, d g is the translation distance of the g-th machine joint point; Construct the control objective function θ based on the control variable group u : i u =ρ1||φ1|| 2 +ρ2||φ2|| 2 +…+r f ||φ f || 2 In the formula, ρ r Reconstruct the motion gain of the joint point of the rth machine, r = 1, 2, ..., f; Adjust the control objective function θ u Let θ u Approaching zero or less than or equal to the threshold θ t , thus completing the first target point; Repeat the above steps until each target point is completed to complete the teaching action.

7. A human-machine teaching control device for a serial robot based on isomer mapping, characterized in that: including processor and storage medium; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method according to any one of claims 1-6.

Images