A vision-based method for a quadruped bionic robot to simulate animal gaits

Abstract

The invention discloses a method for simulating animal gait by a quadruped bionic robot based on vision. The invention combines the motion characteristic parameter results of quadruped animals with the physiological structure characteristics of dogs, establishes a canine-simulating quadruped robot model, and inputs the motion parameters such as the animal joint angles calculated based on a Dpnet network model into a Matlab model system, thereby planning the robot motion trajectory to achieve motion according to the animal ecological mode. The invention can intelligently obtain animal motion images and timely analyze and monitor the health conditions of animals, thereby ensuring the timely acquisition of daily health information of animals and preventing hidden diseases. The invention extracts the motion parameters of animals without contacting target objects, thereby reducing the difficulty and uncertainty of data collection and improving work efficiency. The invention solves the problems of insufficient flexibility and weak reference of bionic robots by simulating the gait of diagonal trotting motion of canine animals through robots.

Description

Technical Field

[0001] The present invention relates to a method for simulating animal motion state by a quadruped bionic robot based on computer vision, in particular to a method for extracting motion parameters based on a Dpnet network and a method for simulating animal gait by a robot based on motion parameters, and belongs to the field of computer vision and image processing. Background Art

[0002] In order to timely discover the health risks of animals when raising dogs on large farms or grazing on pastures, employees are usually responsible for regularly checking the activities of animals. However, manual observation is subjective and has low real-time performance, and has the disadvantages of excessive labor consumption and low efficiency. Therefore, accurately identifying the gait status of animals has a high application value for farm and pasture breeding. At present, the research on animal behavior recognition is relatively mature, but there are still the following problems for the analysis of specific animal gaits: interference caused by different lighting, scene changes and shooting angles in the natural environment, the similarity of the fur of the limbs of the research object itself and the key point positioning error caused by multi-target occlusion. Therefore, the present invention adopts a deep learning method to use the network for automatic detection, reduces manpower and material resources through computer vision, and improves efficiency at the same time. The quadruped bionic robot is a hot research direction in recent years. It is widely used based on its diverse posture forms and wide application range. Compared with the tracked robots with limited gait and the bulky wheeled robots, the quadruped robots with high degrees of freedom have stronger controllability and flexibility. At the same time, for the performance of bionic robots, the footed robots have the advantages of light weight and strong structure, and have a wider application space. Summary of the invention

[0003] The present invention provides a method for simulating the motion state of animals by a quadruped bionic robot based on computer vision, which is intended to be applied to situations where quadruped robots are involved in mine clearance and blasting, carrying heavy objects, and other situations that are difficult to complete by human labor. The present invention first improves the HRnet (High Resolution net) network; secondly, the motion parameters generated by the model are used to apply the physiological information structure of quadruped animals to the quadruped bionic robot, and a full-elbow robot structure is constructed. Then, the DH parameter method is used to perform forward kinematic modeling, the Lagrange equation method is used to calculate the kinetic energy and potential energy of the robot, and the Lagrange equation is solved to obtain the torque of each joint of the robot, and then the foot end trajectory of the robot in the ideal state of forward movement is planned. Finally, a simulation model of the quadruped bionic robot is built through MATLAB, and the change of the distance from the center of mass of the robot to the origin and the change of the robot's speed and acceleration during the forward movement are analyzed based on the simulink module.

[0004] The technical solution of the present invention is as follows: a method for simulating animal motion state by a quadruped bionic robot based on computer vision, wherein the specific steps of the method are as follows:

[0005] Step (1) Experimental data collection: Shoot a video of a quadruped trotting diagonally on the road: the resolution of the video is 1270×720, the shooting device is level with the quadruped and 5-8m away from the quadruped, the shooting angle is between ±270°, and ten 1-minute videos are shot between 8:00-12:00 in the morning and 13:00-18:00 in the afternoon;

[0006] Step (2) Establishing the model structure: Improving on the HRnet high-resolution basic network and establishing a skeleton model;

[0007] Step (3) extracting animal motion parameters: predicting the positions of key points of the quadruped in the video shot in step (1) by using the model established in step (2) to extract the motion parameters of the quadruped;

[0008] Step (4) establishing a quadruped robot model: applying the physical structure information of the quadruped to the overall structural design of the robot, and combining the motion parameters of the quadruped extracted in step (3) to perform forward kinematic modeling and dynamic analysis of the robot;

[0009] Step (5) Simulation of robot diagonal trotting gait: The robot's foot trajectory is planned, and the planned ideal trajectory is compared with the actual motion trajectory of the quadruped bionic robot to obtain a reliable robot model. Finally, the robot model is simulated by MATLAB to simulate the animal motion state.

[0010] In step (1), a video camera is used to shoot a video.

[0011] In step (2), the specific steps of model establishment are as follows:

[0012] Add two paths after the last four branches of the HRnet high-resolution base network output. The first path is set to three layers of mixed dilated convolutions, and the second path is set to a module with both channel and spatial attention mechanisms.

[0013] The first path is specifically as follows: The first path is set to three layers of hybrid dilated convolution. Based on the performance of the first and second branches with fewer channels and more contextual global information, we choose to add an HDC layer after the first two output branches. Assuming that the dilated convolution rate is d and the original convolution kernel size in the network is k×k, then the dilated convolution kernel size n is

[0014] n=k+(k-1)×(d-1) (1)

[0015] Since the dilated convolution will cause the loss of continuity of feature pixel reading while expanding the receptive field of feature information, the pixels involved in the convolution operation are all the effective weight positions of the expanded convolution kernel, which will cause a grid effect and lose some valuable feature information, thus affecting the accuracy of the final network classification prediction. Therefore, dilated convolution with different expansion coefficients is used and the three-layer convolution operation is combined into one path. In order to fill the lost pixels to the maximum extent and ensure the comprehensiveness of information representation, the maximum expansion coefficient of the current layer must not be greater than the original convolution kernel size of the network, that is, the condition is satisfied.

[0016] M i ≤k (2)

[0017] M i =Max[M i+1 -2d i ,M i+1 -2(M i+1 -d i ),d i ] (3)

[0018] In formula (3), M i is the maximum expansion coefficient of the i-th layer, d i is the hole convolution rate of the i-th layer, Max is the maximum value among the three layers, and the middle layer M2≤3, the hole convolution rates of each layer can be obtained as d1=1, d2=2, d3=5, thereby improving the weak feature representation ability and poor information globality brought by ordinary conventional convolution in the first two branches;

[0019] The second path is specifically as follows: a module containing both channel and spatial attention mechanisms is set, specifically, a parallel attention module PAM is added after the output of the third and fourth branches. Based on the fact that the latter two branches have more channels and therefore carry more comprehensive image detail information, the channel attention mechanism is integrated with the spatial attention mechanism, thereby assigning corresponding weights to the representations of different channels and the positions of pixels, so that the pixel information related to classification is more prominent in network training. The output features of the latter two branches are subjected to maximum pooling, average pooling, and median pooling operations, and the obtained feature vectors are subjected to channel and spatial weight calculations; first, a 1×1 convolution is used to adjust the number of channels to 256 and batch normalization is performed on the data. After combining the feature vectors output by the two attention mechanisms, the dimension is reduced. Finally, the weight values ​​of 0-1 of the two branches are obtained and assigned through sigmoid activation. The generated channel and spatial attention maps are calculated as follows:

[0020] Out channel =W(MLP*(in Avgpool +in Maxpool +in Medpool )) (4)

[0021] Out spatial =W(f*(in Avgpool ,in Maxpool ,in Medpool )) (5)

[0022]

[0023] In the above formula, Out channel is the channel attention map, Out spatial is the spatial attention map, W is the sigmoid activation function, MLP is a multilayer perception machine (multilayer perception) used to share weights, f is a 3×3 convolution operation, in represents the output features of the last two branches, Out branch The final branch feature is obtained by multiplying the channel weight and spatial weight with the output feature and then adding them together;

[0024] The two paths use feature fusion to supplement the low-level features with semantic information and add spatial information to the high-level features;

[0025] Through the above improvements to the HRnet high-resolution basic network, the joint positions of the quadruped are first obtained and then connected one by one to form a skeleton model.

[0026] In step (3), the specific steps of extracting motion feature parameters are as follows:

[0027] ①Same-side step parameters:

[0028] After obtaining the coordinate positions of the four corner points in a gait cycle of the research subject in the model, the average walking distance of the ankle joints of the four limbs relative to themselves is calculated by locating the change in the Euclidean distance of the two joint coordinates when the ankle joints of each limb are farthest and closest to the image origin. The f i Frame and f i+1 The coordinates of the ankle joint of the left forelimb of the animal in the frame image are is the coordinates of the right forelimb ankle joint of the animal, is the coordinates of the animal's left hind limb ankle joint, is the coordinate of the right hind limb ankle joint of the animal, and the step length calculation formula is

[0029]

[0030] In the formula is the conversion coefficient between image pixels and actual distance. In this study, the distance advanced by the animal's limbs in one cycle was summed and the average was taken to obtain the stride length;

[0031] ②Joint angle parameters:

[0032] Select the marked key point 3, i.e. the left forelimb knee joint point, and compare and analyze the changes of the knee joints of the three research subjects in the continuous movement cycle. Assume that the model finally recognizes the coordinates of the animal hip joint as (x a ,y a ), the knee joint coordinates are (x b ,y b ), the ankle joint coordinates are (x c ,y c ), then based on the coordinate change between the two joint points, the three distances a, b, and c can be obtained through the square difference of the coordinate vectors. The joint calculation formula is:

[0033]

[0034]

[0035]

[0036]

[0037] The animal's joint angle change parameters are calculated using equations (8) to (11).

[0038] In step (4), the specific steps of establishing the robot model are as follows:

[0039] ①Robot body structure design:

[0040] Through the analysis of the skeleton and muscle texture of quadrupeds, combined with the characteristics of quadrupeds in running motion, a full elbow mechanical structure is used as the body structure of the bionic robot, one degree of freedom is set at the hip joint connecting the first section and the second section, and one degree of freedom is set at the knee joint connecting the second section and the third section. The limbs of the bionic dog robot have a total of eight degrees of freedom.

[0041] ②Robot kinematics modeling

[0042] The robot consists of a world coordinate system {W}, a body coordinate system {B}, and a leg coordinate system {L}; the leg coordinate system also includes a hip joint coordinate system {1}, a knee joint coordinate system {2}, and a foot end coordinate system {3}. The coordinate transformation matrix of two adjacent systems in the coordinate system set by the robot is calculated through the DH parameter. After the plane translation and spatial rotation of the two adjacent coordinate systems, the overall transformation matrix formula is:

[0043]

[0044] in, Represents the coordinate transformation matrix between the nth coordinate system and the n+1th coordinate system. Rot() and Trans represent the rotation and translation operations on the X-axis and Z-axis.i represents the x-axis, α i Represents the rotation angle in the coordinate space, θ i is the angle of rotation of the joint coordinate system, a i Represents the length of the connecting rod between two joints, d i Represents the offset distance between the i-th coordinate system and the i+1-th coordinate system;

[0045] Substituting the DH parameters into equation (10) yields

[0046]

[0047] The rotation matrix of the coordinate system relative to the Z axis is:

[0048]

[0049] The translation matrix of the coordinate system relative to the X-axis is

[0050]

[0051] From equations (12) and (13), we can get the coordinate transformation matrix of the foot end coordinate system {3} of the quadruped bionic robot relative to the body coordinate system {B} as follows:

[0052]

[0053] because The T coordinate transformation matrix can be simplified as the following formula

[0054]

[0055] In the above formula is the rotation matrix, is the translation matrix. According to the above invention, the connecting rod length between the hip joint and the knee joint of the robot is set to be 20 cm, and the connecting rod length between the knee joint and the ankle joint is set to be 18 cm. Then, the forward kinematic modeling formula of the quadruped bionic dog robot is obtained from formula (15):

[0056]

[0057] The robot sets the connecting rod length between the hip joint and the knee joint as l1, and the connecting rod length between the knee joint and the ankle joint as l2;

[0058] ③Robot dynamics analysis

[0059] Assuming that the masses of connecting rod 1 and connecting rod 2 are m1 and m2 respectively, the moment of inertia of connecting rod 1 and connecting rod 2 can be obtained as follows:

[0060]

[0061]

[0062] Then take the center point of each connecting rod as the center of mass, determine the center of mass position of connecting rod 1 and connecting rod 2, and calculate the center of mass coordinates in the x and y directions. The formula is

[0063]

[0064]

[0065] r 2x =l1 cos(θ1(t))+(l2cos((θ1(t)+θ2(t))) / 2 (23)

[0066] r 2y =l1 sin(θ1(t))+(l2sin((θ1(t)+θ2(t))) / 2 (24)

[0067] The resulting velocities of the two connecting rods are

[0068]

[0069]

[0070] is the angular velocity of the first connecting rod, represents the angular velocity of the second connecting rod;

[0071] The total kinetic energy E of the robot is the sum of the kinetic energy of link 1 and link 2. The formula result is

[0072]

[0073] c2 represents cosθ2;

[0074] The total potential energy of the robot is the sum of the potential energies of the two links, and the calculation result is

[0075]

[0076] s1 represents sinθ1,s 12 represents sinθ1θ2;

[0077] From the Lagrange equation, we know that the Lagrange function is equal to the difference between the kinetic energy and potential energy of the robot.

[0078]

[0079] Based on the Lagrangian function l, θ1, θ2, Find the partial derivative and convert the robot's angular acceleration The relationship between the robot joint torque τ1, τ2 is expressed through a matrix, and the change of the relationship matrix can be seen more intuitively:

[0080]

[0081] ④Robot foot trajectory planning

[0082] The composite cycloid and Bezier curve method is used to plan the curve between the starting point and the landing point of the robot foot in a motion cycle. The calculation formula of the composite cycloid is:

[0083]

[0084]

[0085]

[0086] Formula (31) is the trajectory of the swing phase of the quadruped bionic robot in the X and Z directions, and Formula (32) is the trajectory of the support phase of the quadruped bionic robot in the X and Z directions, where x f With x s are the landing point and starting point of the robot in the X-axis direction, z s is the starting point of the robot on the Z axis, T in formula (33) f is the time the robot is in the support phase, T represents a motion cycle,

[0087] The Bezier curve is gradually generated by many control points in a recursive form. Compared with the compound cycloid, the Bezier curve has better speed and acceleration continuity. Therefore, the third-order Bezier curve is used for robot path planning. The formula is:

[0088] B(t)=p0(1-t) 3 +3p1t(1-t) 2 +3p2t 2 (1-t)+p3t 3 ,t∈[0,1] (32)

[0089] In formula (34), p0, p1, p2, and p3 are the four control points of the curve. By adjusting the control points to obtain a better foot trajectory, the four control points are set as the robot starting point (-0.11, -0.36), the highest point (0, -0.27), the second highest point (0.05, -0.28), and the landing point (0.11, -0.36).

[0090] In step (5), the simulink function of MATLAB R2021a is used to simulate the animal motion state of the robot model. The specific steps of the diagonal trotting gait simulation are as follows:

[0091] ① The present invention conducts experiments in the software environment of MATLAB R2021a, and the operating system is Win10 system. The robot simulation model with .slx as the suffix is ​​established through the simulink function, and the overall structure and motion relationship of the quadruped bionic robot are established based on the modules in the simulink library browser tool library. The world reference coordinate system is established through Multibody-Frames and Transforms-World Frame, and on this basis, the commonly used blocks in the tool library are used to add basic modules such as filters or data form conversion to the simulation model, and an oscilloscope is added to the output port of the robot through Simulink-Sinks-Scope. The simulation results of the robot are obtained through the above modeling process;

[0092] ② This section further analyzes the robot's own displacement and speed and other moving parameters. Add a partial derivative module to the simulink model, connect it to the output end of the robot's three coordinate axes ZYZ, and connect the other end of the partial derivative module to an oscilloscope to obtain the speed and acceleration change curve of the quadruped bionic robot during its movement.

[0093] The working principle of the present invention is:

[0094] (1) HRnet high-resolution basic network:

[0095] Based on the fact that the extraction of moving target feature parameters requires accurate positioning of joints, the present invention selects a bottom-up approach, that is, first obtaining the joint positions of quadrupeds and then connecting them one by one to form a skeleton model. In order to solve the disadvantage that the resolution is greatly reduced after downsampling due to the deepening of the network layers, the present invention uses HRnet as the basic network, preprocesses the original image into a uniform size of 256×256 and inputs it into the network. The network contains 4 modules, and the residual unit of each module increases from 1 to the top, where module 3 and module 4 are repeated 4 times and 3 times in the network respectively.

[0096] (2) Dpnet dual-path network

[0097] Based on the design that HRnet uses conventional convolution kernel 3×3 operations to extract features while maintaining high resolution, repeating a large number of convolution operations of the same size will weaken the spatial correlation between image pixels. In view of the fact that the receptive field obtained by the HRnet model is one-sided and fixed, the present invention makes improvements based on HRnet. Considering the integrity and coherence of the low-resolution layers in the network being alternately fused into the final high-resolution, the present invention chooses to add two paths after the last four branches of the HRnet output. The first path is set to a three-layer mixed hole convolution, and the second path is set to a module containing both channel and spatial attention mechanisms.

[0098] The beneficial effects of the present invention are:

[0099] (1) The present invention uses computer vision to intelligently acquire animal motion images and timely analyze and monitor the health status of animals, thereby ensuring timely acquisition of daily health information of animals and preventing potential diseases.

[0100] (2) The present invention extracts the motion parameters of the animal without contacting the target object, which reduces the difficulty and uncertainty of data collection and improves work efficiency.

[0101] (3) The present invention solves the problem of insufficient flexibility and weak reference of some bionic robots by simulating the gait of canine diagonal trotting by the robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0102] Figure 1 is a flow chart of the method of the present invention;

[0103] Figure 2 It is the structural diagram of the Dpnet model;

[0104] Figure 3(a) is a diagram showing the average angles of the hip joints of the four limbs of a biological dog;

[0105] Figure 3(b) is a diagram showing the average angles of the knee joints of the biological dog's limbs;

[0106] Figure 4 It is a robot coordinate system setting diagram in the present invention;

[0107] Figure 5 It is a diagram for establishing a single-leg coordinate system of the bionic robot in the present invention;

[0108] Figure 6 It is the foot end trajectory of the cycloid-based quadruped bionic robot in the present invention;

[0109] Figure 7 It is the foot end trajectory of the quadruped bionic robot based on Bezier curve in the present invention;

[0110] Figure 8It is the actual foot end trajectory of the quadruped bionic robot in the present invention;

[0111] FIG9( a ) is a graph showing the velocity and acceleration of the robot in the X-axis direction according to the present invention;

[0112] FIG9( b ) is a graph showing the velocity and acceleration of the robot in the Y-axis direction according to the present invention;

[0113] FIG9( c ) is a graph showing the velocity and acceleration of the robot in the Y-axis direction according to the present invention. DETAILED DESCRIPTION

[0114] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. The embodiments are only used to specifically illustrate the present invention, and the protection scope of the present invention is not limited to the contents described.

[0115] Embodiment 1: A method for simulating animal gait by a vision-based quadruped bionic robot, such as Figure 1 As shown, the method steps are as follows:

[0116] Step (1) Experimental data collection: A video of a Teddy dog ​​trotting diagonally on a residential road was filmed with a camera: the resolution of the video was 1270×720, the filming device was level with the quadruped and 5-8 m away from the quadruped, the filming angle was between ±270°, and ten 1-min videos were filmed between 8:00-12:00 in the morning and 13:00-18:00 in the afternoon;

[0117] Step (2) Establishing the model structure: Improve the HRnet high-resolution basic network and establish the model; Figure 2 As shown in the figure, the specific steps of model establishment are as follows:

[0118] Add two paths after the last four branches of the HRnet high-resolution base network output. The first path is set to three layers of mixed dilated convolutions, and the second path is set to a module with both channel and spatial attention mechanisms.

[0119] The first path consists of three layers of convolution operations. Based on the performance of the first and second branches with fewer channels and more global context information, we choose to add an HDC layer after the first two output branches. Assuming that the dilated convolution rate is d and the original convolution kernel size in the network is k×k, the dilated convolution kernel size n is

[0120] n=k+(k-1)×(d-1) (1)

[0121] Since the dilated convolution will cause the loss of continuity of feature pixel reading while expanding the receptive field of feature information, the pixels involved in the convolution operation are all the effective weight positions of the expanded convolution kernel, which will cause a grid effect and lose some valuable feature information, thus affecting the accuracy of the final network classification prediction. Therefore, dilated convolution with different expansion coefficients is used and the three-layer convolution operation is combined into one path. In order to fill the lost pixels to the maximum extent and ensure the comprehensiveness of information representation, the maximum expansion coefficient of the current layer must not be greater than the original convolution kernel size of the network, that is, the condition is satisfied.

[0122] M i ≤k (2)

[0123] M i =Max[M i+1 -2d i ,M i+1 -2(M i+1 -d i ),d i ] (3)

[0124] Among them, M i is the maximum expansion coefficient of the i-th layer, d i is the hole convolution rate of the i-th layer, and Max is the maximum value among the three layers, which can improve the weak feature representation ability and poor information globality brought by ordinary conventional convolution in the first two branches;

[0125] The second path is as follows: add a parallel attention module PAM after the output of the third branch and the fourth branch. Based on the fact that the latter two branches have more channels and therefore carry more comprehensive image detail information, the channel attention mechanism is integrated with the spatial attention mechanism, so as to assign corresponding weights to the representations of different channels and the positions of pixels, so that the pixel information related to classification is more prominent in network training. The output features of the latter two branches are subjected to maximum pooling, average pooling and median pooling operations to obtain the feature vectors for channel and space weight calculation; first use 1×1 convolution to adjust the number of channels to 256 and perform data batch normalization, reduce the dimension after combining the feature vectors output by the two attention mechanisms, and finally obtain and assign the weight values ​​of 0-1 to the two branches through sigmoid activation. The generated channel and spatial attention maps are calculated as follows:

[0126] Out channel =W(MLP*(in Avgpool +in Maxpool +in Medpool )) (4)

[0127] Out spatial =W(f*(in Avgpool ,inMaxpool ,in Medpool )) (5)

[0128]

[0129] In the above formula, Out channel is the channel attention map, Out spatial is the spatial attention map, W is the sigmoid activation function, MLP is a multi-layer perceptron for sharing weights, f is a 3×3 convolution operation, in represents the output features of the last two branches, Out branch The final branch feature is obtained by multiplying the channel weight and spatial weight with the output feature and then adding them together;

[0130] The two paths use feature fusion to supplement the low-level features with semantic information and add spatial information to the high-level features;

[0131] Through the above improvements to the HRnet high-resolution basic network, the joint positions of the quadruped are first obtained and then connected one by one to form a skeleton model.

[0132] Step (3) extracting animal motion parameters: The positions of key points of the quadruped are predicted by the model to extract the motion parameters of the quadruped. The specific steps are as follows:

[0133] ①Same-side step parameters:

[0134] After obtaining the coordinate positions of the four corner points in a gait cycle of the research subject in the model, the average walking distance of the ankle joints of the four limbs relative to themselves is calculated by locating the change in the Euclidean distance of the two joint coordinates when the ankle joints of each limb are farthest and closest to the image origin. The f i Frame and f i+1 The coordinates of the ankle joint of the left forelimb of the animal in the frame image are is the coordinates of the right forelimb ankle joint of the animal, is the coordinates of the animal's left hind limb ankle joint, is the coordinate of the right hind limb ankle joint of the animal, and the step length calculation formula is

[0135]

[0136] In the formula is the conversion coefficient between the image pixel and the actual distance. In this embodiment, the distance advanced by the animal's four limbs in one cycle is summed and the average value is taken to obtain the step length, as shown in Table 1;

[0137] Table 1 Parameters of the same-side stride length of Teddy dogs in diagonal trotting

[0138]

[0139] ②Joint angle parameters:

[0140] Select the marked key point 3, i.e. the left forelimb knee joint point, and compare and analyze the changes of the knee joints of the three research subjects in the continuous movement cycle. Assume that the model finally recognizes the coordinates of the animal hip joint as (x a ,y a ), the knee joint coordinates are (x b ,y b ), the ankle joint coordinates are (x c ,y c ), then based on the coordinate change between the two joint points, the three distances a, b, and c can be obtained through the square difference of the coordinate vectors. The joint calculation formula is:

[0141]

[0142]

[0143]

[0144]

[0145] The animal's joint angle change parameters are calculated using equations (8) to (11).

[0146] Step (4) Establishing a quadruped robot model: Designing the overall structure of the robot based on the biological characteristics of a dog, and then performing forward kinematic modeling using the DH parameter method. Using the Lagrange equation method, the kinetic energy and potential energy of the robot are calculated. The Lagrange equation is solved to obtain the torque of each joint of the robot, and then the foot end trajectory is planned under the ideal state of the robot moving forward. The specific steps are as follows:

[0147] ①Robot body structure design:

[0148] Through the analysis of the dog's body bones and muscle texture, and combined with the characteristics of dogs in running, this paper chooses to use a full elbow mechanical structure as the body structure of the bionic dog robot. The present invention sets one degree of freedom at the hip joint connecting the first section and the second section, and sets one degree of freedom at the knee joint connecting the second section and the third section, so the limbs of the bionic dog robot have a total of eight degrees of freedom.

[0149] The canine joint angles extracted based on the dual-path network are shown in FIG3. As shown in FIG3(a) and FIG3(b), the average angles of the hip joints of the four limbs of the Teddy dog ​​are between 20° and 90°, and the average angles of the knee joints of the four limbs are mainly concentrated in the range of 40° to 170°. In this embodiment, the actual measurement of the canine is carried out, and the length information of each part of the biological dog is obtained by observing the physical size. Therefore, the information of each part of the biological dog is summarized as shown in Table 1.

[0150] Table 1 Body parameters of the bionic dog robot

[0151]

[0152] ②Robot kinematics modeling

[0153] The robot of the present invention comprises a world coordinate system {W}, a body coordinate system {B}, and a leg coordinate system {L}; wherein the leg coordinate system further comprises a hip joint coordinate system {1}, a knee joint coordinate system {2}, and a foot end coordinate system {3}. The coordinate system setting diagram of the entire robot is as shown in FIG. Figure 4 The DH parameter table is shown in the following table.

[0154] Table 2 DH parameter table

[0155]

[0156] The coordinate transformation matrix of two adjacent systems in the coordinate system set by the robot is calculated through the DH parameters. After the plane translation and space rotation process of the two adjacent coordinate systems, the overall transformation matrix formula is:

[0157]

[0158] in, Represents the coordinate transformation matrix between the nth coordinate system and the n+1th coordinate system. Rot() and Trans represent the rotation and translation operations on the X-axis and Z-axis. i represents the x-axis, α i Represents the rotation angle in the coordinate space, θ i is the angle of rotation of the joint coordinate system, a i Represents the length of the connecting rod between two joints, d i Represents the offset distance between the i-th coordinate system and the i+1-th coordinate system;

[0159] Substituting the parameters in Table 2 into formula (10) we can obtain

[0160]

[0161] The rotation matrix of the coordinate system relative to the Z axis is:

[0162]

[0163] The translation matrix of the coordinate system relative to the X-axis is

[0164]

[0165] From equations (14) and (15), we can get the coordinate transformation matrix of the foot end coordinate system {3} of the quadruped bionic robot relative to the body coordinate system {B} as follows:

[0166]

[0167] because The T coordinate transformation matrix can be simplified as the following formula

[0168]

[0169] In the above formula is the rotation matrix, is the translation matrix. According to the above invention, the connecting rod length between the hip joint and the knee joint of the robot is set to be 20 cm, and the connecting rod length between the knee joint and the ankle joint is set to be 18 cm. Then, the forward kinematic modeling formula of the quadruped bionic dog robot is obtained from formula (17):

[0170]

[0171] ③Robot dynamics analysis

[0172] Combined with the above results, the present invention uses the Lagrange equation method to analyze the kinetic energy and potential energy of the eight connecting rods of the robot, and establishes the single-leg coordinate system of the quadruped bionic robot as follows: Figure 5 Assume that the masses of connecting rod 1 and connecting rod 2 are m1 and m2 respectively, from which the moment of inertia of connecting rod 1 and connecting rod 2 can be obtained as follows:

[0173]

[0174]

[0175] Then take the center point of each connecting rod as the center of mass, determine the center of mass position of connecting rod 1 and connecting rod 2, and calculate the center of mass coordinates in the x and y directions. The formula is

[0176]

[0177]

[0178] r 2x =l1 cos(θ1(t))+(l2cos((θ1(t)+θ2(t))) / 2 (23)

[0179] r 2y =l1 sin(θ1(t))+(l2sin((θ1(t)+θ2(t))) / 2 (24)

[0180] The resulting velocities of the two connecting rods are

[0181]

[0182]

[0183] The total kinetic energy E of the robot is the sum of the kinetic energy of link 1 and link 2. The formula result is

[0184]

[0185] The total potential energy of the robot is the sum of the potential energies of the two links, and the calculation result is

[0186]

[0187] From the Lagrange equation, we know that the Lagrange function is equal to the difference between the kinetic energy and potential energy of the robot.

[0188]

[0189] Based on the Lagrangian function l, θ1, θ2, Find the partial derivative and convert the robot's angular acceleration The relationship between the robot joint torque τ1, τ2 is expressed through a matrix, and the change of the relationship matrix can be seen more intuitively:

[0190]

[0191] ④Robot foot trajectory planning

[0192] The composite cycloid and Bezier curve method is used to plan the curve between the starting point and the landing point of the robot foot in a motion cycle. The calculation formula of the composite cycloid is:

[0193]

[0194]

[0195]

[0196] Formula (31) is the trajectory of the swing phase of the quadruped bionic robot in the X and Z directions, and Formula (32) is the trajectory of the support phase of the quadruped bionic robot in the X and Z directions, where x f With x s are the landing point and starting point of the robot in the X-axis direction, z s is the starting point of the robot on the Z axis, T in formula (33) f is the time the robot is in the support phase, and T represents a motion cycle;

[0197] The Bezier curve is gradually generated by many control points in a recursive form. Compared with the compound cycloid, the Bezier curve has better speed and acceleration continuity. The present invention uses the third-order Bezier curve for robot path planning. The formula is

[0198] B(t)=p0(1-t) 3 +3p1t(1-t) 2 +3p2t 2 (1-t)+p3t 3 ,t∈[0,1] (34)

[0199] In formula (34), p0, p1, p2, and p3 are the four control points of the curve. The four control points are set as the robot starting point (-0.11, -0.36), the highest point (0, -0.27), the second highest point (0.05, -0.28), and the landing point (0.11, -0.36). By adjusting the control points, a better foot trajectory can be obtained.

[0200] like Figure 6 The figure shows the foot end trajectory of the cycloid-based quadruped bionic robot in the present invention. Figure 7 The figure shows the foot trajectory of the quadruped bionic robot based on the Bezier curve in the present invention. Figure 8 The figure shows the actual foot trajectory of the quadruped bionic robot. Figure 6 and Figure 7 It can be seen that there are certain differences between the robot foot trajectory based on the composite cycloid and Bezier curve methods. Figure 8 We can see that the initial position height of the robot's foot is -0.36m. As the system starts to calculate the trajectory, the robot's foot is lifted up to about -0.27m and begins the first cycle of motion. This shows that the planning method based on the Bezier curve is closer to the actual foot trajectory of the robot. The actual starting point and landing point of the robot's foot are closer than the ideal planning point.

[0201] Step (5) diagonal trotting gait simulation: The robot's foot trajectory is planned, and the planned ideal trajectory is compared with the actual motion trajectory of the quadruped bionic robot to obtain a reliable robot model. Finally, the robot model is simulated by animal motion state using MATLAB. Based on the simulink module, the change in the distance from the robot's center of mass to the origin and the change in the robot's speed and acceleration during the forward movement are analyzed. The specific steps of diagonal trotting gait simulation are as follows:

[0202] ① Experiments were conducted in the software environment of MATLAB R2021a. The operating system was Win10. The robot simulation model with the suffix .slx was established through the simulink function. The overall structure and motion relationship of the quadruped bionic robot were established based on the modules in the simulink library browser tool library. The world reference coordinate system was established through Multibody-Frames and Transforms-World Frame. On this basis, the commonly used blocks in the tool library were used to add basic modules such as filters or data form conversion to the simulation model. An oscilloscope was added to the output port of the robot through Simulink-Sinks-Scope. The simulation results of the robot were obtained through the above modeling process.

[0203] ② Then analyze the robot's own displacement and speed and other moving parameters, add a partial derivative module in the Simulink model, connect it to the output end of the robot's three coordinate axes ZYZ, and connect the other end of the partial derivative module to an oscilloscope to obtain the speed and acceleration change curve of the quadruped bionic robot during the moving process, as shown in Figure 9. Figure 9a is a graph of the robot's travel speed and acceleration in the X-axis direction in the present invention, Figure 9b is a graph of the speed and acceleration of the robot in the Y-axis direction in the present invention, Fig.9c It is a curve diagram of the travel speed and acceleration of the robot in the Y-axis direction in the present invention.

[0204] The specific implementation modes of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above implementation modes, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

Claims

1. A method for simulating animal gait by a quadruped bionic robot based on vision, characterized in that: The specific steps of the method are as follows: Step (1) Experimental data collection: Shoot a video of a quadruped trotting diagonally on the road: the resolution of the video is 1270×720, the shooting device is level with the quadruped and 5-8m away from the quadruped, the shooting angle is between ±270°, and ten 1-minute videos are shot between 8:00-12:00 in the morning and 13:00-18:00 in the afternoon; Step (2) Establishing the model structure: Improve the HRnet high-resolution basic network and establish a skeleton model. The specific steps of model establishment are as follows: Add two paths after the last four branches of the HRnet high-resolution base network output. The first path is set to three layers of mixed dilated convolutions, and the second path is set to a module with both channel and spatial attention mechanisms. The first path consists of three layers of convolution operations. Based on the performance of the first and second branches with fewer channels and more global context information, we choose to add an HDC layer after the first two output branches. Assuming that the dilated convolution rate is d and the original convolution kernel size in the network is k×k, the dilated convolution kernel size n is n=k+(k-1)×(d-1) (1) Since the dilated convolution will cause the loss of continuity of feature pixel reading while expanding the receptive field of feature information, the pixels involved in the convolution operation are all the effective weight positions of the expanded convolution kernel, which will cause a grid effect and lose some valuable feature information, thus affecting the accuracy of the final network classification prediction. Therefore, dilated convolution with different expansion coefficients is used and the three-layer convolution operation is combined into one path. In order to fill the lost pixels to the maximum extent and ensure the comprehensiveness of information representation, the maximum expansion coefficient of the current layer must not be greater than the original convolution kernel size of the network, that is, the condition is satisfied. M i ≤k (2) M i =Max[M i+1 -2d i ,M i+1 -2(M i+1 -d i ),d i ] (3) Among them, M i is the maximum expansion coefficient of the i-th layer, d i is the hole convolution rate of the i-th layer, and Max is the maximum value among the three layers, which can improve the weak feature representation ability and poor information globality brought by ordinary conventional convolution in the first two branches; The second path is as follows: add a parallel attention module PAM after the output of the third branch and the fourth branch. Based on the fact that the latter two branches have more channels and therefore carry more comprehensive image detail information, the channel attention mechanism is integrated with the spatial attention mechanism, so as to assign corresponding weights to the representations of different channels and the positions of pixels, so that the pixel information related to classification is more prominent in network training. The output features of the latter two branches are subjected to maximum pooling, average pooling and median pooling operations to obtain the feature vectors for channel and space weight calculation; first use 1×1 convolution to adjust the number of channels to 256 and perform data batch normalization, reduce the dimension after combining the feature vectors output by the two attention mechanisms, and finally obtain and assign the weight values ​​of 0-1 to the two branches through sigmoid activation. The generated channel and spatial attention maps are calculated as follows: Out channel =W(MLP*(in Avgpool +in Maxpool +in Medpool )) (4) Out spatial =W(f*(in Avgpool ,in Maxpool ,in Medpool )) (5) In the above formula, Out channel is the channel attention map, Out spatial is the spatial attention map, W is the sigmoid activation function, MLP is a multi-layer perceptron for sharing weights, f is a 3×3 convolution operation, in represents the output features of the last two branches, Out branch The final branch feature is obtained by multiplying the channel weight and spatial weight with the output feature and then adding them together; The two paths use feature fusion to supplement the low-level features with semantic information and add spatial information to the high-level features; Through the above improvements to the HRnet high-resolution basic network, the joint positions of the quadruped were first obtained and then connected one by one to form a skeleton model; Step (3) extracting animal motion parameters: predicting the positions of key points of the quadruped in the video shot in step (1) by using the model established in step (2) to extract the motion parameters of the quadruped; Step (4) establishing a quadruped robot model: applying the physical structure information of the quadruped to the overall structural design of the robot, and combining the motion parameters of the quadruped extracted in step (3) to perform forward kinematic modeling and dynamic analysis of the robot; Step (5) Simulation of robot diagonal trotting gait: The robot's foot trajectory is planned, and the planned ideal trajectory is compared with the actual motion trajectory of the quadruped bionic robot to obtain a reliable robot model. Finally, the robot model is simulated by MATLAB to simulate the animal motion state.

2. The method for simulating animal gait by a vision-based quadruped bionic robot according to claim 1, characterized in that: In step (1), a video camera is used to shoot a video.

3. The method for simulating animal gait by a vision-based quadruped bionic robot according to claim 1, characterized in that: In step (3), the specific steps of extracting motion feature parameters are as follows: ①Same-side step parameters: After obtaining the coordinate positions of the four corner points in a gait cycle of the research subject in the model, the average walking distance of the ankle joints of the four limbs relative to themselves is calculated by locating the change in the Euclidean distance of the two joint coordinates when the ankle joints of each limb are farthest and closest to the image origin. The f i Frame and f i+1 The coordinates of the ankle joint of the left forelimb of the animal in the frame image are is the coordinates of the right forelimb ankle joint of the animal, is the coordinates of the animal's left hind limb ankle joint, is the coordinate of the right hind limb ankle joint of the animal, and the step length l is calculated as In the formula is the conversion coefficient between image pixels and actual distance; ②Joint angle parameters: Select any joint point and assume that the animal hip joint coordinates finally recognized by the model are (x a ,y a ), the knee joint coordinates are (x b ,y b ), the ankle joint coordinates are (x c ,y c ), based on the coordinate change between the two joint points, the three distances a, b, and c can be obtained through the square difference of the coordinate vectors, and the calculation formula for the joint angle α is: The animal's joint angle change parameters are calculated using equations (8) to (11).

4. The method for simulating animal gait by a vision-based quadruped bionic robot according to claim 1, characterized in that: In step (4), the specific steps of establishing the quadruped robot model are as follows: ①Robot body structure design: Through the analysis of the skeleton and muscle texture of quadrupeds, combined with the characteristics of quadrupeds in running, the full elbow mechanical structure is used as the body structure of the bionic robot. One degree of freedom is set at the hip joint connecting the first and second sections, and one degree of freedom is set at the knee joint connecting the second and third sections. The limbs of the bionic robot have a total of eight degrees of freedom: ②Robot kinematics modeling The robot consists of a world coordinate system {W}, a body coordinate system {B}, and a leg coordinate system {L}; the leg coordinate system also includes a hip joint coordinate system {1}, a knee joint coordinate system {2}, and a foot end coordinate system {3}. The coordinate transformation matrix of two adjacent systems in the coordinate system set by the robot is calculated by using the DH parameters. After the plane translation and spatial rotation of the two adjacent coordinate systems, the overall transformation matrix formula is: in, Represents the coordinate transformation matrix between the nth coordinate system and the n+1th coordinate system. Rot() and Trans represent the rotation and translation operations on the X-axis and Z-axis. i represents the x-axis, α i Represents the rotation angle in the coordinate space, θ i is the angle of rotation of the joint coordinate system, a i Represents the length of the connecting rod between two joints, d i Represents the offset distance between the i-th coordinate system and the i+1-th coordinate system; Substituting the DH parameters into equation (10) yields The rotation matrix of the coordinate system relative to the Z axis is: The translation matrix of the coordinate system relative to the X-axis is From equations (12) and (13), we can get the coordinate transformation matrix of the foot end coordinate system {3} of the quadruped bionic robot relative to the body coordinate system {B} as follows: because The coordinate transformation matrix can be simplified as the following formula In the above formula is the rotation matrix, is the translation matrix. According to the above invention, the connecting rod length between the hip joint and the knee joint of the robot is set to be 20 cm, and the connecting rod length between the knee joint and the ankle joint is set to be 18 cm. Then, the forward kinematic modeling formula of the quadruped bionic dog robot is obtained from formula (15): The robot sets the connecting rod length between the hip joint and the knee joint as l1, and the connecting rod length between the knee joint and the ankle joint as l2; ③Robot dynamics analysis Assuming that the masses of connecting rod 1 and connecting rod 2 are m1 and m2 respectively, the moment of inertia of connecting rod 1 and connecting rod 2 can be obtained as follows Then take the center point of each connecting rod as the center of mass, determine the center of mass position of connecting rod 1 and connecting rod 2, and calculate the center of mass coordinates in the x and y directions. The formula is r 2x =l1cos(θ1(t))+(l2cos((θ1(t)+θ2(t))) / 2 (23) r 2y =l1sin(θ1(t))+(l2sin((θ1(t)+θ2(t))) / 2 (24) The resulting velocities of the two connecting rods are is the angular velocity of the first connecting rod, represents the angular velocity of the second connecting rod; The total kinetic energy E of the robot is the sum of the kinetic energy of link 1 and link 2. The formula result is c2 represents cosθ2; The total potential energy of the robot is the sum of the potential energies of the two links, and the calculation result is s1 represents sinθ1,s 12 represents sinθ1θ2; From the Lagrange equation, we know that the Lagrange function is equal to the difference between the robot's kinetic energy and potential energy. Based on the Lagrangian function l, θ1, θ2, Find the partial derivative and convert the robot's angular acceleration The relationship between the robot joint torque τ1, τ2 is expressed through a matrix, and the change of the relationship matrix can be seen more intuitively: ④Robot foot trajectory planning The composite cycloid and Bezier curve method is used to plan the curve between the starting point and the landing point of the robot foot in a motion cycle. The calculation formula of the composite cycloid is: Formula (31) is the trajectory of the swing phase of the quadruped bionic robot in the X and Z directions, and Formula (32) is the trajectory of the support phase of the quadruped bionic robot in the X and Z directions, where x f With x s are the landing point and starting point of the robot in the X-axis direction, z s is the starting point of the robot on the Z axis, T in formula (33) f is the time the robot is in the support phase, and T represents a motion cycle; The Bezier curve is gradually generated by many control points in a recursive form. Compared with the compound cycloid, the Bezier curve has better speed and acceleration continuity. Therefore, the third-order Bezier curve is used for robot path planning. The formula is: B(t)=p0(1-t) 3 +3p1t(1-t) 2 +3p2t 2 (1-t)+p3t 3 ,t∈[0,1] (32) In formula (34), p0, p1, p2, and p3 are the four control points of the curve, and a better foot trajectory can be obtained by adjusting the control points.

5. The method for simulating animal gait by a quadruped bionic robot based on vision according to claim 4, characterized in that: The four control points are set as the robot's starting point (-0.11, -0.36), the highest point (0, -0.27), the second highest point (0.05, -0.28), and the landing point (0.11, -0.36).

6. The method for simulating animal gait by a vision-based quadruped bionic robot according to claim 1, characterized in that: In step (5), the simulink function of MATLAB R2021a is used to simulate the animal motion state of the robot model.

Images