A method for evaluating the motion smoothness of a rat robot

Abstract

The present invention discloses a method for evaluating the motion smoothness of a rat robot. A key point sequence capable of representing the motion state of the rat robot is extracted from the motion video sequence of the rat robot. The global feature field, the motion posture field, and the motion speed field are respectively used to measure the smoothness of the rat robot in the motion video sequence from different aspects, and the obtained global feature field score, motion posture field score, and motion speed field score are used to quantify the motion characteristics of the rat robot. The global feature field score, the motion posture field score, and the motion speed field score are fused, and the fusion result is used to represent the overall motion smoothness of the rat robot. The present invention creates a feature fusion field to quantify the motion smoothness. Different feature fields have a certain complementarity, and the most important information is captured simultaneously from the overall law and local law of motion.

Description

Technical Field

[0001] The present invention relates to the technical field of robot motion evaluation, and particularly relates to a method for evaluating the motion fluency of a rat robot. Background Art

[0002] Existing research on animal robots has covered invertebrates (such as insects like bees, tobacco moths, beetles, etc.) and vertebrates (such as rats, pigeons, geckos, etc.). Among them, rats are selected as important model animals in the laboratory due to their superior perception ability, easy reproduction, low cost, and complete understanding of brain structure and function. Currently, electrical stimulation is the main way to control rat robots. During the perception and motion control process of rat robots, environmental information is captured through worn cameras or sensors, and then electrical stimulation command decisions are made based on visual data and sensor data to control the motion of rat robots and complete complex tasks.

[0003] In the absence of electrical stimulation, rats move naturally according to their own will, with coordinated limbs and a stable body. After adding electrical stimulation, due to the influence of individual differences in rats, voltage amplitude, stimulation frequency, site implantation accuracy, etc., their movements may become abnormal. For example, there may be stiff action switching, instant bouncing, twitching, collisions, spastic torsion of the head, uncoordinated limbs, etc., which are significantly different from the characteristics of natural movements. If the rat robot cannot move smoothly, its behavioral movements will become clumsy, which will lead to low work efficiency for tasks.

[0004] Therefore, it is very necessary to evaluate the motion fluency of rat robots. In current experiments, researchers usually rely on empirical observation to judge the state of rat robots, such as observing motion videos or data. However, this manual observation method is subjective, consumes a large amount of time and energy, and lacks quantification. By using the manual observation method to evaluate the motion state, only relative evaluations can be obtained, and accurate numerical values cannot be obtained. This is not conducive to accurately evaluating the motion quality and statistical analysis of the evaluation results. Summary of the Invention

[0005] The purpose of the present invention is to provide a method for evaluating the motion fluency of a rat robot, so as to solve the technical problems in the prior art that the manual observation method is subjective, consumes a large amount of time and energy, and lacks quantification. By using the manual observation method to evaluate the motion state, only relative evaluations can be obtained, and accurate numerical values cannot be obtained. This is not conducive to accurately evaluating the motion quality and statistical analysis of the evaluation results.

[0006] To solve the above technical problems, the present invention provides a method for evaluating the motion fluency of a rat robot, including the following steps:

[0007] Step 100: Extract a key point sequence that can represent the motion state of the rat robot from the motion video sequence of the rat robot;

[0008] Step 200: Use the global feature field GFF, the motion pose field MPF, and the motion velocity field MVF to represent the smoothness of the rat robot in the motion video sequence, and quantify the global feature field GFF, the motion pose field MPF, and the motion velocity field MVF respectively. Use the obtained global feature field score S GFF , the motion pose field score S MPF , and the motion velocity field score S MVF to quantify the motion characteristics of the rat robot;

[0009] Step 300: Fuse the three feature description fields of the global feature field score S GFF , the motion pose field score S MPF , and the motion velocity field score S MVF , and use the fusion result to represent the overall motion smoothness of the rat robot.

[0010] As a preferred embodiment of the present invention, in the step 100, a key point sequence of the rat robot is extracted by using a pose estimation model, wherein the pose estimation model includes a backbone unit for feature extraction and a head unit composed of upsampling modules;

[0011] To obtain the key point sequences of the right front limb, left front limb, right hind limb, left hind limb, head, tail root, neck, and middle of the spine of the rat robot.

[0012] As a preferred embodiment of the present invention, the resolution of the feature map formed by the backbone unit is

[0013] The input of the backbone unit is the batch-processed image I (I ∈ R B×C×H×W ), and the output is the feature map where B represents the batch size, C is the number of channels, H and W respectively represent the height and width of the input image I, and R represents the domain.

[0014] As a preferred embodiment of the present invention, the head unit gradually restores the size of the feature map of the rat robot to by performing multiple upsampling module processes and convolutional neural network operations on the motion video sequence of the rat robot to obtain the sequence of 8 key points of the rat robot. The specific implementation method is:

[0015] Input the feature map F 0 into the i-th upsampling module using the upsampling module and the DOConv layer for processing;

[0016] Establish the correlation relationship of the feature maps between multiple said upsampling modules;

[0017] Use the attention module to calculate the attention map, and fuse the feature map and the attention map to form a fused feature map. Among them, the specific fusion formula of the fused feature map is:

[0018]

[0019] Among them, F i is the fused feature map output by the i-th upsampling module, is the feature map output by the i-th upsampling module. CBAM(·) models the image features of the said feature map through a convolutional neural network to obtain the attention map.

[0020] As a preferred embodiment of the present invention, perform batch normalization operation on the feature map obtained by the said upsampling module to obtain the feature map R represents the domain;

[0021] There is a correlation relationship between multiple said upsampling modules. Among them, the relationship formula of the feature maps of two adjacent said upsampling modules is specifically:

[0022] Among them, (F i-1 ) is the fused feature map output by the (i - 1)-th upsampling module.

[0023] As a preferred embodiment of the present invention, in step 200, use the global feature field GFF to construct the distribution model of each said key point sequence in the motion video sequence to distinguish the normal motion sequence and the abnormal motion sequence of the rat robot in the motion video sequence;

[0024] Taking the limb coordination and body stability of the rat robot as the auxiliary criteria for evaluation, use the motion pose field MPF to represent the pose change of the rat robot between adjacent frames in the motion video sequence, and use the motion velocity field MVF to represent the velocity change of the rat robot in the frequency domain of the motion video sequence.

[0025] As a preferred embodiment of the present invention, in the said step 200, the implementation manner of using the global feature field GFF to construct the distribution model of each said key point sequence is:

[0026] Obtain the motion sequence sample of the rat robot, perform principal component analysis and dimensionality reduction on the motion sequence sample, map the high-dimensional sampling data of each said key point sequence to a low-dimensional space to form a dimensionality-reduced sample, and represent each said key point sequence using two-dimensional coordinates;

[0027] Label the dimensionality-reduced samples of each of the key point sequences within the spatial coordinate axes, calculate the distribution center of the dimensionality-reduced samples, and denote the distribution center as u;

[0028] Based on the probabilities that all abnormal samples are determined to be abnormal, select a radius threshold δ for distinguishing normal motion sequence samples and abnormal motion sequence samples;

[0029] Take the distance between the new sample G new and u as the global feature field score S GFF , to quantify the smoothness of the new sample G to be measured new , where the calculation formula of the global feature field score S GFF is:

[0030] S GFF = ||g new - u||;

[0031] where g new is the dimensionality-reduced sample formed by the new sample G new in the low-dimensional space.

[0032] As a preferred embodiment of the present invention, compare the result obtained from the global feature field score S GFF with the radius threshold δ to determine whether the new sample G to be measured new is a normal operation sample or an abnormal operation sample.

[0033] As a preferred embodiment of the present invention, for the motion video sequence with T frames of the motion pose field MPF, calculate the pose distance between any two adjacent frames p, q ∈ R n*2 to evaluate the motion smoothness of the rat robot, and the specific calculation method is:

[0034] Use M p to represent the distance between each key point in the state of the p frame and other key points, and form a difference matrix of all key point sequences, where M p is specifically an 8*8 matrix;

[0035] Use M q to represent the distance between each key point in the state of the q frame and other key points, and form a difference matrix of all key point sequences, where M q is specifically an 8*8 matrix;

[0036] Calculate the quasi-GW distance between the two groups of pose sequences Mp and Mq:

[0037] d qGW (p,q) = ||M p - Mq || F ;

[0038] wherein, ||.|| F represents the Frobenius norm, and the difference matrix of the p-frame state is denoted as M p ∈R n*n ; M p The element M i,j =||p i -p j || is the Euclidean distance between the i-th and j-th key points in the corresponding pose sequence p.

[0039] As a preferred embodiment of the present invention, the motion pose field MPF is used to measure the pose difference between adjacent frames within the sliding window of the motion video sequence, obtain the i-th frame in each key point sequence of the rat robot, and use the neighborhood window τ, which is expressed as the number of frames before and after the current i-th frame with the i-th frame itself as the center, select the mean value of smooth motion as the threshold σ, find the number of difference values greater than the threshold in more than half of the neighborhood window τ, and the evaluation expression of the motion pose field MPF is:

[0040]

[0041] wherein, σ is the mean value of smooth motion, δ is an indicator function, when the pose difference value is greater than σ, δ is 1; otherwise, δ is 0, τ is the neighborhood window size, both i and j are variables, i = τ, 2,......, T - τ, j = 1, 2,......, 2τ + 1, and T is the total number of frames of the motion sequence;

[0042] The motion pose field MPF can calculate the moment of obvious pose switching in the sequence pose by calculating the pose difference between adjacent two frames.

[0043] As a preferred embodiment of the present invention, the motion velocity field MVF is used to express the jitter phenomenon of the velocity history feature of each key point sequence of the rat robot transformed into the frequency domain. The specific implementation method is:

[0044] Obtain the velocity information of all T frames of each key point sequence, denoted as v T , perform a fast Fourier transform FFT on the velocity of each frame to obtain the frequency domain velocity spectrum V F =FFT(v T ), and MVF measures the jitter by calculating the sum of the amplitudes of the effective frequency band [α, β], expressed as:

[0045]

[0046] where V F (f o)represents a complex number with a frequency of f o S MVF The magnitude of MVF indicates the smoothness of the rat robot's movement in terms of speed.

[0047] As a preferred embodiment of the present invention, obtaining v T is achieved as follows:

[0048] Taking the average coordinates of n key points on the horizontal plane of the rat robot as the centroid, its movement trajectory is converted into a time series composed of two-dimensional centroid coordinates, which contains statistical data such as position, speed, distance, and movement time, and can be used to measure its movement state;

[0049] Among them, the central coordinates of all key points are expressed as: p i Specifically, it is the coordinate of the key point corresponding to the i-th frame;

[0050] Approximately calculate the speed v of the current frame using the finite difference method, and the calculation is v t = ||s t+1 - s t ||, where s t represents the central coordinate at time t.

[0051] As a preferred embodiment of the present invention, in step 300, the global feature field GFF, the motion pose field MPF, and the motion speed field MVF are combined to jointly characterize the motion smoothness, and the motion smoothness is accurately judged by weighting the global features and the significant difference features. The specific implementation method is as follows:

[0052] Step 301: Calculate the smoothness S new of the sample G GFF , S MPF and S MVF under different feature fields respectively, and normalize the values of each field to ensure that the scores of each aspect are within the numerical range of (0, 1);

[0053] Step 302: Use the smoothness evaluation function S FLU to fuse the scores of different feature fields. The calculation formula is:

[0054] S FLU = ηS GFF + λS MPF + μS MVF ;

[0055] Among them, the constraint is η + λ + μ = 1, and η, λ, μ are constants, representing the weights of the corresponding features in the similarity calculation.

[0056] The present invention has the following beneficial effects compared with the prior art:

[0057] In view of the problems of rigid action switching and unnatural movement behavior that occur during the control of a rat robot, the present invention introduces a motion smoothness evaluation index, creates a feature fusion field to quantify motion smoothness, and different description fields have a certain degree of complementarity. Among them, the global feature field GFF establishes a distribution model for the samples formed by the motion video sequence to determine the range of smooth motion and calculates the possibility of the sequence to be tested outside the ideal range. Differential features are extracted through the motion pose field MPF and the motion velocity field MVF, and the kinematic laws are quantified by the pose changes between adjacent frames and the velocity changes in the frequency domain. The most important information is captured simultaneously from the overall and local laws of motion. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only exemplary, and for those of ordinary skill in the art, without creative efforts, other implementation drawings can be obtained according to the provided drawings.

[0059] Figure 1 It is a schematic flowchart of the method for evaluating the motion smoothness of a rat robot provided by an embodiment of the present invention;

[0060] Figure 2 It is a distribution schematic diagram of normal sequence samples and abnormal sequence samples of a rat robot in a motion video sequence provided by an embodiment of the present invention; DETAILED DESCRIPTION OF THE EMBODIMENTS

[0061] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0062] As Figure 1 shown, the present invention provides a method for evaluating the motion smoothness of a rat robot, including the following steps:

[0063] Step 100: Extract a key point sequence from the motion video sequence of the rat robot that can represent the motion state of the rat robot.

[0064] It should be noted that accurately estimating the body key points of the rat robot is the basis for analyzing and evaluating its motion state. However, due to the problem that the area of the rat in the collected motion video sequence is very small, in order to accurately detect its key points for evaluating the motion control state of the rat, a pose estimation model for small target rats is proposed based on saliency detection and structural similarity constraints, aiming to solve the problem of extracting the features of the rat robot in the case of a small foreground, and at the same time having the advantages of high efficiency, small number of parameters, and fast speed.

[0065] Specifically, in the step 100, the key point sequence of the rat robot is extracted by using the pose estimation model. Among them, the pose estimation model includes a backbone unit for feature extraction and a head unit composed of upsampling modules; eight key point sequences of the right front limb, left front limb, right hind limb, left hind limb, head, tail root, neck, and middle of the spine of the rat robot are obtained.

[0066] Among them, the backbone unit specifically uses the pre-trained ResNet-50, which includes a downsampling operation and four pooling layers.

[0067] The input of the backbone unit is the batch-processed image I (I ∈ R B×C×H×W ), and the output is the feature map where B represents the batch size, C is the number of channels, H and W respectively represent the height and width of the input image I, and the resolution of the feature map finally formed by the backbone unit is

[0068] The head unit gradually restores the size of the feature map of the rat robot to by performing multiple (specifically, four) upsampling module processes and convolutional neural network operations on the motion video sequence of the rat robot to obtain the sequence of 8 key points of the rat robot. The specific implementation method is as follows:

[0069] (1) Input the feature map F 0 into the i-th upsampling module for processing, and perform batch normalization operation on the feature map obtained by the upsampling module to obtain the feature map R represents the domain.

[0070] (2) Establish the feature map association relationship between multiple upsampling modules. There is an association relationship between multiple upsampling modules. The feature map relationship formula between two adjacent upsampling modules is specifically:

[0071] Among them, (F i-1) is the fused feature map output by the (i - 1)-th upsampling module.

[0072] (3) Use the attention module to calculate the attention map, and fuse the feature map and the attention map to form a fused feature map. The specific fusion formula of the fused feature map is:

[0073]

[0074] where F i is the fused feature map output by the i-th upsampling module, is the feature map output by the i-th upsampling module. CBAM(·) models the image features of the feature map through a convolutional neural network to obtain the attention map.

[0075] After the above three steps of processing, the predicted heat map of the rat robot is:

[0076] It should be noted that this embodiment proposes an improved upsampling module for the head part. The improved upsampling module does not use a common transposed convolution layer, but introduces the CBAM function and the DOConv module to replace the transposed convolution layer, which can further improve the feature processing ability and execution efficiency of the pose estimation model.

[0077] The pose estimation module designed in this embodiment is mainly used to solve the problem of too small target foreground. Among them, the CBAM function is not only a lightweight general module, which can be well combined with the DOConv module to further improve the model's ability to recognize key point features. Through this method, even in the case of very few reference parameters, the key point features can be accurately recognized.

[0078] Furthermore, in order to facilitate the motion evaluation modeling of the rat robot, this embodiment models the motion pattern based on the spatio-temporal relationship of the key point sequence. Specifically, the problem of kinematic modeling is carried out through commonality and difference. Considering that smooth motion should have a stable pose and good joint coordination, in order to characterize the overall law of the motion sequence, a distribution model of each key point sequence of the rat robot in the motion video sequence is established using the global feature field GFF. At the same time, the motion pose field MPF and the motion velocity field MVF are used as auxiliary criteria at the expert knowledge level to quantify the differential features in space-time (including space and time).

[0079] Step 200: Represent the smoothness of the rat robot in the motion video sequence using the global feature field GFF, the motion pose field MPF, and the motion velocity field MVF, and quantify the global feature field GFF, the motion pose field MPF, and the motion velocity field MVF respectively. Use the obtained global feature field score S GFF, the motion posture field score S MPF , and the motion speed field score S MVF to quantify the motion characteristics of the rat robot.

[0080] In step 200, a distribution model of each key point sequence in the motion video sequence is constructed by using the global feature field GFF to distinguish the normal motion sequence and the abnormal motion sequence of the rat robot in the motion video sequence.

[0081] Taking the limb coordination and body stability of the rat robot as auxiliary evaluation criteria, the motion posture field MPF is used to represent the posture change of the rat robot between adjacent frames in the motion video sequence, and the motion speed field MVF is used to represent the speed change of the rat robot in the frequency domain of the motion video sequence.

[0082] Among them, the implementation method of constructing the distribution model of each key point sequence by using the global feature field GFF is as follows:

[0083] (1) Obtain the motion sequence sample of the rat robot, perform principal component analysis dimensionality reduction on the motion sequence sample, map the high-dimensional sampling data of each key point sequence to a low-dimensional space to form a dimensionality-reduced sample, and represent each key point sequence using two-dimensional coordinates.

[0084] Specifically, all the key point sequences in the motion sequence sample are represented as: P i = [p 1 , p 2 ,... p j ,... p T , and each sample has d = T * n * 2 dimensions; where T represents the total number of frames in the sequence, and p j ∈ R n*2 represents the key point coordinates corresponding to the j-th frame, R represents the domain; n is the number of key point sequences, which takes the value of 8 in this embodiment.

[0085] Among them, in a motion video sequence with a constant number of frames T, each frame of the rat can be represented by eight two-dimensional key point coordinates (the number of key points n = 8). For example, the key point coordinates of the rat corresponding to the i-th frame in the motion video sequence are The key point coordinates of the rat corresponding to the i + 1-th frame are Therefore, a motion can be represented as a sequence of T * n * 2.

[0086] Preprocess all motion sequence samples formed by all key point sequences to ensure that all motion sequence samples are on the same scale; perform principal component analysis on each motion sequence sample to screen out parameters that can retain important energy information, so as to map high-dimensional sampling data to a low-dimensional space.

[0087] It should be particularly noted that in the low-dimensional space, the smooth motions and abnormal motions of the rat robot show different distributions. Therefore, video segments of the motion video sequences of the rat robot and video segments of abnormal motions can be screened out.

[0088] (2) Label the dimensionality-reduced samples of each of the key point sequences within the spatial coordinate axes, and project the original sample P i onto the low-dimensional space formed by the selected eigenvectors to obtain the dimensionality-reduced sample U i , calculate the distribution center of the dimensionality-reduced sample U i , and denote the distribution center as u.

[0089] Based on the probabilities that all abnormal samples are determined to be abnormal, select a radius threshold δ for distinguishing normal motion sequence samples and abnormal motion sequence samples;

[0090] (3) Take the distance between the new sample G new and u as the global feature field score S GFF to quantify the smoothness of the new sample G new to be measured. Among them, the calculation formula for the global feature field score S GFF is:

[0091] S GFF = ||g new - u||;

[0092] where g new is the dimensionality-reduced sample formed by the new sample G new in the low-dimensional space.

[0093] Compare the result obtained from the global feature field score S GFF with the radius threshold δ to determine whether the new sample G new to be measured is a normal operation sample or an abnormal operation sample.

[0094] As Figure 2 shown, the normal samples and abnormal samples in the motion video of the rat robot show a concentric circle distribution law. According to this law, calculate the position where a sample sequence to be measured falls within the concentric circles, so as to judge whether the sequence is smooth or abnormal, and its smoothness can be quantified according to the distance from the central position. The global feature field score S GFF is the quantified smoothness.

[0095] Therefore, after determining the normal distribution range of the rat robot, by comparing the motion video sequence of the rat robot with the normal distribution range, we can not only quantify the smoothness of the motion of the sample, but also identify abnormal samples.

[0096] Among them, the implementation steps for determining the normal distribution range are as follows: screen out the video segments of the motion video sequence of the rat robot and the video segments of abnormal motions, perform PCA dimensionality reduction on all normal motion sequence samples and abnormal motion sequence samples of the rat robot, label the normal motion sequence samples and abnormal motion sequence samples in the two-dimensional space coordinates, and it is found that the normal motions and abnormal motions show a concentric circle distribution.

[0097] Among them, the normal motion samples are distributed in the outer circle, and the abnormal motion samples are distributed near the center. Therefore, it is determined whether it is abnormal according to the distance of the sample from the center point. The closer to the center point, the higher the degree of abnormality; the farther from the center point, the smoother the motion.

[0098] And it is very important to determine the appropriate radius threshold δ for anomaly detection. If the radius is small, abnormal samples may be misjudged as normal; conversely, if the radius is large, normal samples may be misjudged as abnormal.

[0099] Since the video segments of the motion video sequence of the rat robot and the abnormal motions are known to be screened out, we verify the known abnormal sample G abnormal The probability Pr(G abnormal ∈Neg) belonging to the abnormal category (Neg) to determine the radius range. When the radius is δ, the probability Pr(G abnormal ∈Neg)>=Г that all abnormal samples are judged as abnormal. It can be considered that the abnormal samples are correctly identified as abnormal and meet the set probability Г. At this time, the corresponding radius threshold δ can be determined.

[0100] The purpose of identifying abnormal samples is to determine the appropriate radius threshold, so that the method can not only judge whether the sample is smooth motion or abnormal motion, but also quantify the smoothness or abnormality degree according to the distance from the center.

[0101] It should be particularly noted that since there is a risk probability in the identification of abnormal samples and normal samples, although the global feature field GFF can measure significantly abnormal motion segments, it cannot accurately identify a small number of motion segments with unobvious abnormalities. Therefore, in order to further improve the anomaly detection performance, we propose two additional auxiliary indicators, namely the motion pose field MPF and the motion velocity field MVF, as supplementary information to the global features.

[0102] In addition to comparing the commonalities of joint movements, motion stability is also important for motion assessment. In a neighborhood window, significant changes in the poses of two adjacent frames indicate a lack of stability. Therefore, we introduce the Motion Pose Field (MPF) to measure the pose differences between adjacent time steps, thereby detecting smooth motions.

[0103] Construct a skeleton sequence from the eight key points of the rat robot, and abstract the rat robot into a two-dimensional representation described by pose key points. Use the GW distance to establish a difference matrix for the poses of all key points of the rat robot to measure the difference between any two poses. The specific calculation method is as follows:

[0104] For the motion video sequence with T frames, the Motion Pose Field (MPF) calculates the distance between two poses p, q ∈ R of any adjacent frames n*2 to evaluate the motion smoothness of the rat robot. The specific calculation method is as follows:

[0105] Use M p to represent the distance between each key point in the p-frame state and other key points, forming a difference matrix for all key point sequences, where M p is specifically an 8×8 matrix;

[0106] Use M q to represent the distance between each key point in the q-frame state and other key points, forming a difference matrix for all key point sequences, where M q is specifically an 8×8 matrix;

[0107] Calculate the quasi-GW distance between the two sets of pose sequences Mp and Mq:

[0108] d qGW (p, q) = ||M p - M q || F ;

[0109] where ||.|| F represents the Frobenius norm. The difference matrix in the p-frame state is denoted as M p ∈R n*n ; The element M p in M i,j = ||p i - p j || corresponds to the Euclidean distance between the i-th and j-th key points in the corresponding pose p.

[0110] The motion posture field MPF calculates the distance (or difference) between two poses. The pose of the pth frame can be represented by the distance between the eight key points, that is, the distance between the first key point and the 1st, 2nd, 3rd, 4th, 5th, 6th, 7th, and 8th key points is M. 1,j , the distances between the second key point and the first, second, third, fourth, fifth, sixth, seventh, and eighth key points are M 2,j , and so on, until the distance between the 8th key point and the 1st, 2nd, 3rd, 4th, 5th, 6th, 7th, and 8th key points is M respectively 8,j , then, the pose at the i-th moment can be represented by the 8*8 matrix M p Similarly, the pose of the i+1th frame can also be represented by the distance between the eight key points, and finally the 8*8 matrix M can be used. q express.

[0111] The motion posture field MPF evaluates the smoothness by calculating the distance (or difference) between the poses of adjacent frames p and q. If the distance between the poses of adjacent frames is large, it means that the rat's posture changes greatly. If the distance between the poses of adjacent frames is small, it means that the rat's posture changes little and the movement is smoother.

[0112] Under the control of electrical stimulation, if the rat robot moves abnormally, the movement is often more violent and the abnormal reaction is larger, so it can be quantified by the pose distance between adjacent frames. A motion sequence has T frames, so the motion posture field MPF proposes to use the sliding window method to find the number of posture difference values ​​in the window that are greater than the threshold for more than half.

[0113] Therefore, the motion posture field MPF is used to measure the posture difference between adjacent frames in the sliding window of the motion video sequence, and the i-th frame in each key point sequence of the rat robot is obtained. The i-th frame itself is taken as the center, and the neighborhood window τ is used to represent the number of frames before and after the current i-th frame. The mean value of smooth motion is selected as the threshold σ, and the number of difference values ​​in the neighborhood window τ that are more than half greater than the threshold is found. The motion posture field MPF evaluation expression is:

[0114]

[0115] Wherein, σ is the mean value of smooth motion, δ is the indicator function, when the posture difference value is greater than σ, δ is 1; otherwise, δ is 0, τ is the neighborhood window size, i and j are variables, i = τ, 2, ..., T-τ, j = 1, 2, ..., 2τ + 1, T is the total number of frames in the motion sequence;

[0116] The motion pose field MPF can calculate the moments of significant pose switches in the sequence of poses by computing the pose differences between adjacent frames, and can timely detect segments with large differences from natural motions.

[0117] Although the time-domain changes in the rat skeleton structure are important for describing motion smoothness and stability, the frequency-domain changes in the rat skeleton structure are also useful. For example, when a rat is moving, the center of mass can be used to measure the degree of body spasmodic jitter, and the velocity characteristics of the center of mass have superiority in the recognition of video sequences of complex activities. Therefore, the motion velocity field MVF is introduced to extend the velocity history features in the frequency-domain changes.

[0118] To obtain the motion velocity, the average coordinates of n key points on the horizontal plane of the rat robot are used as the center of mass, and its motion trajectory is converted into a time series composed of two-dimensional center-of-mass coordinates, which contains statistical data such as position, velocity, distance, and motion time, and can be used to measure its motion state;

[0119] Among them, the central coordinates of all key points are expressed as: p i represents the coordinates of a certain key point corresponding to the i-th frame.

[0120] The velocity v of the current frame is approximately calculated by the finite difference method, and the calculation is v t =||s t+1 -s t ||, where s t represents the central coordinates at time t.

[0121] Specifically, when three typical abnormal motions such as spasm, collision, and dash occur, these features are significantly different from normal motions. The intuitive motion performance is that it is unable to move forward normally with alternating left and right limbs. Often, the two front limbs and the two hind limbs jump up and land simultaneously respectively. Another manifestation is that when moving, the left and right limbs alternate to move forward, but the limb movements are uncoordinated. One side of the two hind limbs lands gently and shows weakness, and the other side of the hind limb bears the weight, and the gait is short and unstable. The center of mass of the rat's body will sway when moving. Mapping the abnormal motions to the velocity, the characteristics are also different. During spastic motion, the curve of the motion velocity field MVF shows periodic fluctuations. During a collision, the curve shows a large peak. Finally, when the rat robot is abnormally excited and moves at a high speed for a long time, it indicates that it is dashing.

[0122] Due to the significant tremors of the body, the energy of some frequency bands of the velocity is affected. The velocity information of all T frames is denoted as v T . We perform a fast Fourier transform (FFT) on the velocity of each frame to obtain the frequency-domain velocity spectrum V F =FFT(v T)。The motion velocity field MVF measures jitter by calculating the sum of the amplitudes of the effective frequency band [α, β], expressed as:

[0123] where V F (f o ) represents the complex number with frequency f o . The magnitude of S MVF indicates the smoothness of the rat robot's motion in terms of speed.

[0124] Therefore, the spectral analysis method can calculate the energy of each frequency band. Since the normal and abnormal frequency components are different, it can judge whether abnormal motion occurs through the energy distribution characteristics, mapping the jitter changes that are difficult to describe mathematically in the time domain onto the spectral curve.

[0125] Due to the unpredictability and high variability of the abnormal motion of the rat robot, a single index is difficult to comprehensively define and characterize. Therefore, we propose to use the feature fusion field method to detect abnormal motion, that is, combining the global feature field GFF, the motion pose field MPF, and the motion velocity field MVF to jointly characterize the motion smoothness. By establishing an evaluation model, the weights of these three aspects can be automatically adjusted to make the weighted result closer to the true label.

[0126] Step 300: Fuse the three feature description fields of the global feature field GFF, the motion pose field MPF, and the motion velocity field MVF, and use the fusion result to represent the overall motion smoothness of the rat robot.

[0127] The specific fusion method is as follows: In step 300, combine the global feature field GFF, the motion pose field MPF, and the motion velocity field MVF to jointly characterize the motion smoothness, and accurately judge the motion smoothness by weighting the global features and significant difference features. The specific implementation method is:

[0128] Step 301: Calculate the smoothness degrees S new of the sample G GFF , S MPF , and S MVF under different feature fields respectively, and normalize the values of each field to ensure that the scores of each aspect are in the numerical range of (0, 1);

[0129] Step 302: Use the smoothness evaluation function S FLU to fuse the scores of different feature fields. The calculation formula is:

[0130] S FLU = ηS GFF + λS MPF + μS MVF ;

[0131] Among them, the constraint is η + λ + μ = 1, where η, λ, and μ are constants, representing the weights of the corresponding features in the similarity calculation.

[0132] The input of the evaluation model is the evaluation results of three feature fields, namely the global feature field GFF, the motion pose field MPF, and the motion velocity field MVF, and the output of the evaluation model is a regression score used to measure the overall motion smoothness of the rat robot.

[0133] In view of the problems of rigid action switching and unnatural motion behavior that occur during the control of the rat robot, the present invention introduces an evaluation index for motion smoothness, creates a feature fusion field to quantify the motion smoothness, and different description fields have a certain degree of complementarity. Among them, the global feature field GFF establishes a distribution model for the samples formed by the motion video sequence to determine the range of smooth motion and calculates the possibility of the sequence to be tested outside the ideal range. The differential features are extracted through the motion pose field MPF and the motion velocity field MVF, and the kinematic laws are quantified by the pose changes between adjacent frames and the velocity changes in the frequency domain. The most important information is captured simultaneously from the overall and local laws of motion, which has important practical significance for adjusting the control strategy of the animal robot, the smooth switching between motion states, improving the control algorithm, increasing the work efficiency, and increasing the task success rate.

[0134] The above embodiments are only exemplary embodiments of the present application and are not used to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art can make various modifications or equivalent replacements within the essence and protection scope of the present application, and such modifications or equivalent replacements should also be regarded as falling within the protection scope of the present application.

Claims

1. A method for evaluating the smoothness of rat robot movement, characterized in that, Including the following steps: Step 100: Extract a key point sequence that can represent the motion state of the rat robot from the motion video sequence of the rat robot; Step 200: Measure the smoothness of the rat robot in the motion video sequence from different aspects by using the global feature field (GFF), the motion posture field (MPF), and the motion velocity field (MVF) respectively, and quantify the GFF, MPF, and MVF respectively. Use the obtained global feature field score , the motion posture field score , and the motion velocity field score to quantify the motion characteristics of the rat robot; Use the global feature field GFF to construct a distribution model for each key point sequence in the motion video sequence to distinguish the normal motion sequence and the abnormal motion sequence of the rat robot in the motion video sequence; Taking the limb coordination and body stability of the rat robot as auxiliary criteria for evaluation, use the motion pose field MPF to represent the pose change of the rat robot between adjacent frames in the motion video sequence, and use the motion velocity field MVF to represent the velocity change of the rat robot in the frequency domain of the motion video sequence; Step 300: Fuse the global feature field score , the motion posture field score , and the motion speed field score of the three feature description fields, and use the fusion result to represent the overall motion smoothness of the rat robot.

2. The method for evaluating the motion smoothness of a rat robot according to claim 1, wherein In the step 100, a key point sequence of the rat robot is extracted by using a pose estimation model, wherein the pose estimation model includes a backbone unit for feature extraction and a head unit composed of upsampling modules; To obtain eight key point sequences of the right front limb, left front limb, right hind limb, left hind limb, head, tail root, neck and middle of the spine of the rat robot; Among them, the resolution of the feature map formed by the backbone unit is ; The input of the backbone unit is a batch of processed images ( ) and the output is a feature map ( ), where B represents the batch size, C is the number of channels, H and W respectively represent the height and width of the input image and R represents the domain.

3. The method for evaluating the motion smoothness of a rat robot according to claim 2, wherein The head unit gradually restores the feature map size of the rat robot to by processing the motion video sequence of the rat robot through multiple upsampling modules and convolutional neural network operations, so as to obtain a sequence of 8 key points of the rat robot. The specific implementation method is as follows: Use the upsampling module and the DOConv layer to process the feature map Input it into the th upsampling module for processing; Establish the feature map association relationship between multiple upsampling modules; Use the attention module to calculate the attention map, and fuse the feature map and the attention map to form a fused feature map, wherein the specific fusion formula of the fused feature map is: ， Among them, is the fused feature map output by the th upsampling module, is the feature map output by the th upsampling module, and the image features of the feature map are modeled by a convolutional neural network to obtain an attention map; Among them, for the said upsampling module perform batch normalization operation on the obtained feature map to obtain a feature map ; R represents a domain; There is an association relationship between multiple upsampling modules, and the feature map relationship formula between two adjacent upsampling modules is specifically: ; among them, is the fused feature map output by the th upsampling module.

4. The method for evaluating the motion smoothness of a rat robot according to claim 1, wherein In the step 200, the implementation manner of using the global feature field GFF to construct a distribution model for each key point sequence is: Obtain the motion sequence sample of the rat robot, perform principal component analysis and dimensionality reduction on the motion sequence sample, map the high-dimensional sampling data of each key point sequence to a low-dimensional space to form a dimensionality-reduced sample, and represent each key point sequence using two-dimensional coordinates; Label the dimensionality-reduced samples of each of the key point sequences within the spatial coordinate axes, calculate the distribution center of the dimensionality-reduced samples, and denote the distribution center as ; Based on the probabilities of all abnormal samples being determined as abnormal, a radius threshold is selected to distinguish normal motion sequence samples from abnormal motion sequence samples ; Take the distance between the new sample and as the global feature field score to quantify the smoothness of the new sample to be measured . Among them, the calculation formula of the global feature field score is as follows: ； Among them is the dimensionality reduction and cost reduction formed in the low-dimensional space; Compare the global feature field score with the obtained result and the radius threshold to determine whether the new sample to be tested is a normal operating sample or an abnormal operating sample.

5. The method for evaluating the motion smoothness of a rat robot according to claim 1 or 4, wherein For the motion video sequence with T frames, the motion pose field MPF calculates the pose distance between any two adjacent frames to evaluate the motion smoothness of the rat robot. The specific calculation method is as follows: Utilize Represent the distances between each key point in the p-frame state and other key points to form a difference matrix of all key point sequences, where Specifically, it is an 8*8 matrix; It represents the distance between each key point in the q-frame state and other key points, forming a difference matrix for all key point sequences, where Specifically, it is an 8*8 matrix; Calculate the quasi-GW distance between two groups of pose sequences Mp and Mq: ； Among them, represents the Frobenius norm, and the difference matrix of the p-frame state is expressed as ; The elements in correspond to the Euclidean distance between the i-th and j-th key points in the pose sequence .

6. The method for evaluating the motion smoothness of a rat robot according to claim 5, wherein The motion pose field MPF is used to measure the pose difference between adjacent frames within a sliding window of the motion video sequence, obtain the i-th frame in each of the key point sequences of the rat robot, and use a neighborhood window centered on the i-th frame itself denotes the number of frames before and after the current i-th frame, selects the mean value of smooth motion as the threshold σ, and searches for the number of difference values in the neighborhood window where more than half of them are greater than the threshold. The evaluation expression using the motion pose field MPF is as follows: ； Among them, σ is the mean value of smooth motion, and δ is an indicator function. When the pose difference value is greater than σ, δ is 1; otherwise, δ is 0. is the neighborhood window size, both i and j are variables, and i = , 2,......, T - , j = 1, 2,......, , and T is the total number of frames in the motion sequence. The motion pose field MPF can calculate the moment of obvious pose switching in the sequence pose by calculating the pose difference between two adjacent frames.

7. The method for evaluating the motion smoothness of a rat robot according to claim 1 or 6, wherein The motion velocity field MVF is used to express the jitter phenomenon of the velocity history feature of each key point sequence of the rat robot transformed to the frequency domain, and the specific implementation manner is: Obtain the velocity information of all T frames for each key point sequence, denoted as , perform a fast Fourier transform FFT on the velocity of each frame to obtain the frequency-domain velocity spectrum =FFT( ), and MVF measures the jitter by calculating the sum of the amplitudes of the effective frequency band , expressed as: wherein represents a complex number with a frequency of , and the magnitude of indicates the smoothness of the rat robot's movement in terms of speed.

8. The method for evaluating the motion smoothness of a rat robot according to claim 7, wherein Obtain The implementation method is as follows: Taking the average coordinates of n key points on the horizontal plane of the rat robot as the centroid, its motion trajectory is converted into a time series composed of two-dimensional centroid coordinates, which includes position, velocity, distance, and motion time statistical data, and can be used to measure its motion state; Among them, the central coordinates of all key points are expressed as: ; Specifically, they are the key point coordinates corresponding to the i-th frame. The velocity v of the current frame is approximately calculated using the finite difference method as , where represents the central coordinates at time t.

9. The method for evaluating the motion smoothness of a rat robot according to claim 1, wherein In step 300, the global feature field GFF, the motion pose field MPF, and the motion velocity field MVF are combined to jointly characterize the motion smoothness, and the global features and significant difference features are weighted to accurately judge the motion smoothness. The specific implementation method is as follows: Step 301: Calculate respectively the smoothness under different characteristic fields , and , and normalize the values of each field to ensure that the scores of each aspect are in the numerical range of (0, 1); Step 302: Use the fluency evaluation function The scores of different feature fields, and the calculation formula is: ； Among them, the constraints are , 、 、 are constants, representing the weights of the corresponding features in the similarity calculation.

Images