Motion learning method, device, electronic device and storage medium for multi-legged robot

Abstract

The present invention relates to the field of robot motion control technology, and specifically discloses a motion learning method, device, electronic equipment and storage medium for a multi-legged robot, wherein the method comprises the following steps: obtaining different video information about foot-type biological motion; extracting motion key information according to the video information to generate a skeleton sequence image, and extracting scene sequence images according to the video information; extracting skeleton feature information according to the skeleton sequence images to generate a skeleton analysis function, and extracting scene feature information according to the scene sequence images to generate an action decision function; establishing a loss function according to the correlation between the skeleton analysis function and the action decision function; updating the loss function until convergence to obtain a foot decision model; the loss function established by the method can reflect the similarity between the skeleton analysis function and the action decision function, so that the foot decision model generated based on the convergence of the loss function can make an action that is closest to the real action for the scene decision.

Description

Technical Field

[0001] The present application relates to the technical field of robot motion control, and in particular to a motion learning method, device, electronic equipment and storage medium for a multi-legged robot. Background Art

[0002] With the development of science and technology and people's continuous pursuit of a better life, the demand for mobile robots is growing. However, in complex and uneven terrain conditions, the mobility and flexibility of mobile robots on traditional platforms are greatly limited. Multi-legged robots are less affected by harsh and irregular terrain conditions and can adapt to more complex and dangerous scenarios.

[0003] Existing multi-legged robot prototypes are all designed and manufactured using bionic methods, by imitating the joint structure and movement gait of leg-like creatures.

[0004] The actions of a multi-legged robot are generally decided by deploying a foot decision model as a deep learning model. The training process of the existing foot decision model is end-to-end deep learning training based on a pre-calibrated data set. The trained model can only make action decisions based on spatial features, resulting in the decision actions being inaccurate and unrealistic.

[0005] There is currently no effective technical solution to the above problems. Summary of the invention

[0006] The purpose of the present application is to provide a motion learning method, device, electronic device and storage medium for a multi-legged robot, so that the foot decision model can decide the action that is closest to the real action according to the scene space and time characteristics.

[0007] In a first aspect, the present application provides a motion learning method for a multi-legged robot, for generating a foot decision model of the multi-legged robot, the method comprising the following steps:

[0008] Get different video information about foot-like creatures' movements;

[0009] Extracting motion key information according to the video information to generate a skeleton sequence picture, and extracting a scene sequence picture according to the video information;

[0010] Extracting skeleton feature information from the skeleton sequence pictures to generate a skeleton analysis function, and extracting scene feature information from the scene sequence pictures to generate an action decision function;

[0011] Establishing a loss function according to the correlation between the skeleton analysis function and the action decision function;

[0012] The loss function is updated until convergence to obtain the foot decision model.

[0013] The motion learning method of a multi-legged robot of the present application generates, according to video information, a skeleton sequence image based on key motion information acquisition and a scene sequence image used to characterize the relationship between foot movements and scenes, and independently performs feature extraction based on the two sequence images to generate a skeleton analysis function and a motion decision function, both of which can respectively reflect the correlation between foot movements and time and space, and a loss function established based on the correlation between the skeleton analysis function and the motion decision function can reflect the degree of similarity between the skeleton analysis function and the motion decision function, so that the foot decision model generated based on the convergence of the loss function can decide the action that is closest to the real action for the scene, making the control of the multi-legged robot more stable and precise.

[0014] The motion learning method of the multi-legged robot, wherein the step of extracting skeleton feature information according to the skeleton sequence image to generate a skeleton analysis function comprises:

[0015] Performing 3D convolution on the skeleton sequence images to preliminarily extract the skeleton feature information;

[0016] The skeleton feature information is subjected to multi-level feature extraction and downsampling according to the self-attention mechanism and the action time feature to obtain the skeleton analysis function.

[0017] The motion learning method of the multi-legged robot, wherein the step of performing multi-level feature extraction on the skeleton feature information according to the self-attention mechanism and the action time feature and obtaining the skeleton analysis function after downsampling comprises:

[0018] After the skeleton feature information is extracted in sequence through the first self-attention mechanism layer, the first time-depth convolution layer, the second self-attention mechanism layer, the second time-depth convolution layer and the third self-attention mechanism layer, the first full-module connection layer is used for downsampling to obtain the skeleton analysis function.

[0019] In the motion learning method of the multi-legged robot in this example, the above three layers of self-attention mechanism are all for extracting the internal correlation of the data sequence, mainly by weighted representation of the feature data of multi-channel input, so as to obtain the proportion of channels at different times in the time series in subsequent calculations, so as to realize internal correlation extraction, that is, weighted fusion of multiple channels of the previous level feature extraction results (convolution results) is performed to realize internal correlation extraction, so as to realize the spatial feature extraction of skeleton feature information, and combine the time-depth convolution layer to perform practical deep feature extraction, so as to make the skeleton analysis function have accurate spatial features and time-depth features.

[0020] The motion learning method of the multi-legged robot, wherein the step of extracting scene feature information according to the scene sequence pictures to generate an action decision function comprises:

[0021] Performing 3D convolution on the scene sequence pictures to extract the scene feature information;

[0022] The action decision function is obtained by performing multi-level feature extraction on the scene feature information according to the self-attention mechanism and downsampling.

[0023] The motion learning method of the multi-legged robot, wherein the step of performing multi-level feature extraction on the scene feature information according to the self-attention mechanism and downsampling to obtain the action decision function comprises:

[0024] After the scene feature information is extracted in sequence through the fourth self-attention mechanism layer, the third 3D-CNN convolution layer, the fifth self-attention mechanism layer, the 2D-CNN convolution layer and the sixth self-attention mechanism layer, the second full-module connection layer is used for downsampling to obtain the action decision function.

[0025] The motion learning method of the multi-legged robot in this example performs feature extraction through three layers of self-attention mechanism, the third 3D-CNN convolution layer and the 2D-CNN convolution layer, which enables the action decision function to extract scene features and internal correlations while effectively reducing the amount of feature calculation, so that the decision action is associated with the scene features, that is, it has spatial feature characteristics.

[0026] The motion learning method of the multi-legged robot, wherein the step of establishing a loss function according to the correlation between the skeleton analysis function and the action decision function comprises:

[0027] A loss function is established according to the norm difference between the skeleton analysis function and the action decision function to characterize the correlation between the action decision and the joint action generated by the multi-legged robot based on the scene.

[0028] The norm difference of the motion learning method of the multi-legged robot in this example can characterize the similarity between the skeleton analysis function and the action decision function. The action decision function includes the scene and the corresponding decision action of the multi-legged animal. Reducing the norm difference can make the skeleton action gradually tend to the decision action.

[0029] The motion learning method of the multi-legged robot, wherein the step of updating the loss function to convergence to obtain the foot decision model comprises:

[0030] The adaptive gradient descent method with weight decay is adopted to respectively update all characteristic parameters of the skeleton analysis function and all characteristic parameters of the action decision function, so that the loss function converges to generate the foot decision model.

[0031] In a second aspect, the present application further provides a motion learning device for a multi-legged robot, for generating a foot decision model for the multi-legged robot, the device comprising:

[0032] An acquisition module, used to acquire different video information about the motion of the foot-like creature;

[0033] A sequence extraction module, used to extract motion key information according to the video information to generate a skeleton sequence picture, and to extract a scene sequence picture according to the video information;

[0034] A feature extraction module, used to extract skeleton feature information from the skeleton sequence pictures to generate a skeleton analysis function, and to extract scene feature information from the scene sequence pictures to generate an action decision function;

[0035] A modeling module, used for establishing a loss function according to the correlation between the skeleton analysis function and the action decision function;

[0036] An updating module is used to update the loss function until convergence to obtain the foot decision model.

[0037] The motion learning device of the multi-legged robot of the present application uses a sequence extraction module to generate a skeleton sequence image based on key motion information and a scene sequence image for characterizing the relationship between the foot movement and the scene according to video information, and then uses a feature extraction module to independently perform feature extraction based on the two sequence images to generate a skeleton analysis function and a motion decision function, both of which can respectively reflect the correlation between the foot movement and time and space. The loss function established based on the correlation between the skeleton analysis function and the motion decision function can reflect the similarity between the skeleton analysis function and the motion decision function, so that the foot decision model generated based on the convergence of the loss function can decide the action closest to the real action for the scene, making the control of the multi-legged robot more stable and precise.

[0038] In a third aspect, the present application further provides an electronic device, comprising a processor and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps in the method provided in the first aspect are executed.

[0039] In a fourth aspect, the present application further provides a storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps in the method provided in the first aspect are executed.

[0040] From the above, it can be seen that the present application provides a motion learning method, device, electronic device and storage medium for a multi-legged robot, wherein the method generates a skeleton sequence image based on key motion information and a scene sequence image for characterizing the relationship between foot movements and scenes according to video information, and independently performs feature extraction based on the two sequence images to generate a skeleton analysis function and a motion decision function. The loss function established based on the correlation between the skeleton analysis function and the motion decision function can reflect the similarity between the skeleton analysis function and the motion decision function, so that the foot decision model generated based on the convergence of the loss function can decide the action that is closest to the real action for the scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A flowchart of a motion learning method for a multi-legged robot provided in an embodiment of the present application.

[0042] Figure 2 Schematic diagram of the generation process of the loss function.

[0043] Figure 3 Schematic diagram of the structure of the temporal depth convolution layer.

[0044] Figure 4 A schematic diagram of the structure of a motion learning device for a multi-legged robot provided in an embodiment of the present application.

[0045] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0046] Figure numerals: 201, acquisition module; 202, sequence extraction module; 203, feature extraction module; 204, modeling module; 205, update module; 301, processor; 302, memory; 303, communication bus. DETAILED DESCRIPTION

[0047] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0048] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0049] First, please refer to Figure 1-Figure 3 , Figure 1-Figure 3 A motion learning method for a multi-legged robot in some embodiments of the present application is used to generate a foot decision model for the multi-legged robot. The method comprises the following steps:

[0050] S1. Obtain different video information about foot-type biological motion;

[0051] Specifically, the video information is ordinary video data containing the normal movement process of the corresponding footed organism. The motion learning method of the multi-legged robot in the embodiment of the present application is trained and learned based on this type of ordinary video data, which can greatly increase the amount of behavioral data that the robot can learn, so that the foot decision model finally obtained by training is suitable for use in different scenarios.

[0052] More specifically, a podophilic organism is a multi-legged organism, that is, an organism that has more than two feet for movement, and may be an insect, a reptile, a mammal, etc.

[0053] More specifically, the video information is not restricted in the way it is obtained, and can be video information taken from the Internet or shot by users. It has the characteristic of massive data, which can ensure that the method of the embodiment of the present application has enough behavioral data for motion learning, effectively improving the flexibility of multi-legged robot motion learning.

[0054] More specifically, different video information includes video data of heterogeneous motions of multi-legged creatures in the same scene, video data of similar motions of multi-legged creatures in the same scene, video data of heterogeneous motions of multi-legged creatures in different scenes, and video data of similar motions of multi-legged creatures in different scenes; the foot movements of multi-legged creatures are diverse, such as human running, walking, jumping and other behaviors. In order to ensure that the foot decision model finally trained can make appropriate action decisions according to the scene to learn to imitate the corresponding foot movement behaviors, the video information used for training should involve various foot movement behaviors of the corresponding multi-legged creatures in various scenes to improve the learning flexibility and decision accuracy of the foot decision model.

[0055] S2, extracting motion key information based on video information to generate skeleton sequence pictures, and extracting scene sequence pictures based on video information;

[0056] Specifically, the key motion information is the position and posture information of key limbs of the multi-legged creature regarding the foot motion behavior in the video information, such as the position and posture information of the foot joints (hip, knee, ankle) and trunk of the multi-legged creature.

[0057] More specifically, the key motion information is extracted and generated from the video information by a distance transformation-based algorithm, a morphology-based algorithm, a refinement algorithm, a chain code-based algorithm, or a Voronoi diagram-based algorithm. The above algorithms can maturely and accurately extract the key motion information that characterizes the motion behavior characteristics of multi-legged organisms from the video information.

[0058] More specifically, the video information is a series of image frames arranged in a time series, and the movement behavior of multi-legged creatures is expressed as motion key information recorded in different image frames. The motion key information extracted from each image frame of the video information can generate a series of skeleton sequence pictures arranged in a time series. The skeleton sequence pictures are a series of picture data that express the motion key information in the form of skeletons.

[0059] More specifically, the skeleton sequence image is a type of image data that expresses the characteristics of foot movement behavior through a skeleton structure. It can be image data generated based on a preset skeleton model and key motion information, or it can be image data generated by fitting a skeleton based on key motion information, so as to provide clear action structure data for the foot decision model to learn and imitate.

[0060] More specifically, the scene sequence picture is a kind of picture data that combines the scene and the movement behavior of multi-legged creatures. In the embodiment of the present application, it is preferably an image frame extracted from video information.

[0061] S3, extracting skeleton feature information according to the skeleton sequence pictures to generate a skeleton analysis function, and extracting scene feature information according to the scene sequence pictures to generate an action decision function;

[0062] Specifically, this step mainly extracts features from the skeleton sequence images and scene sequence images obtained in step S2 to generate a skeleton analysis function for characterizing the skeleton action of the foot and an action decision function for characterizing the correlation between the scene and the decision action.

[0063] More specifically, the skeleton feature information is feature data used to characterize the skeleton morphological characteristics in the skeleton sequence pictures, and the scene feature information is feature data used to characterize the scene characteristics and the action characteristics of multi-legged creatures in the scene sequence pictures.

[0064] More specifically, the skeleton analysis function and the action decision function are characteristic functions with unified dimensions, which can serve as the data basis for subsequent internal correlation analysis.

[0065] More specifically, the skeleton analysis function reflects the changes in skeleton movements and can accurately reflect the changing relationship between movement and time. The action decision function reflects the correlation between the characteristics of foot movements and scene characteristics and can accurately reflect the correlation between movement and space.

[0066] S4, establishing a loss function according to the correlation between the skeleton analysis function and the action decision function;

[0067] Specifically, the skeleton analysis function is a feature function generated by converting the skeleton sequence image according to multiple undetermined feature parameters, the action decision function is a feature function generated by converting the scene sequence image according to multiple undetermined feature parameters, and the training process of the foot decision model is the process of obtaining these optimal feature parameters.

[0068] More specifically, the loss function is established based on the correlation between the skeleton analysis function and the action decision function, so that the loss function reflects the similarity between the skeleton analysis function and the action decision function, that is, it reflects the similarity between the action decision made by the action decision function and the action behavior corresponding to the skeleton analysis function. The two functions are extracted and generated based on skeleton sequence images and scene sequence images, respectively, so that the method of the embodiment of the present application can quickly compare action decision features and skeleton features, wherein the action decision function reflects the action decision generated based on the scene, and the skeleton analysis function reflects the real action based on the skeleton representation. The loss function established based on the correlation of the two functions reflects whether the action decision in the corresponding scene is consistent with the real action. Using the loss function to train and optimize the foot decision model can make the action decisions made by the foot decision model for the scene gradually tend towards the real action.

[0069] S5. Update the loss function until convergence to obtain the foot decision model.

[0070] Specifically, from the foregoing content, it can be seen that updating the loss function is the process of obtaining the optimal feature parameters corresponding to the skeleton analysis function and the action decision function, that is, obtaining the optimal feature parameters that can make the correlation between the skeleton analysis function and the action decision function reach the highest level, so that the action decision made by the foot decision model for the scene is closest to the real action.

[0071] More specifically, after obtaining the foot decision model, the foot decision model needs to be deployed on the control end of the corresponding multi-legged robot so that the corresponding multi-legged robot can generate appropriate action decisions according to the scene and move.

[0072] More specifically, the deployment process of the foot decision model is to adjust and transplant the foot decision model to the control end of the multi-legged robot according to the kinematics, dynamics algorithm, coordinate transformation and environmental configuration information corresponding to the multi-legged robot.

[0073] The motion learning method for a multi-legged robot in an embodiment of the present application generates, according to video information, a skeleton sequence image based on key motion information and a scene sequence image for characterizing the relationship between foot movements and scenes, and independently performs feature extraction based on the two sequence images to generate a skeleton analysis function and a motion decision function, both of which can respectively reflect the correlation between foot movements and time and space, and a loss function established based on the correlation between the skeleton analysis function and the motion decision function can reflect the degree of similarity between the skeleton analysis function and the motion decision function, so that the foot decision model generated based on the convergence of the loss function can decide the action that is closest to the real action for the scene, making the control of the multi-legged robot more stable and precise.

[0074] The motion learning method for a multi-legged robot in the embodiment of the present application is applicable to a bipedal robot and a quadrupedal robot. Step S1 uses video information of corresponding legged biological motion to perform motion learning for the bipedal robot and the quadrupedal robot. Among them, for the bipedal robot, step S1 mainly uses video information about human motion, such as video information of human walking, running, jumping and other actions; for the quadrupedal robot, step S1 mainly uses video information of common quadruped animal motion, such as video information of gait motions of cats or dogs, such as pacing, trotting and running.

[0075] Since the foot decision model finally obtained by the motion learning method of the multi-legged robot in the embodiment of the present application needs to be deployed on the corresponding multi-legged robot for use, therefore, in some preferred embodiments, the legged creature in the video information in step S1 has a similar joint structure to the multi-legged robot, that is, before executing step S1 or during the execution of step S1, the video information needs to be screened so that the legged creature in the remaining video information has a similar joint structure to the multi-legged robot, wherein a similar joint structure refers to a similar joint relationship and a similar ratio of joint-limb lengths, so that the foot decision model finally obtained is highly consistent with the corresponding multi-legged robot, can be smoothly deployed in the multi-legged robot to make action decisions for the multi-legged robot, and effectively reduce the amount of data in the model training process.

[0076] In some other embodiments, for video information of a multi-legged animal with the same number of legs as a multi-legged robot, when the joint structures are greatly different, in the process of deploying the foot decision model to the multi-legged robot, joint transformation parameters can be added to match the joint structure to achieve deployment control of the multi-legged robot.

[0077] In some other embodiments, the video information also includes video information of the movement of a pod-like creature when its limb is damaged.

[0078] Specifically, the foot decision model is trained in combination with the video information of the movement of the above-mentioned legged limbs when they are damaged, so that the multi-legged robot can imitate the action pattern when the limbs are damaged, so that when the local legged limbs lose control (such as mechanical joint failure or jamming), the robot can make action decisions corresponding to the damaged legged limbs according to the characteristics of the scene, so that the method of the embodiment of the present application has a wider learning range, and the multi-legged robot deployed with the foot decision model generated by the method can adapt to different scenes and states to make appropriate action decisions, which greatly improves the adaptability and flexibility of the multi-legged robot.

[0079] In some preferred embodiments, the step of extracting skeleton feature information from skeleton sequence images to generate a skeleton analysis function includes:

[0080] S31, performing 3D convolution on the skeleton sequence images to preliminarily extract skeleton feature information;

[0081] Specifically, Figure 2 As shown, the skeleton sequence images are sequentially input into the first 3D-CNN layer (3D-CNN1) for 3D convolution to extract skeleton feature information; since the skeleton sequence images are generated by extracting motion key information according to the image frame sequence of the video information, the skeleton sequence images are input into the first 3D-CNN layer according to the time series, so that the extracted skeleton feature information has the time depth feature, that is, the action corresponding to the skeleton feature information is associated with the time series.

[0082] S32. Perform multi-level feature extraction on the skeleton feature information according to the self-attention mechanism and the action time characteristics and obtain the skeleton analysis function after downsampling.

[0083] Specifically, step S31 is to perform preliminary feature extraction on the skeleton sequence image. For the foot decision model as a deep learning model, it is also necessary to perform multi-level feature extraction on the skeleton feature information to extract the internal correlation of the data sequence of these skeleton feature information, and perform downsampling and normalization processing to generate a skeleton analysis function.

[0084] More specifically, the process of extracting the internal correlation of the data sequence of the skeleton feature information is to perform multi-level feature extraction on the skeleton feature information according to the self-attention mechanism and the action time feature, wherein the feature extraction according to the self-attention mechanism (Self-Attention) can perform action space features and sequence internal correlation on the previous level feature extraction result, and the feature extraction according to the action time feature can perform feature extraction on the sequence time axis on the previous level feature extraction result, so that the skeleton analysis function obtained in step S32 has action space features and time depth features, that is, it represents the real action that can reflect the spatial and temporal relationship, which is convenient for the loss function established in step S4 to compare the action features about time and space.

[0085] In some preferred embodiments, the steps of performing multi-level feature extraction on skeleton feature information according to the self-attention mechanism and the action time feature and obtaining the skeleton analysis function after downsampling include:

[0086] S321. After extracting the skeleton feature information in sequence through the first self-attention mechanism layer, the first time-depth convolution layer, the second self-attention mechanism layer, the second time-depth convolution layer and the third self-attention mechanism layer, the first full-module connection layer is used for downsampling to obtain the skeleton analysis function.

[0087] Specifically, Figure 2 As shown in the figure, the change process from the skeleton sequence image to the first output end constitutes a skeleton algorithm network, which is used to generate and output a skeleton analysis function according to the skeleton sequence image; the skeleton feature information passes through the first self-attention mechanism layer (Self-attention1) to extract the features of the internal correlation of the first data sequence, the first time depth convolution layer (Temporal Module1) to extract the features on the time axis of the first sequence, the second self-attention mechanism layer (Self-attention2) to extract the features of the internal correlation of the second data sequence, the second time depth convolution layer (Temporal Module2) to extract the features on the time axis of the second sequence, the third self-attention mechanism layer (Self-attention3) to extract the features of the internal correlation of the third data sequence, and the first full module connection layer (FullConnected1) to downsample, and then the downsampling result is normalized by using the activation function (softmax) to achieve dimensional unification to generate a skeleton analysis function.

[0088] More specifically, step S321 performs feature extraction through three layers of self-attention mechanism and two layers of time-depth convolution, which can make the skeleton action represented by the skeleton analysis function closely related to the spatial features and time-depth features, and as close to the real action as possible. Therefore, the skeleton analysis function can be regarded as a standard real action.

[0089] More specifically, the above three self-attention mechanism layers are all for extracting the internal correlation of the data sequence, mainly by weighted representation of the feature data of multi-channel input, so as to obtain the proportion of channels at different times in the subsequent calculation under the time series, so as to realize the internal correlation extraction, that is, to perform weighted fusion of multiple channels of the previous level feature extraction results (convolution results) to realize the internal correlation extraction, so as to realize the spatial feature extraction of skeleton feature information, and combine the time-depth convolution layer to perform practical deep feature extraction, so as to make the skeleton analysis function have accurate spatial features and time-depth features.

[0090] More specifically, the structures of the two temporal depth convolutional layers are as follows Figure 3 As shown, the data output by multiple channels (C1, C2, C3) of the self-attention mechanism layer are convolved and average pooled according to the time depth, so as to extract the temporal features of the results of spatial feature extraction.

[0091] More specifically, in order to clearly understand the method of the embodiment of the present application, the skeleton analysis function output by the skeleton algorithm network is defined as: O 1 ( f 1 ( x,θ )),in O 1 Identify the activation function of the last layer, f 1 represents the network equivalent function from the first 3D-CNN layer to the first fully connected module layer, i All characteristic parameters of the skeleton algorithm network can be expressed by the following augmented matrix:

[0092] (1)

[0093] in, i ij represents a parameter of a network layer from the 3D-CNN layer to the first full-module connection layer, i=1, 2,……，m ; j=1,2,……,n ; x is the corresponding skeleton sequence picture.

[0094] In some preferred embodiments, the step of extracting scene feature information from scene sequence pictures to generate an action decision function includes:

[0095] S33, performing 3D convolution on the scene sequence pictures to extract scene feature information;

[0096] Specifically, Figure 2 As shown, the scene sequence images are sequentially input into the second 3D-CNN layer (3D-CNN2) for 3D convolution to extract scene feature information; since the scene sequence images contain scenes and multi-legged biological actions, the scene feature information extracted after the scene sequence images are input into the second 3D-CNN layer can characterize the relationship between the scene and the action.

[0097] S34. Perform multi-level feature extraction on the scene feature information according to the self-attention mechanism and obtain the action decision function after downsampling.

[0098] Specifically, step S34 is to perform preliminary feature extraction on the scene sequence images. For the foot decision model as a deep learning model, it is also necessary to perform multi-level feature extraction on the scene feature information to extract the internal correlation of the data sequence of these scene feature information, and perform downsampling and normalization processing to generate an action decision function.

[0099] More specifically, scene feature information does not change significantly over time compared to skeleton feature information, mainly because the scene itself will not change significantly over time. Therefore, for scene feature information, it is necessary to analyze and extract its spatial features to characterize the relationship between scene characteristics and multi-legged biological action characteristics, so that the foot decision model finally obtained can make corresponding action decisions based on the scene characteristics; therefore, the process of extracting the internal correlation of the data sequence of scene feature information is to perform multi-level feature extraction on the scene feature information according to the self-attention mechanism, so that the action decision function obtained in step S34 can reflect the relationship between scene characteristics and multiple groups of biological action characteristics, that is, characterize the decision relationship between the scene and the action, so that the loss function established in step S4 can determine the most appropriate action decision for the scene based on the action decision function, and can obtain the most appropriate skeleton action based on the correlation between the action decision function and the skeleton analysis function, so that the decided action gradually tends to the real action.

[0100] In some preferred embodiments, step S31 and step S33 are preferably enabled and executed simultaneously, and a skeleton analysis function and an action decision function are generated respectively in step S32 and step S34 before executing step S4.

[0101] In some preferred embodiments, the step of performing multi-level feature extraction on scene feature information according to the self-attention mechanism and downsampling to obtain the action decision function comprises:

[0102] S341. After extracting the scene feature information in sequence through the fourth self-attention mechanism layer, the third 3D-CNN convolution layer, the fifth self-attention mechanism layer, the 2D-CNN convolution layer and the sixth self-attention mechanism layer, downsampling is performed using the second full-module connection layer to obtain the action decision function.

[0103] Specifically, Figure 2 As shown in the figure, the change process from the scene sequence picture to the second output end constitutes a scene algorithm network, which is used to generate and output an action decision function according to the scene sequence picture; the scene feature information is sequentially passed through the fourth self-attention mechanism layer (Self-attention4) to extract the feature of the internal correlation of the first data sequence, the third 3D-CNN convolution layer (3D-CNN3) to extract the second 3D convolution feature, the fifth self-attention mechanism layer (Self-attention5) to extract the feature of the internal correlation of the second data sequence, the 2D-CNN convolution layer (2D-CNN) to extract the 2D convolution feature, the sixth self-attention mechanism layer (Self-attention6) to extract the feature of the internal correlation of the third data sequence, and the second full-module connection layer (Full Connected2) to downsample, and then the downsampling result is normalized by the activation function (softmax) to achieve dimensional unification to generate an action decision function.

[0104] More specifically, step S321 performs feature extraction through three layers of self-attention mechanism, the third 3D-CNN convolution layer and the 2D-CNN convolution layer, which enables the action decision function to extract scene features and internal correlations while effectively reducing the amount of feature calculation, so that the decision action is associated with the scene features, that is, it has spatial feature characteristics.

[0105] More specifically, the above three self-attention mechanism layers also weight the feature data of multi-channel input, so as to obtain the proportion of channels at different moments in the time series in subsequent calculations, so as to realize internal correlation extraction.

[0106] More specifically, in order to clearly understand the method of the embodiment of the present application, the action decision function output by the above-mentioned scene algorithm network is defined as: O 2 ( f 2 ( y,φ )),in O 2 Identify the activation function of the last layer, f 2 represents the network equivalent function from the second 3D-CNN layer to the second fully connected module layer, f Represents all characteristic parameters of the scene algorithm network, which can also be represented by an augmented matrix. f ij represents a parameter of a network layer from the second 3D-CNN layer to the second full-module connection layer, i=1,2,……,m ; j=1,2,……,n ; y is the corresponding scene sequence picture.

[0107] In some preferred embodiments, the step of establishing a loss function according to the correlation between the skeleton analysis function and the action decision function includes:

[0108] S41. A loss function is established based on the norm difference between the skeleton analysis function and the action decision function to characterize the correlation between the action decision and the joint action generated by the multi-legged robot based on the scene.

[0109] Specifically, Figure 2 As shown, the loss function based on the skeleton analysis function and the action decision function is used to compare the loss.

[0110] More specifically, the norm difference between the skeleton analysis function and the action decision function is expressed as | O 1 ( f 1 ( x,θ ))- O 2 ( f 2 ( y, f ))|, the norm difference can characterize the similarity between the skeleton analysis function and the action decision function. The action decision function includes the scene and the corresponding multi-legged animal's decision action. Reducing the norm difference can make the skeleton action gradually tend to the decision action.

[0111] More specifically, in the embodiment of the present application, the loss function is preferably:

[0112] (2)

[0113] Among them,||·|| 2 Expressed as the absolute value of the norm.

[0114] Specifically, the characteristic parameters of the corresponding skeleton analysis function and action decision function are obtained through training and learning. i and f , so that the loss function When it converges to the minimum, it means that the skeleton action is closest to the real action; therefore, the learning process of the method in the embodiment of the present application is based on i and f renew , this updating process can use the learning methods of various deep learning models to update the loss function.

[0115] In some preferred embodiments, the step of updating the loss function to converge to obtain the foot decision model includes:

[0116] S51. Adopting the adaptive gradient descent method with weight decay, respectively update all characteristic parameters of the skeleton analysis function and all characteristic parameters of the action decision function, so that the loss function converges to generate a foot decision model.

[0117] Specifically, the method of the embodiment of the present application preferably uses an adaptive gradient descent method to update the loss function To make Fast convergence to generate a foot decision model.

[0118] More specifically, the adaptive gradient descent method update process is as follows:

[0119] For characteristic parameters i and f Set the corresponding gradient blocks respectively and , respectively satisfying:

[0120] (3)

[0121] (4)

[0122] t is the update time, gradient block and Each has an iterative term and , for i The gradient operator of for f The gradient operator satisfies:

[0123] (5)

[0124] (6)

[0125] in α、β is a combination parameter, which is generally set to 0.99. According to formulas (2)-(6), we can get:

[0126] (7)

[0127] (8)

[0128] in, i t and f t They are i and f Characteristic parameters about update time Oh, oh 1 、oh 2is the weight decay parameter, which is generally set to 0.999. e is an infinitesimal compensation value, or 1 ,or 2 is the learning rate, which is generally set to 0.05.

[0129] According to the above iterative terms, the gradient descent is performed to update the loss function. i t and f t After convergence, the corresponding feature parameters are recorded to generate the required foot decision model.

[0130] Second, please refer to Figure 4 , Figure 4 A motion learning device for a multi-legged robot provided in some embodiments of the present application is used to generate a foot decision model for the multi-legged robot, and the device includes:

[0131] An acquisition module 201 is used to acquire different video information about foot-type biological motion;

[0132] A sequence extraction module 202 is used to extract motion key information according to video information to generate skeleton sequence pictures, and to extract scene sequence pictures according to video information;

[0133] A feature extraction module 203 is used to extract skeleton feature information from the skeleton sequence pictures to generate a skeleton analysis function, and to extract scene feature information from the scene sequence pictures to generate an action decision function;

[0134] A modeling module 204, for establishing a loss function according to the correlation between the skeleton analysis function and the action decision function;

[0135] The updating module 205 is used to update the loss function until it converges to obtain the foot decision model.

[0136] The motion learning device of the multi-legged robot in the embodiment of the present application uses a sequence extraction module 202 to generate a skeleton sequence image based on key motion information and a scene sequence image for characterizing the relationship between the foot movement and the scene according to video information, and then uses a feature extraction module 203 to independently perform feature extraction based on the two sequence images to generate a skeleton analysis function and a motion decision function, both of which can respectively reflect the correlation between the foot movement and time and space. The loss function established based on the correlation between the skeleton analysis function and the motion decision function can reflect the similarity between the skeleton analysis function and the motion decision function, so that the foot decision model generated based on the convergence of the loss function can decide the action that is closest to the real action for the scene, making the control of the multi-legged robot more stable and precise.

[0137] In some preferred embodiments, the motion learning device for a multi-legged robot of an embodiment of the present application is used to execute the motion learning method for a multi-legged robot provided in the first aspect above.

[0138] Third, please refer to Figure 5 , Figure 5 This is a structural diagram of an electronic device provided in an embodiment of the present application. The present application provides an electronic device, including: a processor 301 and a memory 302. The processor 301 and the memory 302 are interconnected and communicate with each other through a communication bus 303 and / or other forms of connection mechanisms (not shown). The memory 302 stores a computer program executable by the processor 301. When the computing device is running, the processor 301 executes the computer program to execute the method in any optional implementation of the above-mentioned embodiment.

[0139] In a fourth aspect, an embodiment of the present application provides a storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method in any optional implementation of the above embodiment is executed. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, disk or optical disk.

[0140] In summary, the embodiments of the present application provide a motion learning method, device, electronic device and storage medium for a multi-legged robot, wherein the method generates a skeleton sequence image based on key motion information and a scene sequence image for characterizing the relationship between foot movements and scenes according to video information, and independently performs feature extraction based on the two sequence images to generate a skeleton analysis function and a motion decision function. The loss function established based on the correlation between the skeleton analysis function and the motion decision function can reflect the similarity between the skeleton analysis function and the motion decision function, so that the foot decision model generated based on the convergence of the loss function can decide the action that is closest to the real action for the scene.

[0141] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0142] In addition, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0143] Furthermore, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0144] In this document, relational terms such as first and second, etc. are used merely to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0145] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A motion learning method for a multi-legged robot, used to generate a foot decision model of the multi-legged robot, characterized in that: The method comprises the following steps: Get different video information about foot-like creatures' movements; Extracting key motion information according to the video information to generate a skeleton sequence picture, and extracting a scene sequence picture according to the video information, wherein the scene sequence picture includes a scene and a multi-legged creature action; Extracting skeleton feature information from the skeleton sequence pictures to generate a skeleton analysis function, and extracting scene feature information from the scene sequence pictures to generate an action decision function, wherein the scene feature information represents the relationship between the scene and the action; Establishing a loss function according to the correlation between the skeleton analysis function and the action decision function; updating the loss function until convergence to obtain the foot decision model; The step of extracting skeleton feature information according to the skeleton sequence picture to generate a skeleton analysis function comprises: Performing 3D convolution on the skeleton sequence images to preliminarily extract the skeleton feature information; Perform multi-level feature extraction on the skeleton feature information according to the self-attention mechanism and the action time feature and perform downsampling to obtain the skeleton analysis function; The step of extracting scene feature information according to the scene sequence pictures to generate an action decision function comprises: Performing 3D convolution on the scene sequence pictures to extract the scene feature information; The action decision function is obtained by performing multi-level feature extraction on the scene feature information according to the self-attention mechanism and downsampling.

2. The motion learning method of a multi-legged robot according to claim 1, characterized in that: The step of performing multi-level feature extraction on the skeleton feature information according to the self-attention mechanism and the action time feature and performing downsampling to obtain the skeleton analysis function comprises: After the skeleton feature information is extracted in sequence through the first self-attention mechanism layer, the first time-depth convolution layer, the second self-attention mechanism layer, the second time-depth convolution layer and the third self-attention mechanism layer, the first full-module connection layer is used for downsampling to obtain the skeleton analysis function.

3. The motion learning method of a multi-legged robot according to claim 1, characterized in that: The step of performing multi-level feature extraction on the scene feature information according to the self-attention mechanism and downsampling to obtain the action decision function comprises: After the scene feature information is extracted in sequence through the fourth self-attention mechanism layer, the third 3D-CNN convolution layer, the fifth self-attention mechanism layer, the 2D-CNN convolution layer and the sixth self-attention mechanism layer, the second full-module connection layer is used for downsampling to obtain the action decision function.

4. The motion learning method of a multi-legged robot according to claim 1, characterized in that: The step of establishing a loss function according to the correlation between the skeleton analysis function and the action decision function comprises: A loss function is established according to the norm difference between the skeleton analysis function and the action decision function to characterize the correlation between the action decision and the joint action generated by the multi-legged robot based on the scene.

5. The motion learning method of a multi-legged robot according to claim 1, characterized in that: The step of updating the loss function until convergence to obtain the foot decision model comprises: The adaptive gradient descent method with weight decay is adopted to respectively update all characteristic parameters of the skeleton analysis function and all characteristic parameters of the action decision function, so that the loss function converges to generate the foot decision model.

6. A motion learning device for a multi-legged robot, used to generate a foot decision model for the multi-legged robot, characterized in that: The device comprises: An acquisition module, used to acquire different video information about the motion of the foot-like creature; A sequence extraction module, used to extract key motion information according to the video information to generate a skeleton sequence picture, and to extract a scene sequence picture according to the video information, wherein the scene sequence picture includes a scene and a multi-legged creature action; A feature extraction module, used to extract skeleton feature information from the skeleton sequence pictures to generate a skeleton analysis function, and to extract scene feature information from the scene sequence pictures to generate an action decision function, wherein the scene feature information represents the relationship between the scene and the action; A modeling module, used for establishing a loss function according to the correlation between the skeleton analysis function and the action decision function; An updating module, used for updating the loss function until convergence to obtain the foot decision model; The step of extracting skeleton feature information according to the skeleton sequence picture to generate a skeleton analysis function comprises: Performing 3D convolution on the skeleton sequence images to preliminarily extract the skeleton feature information; Perform multi-level feature extraction on the skeleton feature information according to the self-attention mechanism and the action time feature and perform downsampling to obtain the skeleton analysis function; The step of extracting scene feature information according to the scene sequence pictures to generate an action decision function comprises: Performing 3D convolution on the scene sequence pictures to extract the scene feature information; The action decision function is obtained by performing multi-level feature extraction on the scene feature information according to the self-attention mechanism and downsampling.

7. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the steps in the method according to any one of claims 1 to 5 are executed.

8. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are executed.

Images