Gait imitation method, device, electronic device and storage medium

Abstract

This application relates to a gait imitation method, apparatus, electronic device, and storage medium. The gait imitation method comprises: acquiring target motion data, the target motion data comprising motion data for multiple key postures; determining the similarity of the motion data for the multiple key postures; grouping the multiple key postures based on the similarity; and selecting a posture to be imitated from the grouped key postures for gait imitation. This application solves the problem of poor fluency in the robot's gait imitation process in the prior art, thereby improving the fluency of the robot's gait imitation process.

Description

Technical Field

[0001] The present application relates to the field of robot control technology, and in particular to a gait imitation method, device, electronic device and storage medium. Background Art

[0002] Bipedal robots, with structures similar to the human body, can replace humans in various tasks within human social environments, expanding human capabilities and demonstrating excellent environmental adaptability. In recent years, bipedal robots have been widely used in a variety of fields, including services, healthcare, education, and entertainment, and related research has been steadily expanding.

[0003] The bipedal robot constructs a human motion model by collecting human skeleton information. Furthermore, the bipedal robot's motion model is constructed based on the bipedal robot's own limb structure characteristics. The collected human movements are converted into movements in the robot motion model through a mapping model, enabling the bipedal robot to complete robot gait imitation based on joint configuration data.

[0004] In the above gait imitation process, a large amount of human skeleton data needs to be processed. A large amount of raw data processing will cause the robot's imitation movement to lag, thereby causing the robot's movement imitation process to jitter, and further resulting in poor smoothness of the robot's gait imitation process.

[0005] There is currently no effective solution to the problem of poor fluency in the robot gait imitation process in related technologies. Summary of the Invention

[0006] In this embodiment, a gait imitation method, apparatus, electronic device, and storage medium are provided to solve the problem of poor fluency in the robot gait imitation process in the related art.

[0007] In a first aspect, a gait imitation method is provided in this embodiment, which is applied to a robot. The method includes:

[0008] Acquiring target motion data, wherein the target motion data includes motion data of a plurality of key postures;

[0009] Determining similarity of motion data of a plurality of said key postures;

[0010] grouping the plurality of key gestures based on the similarity;

[0011] Select the posture to be imitated from the grouped key postures to perform gait imitation.

[0012] In some embodiments, obtaining target action data includes:

[0013] Get initial action data;

[0014] The target motion data is determined based on initial motion data and a key posture recognition model, where the key posture recognition model is obtained through reinforcement learning training.

[0015] In some embodiments, determining the target motion data based on the initial motion data and the key gesture recognition model includes:

[0016] determining a similarity threshold based on the key gesture recognition model;

[0017] Determining similarities between multiple actions in the initial action data;

[0018] determining the key gesture based on the similarity and a similarity threshold;

[0019] The target motion data is determined based on the key pose.

[0020] In some embodiments, grouping the plurality of key gestures based on the similarity comprises:

[0021] determining a first Gaussian distribution of a target key pose, wherein the target key pose is any one of the plurality of key poses;

[0022] Determining key postures to be grouped in descending order of similarity to the target key posture, wherein the key postures to be grouped are key postures other than the target key posture;

[0023] Determining a second Gaussian distribution of the key gestures to be grouped;

[0024] The key gestures are grouped based on a divergence between the first Gaussian distribution and the second Gaussian distribution.

[0025] In some embodiments, grouping the plurality of key gestures based on the divergence between the first Gaussian distribution and the second Gaussian distribution includes:

[0026] If the divergence is less than a preset threshold, the key posture to be grouped and the target key posture are classified into one category;

[0027] If the divergence is greater than or equal to the preset threshold, the next key gesture to be grouped is selected for grouping until all key gestures are grouped.

[0028] In some embodiments, after grouping the plurality of key gestures based on the similarity, the method further comprises:

[0029] Distributed storage of grouped key poses.

[0030] In some embodiments, selecting a posture to be imitated from the grouped key postures for gait imitation includes:

[0031] According to the action sequence of key postures, one key posture is selected from each group of key postures as the posture to be imitated;

[0032] Perform Gaussian blur on each key posture to be imitated to obtain the blurred key posture;

[0033] Gait imitation is performed based on the blurred key postures.

[0034] In some embodiments, before performing gait imitation based on the blurred key posture, the method further includes:

[0035] Anti-discretization processing is performed on the blurred key posture.

[0036] In a second aspect, a gait imitation device is provided in this embodiment, comprising:

[0037] An acquisition module, configured to acquire target motion data, wherein the target motion data includes motion data of a plurality of key postures;

[0038] a determination module, configured to determine similarities of motion data of a plurality of key postures;

[0039] a grouping module, configured to group the plurality of key gestures based on the similarity;

[0040] The imitation module is used to select the posture to be imitated from the grouped key postures to perform gait imitation.

[0041] In a third aspect, an electronic device is provided in this embodiment, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the gait imitation method described in the first aspect when executing the computer program.

[0042] In a fourth aspect, a storage medium is provided in this embodiment, on which a computer program is stored. When the program is executed by a processor, the gait imitation method described in the first aspect is implemented.

[0043] Compared with the related art, a gait imitation method provided in this embodiment groups multiple key postures by the similarity of their motion data, so that postures with close similarity can be grouped together. Furthermore, the posture to be imitated is selected from the grouped key postures for gait imitation, so that during the gait imitation process, the robot does not need to process a large amount of acquired raw data, but only needs to process the data of the key postures selected in the group, thereby reducing the amount of data processing, improving data processing efficiency, and further improving the fluency of the robot's gait imitation process.

[0044] The details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0046] Figure 1 This is a usage scenario diagram of a gait imitation method provided in an embodiment of the present application;

[0047] Figure 2 This is a flow chart of a gait imitation method provided in an embodiment of the present application;

[0048] Figure 3 This is a flow chart of an embodiment of a gait imitation method provided by the present application;

[0049] Figure 4 This is a structural block diagram of a gait imitation device provided in an embodiment of the present application;

[0050] Figure 5 This is a schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0052] Unless otherwise defined, the technical terms or scientific terms involved in this application should have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "an", "a", "the", "these" and the like in this application do not indicate quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "plurality" involved in this application refers to two or more. "And / or" describes the relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, and B exists alone. Generally, the character " / " indicates that the related objects are in an "or" relationship. The terms "first," "second," "third," etc. used in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0053] The gait imitation method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, Figure 1 This is a usage scenario diagram of a gait imitation method provided in an embodiment of the present application. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Among them, the terminal 102 can be a robot, or it can be various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented as an independent server or a server cluster consisting of multiple servers.

[0054] Bipedal robots, with structures similar to the human body, can replace humans in various tasks within human social environments, expanding human capabilities and demonstrating excellent environmental adaptability. In recent years, bipedal robots have been widely used in a variety of fields, including services, healthcare, education, and entertainment, and related research has been steadily expanding.

[0055] The bipedal robot constructs a human motion model by collecting human skeleton information. Furthermore, the bipedal robot's motion model is constructed based on the bipedal robot's own limb structure characteristics. The collected human movements are converted into movements in the robot motion model through a mapping model, enabling the bipedal robot to complete robot gait imitation based on joint configuration data.

[0056] In the above gait imitation process, a large amount of human skeleton data needs to be processed. A large amount of raw data processing will cause the robot's imitation movement to lag, thereby causing the robot's movement imitation process to jitter, and further resulting in poor smoothness of the robot's gait imitation process.

[0057] Therefore, how to improve the fluency of the robot's gait imitation process is a problem that needs to be solved.

[0058] In this embodiment, a gait imitation method is provided. Figure 2 This is a flowchart of a gait imitation method provided in an embodiment of the present application. The executor of the method may be an electronic device. Optionally, the electronic device may be a server, a terminal device, or a robot, but the present application is not limited thereto.

[0059] Specifically, such as Figure 2 As shown, the process includes the following steps:

[0060] Step S201: acquiring target motion data, where the target motion data includes motion data of a plurality of key postures.

[0061] Exemplarily, a posture image of the target user is obtained through the robot's RGB-D camera, and motion data of multiple key postures of the target user is obtained through the posture image, and the motion data of the multiple key postures is used as target motion data. Specifically, the motion data of the multiple key postures may include the rotation angle and rotation direction of the target user's joint points, wherein the RGB-D camera refers to a camera that obtains an RGB image and a depth image through a physical ranging method.

[0062] More specifically, the RGB-D camera can collect point cloud data of the target user's joints. Furthermore, the camera's intrinsic parameters can be combined to determine the coordinates of any point in the camera coordinate system, or the camera's intrinsic and extrinsic parameters can be combined to determine the coordinates of any point in the world coordinate system. Furthermore, the rotation angle and direction of the joint can be determined based on the coordinate information of the joint points.

[0063] Step S202 : determining the similarity of the motion data of the multiple key postures.

[0064] Furthermore, the cosine similarity between the rotation angles of the plurality of key postures is determined. Specifically, the similarity between two key postures may be determined based on the rotation angles of the two key postures.

[0065] If the angle data of the first key pose is x1, and the angle data of the second key pose is x2, where the angle data of each key pose includes n angle data, that is, the angle data x1 of the first key pose is a set of n angle data, and the angle data x2 of the second key pose is a set of n angle data, then the cosine similarity between the first key pose and the second key pose can be determined by the following formula:

[0066]

[0067] Among them, Sim represents the cosine similarity between the first key pose and the second key pose, x 1k Represents the kth angle data of the first key posture, x 2k The k-th angle data of the second key posture is represented. Both the first key posture and the second key posture include data of n angles.

[0068] When the value of Sim is closer to 1, it means that the first key posture is more similar to the second key posture, and when it is closer to -1, it means that the first key posture is less similar to the second key posture.

[0069] It should be noted that the angle data of the key posture may refer to a combination of rotation angles of various joints.

[0070] Furthermore, the similarity between any two key postures in the plurality of key postures is traversed by the above formula (1).

[0071] Step S203: grouping the multiple key postures based on similarity.

[0072] Step S204: Select a posture to be imitated from the grouped key postures to perform gait imitation.

[0073] Furthermore, multiple key postures can be grouped according to the similarity between their rotation angles and rotation directions. Specifically, key postures with close similarity can be grouped together, thereby obtaining multiple groups of key postures with large similarity differences.

[0074] Furthermore, a posture to be imitated is selected from the grouped key postures for gait imitation. Specifically, one or more key postures can be selected from each group of key postures for gait imitation.

[0075] In the above implementation process, multiple key postures are grouped according to the similarity of their motion data, so that postures with close similarity can be grouped together. Furthermore, the postures to be imitated are selected from the grouped key postures for gait imitation, so that during the gait imitation process, the robot does not need to process a large amount of acquired raw data, but only needs to process the data of the key postures selected in the group, thereby reducing the amount of data processing, improving data processing efficiency, and further improving the fluency of the robot's gait imitation process.

[0076] In some embodiments, obtaining target motion data may include the following steps:

[0077] Step 1: Get initial motion data.

[0078] Step 2: Determine the target motion data based on the initial motion data and the key posture recognition model, which is trained through reinforcement learning.

[0079] Exemplarily, a posture image of the target user is acquired through the robot's RGB-D camera, and initial motion data of the target user is acquired through the posture image. The initial motion data includes the rotation angle and rotation direction of the target user's joints.

[0080] Furthermore, target motion data is determined based on the initial motion data and a key posture recognition model trained through reinforcement learning. Specifically, multiple target motion data are determined based on the rotation angle and rotation direction of the target user's joints and the key posture recognition model, thereby determining the multiple target motion data as multiple key postures.

[0081] In the above implementation process, the target motion data is determined according to the initial motion data and the key posture recognition model, so that the motion data corresponding to the key posture can be screened out from the initial motion data.

[0082] In some embodiments, determining target motion data based on the initial motion data and the key gesture recognition model may include the following steps:

[0083] Step 1: Determine the similarity threshold based on the key gesture recognition model.

[0084] Step 2: Determine the similarity between multiple actions in the initial action data.

[0085] Step 3: Determine key poses based on similarity and similarity threshold.

[0086] Step 4: Determine target motion data based on key poses.

[0087] For example, the similarity threshold Q is determined by a key posture recognition model, where the key posture recognition model can be trained using the Q-Learning algorithm in reinforcement learning. The reward function is set based on the robot's imitation performance. The better the robot's imitation performance, the higher the reward. The training method of the Q-Learning algorithm is as follows:

[0088] Q(s t ,a t )←Q(s t ,a t )+α(r t+1 +γmaxQ(s t+1 ,a)-Q(s t ,a t ))(2)

[0089] Among them, Q(s,a) represents the reward obtained by the robot when taking action a in state s, γ(0≤γ≤1) represents the time discount factor, α(0≤α≤1) represents the learning rate, t represents the learning step size, maxQ(s t+1 ,a) indicates that the robot is in state S t+1 The biggest reward ever received.

[0090] Furthermore, the similarity between two consecutive actions in the initial action data is determined.

[0091] Specifically, the rotation angle of each joint is calculated through inverse kinematics, and then the difference between the angle of the joint at the current moment and the angle of the joint at the previous moment is determined. The difference is the similarity between the current posture and the previous posture.

[0092] More specifically, the positions of the target user's joints at different acquisition times are identified based on the collected image data of the target user, and the angles of the joints are determined based on the positions at different acquisition times. For example, based on the position of the j joint at time i (x i ,y i ), and the position of joint j at time i+1 (x i+1 ,y i+1 ), determine the angle of joint j at time i+1, and each moment can correspond to a posture.

[0093] Furthermore, the difference between the angle of joint j in pose i and the angle of joint j in pose i+1 can be determined by the following formula:

[0094]

[0095] Among them, ρ i,i+1 represents the difference between the angle of joint j in pose i and the angle of joint j in pose i+1, n represents the number of initial actions, σ irepresents the standard deviation of posture i, σ i+1 represents the standard deviation of posture i+1, β i,j represents the angle of pose i in joint j, β i+1,j represents the angle of pose i+1 in joint j, represents the average value of all pose angles of joint j.

[0096] Furthermore, the difference between the angle of the joint at the current moment and the angle of the joint at the previous moment is ρ i,i+1 When the similarity is greater than the threshold Q, the posture at the current moment is determined as the key posture, that is, the action data at the current moment is determined as the target action data, so that multiple key postures can be determined from the initial action.

[0097] In the above implementation process, a similarity threshold is determined by the key gesture recognition model. Further, based on the similarity between multiple actions and the similarity threshold, multiple key gestures are determined from multiple actions, thereby achieving the screening of target action data.

[0098] In some embodiments, grouping a plurality of key gestures based on similarity may include the following steps:

[0099] Step 1: Determine the first Gaussian distribution of the target key pose, where the target key pose is any one of multiple key poses.

[0100] Step 2: Determine the key poses to be grouped in descending order based on the similarity with the target key pose. The key poses to be grouped are the key poses other than the target key pose.

[0101] Step 3: Determine the second Gaussian distribution of the key poses to be grouped.

[0102] Step 4: Group multiple key poses based on the divergence between the first Gaussian distribution and the second Gaussian distribution.

[0103] Exemplarily, any one of the multiple key postures is determined as a target key posture, and further, a first Gaussian distribution of the target key posture is determined.

[0104] Furthermore, similarities between the key postures other than the target key posture and the target key posture are determined, and the other key postures are sequentially determined as key postures to be grouped in descending order of similarity to the target key posture, and a second Gaussian distribution of the key postures to be grouped is determined.

[0105] Furthermore, the divergence between the first Gaussian distribution and the second Gaussian distribution is determined. Specifically, the divergence between the first Gaussian distribution and the second Gaussian distribution can be determined by the following formula:

[0106]

[0107] in, represents the divergence between the first Gaussian distribution and the second Gaussian distribution, t represents the time corresponding to the target key posture, n represents the data dimension, represents the first Gaussian distribution, p(x|θ i ) represents the second Gaussian distribution, Y i =Y i +y t , Y i Represents the grouping set corresponding to the i-th data, y t Represents the data of the target key posture selected at time t, θ i represents the i-th dimension of the target key pose, Represents the i-th dimension of the key pose to be grouped.

[0108] Furthermore, the key gestures to be grouped are grouped according to the divergence between the first Gaussian distribution and the second Gaussian distribution.

[0109] In the above implementation process, the key postures to be grouped are grouped according to the divergence of the Gaussian distribution between the target key posture and the key posture to be grouped, so that each group of key postures has a certain similarity, and then the similarity between each group of key postures is determined by the divergence.

[0110] In some embodiments, grouping the plurality of key gestures based on the divergence between the first Gaussian distribution and the second Gaussian distribution may include the following steps:

[0111] Step 1: If the divergence is less than the preset threshold, the key poses to be grouped and the target key poses are classified into one category.

[0112] Step 2: If the divergence is greater than or equal to the preset threshold, the next key posture to be grouped is selected for grouping until all key postures are grouped.

[0113] Exemplarily, a preset threshold η is determined, where the preset threshold η may be obtained through testing or may be preset.

[0114] If the divergence If the value is less than the preset threshold η, the current key posture to be grouped and the target key posture are classified into one category.

[0115] If the divergence If the value is greater than or equal to the preset threshold η, the current key posture to be grouped is determined as an ungrouped key posture, and the divergence between the Gaussian distribution of the next key posture to be grouped and the Gaussian distribution of the target key posture is determined, and the next key posture to be grouped is grouped according to the divergence until all the key postures to be grouped are grouped.

[0116] If the divergence between the Gaussian distribution of no key pose and the Gaussian distribution of the ungrouped key pose is less than a preset threshold, the grouped key poses are determined to be a group, and the parameters of the Gaussian distribution of the key poses can be updated by the expectation maximization algorithm (EM algorithm).

[0117] In the above implementation process, the distribution of any two key postures is further determined based on the divergence between the Gaussian distribution of the target key posture and the Gaussian distribution of the key postures to be grouped, and the key postures with a divergence less than a preset threshold are determined as a group, further improving the similarity between each group of key postures.

[0118] In some of the embodiments, after all key gestures are grouped, the following steps may be further included:

[0119] Step 1: Determine the Gaussian distribution of each set of key poses.

[0120] Step 2: Based on the preset conditions and the Gaussian distribution of each group of key postures, multiple groups of key postures are combined to obtain target groups.

[0121] For example, after all key gestures are grouped, the Gaussian distribution of each group of key gestures can be determined. Furthermore, if the Gaussian distributions of any two groups of key gestures meet preset conditions, the two groups of key gestures are merged into one group, and the parameters of the Gaussian distribution of the merged key gestures can be calculated by maximum likelihood estimation.

[0122] Specifically, the preconditions are:

[0123]

[0124] in, represents the preconditioning function for the first set of key poses, Represents the preset condition function of the second set of key poses, μ a Represents the mathematical expectation sequence of the Gaussian distribution of the first set of key postures, μ b The mathematical expectation sequence of the Gaussian distribution representing the second set of key postures, is a constant.

[0125] In the above implementation process, two groups of key postures that meet the preset conditions are merged into one group, so that the two groups of key postures with high similarity are merged.

[0126] In some of the embodiments, after grouping the multiple key gestures based on similarity, the method further includes: distributing and storing the grouped key gestures.

[0127] For example, after grouping multiple key gestures based on similarity, the grouped key gestures can also be distributedly stored, thereby distributing the multiple groups of key gestures across multiple other robotic devices. Specifically, when a robot receives data, it determines whether the data already exists or is new data in its database. If the data is new, it will be saved in the database, and an ID will be assigned to each group of key gesture data to distinguish each group of key gestures; if the data already exists, it will not be saved. Distributed storage solves the capacity, performance, and other bottlenecks of single-machine storage, as well as issues such as availability and scalability. By distributing data across multiple robotic devices, it provides large-capacity, high-performance, high-availability, and scalable storage services for large-scale storage applications.

[0128] When robots store a large amount of motion memory, it often leads to problems of increased system costs and decreased efficiency. In the above implementation process, distributed memory storage is introduced to improve the robot's memory storage efficiency, thereby reducing system costs.

[0129] In some embodiments, selecting a posture to be imitated from the grouped key postures for gait imitation may include the following steps:

[0130] Step 1: According to the action sequence of key poses, select a key pose from each group of key poses as the pose to be imitated.

[0131] Step 2: Perform Gaussian blur on each key pose to be imitated to obtain the blurred key pose.

[0132] Step 3: Perform gait imitation based on the blurred key poses.

[0133] For example, when the motion data of the key postures are obtained, the motion sequence of each key posture is also obtained. Therefore, when selecting the posture to be imitated from the grouped key postures for gait imitation, one key posture can be selected from each group of key postures as the posture to be imitated according to the motion sequence of the key postures.

[0134] Furthermore, each posture to be imitated is Gaussian blurred to obtain the blurred key posture. Specifically, the existing Gaussian normal distribution density function is:

[0135]

[0136] Where μ is the mean of x and σ is the standard deviation of x.

[0137] Therefore, the key pose after blurring can be determined by the following formula:

[0138]

[0139] Where x represents the horizontal coordinate of the blurred key pose, y represents the vertical coordinate of the blurred key pose, and σ is the standard deviation of x.

[0140] Furthermore, gait imitation is performed based on the blurred key postures.

[0141] In the above implementation process, Gaussian blurring is performed on each posture to be imitated, which can reduce the jitter between continuous action data and further improve the smoothness of the robot's gait imitation.

[0142] In some of the embodiments, before gait imitation is performed based on the blurred key posture, the method further includes: performing anti-discretization processing on the blurred key posture.

[0143] For example, since each key posture is stored in the form of discrete data, when gait simulation is performed, the discrete data of the grouped key postures is also obtained, which causes the robot to have posture unsmoothness and delay when imitating gait. In order to maintain the smoothness and stability of gait simulation, the blurred key postures are de-discretized.

[0144] Specifically, the data corresponding to the blurred key posture of the robot's gait imitation is After N perturbations, a set of perturbation sequences is obtained: {X (1) ,X (2) ,…,X (N)}.

[0145] Furthermore, the error sequence Δ between the data before and after the disturbance is determined k .

[0146] Furthermore, according to the error sequence Δ k , use Gaussian distribution to calculate the weight of each key posture and get the weight sequence in, The calculation formula is:

[0147]

[0148] in, Represents the N values after disturbance and the corresponding values before disturbance The error between .

[0149] Furthermore, the anti-discretized value of each blurred key posture is determined by the perturbed value and the corresponding weight. Specifically, the anti-discretized value of each blurred key posture can be determined by the following formula:

[0150]

[0151] in, Indicates the anti-discretized value corresponding to the blurred key posture, represents the key pose after blurring, represents the weight corresponding to the blurred key pose, and N represents the number of perturbations.

[0152] In the above implementation process, each blurred key posture is discretized, thereby further improving the smoothness and stability of the robot's gait imitation.

[0153] Figure 3 This is a flow chart of an embodiment of a gait imitation method provided by the present application, such as Figure 3 As shown, the flow chart includes:

[0154] Step S301: Determine the motion data of the key posture.

[0155] Specifically, the initial motion data is obtained through an RGB-D camera, and the key posture recognition model obtained by reinforcement learning and the above formula (3) are used to filter out the key posture motion data from the initial motion data.

[0156] Step S302: Determine the similarity of the motion data of multiple key postures.

[0157] Furthermore, the similarity of the action data of any two key postures is obtained through the above formula (1).

[0158] Step S303: Grouping multiple key gestures based on similarity.

[0159] Furthermore, multiple key postures are grouped according to the similarity and the above formula (4).

[0160] Step S304: Distributed storage of grouped key posture data.

[0161] Furthermore, the grouped key postures are distributedly stored on multiple storage devices. Specifically, the multiple storage devices may be multiple robots or other devices, which are not limited here.

[0162] For example, if the multiple storage devices are multiple robots, when a robot receives data, it will determine whether the data already exists or is new in its database. If the data is new, it will be saved to the database and an ID will be assigned to each new key pose set. If the data already exists, it will not be saved.

[0163] Step S305: determining the posture data to be replayed from the grouped key posture data, and performing Gaussian blur on the posture data to be replayed.

[0164] Furthermore, according to the action sequence of the key gestures, one key gesture is selected from each group of key gestures as the gesture data to be replayed, and Gaussian blur processing is performed on each gesture data to be replayed according to the above formula (7).

[0165] Step S306: collaboratively optimize the posture data to be replayed after Gaussian blurring, and perform posture replay based on the optimized data.

[0166] Furthermore, according to the above formulas (8) and (9), the posture data to be replayed after Gaussian blurring is subjected to anti-discrete processing, that is, collaborative processing, so as to obtain processed posture data. Further, posture replay is performed based on the processed posture data.

[0167] It should be noted that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0168] This embodiment also provides a gait mimicking device for implementing the aforementioned embodiments and preferred embodiments. Details already described will not be repeated. The terms "module," "unit," "subunit," etc., used below, may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented using software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0169] Figure 4 This is a structural block diagram of a gait imitation device provided in an embodiment of the present application. Figure 4 As shown, the device includes:

[0170] The acquisition module 401 is used to acquire target motion data, where the target motion data includes motion data of multiple key postures.

[0171] The determination module 402 is configured to determine the similarity of the motion data of multiple key postures.

[0172] The grouping module 403 is configured to group the multiple key gestures based on similarity.

[0173] The imitation module 404 is used to select a posture to be imitated from the grouped key postures to perform gait imitation.

[0174] In some embodiments, the acquisition module 401 is specifically configured to:

[0175] Get initial action data;

[0176] The target motion data is determined based on the initial motion data and the key posture recognition model, and the key posture recognition model is trained through reinforcement learning.

[0177] In some embodiments, the acquisition module 401 is specifically configured to:

[0178] Determining a similarity threshold based on a key gesture recognition model;

[0179] Determine the similarity between multiple actions in the initial action data;

[0180] Determine key poses based on similarity and similarity threshold;

[0181] Determine target motion data based on key poses.

[0182] In some embodiments, the grouping module 403 is specifically configured to:

[0183] Determine a first Gaussian distribution of a target key pose, where the target key pose is any one of the multiple key poses;

[0184] Determine the key postures to be grouped in descending order based on the similarity with the target key posture, and the key postures to be grouped are the key postures other than the target key posture;

[0185] Determine the second Gaussian distribution of key gestures to be grouped;

[0186] The plurality of key poses are grouped based on the divergence between the first Gaussian distribution and the second Gaussian distribution.

[0187] In some embodiments, the grouping module 403 is specifically configured to:

[0188] If the divergence is less than the preset threshold, the key posture to be grouped and the target key posture are classified into one category;

[0189] If the divergence is greater than or equal to the preset threshold, the next key posture to be grouped is selected for grouping until all key postures are grouped.

[0190] In some embodiments, the grouping module 403 is further configured to:

[0191] Distributed storage of grouped key poses.

[0192] In some embodiments, the imitation module 404 is specifically configured to:

[0193] According to the action sequence of key postures, one key posture is selected from each group of key postures as the posture to be imitated;

[0194] Perform Gaussian blur on each key posture to be imitated to obtain the blurred key posture;

[0195] Gait imitation based on blurred key poses.

[0196] In some embodiments, the simulation module 404 is further configured to:

[0197] De-discretize the blurred key poses.

[0198] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 5 As shown, Figure 5 This is a schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. The computer device includes a processor, a memory, and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store endoscope light source brightness adjustment data. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a gait imitation method is implemented.

[0199] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0200] In one embodiment, an electronic device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0201] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0202] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0203] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0204] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0205] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A gait imitation method, applied to a robot, characterized in that: The method comprises: Acquiring target motion data, wherein the target motion data includes motion data of a plurality of key postures; Determining similarity of motion data of a plurality of said key postures; grouping the plurality of key gestures based on the similarity; Select the key posture to be imitated from the grouped key postures to perform gait imitation; The grouping of the plurality of key gestures based on the similarity comprises: determining a first Gaussian distribution of a target key pose, wherein the target key pose is any one of the plurality of key poses; Determining key postures to be grouped in descending order of similarity to the target key posture, wherein the key postures to be grouped are key postures other than the target key posture; Determining a second Gaussian distribution of the key gestures to be grouped; grouping the plurality of key gestures based on a divergence between the first Gaussian distribution and the second Gaussian distribution; The acquiring target action data comprises: Get initial action data; Determining the target motion data based on initial motion data and a key posture recognition model, wherein the key posture recognition model is obtained through reinforcement learning training; Determining the target motion data based on the initial motion data and the key posture recognition model includes: determining a similarity threshold based on the key gesture recognition model; Determining similarities between multiple actions in the initial action data; determining the key gesture based on the similarity and a similarity threshold; The target motion data is determined based on the key pose.

2. The method according to claim 1, characterized in that The grouping of the plurality of key gestures based on the divergence between the first Gaussian distribution and the second Gaussian distribution includes: If the divergence is less than a preset threshold, the key posture to be grouped and the target key posture are classified into one category; If the divergence is greater than or equal to the preset threshold, the next key gesture to be grouped is selected for grouping until all key gestures are grouped.

3. The method according to claim 1, characterized in that After grouping the plurality of key gestures based on the similarity, the method further includes: Distributed storage of grouped key poses.

4. The method according to claim 1, wherein The step of selecting a key posture to be imitated from the grouped key postures to perform gait imitation includes: According to the action sequence of the key postures, one key posture is selected from each group of key postures as the key posture to be imitated; Perform Gaussian blur on each key posture to be imitated to obtain the blurred key posture; Gait imitation is performed based on the blurred key postures.

5. The method according to claim 4, characterized in that Before performing gait imitation based on the blurred key posture, the method further includes: Anti-discretization processing is performed on the blurred key posture.

6. A gait imitation device, characterized in that: include: An acquisition module, configured to acquire target motion data, wherein the target motion data includes motion data of a plurality of key postures; a determination module, configured to determine similarities of motion data of a plurality of key postures; a grouping module, configured to group the plurality of key gestures based on the similarity; An imitation module is used to select a key posture to be imitated from the grouped key postures to perform gait imitation; The grouping module is further configured to determine a first Gaussian distribution of a target key posture, wherein the target key posture is any one of the plurality of key postures; determine key postures to be grouped in descending order of similarity to the target key posture, wherein the key postures to be grouped are key postures other than the target key posture; Determining a second Gaussian distribution of the key gestures to be grouped; grouping the plurality of key gestures based on a divergence between the first Gaussian distribution and the second Gaussian distribution; The acquiring target motion data comprises: acquiring initial motion data; determining the target motion data based on the initial motion data and a key posture recognition model, wherein the key posture recognition model is obtained through reinforcement learning training; Determining the target action data based on the initial action data and the key posture recognition model includes: determining a similarity threshold based on the key posture recognition model; determining the similarity between multiple actions in the initial action data; determining the key posture based on the similarity and the similarity threshold; and determining the target action data based on the key posture.

7. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the gait imitation method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the gait imitation method according to any one of claims 1 to 5 are implemented.

Images