Human Posture Restoration Method, Device, Terminal Device and Storage Medium

Abstract

This application is applicable to the field of motion capture technology, and provides a human body pose restoration method, device, terminal device, and storage medium. The human body pose restoration method includes: obtaining target motion data related to a target motion scene, where the target motion scene refers to the scene to which the human body's motion belongs; determining a target installation position of a motion capture device on the human body according to the target motion data; training a neural network according to the target motion data and the target installation position on the human body; obtaining motion data captured by the motion capture device when the motion capture device is installed at the target installation position; and inputting the motion data captured by the motion capture device into the trained neural network to restore the motion pose of the human body. Using the above method can solve the problems in the prior art that the motion capture device is complex to wear and the restoration accuracy is not high for specific motion scenes.

Description

Technical Field

[0001] This application belongs to the technical field of motion capture, and particularly relates to a human body pose restoration method, device, terminal device, and storage medium. Background Art

[0002] Motion capture technology has currently been widely applied in fields such as gait analysis, biomechanics, medical rehabilitation, human-computer interaction and control. More and more motion capture devices appear in people's lives. For example, motion capture devices for animation production, motion capture devices for realizing the interaction between humans and virtual environments, and motion capture devices for analyzing human physical training, etc. However, in order to obtain more human motion data, existing motion capture devices are installed in excessive numbers on the human body, which not only has problems of time-consuming wearing and complex installation, but also causes a burden on the normal movement of the human body, resulting in inaccurate measurement of the obtained human motion data. Therefore, how to make the motion capture device easy to wear and not affect the normal movement of the human body has become an important problem that needs to be solved urgently. Summary of the Invention

[0003] Embodiments of this application provide a human body pose restoration method, device, terminal device, and storage medium, which can solve the problems of complex wearing and inaccurate measurement of motion data in the prior art.

[0004] The first aspect of the embodiments of this application provides a human body pose restoration method, and the human body pose restoration method includes:

[0005] Obtain target motion data related to a target motion scene, where the target motion scene refers to the scene to which the human body's motion belongs;

[0006] Determine the target installation position of the motion capture device on the human body according to the target motion data;

[0007] When the motion capture device is already installed at the target installation position, obtain the motion data captured by the motion capture device;

[0008] Train a neural network according to the target motion data and the target installation position on the human body;

[0009] Input the motion data captured by the motion capture device into the trained neural network to restore the motion pose of the human body.

[0010] The second aspect of the embodiments of this application provides a human body pose restoration device, and the human body pose restoration device includes:

[0011] A target acquisition module, configured to obtain target motion data related to a target motion scene, where the target motion scene refers to the scene to which the human body's motion belongs;

[0012] A position determination module, configured to determine a target installation position of the motion capture device on the human body according to the target motion data;

[0013] A motion data acquisition module, configured to acquire motion data captured by the motion capture device when the motion capture device is installed at the target installation position;

[0014] A training module, configured to train a neural network according to the target motion data and the target installation position on the human body;

[0015] A restoration module, configured to input the motion data captured by the motion capture device into the trained neural network to restore the motion posture of the human body.

[0016] A third aspect of the embodiments of the present application provides a terminal device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, where when the processor executes the computer program, the human body posture restoration method described in the first aspect above is implemented.

[0017] A fourth aspect of the embodiments of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the human body posture restoration method described in the first aspect above is implemented.

[0018] A fifth aspect of the embodiments of the present application provides a computer program product, when the computer program product runs on a terminal device, enabling the terminal device to execute the human body posture restoration method described in the first aspect above.

[0019] The beneficial effects of the embodiments of the present application compared with the prior art are as follows: In the embodiments of the present application, first, target motion data related to the target motion scenario is obtained. Based on this target motion data, the target installation position of the motion capture device on the human body can be determined. This target installation position is the optimal installation position of the motion capture device on the human body. According to this target installation position, the number of motion capture devices to be installed on the human body can be obtained. This number is less than the number of motion capture devices installed on all parts of the human body in the prior art. Therefore, the number of motion capture devices to be installed in the present application is less, which can save the installation cost and is more convenient to wear. Secondly, since the target installation position is the optimal installation position of the motion capture device on the human body, these positions can obtain the human body motion information to the greatest extent under the condition of a certain number of sensors, so that the motion data captured by the motion capture device is motion data with higher accuracy in restoring the human body posture. Finally, by inputting the motion data captured by the motion capture device into the trained neural network, the motion posture of the human body can be restored. By adopting the above technical solutions, the problems of complex wearing of motion capture devices and low restoration accuracy in specific application scenarios in the prior art can be solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the prior art description. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0021] Figure 1 is a schematic flowchart of a human body posture restoration method provided in Embodiment 1 of the present application;

[0022] Figure 2 is a schematic flowchart of a human body posture restoration method provided in Embodiment 2 of the present application;

[0023] Figure 3 is a schematic diagram of the positions to be installed of the motion capture device on the human body;

[0024] Figure 4 is a schematic diagram of the target installation positions of the motion capture device on the human body;

[0025] Figure 5 is a schematic structural diagram of a human body posture restoration device provided in Embodiment 3 of the present application;

[0026] Figure 6 is a schematic structural diagram of a terminal device provided in Embodiment 4 of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0027] In the following description, specific details such as specific system architectures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present application. However, those skilled in the art should understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0028] It should be understood that when used in the specification and appended claims of the present application, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0029] It should also be understood that the term "and / or" used in the specification and appended claims of the present application refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0030] In addition, in the description of the specification and appended claims of the present application, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0031] The reference to "one embodiment" or "some embodiments" etc. described in the specification of the present application means that a specific feature, structure, or characteristic described in connection with that embodiment is included in one or more embodiments of the present application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "comprising", "including", "having", and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0032] It should be understood that the magnitude of the sequence numbers of the steps in this embodiment does not mean the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0033] To illustrate the technical solution of the present application, specific embodiments are used below for illustration.

[0034] Refer to Figure 1 , which shows a schematic flowchart of a human pose restoration method provided in the first embodiment of the present application. This human pose restoration method is applied to a terminal device. As shown in the figure, the human pose restoration method may include the following steps:

[0035] Step 101, obtain target motion data related to a target motion scenario.

[0036] The target motion scenario refers to the scenario to which the human body's motion belongs, and the target motion scenario includes, but is not limited to, motion scenarios such as walking, running, jumping, playing basketball, playing table tennis, etc. The target motion data refers to the motion data related to the target motion scenario obtained from the human motion database, and the data types of the target motion data include, but are not limited to, acceleration, attitude angle, speed, etc.

[0037] Optionally, obtaining target motion data related to a target motion scenario includes:

[0038] Obtain sample motion data related to the target motion scenario, where the quantity of the sample motion data is less than that of the target motion data;

[0039] According to the sample motion data, obtain the target motion data from the human motion database.

[0040] In the embodiments of the present application, the sample motion data related to the target motion scenario can be randomly collected according to the target motion scenario. For example, if the target motion scenario is a walking motion scenario, the sample motion data can be collected from various walking tests, including the gait data of healthy subjects and the gait data of patients.

[0041] Among them, the sample motion data related to the target motion scenario is used as reference data. The purpose of using the sample motion data as reference data is: when obtaining the target motion data, a certain reference standard is given, so as to obtain more target motion data from the human motion database based on less sample motion data. Training the neural network with more target motion data can improve the accuracy of the neural network. Using more target motion data to determine the target installation position of the motion capture device on the human body can improve the accuracy of the target installation position.

[0042] Optionally, according to the sample motion data, obtaining the target motion data from the human motion database includes:

[0043] Obtain candidate motion data from the human motion database, where the quantity of the candidate motion data is less than or equal to the quantity of the motion data in the human motion database;

[0044] According to the candidate motion data and the sample motion data, use a heuristic algorithm to calculate the value function of the candidate motion data sequence combination one by one;

[0045] Determine the candidate motion data sequence combination when the value function takes the maximum value as the target motion data.

[0046] In an embodiment of the present application, first, motion data is obtained from a human motion database. This motion data can be referred to as the selected motion data in the human motion database. The selected motion data in the human database can be directly used as candidate motion data, and the candidate motion data includes candidate motion sequences. According to the candidate motion data and sample data, the value function of the candidate motion data sequence combination is calculated. When the value function of the candidate motion data sequence combination reaches the maximum value, the candidate motion data sequence combination is determined as the target motion data. Among them, the candidate motion data sequence combination is a combination of any data sequences in the candidate motion data.

[0047] Optionally, obtaining candidate motion data from the human motion database further includes:

[0048] Calculating the similarity between the sample motion data and each motion data in the human motion database;

[0049] If there is motion data in the human motion database whose similarity to the sample motion data is greater than the similarity threshold, then obtain the number of data frames of the data sequence in the motion data whose similarity to the sample motion data is greater than the similarity threshold;

[0050] If there is a data sequence in the motion data whose similarity to the sample motion data is greater than the similarity threshold and the number of data frames of the data sequence is greater than the preset number, then determine the data sequence as candidate motion data.

[0051] In an embodiment of the present application, first, motion data is obtained from a human motion database. This motion data can be referred to as the selected motion data in the human motion database. Then, sample data is obtained from the sample motion data, and the similarity between the sample motion data and the selected motion data in the human motion database is calculated. If the similarity between the sample motion data and the selected motion data in the human motion database is greater than the similarity threshold, then determine the selected motion data as candidate motion data. Secondly, continue to obtain motion data from the human motion database and perform the above similarity judgment, and determine the motion data that satisfies the similarity to the sample data being greater than the similarity threshold as candidate motion data until all the motion data in the human motion database has been obtained and judged, so as to determine all the candidate motion data in the human motion database. Finally, according to the candidate motion data and sample data, the value function of the candidate motion data sequence combination is calculated. When the value function of the candidate motion data sequence combination reaches the maximum value, the candidate motion data sequence combination is determined as the target motion data. Among them, the candidate motion data sequence combination is a combination of any data sequences in the candidate motion data.

[0052] Exemplarily, assume that the motion data set contained in the human motion database is D A , since the period of most motion data is 50 frames, D can be divided into units of 50 frames ADivide it into F equal parts, and at the same time divide the sample motion data into E equal parts with 50 frames as a unit, where both F and E are positive integers. First, randomly select a data unit from the F equal parts as data unit 1. At the same time, select a data unit from the E equal parts of the sample motion data as data unit 2. Secondly, use the cosine angle to calculate the similarity between this data unit 1 and data unit 2, and the specific calculation method of the similarity can be expressed as follows:

[0053]

[0054] Among them, N refers to the number of frames in the data unit, and generally N is taken as 50; u1 is data unit 1, and u2 is data unit 2; x 1i refers to the i-th frame motion data in data unit 1, and x 2i refers to the i-th frame motion data in data unit 2; <x 1i ,x 2i > refers to the dot product calculation between x 1i and x 2i ; ||x 1i ||, ||x 2i || respectively refer to the length of the i-th frame motion data in data unit 1 and the length of the i-th frame motion data in data unit 2.

[0055] If the value of d(u1, u2) is greater than the similarity threshold, then determine that data unit 1 is a unit in the candidate motion data. Among them, the similarity threshold is generally set to 0.89. Traverse all data units in the human motion database, and take the data units with a similarity greater than 0.89 to the sample data unit to form the candidate motion data.

[0056] It should be noted that the data type in the selected data unit 1 is the same as the data type of the selected data unit 2, so as to avoid large errors caused by mutual comparison between different data types when obtaining the target motion data. For example, if the data type of data unit 1 is the pose angle, then the data type of the selected data unit 2 is also the pose angle.

[0057] It should also be noted that in the embodiment of the present application, the motion data in the human motion database is included in the motion data set, and the motion data set is included in the human motion database. Among them, the motion data includes data sequences, and each data sequence includes H data frames, where H is a positive integer.

[0058] In an embodiment of the present application, first, motion data with a similarity greater than a similarity threshold to the sample data is obtained from a human motion database, and then the number of data frames in the data sequence of the motion data is obtained. Secondly, the number of data frames in the data sequence of the obtained motion data is compared with a preset number. If the number of data frames in the data sequence of the obtained motion data is greater than the preset number, then the data sequence is determined as candidate motion data.

[0059] Among them, the motion data with a similarity greater than the similarity threshold to the sample motion data is the motion data that meets the similarity condition. After the candidate motion data is operated by a value function, target motion data can be obtained, and the target motion data can be used for the training of Bi-RNN in the process of human pose restoration. Since the data sequence length in Bi-RNN is 300 frames, if the input sequence length is less than 300 frames of data, it will cause inaccurate training of Bi-RNN. Therefore, motion data with a sequence length greater than 300 frames should be selected from the motion data that meets the similarity condition. Also, since the motion data with a shorter sequence length cannot express the coherence of an action in human motion and has a different motion type from the sample motion data, the motion data with a shorter sequence length is redundant data in the motion data that meets the similarity condition and can be deleted to improve the accuracy of the neural network and the accuracy of the target installation position. Therefore, motion data with a longer sequence length can be selected from the motion data that meets the similarity condition. For example, motion data with a sequence length of more than 1200 frames can be selected as the target motion data, that is, the above preset number is set to 1200.

[0060] Optionally, according to the candidate motion data and the sample motion data, calculating the value function of the candidate motion data sequence combination one by one using a heuristic algorithm includes:

[0061] Obtaining candidate data features and sample data features, where the candidate data features refer to the data features of the candidate motion data, and the sample data features refer to the data features of the sample motion data;

[0062] According to the candidate data features and the sample data features, calculating the probability distribution difference between the candidate motion data and the sample motion data;

[0063] According to the candidate data features, calculating the information amount of the candidate motion data;

[0064] According to the probability distribution difference between the candidate motion data and the sample motion data, the information amount of the candidate motion data, and the number of data frames included in the candidate motion data, calculating the value function of the candidate motion data sequence combination one by one using a heuristic algorithm.

[0065] In the embodiments of the present application, each of the above data frames includes L data features, where L is a positive integer, and the direction and magnitude of the human body movement can be distinguished according to the data features. The candidate data features are a subset of the candidate motion data, and the candidate data features are the smallest units in the candidate motion data. Similarly, the sample data features are the smallest units in the sample motion data.

[0066] In a specific implementation, each candidate data feature and each sample data feature have marginal distribution probabilities. First, calculate the marginal distribution probability of each candidate data feature in the candidate motion data, and at the same time calculate the marginal distribution probability of each sample data feature in the sample motion data. Secondly, summing the marginal distribution probabilities of each candidate data feature can obtain the joint probability distribution of the candidate motion data, and summing the marginal distribution probabilities of each sample data feature can obtain the joint probability distribution of the sample motion data. Finally, subtracting the joint probability distribution of the candidate motion data from the joint probability distribution of the sample motion data can obtain the probability distribution difference between the candidate motion data and the sample motion data.

[0067] Exemplarily, the present application uses relative entropy (also known as Kullback-Leibler divergence) to measure the difference in marginal probability distributions between candidate data features and sample data features. Specifically, the probability distribution difference between the candidate motion data and the sample motion data can be calculated through the following formula:

[0068]

[0069] where D B refers to the candidate motion data, and D ref refers to the sample motion data; N f refers to the number of candidate data features; in the present application, each feature is divided into 20 intervals according to the maximum and minimum values of each feature in the candidate motion data and the sample motion data, that is, x is divided into 20 intervals according to the distribution range of x, χ refers to one of the intervals, and x refers to the value in the interval; P i (x) refers to the marginal probability distribution of the i-th candidate data feature of the candidate motion data D B , and Q i (x) refers to the marginal probability distribution of the i-th sample data feature of the sample motion data D ref . The specific calculation process of P i (x) can be expressed as follows:

[0070]

[0071] where K refers to the number of data sequences of the candidate motion data; l k refers to the number of data frames included in the k-th data sequence in the candidate motion data; Ni The specific calculation method of (x) can be expressed by the following formula:

[0072]

[0073] where δ kji (x) refers to the i-th data feature in the j-th frame of the k-th data sequence in the candidate motion data. The specific calculation method can be expressed by the following formula:

[0074]

[0075] In the embodiments of the present application, the calculation method of Q i (y) is the same as that of the above P i (x). The specific calculation process can be expressed as follows:

[0076]

[0077]

[0078]

[0079] where K q refers to the number of data sequences of the sample motion data; l kq refers to the number of data frames included in the k-th data sequence of the sample motion data, and δ kqji (x) refers to the i-th data feature in the j-th frame of the k-th data sequence of the sample motion data.

[0080] In the embodiments of the present application, the information amount of the candidate motion data may refer to the information entropy of the candidate motion data. In order to ensure that the information amount of the candidate motion data remains unchanged while reducing the number of candidate motion data to obtain more accurate target motion data, redundant data of the candidate motion data can be deleted to improve the operation efficiency of the system.

[0081] Exemplarily, the method for calculating the information entropy of the candidate motion data can be specifically implemented in the following manner:

[0082]

[0083] where H(D B ) refers to the information amount of the candidate motion data; N f refers to the number of candidate data features.

[0084] The number of data frames included in the candidate motion data can be obtained through the following calculation formula:

[0085]

[0086] where N(DB ) refers to the number of data frames included in the candidate motion data.

[0087] According to the above formula, using a heuristic algorithm, the value function of the data sequence in the candidate motion data can be specifically expressed as:

[0088]

[0089] Among them, α refers to the value coefficient; N f refers to the number of candidate data features and sample data features; when D selected from the candidate motion data D B maximizes the above formula, D is the best, and determine D at this time as the target motion data, where D refers to the candidate motion data sequence combination.

[0090] It should be noted that this application can select data with a smaller number of data frames according to ; according to the redundant data existing in the candidate motion data can be deleted, and then the best candidate motion data sequence combination D can be obtained through a greedy algorithm (a kind of heuristic algorithm), and D is determined as the target motion data.

[0091] Step 102: Determine the target installation position of the motion capture device on the human body according to the target motion data.

[0092] In the embodiment of this application, the motion capture device may refer to an inertial measurement unit (Inertial Measurement Unit, IMU). The IMU mainly consists of three micro-electro-mechanical system (MEMS) acceleration sensors, three gyroscopes and a solution circuit. The MEMS acceleration sensor is a sensor that measures the inertial force of the sensing mass and is usually composed of a standard mass block (sensing element) and a detection circuit.

[0093] In the embodiments of the present application, according to the idea of selecting as few motion capture devices as possible to accurately restore the human motion posture, the target installation positions of the motion capture devices on the human body are determined. First, when selecting the positions of the motion capture devices, the amount of motion data captured by the motion capture devices at different positions needs to be considered. After selecting a position, the subsequent positions can be selected according to the amount of motion data captured by the motion capture device at this position. Therefore, it is necessary to maximize the correlation between the motion capture devices at the selected positions and the motion capture devices at the unselected positions. Secondly, although there is a correlation between the motion capture devices at different positions, there is also redundant data information among the motion data captured by the motion capture devices at the selected positions. In order to enable the motion capture devices at the target installation positions to accurately restore the human motion posture, the maximum correlation and minimum redundancy feature selection algorithm can be used to determine the target installation positions of the motion capture devices on the human body.

[0094] Step 103, when the motion capture devices are already installed at the target installation positions, obtain the motion data of the motion capture devices.

[0095] In the embodiments of the present application, when the motion capture devices are already installed at the target installation positions, the motion data captured by the motion capture devices includes the acceleration W a S relative to the world coordinate system and the rotation matrix where the rotation matrix refers to the orientation of the motion capture device in the world coordinate system, and at the same time, obtain the orientation of each bone of the human body in the current posture By matching the orientation of the motion capture device in the world coordinate system with the known orientation of each bone in the current posture the rotation matrix from the motion capture device to the human body can be determined. The determination of the rotation matrix from the motion capture device to the human body can be specifically expressed as follows:

[0096]

[0097] where refers to the rotation matrix from the human body to the motion capture device, and calculating the inverse matrix of can obtain the rotation matrix from the motion capture device to the human body.

[0098] Since the direction of the motion data input to the neural network should be consistent with the motion direction of the human body, that is, no matter which direction the human body moves in, the direction of the motion data captured by the motion capture device should be consistent with the direction of the motion data input to the neural network, so it is necessary to standardize the motion capture devices installed on the human bones relative to the human spine. The calculation of the standardized orientation and acceleration of each bone can be expressed as follows:

[0099]

[0100]

[0101] Among them, refers to the rotation matrix of the world coordinate system relative to the root bone; refers to the normalized rotation matrix of each bone; Root a B refers to the normalized speed of each bone; W a B refers to the acceleration of the bone relative to the world coordinate system; W a Root refers to the acceleration of the root bone relative to the world coordinate system, where the root bone refers to the human spine.

[0102] Optionally, after obtaining the motion data captured by the motion capture device, it further includes:

[0103] Normalize the motion data captured by the motion capture device according to the preset posture of the human bones to obtain the normalized motion data. The posture of the human bones refers to the rotation matrix of the local coordinate system on the human bones relative to the world coordinate system;

[0104] In the embodiments of the present application, the rotation matrix of the local coordinate system on the human bones relative to the world coordinate system is composed of the rotation angles of the local coordinate system on the human bones relative to the world coordinate system. Assume that there are W bones on the human body where the motion capture device can be installed, and the normalized motion data includes the normalized rotation matrices and accelerations of V bones, where both W and V are positive integers, and V is less than W.

[0105] Step 104, train the neural network according to the target motion data and the target installation positions on the human body.

[0106] In the embodiments of the present application, the target motion data can be used for training the neural network, and this neural network is used to restore the motion postures of the human body. Among them, the human motion database includes motion data in different motion scenarios, and the data in the human motion database is random and arbitrary, without labels corresponding to the motion types. Therefore, in the target motion scenario, if the motion data in the human motion database is directly used to train the neural network, the trained neural network has no pertinence and cannot accurately restore the motion postures of the human body in the target motion scenario. However, in the present application, by using the target motion data related to the target motion scenario in the human motion database and the motion data captured by the motion capture device at the target installation positions on the human body to train the neural network, the neural network can be trained specifically to ensure that the neural network can more accurately restore the motion postures of the human body in the target motion scenario.

[0107] Among them, the neural network trained according to the target motion data and the target installation positions on the human body includes, but is not limited to, a bidirectional recursive neural network (Bi-RNN).

[0108] Step 105: Input the motion data captured by the motion capture device into the trained neural network to restore the motion posture of the human body.

[0109] In the embodiment of the present application, the motion data captured by the motion capture device includes the normalized rotation matrices and accelerations of V bones. Input the above-mentioned normalized rotation matrices and accelerations of the V bones into the trained neural network, and the output is the normalized axis angles and accelerations of the remaining bones. By obtaining the normalized axis angles and accelerations of all the bones of the human body, the motion posture of the human body can be restored, where the remaining bones are the remaining bones obtained by subtracting the V bones from the bones on which the motion capture device can be installed.

[0110] Among them, the normalized axis angle refers to the rotation axis and rotation angle of the local coordinate system on the human bone relative to the world coordinate system.

[0111] In the embodiment of the present application, first, obtain the target motion data related to the target motion scene. According to the target motion data, the number of motion capture devices to be installed on the human body and the target installation positions of the motion capture devices can be determined. The target installation position is the optimal installation position of the motion capture device on the human body. According to the installation position, the number of motion capture devices to be installed on the human body can be obtained. This number is less than the number of motion capture devices installed on each part of the human body in the prior art. The small number of motion capture devices can save the installation cost and is convenient to wear. Finally, input the motion data captured by the motion capture device into the trained neural network, and output the measurement data at the bones where the motion capture devices are not installed, and then the motion posture of the human body can be restored. By adopting the above technical solution, the running speed of the restoration model in the existing human posture restoration method can be improved, and the restoration accuracy can be improved.

[0112] Refer to Figure 2 , which shows a schematic flowchart of a human posture restoration method provided in the second embodiment of the present application. As shown in the figure, the human posture restoration method may include the following steps:

[0113] Step 201: Obtain the target motion data related to the target motion scene.

[0114] Step 201 in this embodiment is similar to Step 101 in the foregoing embodiment and can be referred to each other. This embodiment will not be elaborated herein.

[0115] Step 202: Obtain the to-be-installed positions of the motion capture devices on the human body, and determine that the motion capture device with the highest correlation of the motion data captured by all the to-be-installed motion capture devices except itself among all the to-be-installed motion capture devices is the first motion capture device.

[0116] In the embodiment of the present application, to obtain the to-be-installed positions of the motion capture devices on the human body, it is necessary to perform a normalization transformation of the motion capture devices installed on the human bones relative to the root bone. Since the motion data captured by all motion capture devices except those installed on the human hip need to be converted into a normalized format relative to the hip, the to-be-installed positions refer to the installation positions of the remaining to-be-installed devices except the hip.

[0117] Specifically, as Figure 3 shown, there can be 20 to-be-installed positions of the motion capture devices on the human body, including hands, forearms, arms, legs, thighs, feet, shoulders, spines, heads, necks, etc. The hip, as a necessary installation position, does not belong to the to-be-installed positions in the embodiment of the present application.

[0118] Exemplarily, the to-be-installed motion capture devices at the 20 to-be-installed positions are all the to-be-installed motion capture devices in the embodiment of the present application. The installation position of the first motion capture device can be determined by calculating the correlation between all the to-be-installed motion capture devices and other to-be-installed motion capture devices except itself. For example, assume that any motion capture device among all the to-be-installed motion capture devices is X, and the other to-be-installed motion capture devices are Y. Then the calculation method of the correlation can be specifically expressed as follows:

[0119]

[0120] Among them, Rele(X; Y) refers to the correlation between X and Y; range(X) refers to the rotation amplitude of the position of the first motion capture device, range(Y) refers to the rotation amplitude of the position of the motion capture device Y; S refers to the dimension of the data captured by each motion capture device; x i is the i-th motion data captured by the first motion capture device, y j is the i-th motion data captured by the motion capture device at the Y position, and MIC(x i , y j ) refers to the maximum information coefficient of x i and y j . The specific calculation method of MIC(x i , y j ) is as follows:

[0121]

[0122]

[0123] Among them, B is a parameter, usually taking the value of x i or y j to the 0.6th power of the number of data frames; p(x i , y j ) is the joint probability density function of the i-th motion data captured at the X position and the Y position respectively, and p(x i ) is the probability density function of the i-th motion data captured at the X position, and p(y j ) is the probability density function of the i-th motion data captured at the Y position.

[0124] According to the above method, when the correlation between any one of the motion capture devices to be installed and all other devices to be installed is the largest for X, the X is determined as the first motion capture device, and the remaining 19 motion capture devices are the second motion capture devices.

[0125] It should be understood that in the embodiments of the present application, the to-be-installed positions of the motion capture devices are obtained, and according to the target motion data at the to-be-installed positions, the target installation positions are determined. The motion data captured by the motion capture devices installed at the target installation positions can be obtained according to the target installation positions, and the motion data is input into the trained neural network. The neural network can output the motion data at other installation positions except the target installation position among all the to-be-installed positions. According to the motion data captured at the target installation position and the motion data output by the neural network, the human body posture can be restored.

[0126] Step 203: Calculate the correlation between all the second motion capture devices and the motion data captured by all other second motion capture devices except themselves, and determine the redundancy between all the second motion capture devices and all the first motion capture devices.

[0127] In the embodiments of the present application, since the number of variables at other installation positions output by the above neural network is greater than 1, when calculating the correlation between all the second motion capture devices and all other second motion capture devices except themselves, if the average value of the correlation is directly calculated according to the number of selected sensors, the normalization effect cannot be achieved. Therefore, the correlation between all the second motion capture devices and all other second motion capture devices except themselves can be specifically calculated by the following method:

[0128]

[0129] Among them, |S| refers to the number of motion capture devices selected from the second motion capture devices, generally taking the value of 1; |C| refers to the number of motion capture devices not selected in the second motion capture devices; s i refers to the i-th motion data captured by the motion capture devices selected from the second motion capture devices, and cj refers to the j-th motion data captured by the motion capture devices not selected in the second motion capture device; D(S, C) refers to the value of the correlation between the motion capture devices selected in the second motion capture device and the motion capture devices not selected.

[0130] In the embodiments of the present application, the redundancy between all the second motion capture devices and all the first motion capture devices is determined. All the first motion capture devices refer to all the selected first motion capture devices. For example, if the number of all the first motion capture devices is two, then the redundancy between all the second motion capture devices and all the first motion capture devices is calculated by calculating the redundancy between the two first motion capture devices and all the second motion capture devices respectively, and then accumulating them.

[0131] Exemplarily, when calculating the redundancy between all the second motion capture devices and all the first motion capture devices, the smaller the value of the redundancy, the lower the redundancy between the motion capture devices selected from the second motion capture device and all the selected first motion capture devices. The specific calculation method of the redundancy can be expressed as follows:

[0132]

[0133] where s i refers to the i-th motion data captured by the motion capture devices selected from the second motion capture device, and s j refers to the j-th motion data captured by all the first motion capture devices; R(S) refers to the value of the redundancy between all the second motion capture devices and all the first motion capture devices.

[0134] Step 204, for the i-th second motion capture device among all the second motion capture devices, determine that the difference between the correlation and redundancy of the i-th second motion capture device is the value function of the i-th second motion capture device.

[0135] In the embodiments of the present application, the i-th second motion capture device is any one of all the second motion capture devices. Determining the value function of the i-th second motion capture device means calculating the respective value functions of all the second motion capture devices in sequence. Among them, the larger the value function of the i-th second motion capture device, the greater the measurement information amount of all the first motion capture devices after taking the i-th second motion capture device as the first motion capture device. The greater the measurement information amount, the more accurate the motion data of the other motion capture devices to be installed output after inputting into the neural network. Therefore, the restored human body posture is also more accurate. The value function of the i-th second motion capture device can be represented by the measurement information amount of the i-th second motion capture device. The specific value function of the i-th second motion capture device can be expressed as follows:

[0136] maxφ i (D i ,R i ),φ i =D i -R i

[0137] Among them, φ i refers to the value function of the i-th second motion capture device, D i refers to the correlation value of the i-th second motion capture device, R i refers to the redundancy value of the i-th second motion capture device.

[0138] Step 205: Add the motion capture device with the largest value function among all the second motion capture devices to the first motion capture device.

[0139] In the embodiment of the present application, after calculating the value functions of all the second motion capture devices according to Step 204, the second motion capture device with the largest value function is added to the first capture motion device. For example, the original number of the first capture motion devices is 2, and the number of the second motion capture devices is 18. After Step 205, the number of the first motion capture devices becomes 3, and the number of the second motion capture devices becomes 17. That is, at this time, only 3 first motion capture devices are installed on the human body to capture motion data, and the neural network outputs the motion data of 17 motion capture devices among the second motion capture devices, where the motion data output by the neural network is a predicted value.

[0140] Step 206: Determine whether the measurement errors of all other second motion capture devices and the total number of all first motion capture devices meet a preset condition.

[0141] In the embodiment of the present application, the measurement errors of all other second motion capture devices refer to the difference between the true value and the predicted value captured by the second motion capture devices. The total number of all first motion capture devices is the total number of the first motion capture devices obtained according to Step 205. The preset condition refers to that the measurement errors of all other second motion capture devices and the total number of all first motion capture devices meet a corresponding relationship, and the corresponding relationship can be specifically expressed as:

[0142]

[0143] From the above formula, a relationship graph between the measurement errors of all other second motion capture devices and the total number of all first motion capture devices can be obtained. According to the relationship graph, a point where the slope changes abruptly can be obtained, and the point where the slope changes abruptly is the preset condition that the measurement errors of all other second motion capture devices and the total number of all first motion capture devices need to satisfy. If the measurement errors of all other second motion capture devices and the total number of all first motion capture devices do not satisfy the preset condition, then return to execute step 203; if the measurement errors of all other second motion capture devices and the total number of all first motion capture devices satisfy the preset condition, then execute the following step 207.

[0144] Step 207, determine the installation positions of all first motion capture devices as the target installation positions.

[0145] In the embodiment of the present application, when the measurement errors of all other second motion capture devices and the total number of all first motion capture devices satisfy the preset condition, the total number of all first motion captures at this time can be obtained, and the number of all first motion capture devices installed on the human body can be determined.

[0146] Exemplarily, when there are 15 remaining second motion capture devices among all other second motion capture devices and the total number of all first motion capture devices is 5, obtain the measurement errors between the true motion data captured by the 15 other second motion capture devices and the predicted motion data output by the neural network. If the point corresponding to the measurement error and the total number of all first motion capture devices in the relationship graph is the point where the slope changes abruptly, then determine the installation positions of these 5 first motion capture devices as the target installation positions. As Figure 4 shown is the target installation position of the motion capture device on the human body. Since the hip is a necessary installation position, the target installation position determined in the present application is the other 5 installation positions except the hip.

[0147] Step 208, in the case where the motion capture device is already installed at the target installation position, obtain the motion data of the motion capture device.

[0148] Step 208 in this embodiment is similar to step 103 in the foregoing embodiment and can be referred to each other. This embodiment will not be elaborated here.

[0149] Step 209, input the motion data captured by the motion capture device into the trained neural network to restore the motion posture of the human body.

[0150] Step 209 in this embodiment is similar to step 104 in the foregoing embodiment and can be referred to each other. This embodiment will not be elaborated here.

[0151] In the embodiments of the present application, in order to improve the accuracy of measuring motion data by a motion capture device, reduce the number of motion capture devices to be worn, and make it more convenient for the subject to wear, an incremental search algorithm based on maximum correlation and minimum redundancy is used to optimize the number and installation positions of the motion capture devices. The number of installed motion capture devices is less than that in the prior art where motion capture devices are installed on all parts of the human body. The small number of motion capture devices can save the installation cost, and at the same time, the installation positions of the motion capture devices determined by the optimization algorithm can accurately restore the human body posture.

[0152] Referring to Figure 5 , a schematic structural diagram of a human body posture restoration device provided in the third embodiment of the present application is shown. For the convenience of description, only the parts related to the embodiments of the present application are shown. The human body posture restoration device may specifically include the following modules:

[0153] A target acquisition module 501, configured to acquire target motion data related to a target motion scenario, where the target motion scenario refers to the scenario to which the motion of the human body belongs;

[0154] A position determination module 502, configured to determine the target installation position of the motion capture device on the human body according to the target motion data;

[0155] A motion data acquisition module 503, configured to acquire the motion data captured by the motion capture device when the motion capture device is installed at the target installation position;

[0156] A training module 504, configured to train a neural network according to the target motion data and the target installation position on the human body.

[0157] A restoration module 505, configured to input the motion data captured by the motion capture device into the trained neural network to restore the motion posture of the human body.

[0158] In the embodiments of the present application, the target data acquisition module 501 may specifically include the following sub-modules:

[0159] A sample acquisition sub-module, configured to acquire sample motion data related to the target motion scenario, where the number of the sample motion data is less than the number of the target motion data;

[0160] A target acquisition sub-module, configured to acquire the target motion data from the human body motion database according to the sample motion data.

[0161] Optionally, the target acquisition sub-module may specifically include the following units:

[0162] A candidate acquisition unit, configured to acquire candidate motion data from the human body motion database, where the number of the candidate motion data is less than or equal to the number of the motion data in the human body motion database;

[0163] A function calculation unit, configured to calculate the value function of the candidate motion data sequence combination one by one according to the candidate motion data and the sample motion data by using a heuristic algorithm;

[0164] A target determination unit, configured to determine the candidate motion data sequence combination when the value function takes the maximum value as the target motion data.

[0165] Optionally, the candidate acquisition unit may specifically be configured to:

[0166] Determine the motion data in the human motion database as the candidate motion data;

[0167] Or calculate the similarity between the sample motion data and each motion data in the human motion database;

[0168] If there is motion data in the human motion database whose similarity to the sample motion data is greater than the similarity threshold, obtain the number of data frames of the data sequence in the motion data whose similarity to the sample motion data is greater than the similarity threshold;

[0169] If there is a data sequence in the motion data whose similarity to the sample motion data is greater than the similarity threshold and the number of data frames of the data sequence is greater than the preset number, determine the data sequence as the candidate motion data.

[0170] Optionally, the function calculation unit is specifically configured to:

[0171] Obtain candidate data features and sample data features, where the candidate data features refer to the data features of the candidate motion data, and the sample data features refer to the data features of the sample motion data;

[0172] Calculate the probability distribution difference between the candidate motion data and the sample motion data according to the candidate data features and the sample data features;

[0173] Calculate the information amount of the candidate motion data according to the candidate data features;

[0174] Calculate the value function of the candidate motion data sequence combination one by one according to the probability distribution difference between the candidate motion data and the sample motion data, the information amount of the candidate motion data, and the number of data frames included in the candidate motion data by using a heuristic algorithm.

[0175] In the embodiment of the present application, the position determination module 502 may specifically include the following sub-modules:

[0176] The first device determination sub-module is used to obtain the to-be-installed positions of the motion capture devices on the human body, and determine the motion capture device with the highest correlation of the motion data captured by all the to-be-installed motion capture devices except itself among all the to-be-installed motion capture devices as the first motion capture device. The motion capture devices include the first motion capture device and the second motion capture device, and the second motion capture device refers to the motion capture devices at all other to-be-installed positions except the one installed on the first motion capture device.

[0177] The calculation sub-module is used to calculate the correlation of the motion data captured by all the second motion capture devices with the motion data captured by all the other second motion capture devices except itself according to the target motion data, and determine the redundancy between all the second motion capture devices and all the first motion capture devices.

[0178] The function determination sub-module is used to, for the i-th second motion capture device among all the second motion capture devices, determine the difference between the correlation and redundancy of the i-th second motion capture device as the value function of the i-th second motion capture device, where the i-th second motion capture device is any one of all the second motion capture devices.

[0179] The judgment sub-module is used to add the motion capture device with the largest value function among all the second motion capture devices to the first motion capture device, and judge whether the measurement errors of all the other second motion capture devices and the total number of all the first motion capture devices meet the preset conditions.

[0180] The return execution sub-module is used to, if the measurement errors of all the other second motion capture devices and the total number of all the first motion capture devices do not meet the preset conditions, return to execute calculating the correlation of all the second motion capture devices with the motion data captured by all the other second motion capture devices except themselves according to the target motion data, and determining the redundancy between the second motion capture devices and all the first motion capture devices.

[0181] The target determination sub-module is used to, if the measurement errors of the other second motion capture devices and the total number of all the first motion capture devices meet the preset conditions, determine the installation positions of all the first motion capture devices as the target installation positions.

[0182] In the embodiments of the present application, the above human body posture restoration device further includes:

[0183] The normalization module is used to perform normalization processing on the motion data captured by the motion capture device according to the preset posture of the human body bones, and obtain the normalized motion data. The posture of the human body bones refers to the rotation matrix of the local coordinate system on the human body bones relative to the world coordinate system.

[0184] The input module is used to input the normalized motion data into the trained neural network.

[0185] The human body posture restoration device provided by the embodiments of the present application can be applied in the foregoing method embodiments. For details, refer to the descriptions of the foregoing method embodiments and will not be elaborated here.

[0186] Figure 6 It is a schematic structural diagram of a terminal device provided in Embodiment 4 of the present application. As Figure 6 shown, the terminal device 600 of this embodiment includes: at least one processor 610 ( Figure 6 only one is shown in the figure), a processor, a memory 620, and a computer program 621 stored in the memory 620 and executable on the at least one processor 610. When the processor 610 executes the computer program 621, the steps in any of the foregoing human body posture restoration method embodiments are implemented.

[0187] The terminal device 600 may be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The terminal device may include, but is not limited to, a processor 610 and a memory 620. Those skilled in the art can understand that Figure 6 merely examples of the terminal device 600 do not constitute a limitation on the terminal device 600, and may include more or fewer components than those shown in the figure, or combine certain components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0188] The so-called processor 610 may be a central processing unit (CPU), and the processor 610 may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0189] The memory 620 may be an internal storage unit of the terminal device 600 in some embodiments, such as the hard disk or memory of the terminal device 600. The memory 620 may also be an external storage device of the terminal device 600 in other embodiments, such as a plug-in hard disk equipped on the terminal device 600, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. Further, the memory 620 may also include both the internal storage unit of the terminal device 600 and the external storage device. The memory 620 is used to store an operating system, application programs, a Boot Loader, data, and other programs, such as the program code of the computer program, etc. The memory 620 may also be used to temporarily store data that has been output or will be output.

[0190] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the above division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiments can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of the present application. The specific working processes of the units and modules in the above system can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0191] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0192] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or by a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0193] In the embodiments provided in the present application, it should be understood that the disclosed device / terminal device and method can be implemented in other ways. For example, the device / terminal device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in electrical, mechanical or other forms.

[0194] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0195] In addition, each functional unit in the various embodiments of the present application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0196] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above-mentioned embodiment methods of the present application, it can also be completed by a computer program instructing relevant hardware. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium can include: any entity or device that can carry the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0197] The implementation of all or part of the processes in the method of the above embodiments in this application can also be completed by a computer program product. When the computer program product runs on a terminal device, the terminal device is caused to execute the steps in the above various method embodiments when executed.

[0198] The above embodiments are only used to illustrate the technical solutions of this application, rather than to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A human body posture restoration method, characterized in that, The human body posture restoration method includes: Obtaining target motion data related to a target motion scenario, where the target motion scenario refers to the scenario to which the human body's motion belongs; Determining a target installation position of a motion capture device on the human body according to the target motion data; When the motion capture device is installed at the target installation position, obtaining the motion data captured by the motion capture device; Training a neural network according to the target motion data and the target installation position on the human body; Inputting the motion data captured by the motion capture device into the trained neural network to restore the motion posture of the human body; The determining the target installation position of the motion capture device on the human body according to the target motion data includes: Obtaining the to-be-installed positions of the motion capture device on the human body, and determining the motion capture device with the greatest correlation with the motion data captured by all other to-be-installed motion capture devices except itself among all to-be-installed motion capture devices as the first motion capture device. The motion capture device includes a first motion capture device and a second motion capture device, and the second motion capture device refers to the motion capture devices at all other to-be-installed positions except the installation position of the first motion capture device; Calculating the correlation between all the second motion capture devices and the motion data captured by all other second motion capture devices except themselves according to the target motion data, and determining the redundancy between all the second motion capture devices and all the first motion capture devices; For the i-th second motion capture device among all the second motion capture devices, determining the difference between the correlation and redundancy of the i-th second motion capture device as the value function of the i-th second motion capture device, where the i-th second motion capture device is any one of all the second motion capture devices; Adding the motion capture device with the largest value function among all the second motion capture devices to the first motion capture device, and determining whether the measurement errors of all other second motion capture devices and the total number of all the first motion capture devices meet a preset condition; If the measurement errors of all other second motion capture devices and the total number of all the first motion capture devices meet the preset condition, determining the installation positions of all the first motion capture devices as the target installation positions.

2. The human body posture restoration method according to claim 1, characterized in that The obtaining the target motion data related to the target motion scenario includes: Obtaining sample motion data related to the target motion scenario, where the quantity of the sample motion data is less than the quantity of the target motion data; Obtaining the target motion data from a human body motion database according to the sample motion data.

3. The human body posture restoration method according to claim 2, wherein, The obtaining the target motion data from the human body motion database according to the sample motion data includes: Obtaining candidate motion data from the human body motion database, where the quantity of the candidate motion data is less than or equal to the quantity of the motion data in the human body motion database; Calculating the value function of each candidate motion data sequence combination one by one according to the candidate motion data and the sample motion data by using a heuristic algorithm; Determine the combination of candidate motion data sequences when the value function reaches the maximum value as the target motion data.

4. The human body posture restoration method according to claim 3, characterized in that, The obtaining of candidate motion data from the human motion database includes: Determine the motion data in the human motion database as candidate motion data; Or calculate the similarity between the sample motion data and each motion data in the human motion database; If there is motion data in the human motion database whose similarity to the sample motion data is greater than the similarity threshold, obtain the number of data frames of the data sequence in the motion data whose similarity to the sample motion data is greater than the similarity threshold; If there is a data sequence in the motion data whose similarity to the sample motion data is greater than the similarity threshold and the number of data frames of the data sequence is greater than the preset number, determine the data sequence as the candidate motion data.

5. The human body pose restoration method according to claim 3 or 4, characterized in that The step of calculating the value function of each candidate motion data sequence combination using a heuristic algorithm according to the candidate motion data and the sample motion data includes: Obtain candidate data features and sample data features, where the candidate data features refer to the data features of the candidate motion data, and the sample data features refer to the data features of the sample motion data; Calculate the probability distribution difference between the candidate motion data and the sample motion data according to the candidate data features and the sample data features; Calculate the information content of the candidate motion data according to the candidate data features; Calculate the value function of each candidate motion data sequence combination using a heuristic algorithm according to the probability distribution difference between the candidate motion data and the sample motion data, the information content of the candidate motion data, and the number of data frames included in the candidate motion data.

6. The human body posture restoration method according to claim 1, wherein, The step of determining the target installation position of the motion capture device on the human body according to the target motion data further includes: If the measurement error of the other second motion capture devices does not meet the preset condition with the total number of all the first motion capture devices, return to execute calculating the correlation between all the second motion capture devices and the motion data captured by all the other second motion capture devices except itself according to the target motion data, and determine the redundancy between all the second motion capture devices and all the first motion capture devices.

7. The human body posture restoration method according to claim 1, characterized in that, After obtaining the motion data captured by the motion capture device, it further includes: Normalize the motion data captured by the motion capture device according to the preset posture of the human skeleton to obtain the normalized motion data; The step of inputting the motion data captured by the motion capture device into the trained neural network includes: inputting the normalized motion data into the trained neural network.

8. A human body posture restoration device, characterized in that, The human posture restoration device includes: A target acquisition module, configured to acquire target motion data related to a target motion scenario, where the target motion scenario refers to the scenario to which the human motion belongs; A position determination module, configured to determine the target installation position of the motion capture device on the human body according to the target motion data. A motion data acquisition module, configured to acquire motion data captured by the motion capture device when the motion capture device is already installed at the target installation position; A training module, configured to train a neural network according to the target motion data and the target installation position on the human body; A restoration module, configured to input the motion data captured by the motion capture device into the trained neural network to restore the motion posture of the human body; The position determination module further includes: A first device determination sub-module, configured to obtain the to-be-installed position of the motion capture device on the human body, and determine the motion capture device with the highest correlation with the motion data captured by all other to-be-installed motion capture devices except itself among all to-be-installed motion capture devices as the first motion capture device. The motion capture devices include the first motion capture device and the second motion capture device. The second motion capture device refers to the motion capture devices installed at all other to-be-installed positions except the position where the first motion capture device is installed; A calculation sub-module, configured to calculate the correlation between all second motion capture devices and the motion data captured by all other second motion capture devices except themselves according to the target motion data, and determine the redundancy between all second motion capture devices and all first motion capture devices; A function determination sub-module, configured to, for the i-th second motion capture device among all second motion capture devices, determine that the difference between the correlation and redundancy of the i-th second motion capture device is the value function of the i-th second motion capture device, where the i-th second motion capture device is any one of all second motion capture devices; A judgment sub-module, configured to add the motion capture device with the largest value function among all second motion capture devices to the first motion capture device, and judge whether the measurement errors of all other second motion capture devices and the total number of all first motion capture devices meet a preset condition; A target determination sub-module, configured to, if the measurement errors of other second motion capture devices and the total number of all first motion capture devices meet the preset condition, determine the installation positions of all first motion capture devices as the target installation positions.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, the method according to any one of claims 1 to 7 is implemented.

Images