Multi-IMU motion capture method, system and medium based on deep learning network

Abstract

The present invention relates to the technical field of human-computer interaction devices, and in particular to a multi-IMU motion capture method, system and medium based on a deep learning network. The present invention includes the following steps: S1, capturing multi-IMU motions: wearing multiple IMUs on key parts of an SMPL human body model in advance; applying Gaussian white noise to multivariate features through the Mujoco physics engine to simulate real IMU data; S2, building a deep learning network: the deep learning network inputs multivariate features, obtains spatial features through forward TCN and reverse TCN learning, and then fuses the spatial features, obtains temporal features through forward GRU and reverse GRU, and the temporal features are fused and input into a multi-head attention layer to obtain human body posture parameters; S3, physical constraints on human body posture: using the PBDL loss function, applying the deep learning network after physical constraints, to achieve accurate recognition of human body posture, in line with the physical laws of the natural world.

Description

Technical Field

[0001] The present invention relates to the technical field of human-computer interaction devices, and in particular to a multi-IMU motion capture method, system and medium based on a deep learning network. Background Art

[0002] Human motion capture refers to the process of identifying and transcribing real-life human motion into digital models with the help of sensor data. It plays a vital role in computer vision, robotics, virtual reality, health monitoring and other fields. Traditional human motion capture methods mainly rely on external devices such as cameras or depth cameras, but these devices have problems such as high cost, susceptibility to environmental interference, and privacy leakage. Therefore, in recent years, human posture recognition methods based on inertial measurement units (IMUs) have received increasing attention. IMU is a small device that can measure physical quantities such as acceleration, angular velocity, and magnetometers. It can be tied to various parts of the human body, such as the upper arm, forearm, thigh, calf, waist, and back, so as to obtain human motion information. IMU has the advantages of low cost and strong portability, but it also faces challenges such as data noise, drift, high dimensionality, and complex timing.

[0003] For example, the Chinese patent publication number is CN115868967A, which discloses a human motion capture method, system and storage medium based on IMU. It converts the human motion data collected by IMU into coordinate system, low-pass filters, quaternion normalization, converts to mirror posture, performs frame rate interpolation processing, and matrix transformation to recursively infer the posture of active joints to calculate the joint angle. The joint angles of all joints in the human motion process can be calculated based on the data obtained by IMU, which is used to achieve accurate capture and recognition of human motion, thereby providing a more scientific diagnostic basis for the treatment of patients. However, the data of this solution is imported into unity software, and there are still problems such as the deep learning network is not fully optimized, the loss function does not consider physical constraints, the recognition accuracy is insufficient, and the application field is limited. Summary of the invention

[0004] The technical problem to be solved by the present invention is to overcome the deficiencies of the prior art and provide a multi-IMU motion capture method, system and medium based on a deep learning network.

[0005] The technical solution of the present invention is:

[0006] A multi-IMU motion capture method based on a deep learning network comprises the following steps:

[0007] S1. Capture of multi-IMU actions: Multiple IMUs are pre-installed on key parts of the SMPL human body model to collect multi-variable features of human body actions. Gaussian white noise is applied to the multi-variable features through the Mujoco physics engine to simulate real IMU data. The simulation training data set is obtained by transforming the actions of the SMPL human body model.

[0008] S2. Build a deep learning network: The deep learning network inputs multivariate features, obtains spatial features through forward TCN and reverse TCN learning, and then fuses the spatial features, obtains temporal features through forward GRU and reverse GRU, and the temporal features are fused and input into the multi-head attention layer to obtain human body posture parameters;

[0009] S3. Physical constraints on human posture: Using the PBDL loss function, physical constraints are imposed on human posture, including the range of joint angles, maintenance of joint lengths, and balance of the center of gravity. After applying physical constraints to the deep learning network, the human posture parameters are more in line with the physical laws of the natural world.

[0010] This technical solution improves the accuracy and robustness of human posture recognition by capturing multiple IMU actions: by wearing multiple IMUs at key parts of the SMPL human body model and using a deep learning network to extract spatiotemporal features, it can more accurately capture and recognize human movements. At the same time, by using a physical engine to apply Gaussian white noise to simulate real IMU data, the robustness of the model can be increased, making it better adapted to the data distribution in the real environment; This technical solution uses the PBDL loss function to take into account the physical constraints of human posture: by using the PBDL loss function, not only the accuracy of classification is considered, but also the physical constraints of human movement, such as the range of joint angles, the maintenance of joint lengths, the balance of the center of gravity, etc.; it is used to improve the generalization ability and robustness of deep learning networks, so that human posture parameters are more in line with the physical laws of the natural world; This technical solution can promote the application of human posture capture technology in multiple fields: it can accurately and robustly capture human movements, and is widely used in computer vision, robotics, virtual reality, health monitoring and other fields, and is used to promote the development of human posture capture technology, and provide new ideas and methods for research and application in related fields.

[0011] In some embodiments, in S1, the capture of multiple IMU actions includes the following steps:

[0012] S11, randomly select a human posture parameter from a real motion capture dataset;

[0013] S12, substituting the human body posture parameters into the SMPL human body model, wherein the SMPL human body model has at least one IMU attached to key parts including the head, chest, left hand, right hand, left foot, and right foot;

[0014] S13, importing the multivariate features of human motion collected by IMU into the Mujoco physics engine, and applying Gaussian white noise to the attitude quaternion and linear acceleration of each IMU location to simulate real IMU data;

[0015] S14, recording the attitude quaternion and acceleration of each IMU, as well as the human body attitude parameters, as a data sample;

[0016] S15. Repeat the processes from S11 to S14 until a sufficient number of data samples are generated to form a simulation data set.

[0017] In this embodiment, the capture of multiple IMU actions can generate a simulated data set with a distribution similar to that of IMU data collected in real scenes, which is used to train and verify deep learning networks, improve the accuracy and robustness of human posture recognition, and promote progress in computer vision, robotics, virtual reality, health monitoring and other fields.

[0018] In some embodiments, in S2, the deep learning network is divided into the following levels:

[0019] Input layer: multivariate features, including IMU attitude quaternion and three-axis linear acceleration;

[0020] Middle layer: forward TCN and reverse TCN, spatial feature fusion layer, forward GRU and reverse GRU, temporal feature fusion layer and multi-head attention layer;

[0021] Output layer: human body posture parameters.

[0022] In some of the embodiments, in the middle layer of S2, the forward TCN and the reverse TCN are two independent spatiotemporal convolutional networks, which perform spatial feature extraction on the input multivariate features from the forward and backward directions respectively; each TCN is composed of multiple residual blocks, each residual block includes a causal convolution layer, a batch normalization layer, an activation function layer and a downsampling layer; the outputs of the forward TCN and the reverse TCN are forward spatial features and reverse spatial features, respectively, and their dimensions are N×C×T, where: N is the batch size, C is the number of channels, and T is the time step;

[0023] The spatial feature fusion layer is a fully connected layer, which concatenates and linearly transforms the forward spatial features and the reverse spatial features to obtain the fused spatial features, whose dimension is N×D×T, where D is the feature dimension;

[0024] The forward GRU and the reverse GRU are two independent gated recurrent units, which extract temporal features from the fused spatial features from the forward and backward directions respectively; each GRU consists of multiple recurrent units, each of which includes a reset gate, an update gate, and a candidate state; the outputs of the forward GRU and the reverse GRU are the forward temporal features and the reverse temporal features, respectively, with a dimension of N×D×T;

[0025] The time series feature fusion layer is a fully connected layer, which concatenates and linearly transforms the forward time series features and the reverse time series features to obtain the fused time series features, whose dimension is N×D×T;

[0026] The multi-head attention layer is a self-attention mechanism, which performs autocorrelation and weighted summation on the fused time series features to obtain the final human posture prediction value, whose dimension is N×P×T, where: P is the number of human posture parameters; the multi-head attention layer is composed of multiple attention heads, each of which includes a query matrix, a key matrix, a value matrix, a scaled dot product attention and a fully connected layer; the output of the multi-head attention layer is the concatenation and linear transformation of the outputs of multiple attention heads.

[0027] In some embodiments, in the physical constraints of the human body posture of S3, the PBDL loss function is divided into the following losses:

[0028] S31, Posture loss: Posture loss is the mean square error between the predicted value and the true value of the human body posture, which is used to measure the accuracy of posture estimation. Its formula is:

[0029]

[0030] Where: N is the batch size, P is the number of human posture parameters, T is the time step, is the predicted value of the p-th posture parameter of the n-th sample at the t-th time step, is the true value of the p-th posture parameter of the n-th sample at the t-th time step;

[0031] S32, Angle loss: Angle loss is the range constraint of the human body joint angle, which is used to ensure the rationality of the posture. Its formula is:

[0032]

[0033] Where: K is the number of human joints, is the angle of the kth joint of the nth sample at the tth time step, is the reference angle of the kth joint, is the angle range of the kth joint, |⋅| is the absolute value operation, and max(0,⋅) is the ReLU operation;

[0034] S33, Length loss: Length loss is a constraint on the length of human joints, used to ensure the consistency of posture. The formula is:

[0035]

[0036] Where: M is the number of human joint chains, is the length of the mth joint chain of the nth sample at the tth time step, is the standard length of the mth joint chain;

[0037] S34, balance loss: Balance loss is the balance constraint of the human body's center of gravity, which is used to ensure the stability of the posture. The formula is:

[0038]

[0039] in: is the centroid position of the nth sample at the tth time step, is the equilibrium point position at the tth time step;

[0040] In summary, the PBDL loss function is the weighted sum of S31 to S34 above, and its formula is:

[0041]

[0042] Among them: α, β, γ, δ are the weight coefficients of posture loss, angle loss, length loss and balance loss respectively, which are adjusted according to actual conditions.

[0043] In some of the embodiments, the multi-IMU human motion capture method further includes the following post-processing steps:

[0044] S4. Experimental verification of algorithm testing: The human posture parameters obtained in S3 were experimented on a public human posture recognition dataset. The experimental verification showed that the generalization ability and robustness of the deep learning network were improved and it had higher recognition accuracy.

[0045] The technical solution of the present invention is:

[0046] A multi-IMU human motion capture system based on deep learning network, comprising:

[0047] Multiple IMUs, worn at least on key parts including the head, chest, left hand, right hand, left foot, and right foot, to obtain attitude quaternions and acceleration;

[0048] A processor is used to execute the multi-IMU motion capture method based on deep learning network.

[0049] The technical solution of the present invention is:

[0050] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the multi-IMU motion capture method based on a deep learning network.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] (1) Optimizing the deep learning network can effectively extract spatiotemporal features from IMU data, thereby achieving accurate recognition of human posture;

[0053] (2) Applying the PBDL loss function not only considers the classification accuracy, but also the physical constraints of human motion, such as momentum conservation and energy conservation, thereby improving the generalization ability and robustness of the deep learning network;

[0054] (3) The present invention has high recognition accuracy and can be widely used in computer vision, robotics, virtual reality, health monitoring and other fields. It has high practical value and innovation. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0056] Figure 1 It is a flow diagram of the method of the present invention.

[0057] Figure 2 It is a structural principle block diagram of the system of the present invention.

[0058] Figure 3 It is a flowchart of the deep learning network of the present invention.

[0059] Figure 4 Schematic diagram of wearing the IMU of the system of the present invention. DETAILED DESCRIPTION

[0060] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0061] Example 1

[0062] like Figure 1and Figure 2 As shown, this embodiment provides a multi-IMU motion capture method based on a deep learning network, comprising the following steps:

[0063] S1. Capture of multi-IMU actions: Multiple IMUs are pre-installed on key parts of the SMPL human body model to collect multi-variable features of human body actions. Gaussian white noise is applied to the multi-variable features through the Mujoco physics engine to simulate real IMU data. The simulation training data set is obtained by transforming the actions of the SMPL human body model.

[0064] S2. Build a deep learning network: The deep learning network inputs multivariate features, obtains spatial features through forward TCN and reverse TCN learning, and then fuses the spatial features, obtains temporal features through forward GRU and reverse GRU, and the temporal features are fused and input into the multi-head attention layer to obtain human body posture parameters;

[0065] S3. Physical constraints on human posture: Using the PBDL loss function, physical constraints are imposed on human posture, including the range of joint angles, maintenance of joint lengths, and balance of the center of gravity. After applying physical constraints to the deep learning network, the human posture parameters are more in line with the physical laws of the natural world.

[0066] like Figures 1 to 4 As shown, the technical solution improves the accuracy and robustness of human posture recognition by capturing multiple IMU actions: by wearing multiple IMUs at key parts of the SMPL human body model and using a deep learning network to extract spatiotemporal features, human body movements can be captured and recognized more accurately. At the same time, the physical engine is used to apply Gaussian white noise to simulate real IMU data, which can increase the robustness of the model and make it better adapt to the data distribution in the real environment; the technical solution uses the PBDL loss function to take into account the physical constraints of human posture: by using the PBDL loss function, not only the accuracy of classification is considered, but also the physical constraints of human movement, such as the range of joint angles, the maintenance of joint lengths, the balance of the center of gravity, etc.; it is used to improve the generalization ability and robustness of deep learning networks, so that human posture parameters are more in line with the physical laws of the natural world; the technical solution can promote the application of human posture capture technology in multiple fields: it can accurately and robustly capture human movements, and is widely used in computer vision, robotics, virtual reality, health monitoring and other fields, to promote the development of human posture capture technology, and provide new ideas and methods for research and application in related fields.

[0067] In some embodiments, in S1, the capture of multiple IMU actions includes the following steps:

[0068] S11, randomly select a human posture parameter from a real motion capture dataset;

[0069] S12, substituting the human body posture parameters into the SMPL human body model, wherein the SMPL human body model has at least one IMU attached to key parts including the head, chest, left hand, right hand, left foot, and right foot;

[0070] S13, importing the multivariate features of human motion collected by IMU into the Mujoco physics engine, and applying Gaussian white noise to the attitude quaternion and linear acceleration of each IMU location to simulate real IMU data;

[0071] S14, recording the attitude quaternion and acceleration of each IMU, as well as the human body attitude parameters, as a data sample;

[0072] S15. Repeat the processes from S11 to S14 until a sufficient number of data samples are generated to form a simulation data set.

[0073] In this embodiment, the capture of multiple IMU actions can generate a simulated data set with a distribution similar to that of IMU data collected in real scenes, which is used to train and verify deep learning networks, improve the accuracy and robustness of human posture recognition, and promote progress in computer vision, robotics, virtual reality, health monitoring and other fields.

[0074] like Figure 3 As shown, in S2, the deep learning network is divided into the following levels:

[0075] Input layer: multivariate features, including IMU attitude quaternion and three-axis linear acceleration;

[0076] Middle layer: forward TCN and reverse TCN, spatial feature fusion layer, forward GRU and reverse GRU, temporal feature fusion layer and multi-head attention layer;

[0077] Output layer: human body posture parameters.

[0078] like Figure 3 As shown, in the middle layer of S2, the forward TCN and the reverse TCN are two independent spatiotemporal convolutional networks, which extract spatial features from the input multivariate features from the forward and backward directions respectively; each TCN is composed of multiple residual blocks, each residual block includes a causal convolution layer, a batch normalization layer, an activation function layer and a downsampling layer; the outputs of the forward TCN and the reverse TCN are the forward spatial features and the reverse spatial features, respectively, and their dimensions are N×C×T, where: N is the batch size, C is the number of channels, and T is the time step;

[0079] The spatial feature fusion layer is a fully connected layer, which concatenates and linearly transforms the forward spatial features and the reverse spatial features to obtain the fused spatial features, whose dimension is N×D×T, where D is the feature dimension;

[0080] The forward GRU and the reverse GRU are two independent gated recurrent units, which extract temporal features from the fused spatial features from the forward and backward directions respectively; each GRU consists of multiple recurrent units, each of which includes a reset gate, an update gate, and a candidate state; the outputs of the forward GRU and the reverse GRU are the forward temporal features and the reverse temporal features, respectively, with a dimension of N×D×T;

[0081] The time series feature fusion layer is a fully connected layer, which concatenates and linearly transforms the forward time series features and the reverse time series features to obtain the fused time series features, whose dimension is N×D×T;

[0082] The multi-head attention layer is a self-attention mechanism, which performs autocorrelation and weighted summation on the fused time series features to obtain the final human posture prediction value, whose dimension is N×P×T, where: P is the number of human posture parameters; the multi-head attention layer is composed of multiple attention heads, each of which includes a query matrix, a key matrix, a value matrix, a scaled dot product attention and a fully connected layer; the output of the multi-head attention layer is the concatenation and linear transformation of the outputs of multiple attention heads.

[0083] In some embodiments, in the physical constraints of the human body posture of S3, the PBDL loss function is divided into the following losses:

[0084] S31, Posture loss: Posture loss is the mean square error between the predicted value and the true value of the human body posture, which is used to measure the accuracy of posture estimation. Its formula is:

[0085]

[0086] Where: N is the batch size, P is the number of human posture parameters, T is the time step, is the predicted value of the p-th posture parameter of the n-th sample at the t-th time step, is the true value of the p-th posture parameter of the n-th sample at the t-th time step;

[0087] S32, Angle loss: Angle loss is the range constraint of the human body joint angle, which is used to ensure the rationality of the posture. Its formula is:

[0088]

[0089] Where: K is the number of human joints, is the angle of the kth joint of the nth sample at the tth time step, is the reference angle of the kth joint, is the angle range of the kth joint, |⋅| is the absolute value operation, and max(0,⋅) is the ReLU operation;

[0090] S33, Length loss: Length loss is a constraint on the length of human joints, used to ensure the consistency of posture. The formula is:

[0091]

[0092] Where: M is the number of human joint chains, is the length of the mth joint chain of the nth sample at the tth time step, is the standard length of the mth joint chain;

[0093] S34, balance loss: Balance loss is the balance constraint of the human body's center of gravity, which is used to ensure the stability of the posture. The formula is:

[0094]

[0095] in: is the centroid position of the nth sample at the tth time step, is the equilibrium point position at the tth time step;

[0096] In summary, the PBDL loss function is the weighted sum of S31 to S34 above, and its formula is:

[0097]

[0098] Among them: α, β, γ, δ are the weight coefficients of posture loss, angle loss, length loss and balance loss respectively, which are adjusted according to actual conditions.

[0099] In some of the embodiments, the multi-IMU human motion capture method further includes the following post-processing steps:

[0100] S4. Experimental verification of algorithm testing: The human posture parameters obtained in S3 were experimented on a public human posture recognition dataset. The experimental verification showed that the generalization ability and robustness of the deep learning network were improved and it had higher recognition accuracy.

[0101] Example 2

[0102] Based on Example 1, Figures 2 to 4 As shown, this embodiment provides a multi-IMU human motion capture system based on a deep learning network, including:

[0103] Multiple IMUs, worn at least on key parts including the head, chest, left hand, right hand, left foot, and right foot, to obtain attitude quaternions and acceleration;

[0104] A processor is used to execute the multi-IMU motion capture method based on deep learning network.

[0105] Example 3

[0106] Based on Example 1, this embodiment provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the multi-IMU motion capture method based on a deep learning network is described.

[0107] It should be understood that the present invention is not limited to the specific configurations and processes described above and shown in the drawings.

[0108] For the sake of brevity, a detailed description of the known methods is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0109] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), suitable firmware, a plug-in, a function card or the like. When implemented in software, the elements of the present invention are programs or code segments that are used to perform the required tasks. Programs or code segments can be stored in a computer-readable storage medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave.

[0110] "Computer-readable storage media" may include any media that can store or transmit information.

[0111] Examples of computer-readable storage media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. The code segments can be downloaded via a computer network such as the Internet, an intranet, etc.

[0112] Although the present invention has been described in detail by referring to the accompanying drawings and in combination with the preferred embodiments, the present invention is not limited thereto. Without departing from the spirit and essence of the present invention, a person of ordinary skill in the art may make various equivalent modifications or substitutions to the embodiments of the present invention, and these modifications or substitutions shall be within the scope of the present invention. Any person of ordinary skill in the art may easily think of changes or substitutions within the technical scope disclosed by the present invention, and these shall be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A multi-IMU motion capture method based on deep learning network, characterized in that: The steps include: S1. Capture of multi-IMU actions: Multiple IMUs are pre-installed on key parts of the SMPL human body model to collect multi-variable features of human body actions. Gaussian white noise is applied to the multi-variable features through the Mujoco physics engine to simulate real IMU data. The simulation training data set is obtained by transforming the actions of the SMPL human body model. S2. Build a deep learning network: The deep learning network inputs multivariate features, obtains spatial features through forward TCN and reverse TCN learning, and then fuses the spatial features, obtains temporal features through forward GRU and reverse GRU, and the temporal features are fused and input into the multi-head attention layer to obtain human body posture parameters; S3. Physical constraints on human posture: Using the PBDL loss function, physical constraints are imposed on human posture, including the range of joint angles, the maintenance of joint lengths, and the balance of the center of gravity. After applying physical constraints to the deep learning network, the human posture parameters are more in line with the physical laws of the natural world. In S1, the capture of multiple IMU actions includes the following steps: S11, randomly select a human posture parameter from a real motion capture dataset; S12, substituting the human body posture parameters into the SMPL human body model, wherein the SMPL human body model has at least one IMU attached to key parts including the head, chest, left hand, right hand, left foot, and right foot; S13, importing the multivariate features of human motion collected by IMU into the Mujoco physics engine, and applying Gaussian white noise to the attitude quaternion and linear acceleration of each IMU location to simulate real IMU data; S14, recording the attitude quaternion and acceleration of each IMU, as well as the human body attitude parameters, as a data sample; S15, repeating the process from S11 to S14 until a sufficient number of data samples are generated to form a simulation data set; In S2, the deep learning network is divided into the following levels: Input layer: multivariate features, including IMU attitude quaternion and three-axis linear acceleration; Middle layer: forward TCN and reverse TCN, spatial feature fusion layer, forward GRU and reverse GRU, temporal feature fusion layer and multi-head attention layer; Output layer: human body posture parameters; In the middle layer of S2, the forward TCN and the reverse TCN are two independent spatiotemporal convolutional networks, which extract spatial features from the input multivariate features from the forward and backward directions respectively; each TCN is composed of multiple residual blocks, each residual block includes a causal convolution layer, a batch normalization layer, an activation function layer and a downsampling layer; the outputs of the forward TCN and the reverse TCN are forward spatial features and reverse spatial features, respectively, and their dimensions are N×C×T, where: N is the batch size, C is the number of channels, and T is the time step; The spatial feature fusion layer is a fully connected layer, which concatenates and linearly transforms the forward spatial features and the reverse spatial features to obtain the fused spatial features, whose dimension is N×D×T, where D is the feature dimension; The forward GRU and the reverse GRU are two independent gated recurrent units, which extract temporal features from the fused spatial features from the forward and backward directions respectively; each GRU consists of multiple recurrent units, each of which includes a reset gate, an update gate, and a candidate state; the outputs of the forward GRU and the reverse GRU are the forward temporal features and the reverse temporal features, respectively, with a dimension of N×D×T; The time series feature fusion layer is a fully connected layer, which concatenates and linearly transforms the forward time series features and the reverse time series features to obtain the fused time series features, whose dimension is N×D×T; The multi-head attention layer is a self-attention mechanism, which performs autocorrelation and weighted summation on the fused time series features to obtain the final human posture prediction value, whose dimension is N×P×T, where: P is the number of human posture parameters; the multi-head attention layer consists of multiple attention heads, each of which includes a query matrix, a key matrix, a value matrix, a scaled dot product attention and a fully connected layer; the output of the multi-head attention layer is the concatenation and linear transformation of the outputs of multiple attention heads; In the physical constraints of the human body posture of S3, the PBDL loss function is divided into the following losses: S31, Posture loss: Posture loss is the mean square error between the predicted value and the true value of the human body posture, which is used to measure the accuracy of posture estimation. Its formula is: Where: N is the batch size, P is the number of human posture parameters, T is the time step, is the predicted value of the p-th posture parameter of the n-th sample at the t-th time step, is the true value of the p-th posture parameter of the n-th sample at the t-th time step; S32, Angle loss: Angle loss is the range constraint of the human body joint angle, which is used to ensure the rationality of the posture. Its formula is: Where: K is the number of human joints, is the angle of the kth joint of the nth sample at the tth time step, is the reference angle of the kth joint, is the angle range of the kth joint, |⋅| is the absolute value operation, and max(0,⋅) is the ReLU operation; S33, Length loss: Length loss is a constraint on the length of human joints, used to ensure the consistency of posture. The formula is: Where: M is the number of human joint chains, is the length of the mth joint chain of the nth sample at the tth time step, is the standard length of the mth joint chain; S34, balance loss: Balance loss is the balance constraint of the human body's center of gravity, which is used to ensure the stability of the posture. The formula is: in: is the centroid position of the nth sample at the tth time step, is the equilibrium point position at the tth time step; In summary, the PBDL loss function is the weighted sum of S31 to S34 above, and its formula is: Among them: α, β, γ, δ are the weight coefficients of posture loss, angle loss, length loss and balance loss respectively, which are adjusted according to actual conditions.

2. The multi-IMU motion capture method based on deep learning network as claimed in claim 1, characterized in that: The multi-IMU human motion capture method also includes the following post-processing steps: S4. Experimental verification of algorithm testing: The human posture parameters obtained in S3 are used to conduct experiments in a public human posture recognition dataset.

3. A multi-IMU human motion capture system based on deep learning network, characterized in that: include: Multiple IMUs, worn at least on key parts including the head, chest, left hand, right hand, left foot, and right foot, to obtain attitude quaternions and acceleration; A processor, used to execute the multi-IMU motion capture method based on deep learning network as described in any one of claims 1-2.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the multi-IMU motion capture method based on a deep learning network described in any one of claims 1-2 is implemented.

Images