Pose processing method, apparatus, system, and storage medium
By combining memory networks and graph convolutional networks, a graph structure is constructed for attitude data processing, which solves the problem of low accuracy in sensor motion tracking and achieves high-precision attitude recognition.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHENZHEN INST OF ADVANCED TECH
- Filing Date
- 2022-08-16
- Publication Date
- 2026-07-07
AI Technical Summary
Existing technologies that rely on sensors for motion tracking are not very accurate and struggle to achieve high-precision attitude recognition.
By combining memory networks and graph convolutional networks, we acquire pose data from different parts of the target object, construct a graph structure, and use graph convolutional networks to extract spatiotemporal features for pose reconstruction, thereby improving the accuracy of pose recognition.
By constructing a graph-structured auxiliary graph convolutional network to extract spatiotemporal features, the accuracy of pose tracking is improved, errors caused by sensor spatial location information are reduced, and more accurate pose recognition is achieved.
Smart Images
Figure CN115471908B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, and in particular to a pose processing method, apparatus, system, and storage medium. Background Technology
[0002] High-precision motion tracking can meet the needs of most fields, such as VR games, movie special effects, biology, and kinematic analysis. Therefore, the demand for high-precision motion tracking has been increasing in recent years, especially for motion tracking based on sensors.
[0003] However, the motion tracking methods that rely on sensors typically use only simple time-series networks to process the data from the sensor signals, resulting in low tracking accuracy. Summary of the Invention
[0004] The main technical problem addressed by this application is to provide a posture processing method, apparatus, system, and storage medium that can accurately acquire the posture of a target object.
[0005] To solve the above-mentioned technical problems, one technical solution adopted in this application is: to provide a posture processing method, which includes: acquiring posture data of different parts of a target object at the same time;
[0006] The memory network is used to extract features from the pose data to obtain temporal features; based on the graph structure, a graph convolutional network is used to extract features from the temporal features to obtain spatiotemporal features; the graph structure is obtained based on the baseline pose data of the target object; pose reconstruction is performed based on the spatiotemporal features to obtain the pose map of the target object.
[0007] Among them, the use of a memory network to extract features from the pose data to obtain temporal features includes: in the memory network, using the parameter matrix, offset, and temporal features from the previous time step to extract features from the pose data at the current time step to obtain the temporal features at the current time step.
[0008] Specifically, before extracting features from temporal features using graph convolutional networks based on graph structures to obtain spatiotemporal features, the process includes: fusing pose data and temporal features to obtain enhanced temporal features; and extracting features from temporal features using graph convolutional networks based on graph structures to obtain spatiotemporal features, including: extracting features from enhanced temporal features using graph convolutional networks based on graph structures to obtain spatiotemporal features.
[0009] Specifically, based on the graph structure, a graph convolutional network is used to extract features from the enhanced temporal features to obtain spatiotemporal features. This includes: extracting features from the enhanced temporal features using a graph convolutional network to obtain initial temporal features; and fusing the initial temporal features with the adjacency matrix corresponding to the graph structure to obtain spatiotemporal features.
[0010] Before performing attitude reconstruction based on spatiotemporal features to obtain the attitude map of the target object, the process includes: fusing attitude data and spatiotemporal features to obtain enhanced spatiotemporal features; and performing attitude reconstruction based on spatiotemporal features to obtain the attitude map of the target object, which includes: performing attitude reconstruction based on enhanced spatiotemporal features to obtain the attitude map of the target object.
[0011] The process of reconstructing pose based on spatiotemporal features to obtain the pose map of the target object includes: using a fully connected layer to predict the spatiotemporal features to obtain the spatial coordinates corresponding to each pose data; and obtaining the pose map based on the spatial coordinates.
[0012] The graph structure is derived from the target object's baseline pose data and includes:
[0013] Obtain the reference pose data corresponding to different parts of the target object; map the part corresponding to each reference pose data to a node in the graph structure; use the Euclidean distance between the target node and the other nodes to determine the connection relationship between the target node and the other nodes; connect the target node and the other nodes according to the connection relationship to form a graph structure.
[0014] Before using a memory network to extract features from pose data and obtain temporal features, the process includes normalizing the pose data.
[0015] To address the aforementioned technical problems, another technical solution adopted in this application is: providing a posture processing device, which includes an acquisition module for acquiring posture data of different parts of a target object at the same time; a first extraction module for extracting features from the posture data using a memory network to obtain temporal features; a second extraction module for extracting features from the temporal features using a graph convolutional network based on a graph structure to obtain spatiotemporal features; the graph structure is obtained based on the baseline posture data of the target object; and a remodeling module for reconstructing the posture based on the spatiotemporal features to obtain the posture map of the target object.
[0016] To solve the above-mentioned technical problems, another technical solution adopted in this application is: to provide an attitude processing system, the attitude processing system including: a data acquisition unit, which is set at different parts of the target object to collect attitude data of the response parts of the target object; and an attitude processing device, which is communicatively connected to the data acquisition unit, as described above.
[0017] To solve the above-mentioned technical problems, another technical solution adopted in this application is to provide a computer-readable storage medium that stores program data, which, when executed by a processor, is used to implement the attitude processing method described above.
[0018] The beneficial effects of this application are as follows: Unlike the prior art, this application uses a graph structure-assisted graph convolutional network with spatial location information to extract features from temporal features, thereby obtaining spatiotemporal features with both temporal and spatial information. Then, a more accurate pose map is reconstructed based on the spatiotemporal features, which can improve the accuracy of pose recognition of target objects and solve the problem of low accuracy in pose tracking. Attached Figure Description
[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:
[0020] Figure 1 This is a flowchart illustrating the first embodiment of the attitude processing method provided in this application;
[0021] Figure 2 This is a structural schematic diagram of the key point image provided in this application;
[0022] Figure 3 This is a schematic diagram of the training of the network model provided in this application;
[0023] Figure 4 This is a structural diagram of a motion tracking model based on Long Short-Term Memory (LSTM) networks and graph convolutional networks.
[0024] Figure 5 This is a schematic diagram of the structure of an embodiment of the attitude processing system provided in this application;
[0025] Figure 6 This is a schematic diagram of the structure of an embodiment of the attitude processing device provided in this application;
[0026] Figure 7 This is a schematic diagram of an embodiment of the computer-readable storage medium provided in this application. Detailed Implementation
[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.
[0028] like Figure 1 As shown, the pose processing method described in this application may include: Step 100: Acquiring pose data of different parts of the target object at the same time. Step 200: Extracting features from the pose data using a memory network to obtain temporal features. Step 300: Extracting features from the temporal features using a graph convolutional network based on a graph structure to obtain spatiotemporal features; the graph structure is obtained based on the baseline pose data of the target object. Step 400: Reconstructing the pose based on the spatiotemporal features to obtain the pose map of the target object.
[0029] In other words, this application adds the construction of a graph structure based on the reference pose data of the target object, and then uses a graph convolutional network to extract features from the temporal features based on the graph structure to obtain spatiotemporal features. This fully utilizes the detection dimension of the sensor's spatial position information, avoids errors caused by the sensor's spatial position information to the processed data, and thus improves the accuracy of target object pose recognition.
[0030] The first embodiment of the attitude processing method of this application will be described in detail below.
[0031] Step 100: Obtain the pose data of different parts of the target object at the same time.
[0032] The target object can be a person or other object, such as an animal. The target object mentioned in the embodiments of this application is mainly a person.
[0033] Taking a person as an example, different parts of the target object can be various parts of the human body, including but not limited to the elbow joint, knee joint, hip joint, spine, shoulder, and hand.
[0034] Optionally, sensors can be installed on the target object to acquire attitude data. These sensors can be accelerometers, gyroscopes, magnetometers, or other sensors capable of accurately collecting information about human movement.
[0035] However, it should be noted that since soft sensors are relatively inexpensive, the sensors used in this application are mainly soft sensors.
[0036] In addition to conventional strain sensors, posture data can also be obtained from data with geometric positional relationships, such as human joint angles and finger angles.
[0037] Generally, after acquiring pose data, feature extraction is performed on the pose data. However, in the field of machine learning, different features are different evaluation metrics with different dimensions and units of measurement, which will affect the results of subsequent data analysis. Therefore, in order to eliminate the influence of dimensions between various features, it is necessary to standardize the acquired data first, so that the acquired data conforms to a standard normal distribution, and at the same time speed up the subsequent training.
[0038] Specifically, in some embodiments, after acquiring the attitude data, the attitude data can be normalized first.
[0039] This application does not limit the specific implementation of normalization.
[0040] Step 200: Feature extraction is performed on the pose data using a memory network to obtain temporal features. The memory network can include Long Short-Term Memory (LSTM) networks and Bidirectional Memory networks, with LSTM networks effectively addressing the issues of cross-temporal memory and gradient vanishing. Therefore, this application primarily uses Long Short-Term Memory (LSTM) networks.
[0041] For example, in a memory network, the pose data at the current moment is extracted using the parameter matrix, offset, and temporal features from the previous moment to obtain the temporal features at the current moment.
[0042] Specifically, the formula can be used: H t =σ(W·[H t-1 ,X t ])+b extracts features from the attitude data at the current moment to obtain the temporal features at the current moment.
[0043] Where σ is the activation function, W is the weight matrix, b is the offset, and H... t-1 X represents the temporal characteristics of the previous moment. t H represents the attitude data at the current moment. t This represents the temporal characteristics at the current moment.
[0044] To facilitate subsequent reference to the formula, this application denotes the attitude data as X, uses a memory network to extract features from the attitude data X, and denotes the resulting temporal features as H. The above formula can then be replaced by H = LSTM(X).
[0045] In addition, to retain more temporal features, some embodiments can automatically analyze and synthesize the obtained temporal features under certain standards after obtaining them. For example, attitude data and temporal features can be fused to obtain enhanced temporal features.
[0046] Specifically, the formula can be used: The attitude data and temporal features are fused and stitched together.
[0047] Among them, H in To enhance the temporal features, X represents the pose data, and LSTM(X) represents the temporal features.
[0048] By fusing pose data and temporal features, enhanced temporal features are obtained, which can contain more relevant information from the pose data and will not lose relevant information from the pose data due to feature extraction.
[0049] Since memory networks can generally only extract temporal information from the acquired pose data, but processing only the temporal dimension may result in omissions in the acquired data or errors in subsequent analysis and processing, after obtaining temporal features through the aforementioned memory network and fusing pose data and temporal features to obtain enhanced temporal features, this application introduces a graph convolutional network to extract spatial information from the pose data in order to reduce tracking errors and obtain more accurate poses of the target object, in order to obtain spatiotemporal features, as shown in step 300.
[0050] Step 300: Based on the graph structure, use a graph convolutional network to extract features from the temporal features to obtain spatiotemporal features.
[0051] The graph structure is obtained based on the baseline pose data of the target object.
[0052] Among them, the reference attitude data is the attitude data of the target object in its normal state, that is, the target object has not yet undergone changes in angle such as translation or tilt.
[0053] Optionally, in some embodiments, the graph structure can be obtained through the following steps:
[0054] Step 301: Obtain the reference pose data corresponding to different parts of the target object;
[0055] Step 302: Map the part corresponding to each reference attitude data to a node in the graph structure;
[0056] Step 303: Determine the connection relationship between the target node and the other nodes using the Euclidean distance between them;
[0057] Step 304: Connect the target node with the remaining nodes according to the connection relationship to form a graph structure.
[0058] For the process of forming the graph structure described in steps 301 to 304, please refer to... Figure 2 , Figure 2The diagram below illustrates the graph structure generated based on the K-nearest neighbor method:
[0059] The explanation is based on the target audience being people. Figure 2 In this context, "sensor" refers to a sensor, and 01 to 20 represent the nodes of each sensor. Each sensor node corresponds to a different part of the human body. For example, 01 and 02 correspond to the elbow joint, 03 and 04 correspond to the trapezius muscle, 05 and 06 correspond to the pectoralis major muscle, 07, 08, 09, and 10 correspond to the upper back, 11, 12, 13, and 14 correspond to the muscles along the spine in the lower upper body, 15, 16, 17, and 18 correspond to the back and side of the hip joint, respectively, and 19 and 20 correspond to the knee joint.
[0060] Alternatively, the k-nearest neighbor method can be used to construct the graph structure. Specifically, the actual three-dimensional coordinates of each node in space can be calculated first, and the Euclidean distance between each target node and the other nodes can be calculated using the following formula:
[0061]
[0062] Where, d ij Let x represent the Euclidean distance between node i and node j, where x i Let y be the x-axis coordinate of the i-th node. i Let z be the ordinate of the i-th node. i Let x be the vertical coordinate of the i-th node. Similarly, x... j Let y be the x-axis coordinate of the j-th node. j Let z be the y-axis coordinate of the j-th node. j Let be the vertical coordinate of the j-th node. Connect each target node to its k nearest neighbors using Euclidean distance, forming k undirected edges. Generally, k is 2. However, it should be noted that when multiple neighbors are close to the target node, the value of k can float upwards according to the set range; for example, k could be 3 or 4.
[0063] For example, with 07 as the target node and the remaining 19 as the other nodes, since... Figure 2 07 and 09 are relatively close. Therefore, when 07 is the target node, the value of k can be calculated to be 3, that is, 07 has 3 nearest other nodes, namely 09, 03 and 05. The connection line between 07 and these three nodes 09, 03 and 05 is an undirected edge.
[0064] Optionally, after further enhancing the temporal features to obtain enhanced temporal features, some embodiments may be based on graph structures and utilize graph convolutional networks to extract features from the enhanced temporal features to obtain spatiotemporal features.
[0065] For example, extracting spatiotemporal features from enhanced temporal features using convolutional networks can include the following sub-steps:
[0066] Step 310: Use a graph convolutional network to extract features from the enhanced temporal features to obtain the initial temporal features;
[0067] Step 320: Use the adjacency matrix corresponding to the graph structure to fuse with the initial temporal features to obtain spatiotemporal features.
[0068] The graph convolutional network is (GCN).
[0069] The adjacency matrix corresponding to the graph structure can be represented by A∈R. N×N This means that the formula for A is:
[0070]
[0071] Among them, A ij =1 means that node i is adjacent to node j, A ij =0 means that node i and node j are not adjacent, (v i ,v j Let ) represent the undirected edge connecting node i and node j, E be the set of all undirected edges, and define the degree matrix D. ii =∑ j A ij , node v i Degree D ii This indicates the number of undirected edges associated with this node.
[0072] For example, suppose a node has 3 undirected edges, and the degree D of this node is... ii The value is 3.
[0073] Similarly, in order to reduce noise and improve training speed, some implementations may normalize the adjacency matrix A.
[0074] For example, normalizing the adjacency matrix A results in... Then, the formula can be used: The adjacency matrix is fused with the initial temporal features to obtain the spatiotemporal features.
[0075] in, I is the identity matrix. W is the weight matrix, H cs For the initial time series features, H sk It is a spatiotemporal characteristic.
[0076] It should be noted that the identity matrix I is used to introduce self-connection.
[0077] Furthermore, to retain more features, some embodiments can automatically analyze and synthesize the obtained spatiotemporal features under certain standards after obtaining them. For example, attitude data and spatiotemporal features can be fused to obtain enhanced spatiotemporal features.
[0078] Specifically, the formula can be used: The attitude data and spatiotemporal features are fused to obtain enhanced spatiotemporal features.
[0079] Where S represents the enhanced spatiotemporal features, X represents the pose data, and H represents the pose data. sk It is a spatiotemporal characteristic.
[0080] Step 400: Reconstruct the pose based on spatiotemporal features to obtain the pose map of the target object.
[0081] Optionally, in some embodiments, pose reconstruction based on spatiotemporal features to obtain a pose map of the target object may include the following sub-steps:
[0082] Step 401: Use a fully connected layer to predict the spatiotemporal features and obtain the spatial coordinates corresponding to each pose data.
[0083] Among them, the fully connected layer can use the modified linear activation function (ReLU) to predict spatiotemporal features. Since each node in the fully connected layer is connected to all nodes in the previous layer to integrate the features extracted earlier, the fully connected layer generally has the most parameters, with parameter sizes of 128, 64, and M*3 respectively. The last parameter M*3 corresponds to the spatial coordinates (x, y, z) of M tracking points.
[0084] Step 402: Obtain the attitude diagram based on the spatial coordinates.
[0085] Specifically, a pose diagram is obtained by performing three-dimensional reconstruction based on spatial coordinates.
[0086] Optionally, in some embodiments, after further enhancing the spatiotemporal features to obtain enhanced spatiotemporal features, pose reconstruction can be performed based on the enhanced spatiotemporal features to obtain the pose map of the target object.
[0087] Based on the above embodiments, the training of the network model involved in this application will be described below. For details, please refer to [link / reference needed]. Figure 3 and Figure 4 , Figure 3 This is a flowchart illustrating the human motion tracking process for this applicant. Figure 4 The motion tracking model structure based on Long Short-Term Memory networks and graph convolutional networks can specifically include the following steps:
[0088] 1) Data preprocessing;
[0089] Specifically, bending sensors are installed on human joints to collect posture data. The collected data signals are preprocessed and collected to obtain a dataset.
[0090] In this embodiment, the publicly available datasets used are the DeepFull-Bodydataset and the Stretch-SensingGlovedataset. The DeepFull-Bodydataset provides whole-body microfluidic sensor data of an adult in three different movements, while the Stretch-SensingGlovedataset provides capacitive stretch sensor data of one hand of ten adults. Both datasets cover whole-body motion tracking and hand motion tracking.
[0091] 2) Dataset partitioning;
[0092] For example, when testing the first dataset, the training and test sets are divided according to a specified partition. When testing the second dataset, the data is randomly shuffled and divided into training and test sets in a 9:1 ratio.
[0093] Operations are performed using a long short-term memory network;
[0094] Specifically, the sample data, i.e., the pose data X, from the training set is input into the Long Short-Term Memory network for feature extraction, and the output is the temporal feature H.
[0095] For example, the pose data at the current moment can be used to extract features from the parameter matrix, offset, and temporal features of the previous moment to obtain the temporal features of the current moment.
[0096] Specifically, the formula can be used: H t =σ(W·[H t-1 ,X t ])+b extracts features from the attitude data at the current moment to obtain the temporal features at the current moment.
[0097] Where σ is the activation function, W is the weight matrix, b is the offset, and H... t-1 X represents the temporal characteristics of the previous moment. t H represents the attitude data at time t, i.e., the current time. t This represents the temporal characteristics at the current moment. X t-1 Let X be the attitude data at time (t-1). t-2 The attitude data is at time (t-2).
[0098] To retain more temporal features, pose data and temporal features can be fused and stitched together.
[0099] For example, a formula can be used: The attitude data and temporal features are fused and stitched together.
[0100] Among them, H in To enhance the temporal features, X represents the pose data, and H represents the temporal features.
[0101] 5) Perform computations using graph convolutional networks;
[0102] Specifically, enhance the temporal features H in The input graph is processed by a convolutional network to extract features and output initial temporal features.
[0103] 6) Generate a graph structure from the preprocessed data;
[0104] Specifically, the baseline pose data in the training set is obtained; the part corresponding to each baseline pose data is mapped to a node in the graph structure; the connection relationship between the target node and the other nodes is determined by using the Euclidean distance between the target node and the other nodes; and the target node and the other nodes are connected according to the connection relationship to form a graph structure.
[0105] 7) Obtain the adjacency matrix of the graph structure and normalize the adjacency matrix.
[0106] 8) Output the initial time-series features H cs Adjacency matrix after normalization The graph is input again into a convolutional network for fusion, and the spatiotemporal features H are output. sk .
[0107] 9) The attitude data and spatiotemporal features are fused to obtain enhanced spatiotemporal features.
[0108] 10) Input the obtained enhanced spatiotemporal features into the fully connected layer for 3D reconstruction, output the posture map of human motion, and complete the training.
[0109] It is important to note that after each training iteration, the network parameters of the graph convolutional network are updated, and then the next training iteration is performed until the accuracy of the motion tracking model reaches the required level, at which point the training ends.
[0110] 11) Validate the effectiveness of the trained model using the pose data of the test set. The evaluation metric is the mean error of data tracking (RMSE), and the evaluation model is obtained.
[0111] See Figure 5 , Figure 5 This is a schematic diagram of an embodiment of the attitude processing system provided in this application. The attitude processing system 10 includes a data acquisition unit 01 and an attitude processing device 02.
[0112] The data acquisition unit 01 is set at different parts of the target object to collect the posture data of the target object's response parts; the posture processing device 02 is communicatively connected to the data acquisition unit 01.
[0113] See Figure 6 , Figure 6 This is a schematic diagram of an embodiment of the attitude processing device provided in this application. The attitude processing device 02 includes an acquisition module 001, a first extraction module 002, a second extraction module 003, and a remodeling module 004. The functions of each module are as follows:
[0114] Get module 001, used to obtain the posture data of different parts of the target object at the same time;
[0115] The first extraction module 002 is used to extract features from pose data using a memory network to obtain temporal features;
[0116] The second extraction module 003 is used to extract temporal features based on graph structure and using graph convolutional networks to obtain spatiotemporal features; the graph structure is obtained based on the baseline pose data of the target object;
[0117] Remodeling group 004 is used for pose reconstruction based on spatiotemporal features to obtain the pose map of the target object.
[0118] It is understood that each module is also used to implement the methods of any of the above embodiments.
[0119] See Figure 7 , Figure 7 This is a schematic diagram of an embodiment of the computer-readable storage medium provided in this application. The computer-readable storage medium 140 stores program data 141, which, when executed by a processor, is used to implement the following method:
[0120] The process involves acquiring pose data of different parts of the target object at the same time; extracting features from the pose data using a memory network to obtain temporal features; extracting features from the temporal features using a graph convolutional network based on a graph structure to obtain spatiotemporal features; obtaining the graph structure based on the baseline pose data of the target object; and reconstructing the pose based on the spatiotemporal features to obtain the pose map of the target object.
[0121] It is understood that when the program data 141 is executed by the processor, it is also used to implement the method of any of the above embodiments.
[0122] When the embodiments of this application are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0123] The above are merely embodiments of this application and do not limit the scope of this patent application. Any equivalent structural or procedural changes made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of this application.
Claims
1. A posture processing method, characterized in that, The method includes: Acquire the pose data of different parts of the target object at the same time; The pose data is used to extract features to obtain temporal features; The attitude data and the temporal features are fused to obtain enhanced temporal features; Based on the graph structure, a graph convolutional network is used to extract features from the enhanced temporal features to obtain spatiotemporal features. The graph structure is obtained based on the reference pose data of the target object, including: acquiring reference pose data corresponding to different parts of the target object; mapping each part corresponding to the reference pose data to a node in the graph structure; determining the connection relationship between the target node and the other nodes using the Euclidean distance between the target node and the other nodes; and connecting the target node and the other nodes according to the connection relationship to form the graph structure. The attitude data and the spatiotemporal features are fused to obtain enhanced spatiotemporal features; The enhanced spatiotemporal features are predicted using a fully connected layer to obtain the spatial coordinates corresponding to each of the pose data. The pose diagram of the target object is obtained based on the spatial coordinates.
2. The method according to claim 1, characterized in that, The step of using a memory network to extract features from the pose data to obtain temporal features includes: In the memory network, the pose data at the current time is used to extract features by utilizing the parameter matrix, offset, and temporal features from the previous time step, thereby obtaining the temporal features at the current time step.
3. The method according to claim 1, characterized in that, The method based on a graph structure, using a graph convolutional network to extract features from the enhanced temporal features, yields the spatiotemporal features, including: The enhanced temporal features are extracted using a graph convolutional network to obtain initial temporal features; The spatiotemporal features are obtained by fusing the adjacency matrix corresponding to the graph structure with the initial temporal features.
4. The method according to claim 1, characterized in that, Before extracting features from the pose data using a memory network to obtain temporal features, the process includes: The attitude data is then normalized.
5. An attitude processing device, characterized in that, The attitude processing device includes: The acquisition module is used to acquire the posture data of different parts of the target object at the same time. The first extraction module is used to extract features from the pose data using a memory network to obtain temporal features; and to fuse the pose data and the temporal features to obtain enhanced temporal features. The second extraction module is used to extract features from the enhanced temporal features based on a graph structure using a graph convolutional network to obtain spatiotemporal features. The graph structure is obtained based on the reference pose data of the target object, including: acquiring reference pose data corresponding to different parts of the target object; mapping each part corresponding to the reference pose data to a node in the graph structure; determining the connection relationship between the target node and the other nodes using the Euclidean distance between the target node and the other nodes; and connecting the target node and the other nodes according to the connection relationship to form the graph structure. A remodeling group is used to fuse the pose data and the spatiotemporal features to obtain enhanced spatiotemporal features; a fully connected layer is used to predict the enhanced spatiotemporal features to obtain the spatial coordinates corresponding to each pose data; and the pose map of the target object is obtained based on the spatial coordinates.
6. An attitude processing system, characterized in that, The attitude processing system includes: A data acquisition unit is disposed at different parts of the target object to collect the posture data of the response parts of the target object; An attitude processing device is communicatively connected to the data acquisition unit, as described in claim 5.
7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program data, which, when executed by a processor, is used to implement the attitude processing method as described in any one of claims 1-4.