Object pose prediction method, apparatus, processor and electronic device

Abstract

This application discloses a method, apparatus, processor, and electronic device for predicting object pose. Relating to the field of artificial intelligence, the method includes: acquiring N reference object poses of a target object within a target time period; detecting target action features of the target object based on a first object pose, a second object pose, and a first time step, wherein the target action features characterize the continuous transformation trajectory of the target object's action form within the target time period, the N reference object poses include the first object pose and the second object pose, and the first time step indicates the time interval between the target object's change from the first object pose to the second object pose; and predicting the target object pose at a target time based on the target action features, candidate object poses, and the second time step. This application solves the problem of low prediction efficiency for object pose in related technologies.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and more specifically, to a method, apparatus, processor, and electronic device for predicting the pose of an object. Background Technology

[0002] Understanding and predicting object actions is a crucial topic in computer vision, with wide applications in areas such as human-computer interaction, augmented reality, and animation. The goal of object action prediction is to predict actions in future frames given past frames. Current common methods extract the correlations between object actions across frames and predict future frame actions based on these correlations. This method requires that the past frames used for prediction be of equal frequency, meaning the image acquisition equipment must be stable and error-free. However, in practical applications, image acquisition equipment may experience non-uniform frequency acquisition due to malfunctions or sluggishness, leading to inaccurate prediction results. Furthermore, due to the inherent prediction principle, this method can only predict object actions in future frames with a frame rate matching that of past frames.

[0003] There is currently no effective solution to the problem of low prediction efficiency of object pose in related technologies. Summary of the Invention

[0004] The main objective of this application is to provide a method, apparatus, processor, and electronic device for predicting object pose, in order to solve the problem of low prediction efficiency of object pose in related technologies.

[0005] To achieve the above objectives, according to one aspect of this application, a method for predicting object pose is provided. The method includes: acquiring N reference object poses of a target object within a target time period, wherein the reference object poses characterize the action form of the target object at a corresponding moment, and N is an integer greater than 1; detecting target action features of the target object based on a first object pose, a second object pose, and a first time step, wherein the target action features characterize the continuous change trajectory of the target object's action form within the target time period, the N reference object poses include the first object pose and the second object pose, and the first time step indicates the time interval between the change of the target object from the first object pose to the second object pose; predicting the target object pose of the target object at a target moment based on the target action features, candidate object poses, and the second time step, wherein the N reference object poses include the candidate object poses, the target moment is after the target time period, the second time step indicates the time interval between the candidate moment corresponding to the candidate object pose and the target moment, and the candidate moment is the moment preceding the target moment.

[0006] Optionally, detecting the target action features of the target object based on the first object pose, the second object pose, and the first time step includes: inputting the first object pose, the second object pose, and the first time step into a target feature generation network, wherein the target feature generation network is used to extract features from the first object pose and the second object pose at the first time step based on ordinary differential equations to obtain and output the target action features; and obtaining the target action features output by the target feature generation network.

[0007] Optionally, inputting the first object pose, the second object pose, and the first time step into the target feature generation network includes: extracting features from the first object pose to obtain first object pose features, and extracting features from the second object pose to obtain second object pose features, wherein the first object pose features are used to characterize the motion state of the target object at a first moment corresponding to the first object pose, the second object pose features are used to characterize the motion state of the target object at a second moment corresponding to the second object pose, and the first time step is the time interval between the first moment and the second moment; inputting the first object pose features, the second object pose features, and the first time step into a first gated recurrent unit, wherein the target feature generation network includes the first gated recurrent unit, the first gated recurrent unit is used to extract features from the first object pose features, the second object pose features, and the first time step based on ordinary differential equations to obtain hidden state features of the target object within a target time period, and the target action features include the hidden state features.

[0008] Optionally, feature extraction is performed on the first object's pose to obtain first object pose features, and feature extraction is performed on the second object's pose to obtain second object pose features, including: performing an exponential mapping on the first joint position of the target object at the first time moment to obtain first joint features, wherein the first object pose includes the first joint position; determining the first joint velocity features of the joint at the first time moment based on the first joint features, wherein the first joint velocity features are used to characterize the movement velocity value and velocity direction of the joint at the first time moment; performing position encoding on the temporal position of the first time moment within the target time period to obtain first position features; and combining the first joint features and the first joint velocity features... The first object pose feature is obtained by concatenating the first position feature with the second object position feature; the second joint position of the target object at the second time moment is exponentially mapped to obtain the second joint feature, wherein the second object pose includes the second joint position; the second joint velocity feature of the joint at the second time moment is determined based on the second joint feature, wherein the second joint velocity feature is used to characterize the movement velocity value and velocity direction of the joint at the second time moment; the temporal position of the second time moment within the target time period is positionally encoded to obtain the second position feature; the second joint feature, the second joint velocity feature and the second position feature are concatenated to obtain the second object pose feature.

[0009] Optionally, predicting the target object's pose at the target time based on the target action features, candidate object poses, and the second time step includes: inputting the target action features, candidate object poses, and the second time step into a target pose prediction network, wherein the target pose prediction network is used to calculate the target action features, candidate object poses, and the second time step based on ordinary differential equations to obtain the target object pose; and obtaining the target object pose output by the target pose prediction network.

[0010] Optionally, inputting the target action features, the candidate object pose, and the second time step into the target pose prediction network includes: inputting the target action features, the candidate object pose, and the second time step into a second gated recurrent unit to obtain a third object pose feature of the target object at the target time, wherein the target pose prediction network includes the second gated recurrent unit, which is used to calculate the target action features, the candidate object pose, and the second time step based on ordinary differential equations to obtain the third object pose feature, which is used to characterize the motion state of the target object at the target time; and performing feature transformation on the third object pose feature to obtain the target object pose.

[0011] Optionally, the target feature generation network is generated by the following method: constructing an initial pose prediction model, wherein the initial pose prediction model includes an initial feature generation network and an initial pose prediction network connected in sequence; training the initial pose prediction model using object pose samples of a third object pose labeled with the time to be predicted to obtain a target pose prediction model, wherein the target pose prediction model includes the target feature generation network and the target pose prediction network connected in sequence, the target feature generation network is used to detect the target action features based on the first object pose, the second object pose, and the first time step, and the target pose prediction network is used to predict the target object pose based on the target action features, the candidate object pose, and the second time step.

[0012] Optionally, training the initial pose prediction model using object pose samples labeled with the object pose at the time to be predicted to obtain the target pose prediction model includes: inputting the object pose samples into the initial pose prediction model to obtain the fourth object pose at the time to be predicted output by the initial pose prediction model; calculating the quaternion loss value of the initial pose prediction model using the third object pose and the fourth object pose; and adjusting the model parameters of the initial pose prediction model according to the quaternion loss value until the initial pose prediction model converges to obtain the target pose prediction model.

[0013] To achieve the above objectives, according to another aspect of this application, an object pose prediction device is provided. The device includes: an acquisition module, configured to acquire N reference object poses of a target object within a target time period, wherein the reference object poses characterize the action form of the target object at a corresponding moment, and N is an integer greater than 1; a detection module, configured to detect target action features of the target object based on a first object pose, a second object pose, and a first time step, wherein the target action features characterize the continuous change trajectory of the target object's action form within the target time period, the N reference object poses include the first object pose and the second object pose, and the first time step indicates the time interval between the change of the target object from the first object pose to the second object pose; and a prediction module, configured to predict the target object pose of the target object at a target moment based on the target action features, candidate object poses, and the second time step, wherein the N reference object poses include the candidate object poses, the target moment is located after the target time period, the second time step indicates the time interval between the candidate moment corresponding to the candidate object pose and the target moment, and the candidate moment is the moment preceding the target moment.

[0014] This application employs the following steps: obtaining N reference object postures of the target object within a target time period, wherein the reference object postures are used to characterize the action form of the target object at the corresponding time, and N is an integer greater than 1; detecting target action features of the target object based on a first object posture, a second object posture, and a first time step, wherein the target action features are used to characterize the continuous change trajectory of the target object's action form within the target time period, the N reference object postures include the first object posture and the second object posture, and the first time step is used to indicate the time interval between the target object changing from the first object posture to the second object posture; predicting the target object's position within the target time period based on the target action features, candidate object postures, and the second time step. The target object pose at a given time, wherein N reference object poses include candidate object poses, the target time is located after the target time period, and the second time step is used to indicate the time interval between the candidate time corresponding to the candidate object pose and the target time. The candidate time is the time before the target time, that is, the time interval between the target object changing from the first object pose to the second object pose is the first time step. By detecting the first object pose, the second object pose, and the first time step, target action features representing the continuous transformation trajectory of the target object's action form within the target time period are obtained. Therefore, based on these target object features, the object pose of the target object at any time after the target time period can be accurately predicted, solving the problem of low prediction efficiency of object pose in related technologies. This achieves the effect of improving the prediction efficiency of object pose. Attached Figure Description

[0015] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0016] Figure 1 This is a flowchart of an object pose prediction method provided according to an embodiment of this application;

[0017] Figure 2 This is a schematic diagram of an optional object pose prediction according to an embodiment of this application;

[0018] Figure 3 This is a schematic diagram of an optional target feature generation network structure according to an embodiment of this application;

[0019] Figure 4 This is a schematic diagram of an optional target pose prediction network structure according to an embodiment of this application;

[0020] Figure 5 This is a schematic diagram of an optional model training according to an embodiment of this application;

[0021] Figure 6 This is an optional model training flowchart according to an embodiment of this application;

[0022] Figure 7 This is a schematic diagram of an object pose prediction device according to an embodiment of this application;

[0023] Figure 8 This is a schematic diagram of an electronic device according to an embodiment of the present application. Detailed Implementation

[0024] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0025] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0027] The present invention will now be described in conjunction with preferred implementation steps. Figure 1 This is a flowchart of an object pose prediction method provided according to an embodiment of this application, such as... Figure 1 As shown, the method includes the following steps:

[0028] Step S101: Obtain N reference object poses of the target object within the target time period, wherein the reference object poses are used to characterize the action form of the target object at the corresponding time, and N is an integer greater than 1;

[0029] Step S102: Detect the target action features of the target object based on the first object pose, the second object pose, and the first time step. The target action features are used to characterize the continuous change trajectory of the target object's action form within the target time period. The N reference object poses include the first object pose and the second object pose. The first time step is used to indicate the time interval between the target object changing from the first object pose to the second object pose.

[0030] Step S103: Predict the target object posture at the target time based on the target action features, candidate object postures and the second time step, wherein the N reference object postures include the candidate object postures, the target time is located after the target time period, the second time step is used to indicate the time interval between the candidate time corresponding to the candidate object posture and the target time, and the candidate time is the previous time of the target time.

[0031] The object pose prediction method provided in this application embodiment obtains N reference object poses of a target object within a target time period, wherein the reference object poses are used to characterize the action form of the target object at the corresponding time, and N is an integer greater than 1; detects target action features of the target object based on a first object pose, a second object pose, and a first time step, wherein the target action features are used to characterize the continuous change trajectory of the target object's action form within the target time period, the N reference object poses include the first object pose and the second object pose, and the first time step is used to indicate the time interval between the target object changing from the first object pose to the second object pose; predicts the target object pose of the target object at a target time based on the target action features, candidate object poses, and the second time step, wherein the N reference object poses include the candidate object poses, the target time is located after the target time period, the second time step is used to indicate the time interval between the candidate time corresponding to the candidate object pose and the target time, and the candidate time is the time before the target time. This solves the problem of low prediction efficiency of object pose in related technologies, thereby improving the prediction efficiency of object pose.

[0032] In the embodiment described in step S101 above, the target object is an object with mobility attributes, which may include, but is not limited to, animals (such as people, cats, dogs, etc.), plants (such as flowers, grass, etc.), or transportation equipment (such as cars, boats, etc.). This solution does not limit this.

[0033] Optionally, in this embodiment, the pose of the reference object may be, but is not limited to, the pose obtained by recognizing the target object through the pose recognition device. The pose of the target object may be, but is not limited to, the image of the target object, or it may be the coordinates of the key points of the target object (the key point coordinates may be obtained by directly recognizing the target object through the recognition device, or they may be obtained by recognizing the image of the target object collected). This solution does not limit this.

[0034] Optionally, in this embodiment, the N reference object poses can be the object poses of the target object at any time within the target time period. When N is greater than or equal to 3, the N reference object poses can be reference object poses with equal frequency, or they can be reference object poses with unequal frequency. This solution does not limit this.

[0035] In the embodiment described in step S102 above, the detection of target object features can be, but is not limited to, obtained by differentiating the first pose, the second pose, and the first time step, or it can be obtained by using a trained target feature generation network to detect the first object pose, the second object pose, and the first time step. The target feature generation network is used to extract features from the first object pose and the second object pose at the first time step based on ordinary differential equations. This scheme does not limit the method for detecting target object features.

[0036] Optionally, in this embodiment, the first object pose and the second object pose can be object poses that are temporally adjacent among N reference object poses, or they can be object poses that are not temporally adjacent among N reference object poses. This solution does not limit this.

[0037] In the embodiment provided in step S103 above, the second time step can be any natural number greater than zero. It can be the same as the first time step or a different value. This solution does not limit this.

[0038] Optionally, in this embodiment, the target object pose at the target time can be obtained, but is not limited to, by solving the differential solution of the target action features, candidate object poses and the second time step, or it can be obtained by using a trained target pose prediction network to detect the target action features, candidate object poses and the second time step. The target pose prediction network is used to calculate the target action features, candidate object poses and the second time step based on ordinary differential equations to obtain the target object pose. This solution does not limit this.

[0039] Optionally, in this embodiment, the candidate time can be the last time of the target time period, or it can be another time within the target time period. This solution does not limit this.

[0040] In the above embodiments, the target object features can be detected using a trained target feature generation network, and the target object pose can be predicted using a trained target pose prediction network. Figure 2 This is a schematic diagram of an optional object pose prediction according to an embodiment of this application, such as... Figure 2 As shown, the target feature generation network and the target pose prediction network are connected in sequence. The first object pose, the second object pose, and the first time step are input into the target feature generation network to obtain the target action features output by the target feature generation network. The target action features, the candidate object pose, and the second time step are input into the target pose prediction network to obtain the target object pose output by the target pose prediction network. In this embodiment, when detecting the target action features, the target feature generation network needs to input at least the first object pose, the second object pose, and the first time step. When predicting the target action pose, the target pose prediction network needs to input at least the target action features, the candidate object pose, and the second time step.

[0041] As an optional embodiment, detecting the target action features of the target object based on the first object pose, the second object pose, and the first time step includes:

[0042] The first object pose, the second object pose, and the first time step are input into the target feature generation network, wherein the target feature generation network is used to extract features from the first object pose and the second object pose at the first time step based on ordinary differential equations, to obtain the target action features and output them.

[0043] The target action features are obtained from the target feature generation network output.

[0044] Optionally, in this embodiment, the first object pose and the second object pose can be directly input into the target feature generation network, or features can be extracted from the first object pose and the second object pose to obtain object pose features whose feature dimensions match the feature dimensions required by the target feature generation network, and the extracted object pose features can be input into the target feature generation network module.

[0045] As an optional embodiment, the step of inputting the first object pose, the second object pose, and the first time step into the target feature generation network includes:

[0046] The first object pose is used to extract features to obtain the first object pose features, and the second object pose is used to extract features to obtain the second object pose features. The first object pose features are used to characterize the motion state of the target object at a first moment corresponding to the first object pose, and the second object pose features are used to characterize the motion state of the target object at a second moment corresponding to the second object pose. The first time step is the time interval between the first moment and the second moment.

[0047] The first object pose features, the second object pose features, and the first time step are input into the first gated recurrent unit. The target feature generation network includes the first gated recurrent unit, which is used to extract features from the first object pose features, the second object pose features, and the first time step based on ordinary differential equations to obtain the hidden state features of the target object in the target time period. The target action features include the hidden state features.

[0048] Optionally, in this embodiment, the motion state may include, but is not limited to, the joint position, the moving speed value of the joint at the corresponding time, the velocity direction of the joint at the corresponding time, the moving speed value of the joint angle at the corresponding time, the velocity direction of the joint angle at the corresponding time, etc. This solution does not limit these.

[0049] Optionally, in this embodiment, feature extraction can be obtained by processing the first object pose or the second object pose using a trained feature extraction model, or by performing mathematical operations on the first object pose or the second object pose to obtain object pose features. This solution does not limit this.

[0050] Figure 3 This is a schematic diagram of an optional target feature generation network structure according to an embodiment of this application, such as... Figure 3 As shown, the target feature generation network can be an ordinary differential equation RNN encoder, which includes a gated recurrent unit. The gated recurrent unit is used to extract features from the input data based on the ordinary differential equation. As shown in the figure, the poses of N reference objects are sorted chronologically, and the input features are fed into the RNN encoder to extract features. The sorted N reference object poses are then sequentially input into the target feature generation network. The gated recurrent unit extracts the hidden states from the input reference object poses, ultimately obtaining the hidden states h of the N reference object poses within the target time period. N (i.e., target working characteristics) It can be seen that in this embodiment, the poses of each of the N reference objects are not of equal frequency, that is, the time step values ​​between two adjacent reference object poses are not equal.

[0051] As an optional embodiment, feature extraction is performed on the pose of a first object to obtain first object pose features, and feature extraction is performed on the pose of a second object to obtain second object pose features, including:

[0052] An exponential mapping is performed on the position of the first joint of the target object at the first time moment to obtain a first joint feature, wherein the first object posture includes the first joint position; a first joint velocity feature is determined based on the first joint feature at the first time moment, wherein the first joint velocity feature is used to characterize the movement speed value and velocity direction of the joint at the first time moment; the temporal position of the first time moment within the target time period is positionally encoded to obtain a first position feature; the first joint feature, the first joint velocity feature, and the first position feature are concatenated to obtain the first object posture feature;

[0053] An exponential mapping is performed on the second joint position of the target object at the second time moment to obtain a second joint feature, wherein the second object posture includes the second joint position; a second joint velocity feature is determined based on the second joint feature at the second time moment, wherein the second joint velocity feature is used to characterize the movement speed value and velocity direction of the joint at the second time moment; the temporal position of the second time moment within the target time period is position-encoded to obtain a second position feature; the second joint feature, the second joint velocity feature, and the second position feature are concatenated to obtain the second object posture feature.

[0054] Optionally, in this embodiment, the exponential mapping can be performed directly on the joint position, or it can be performed on the joint angle after determining the corresponding joint angle based on the joint position. This solution does not limit this.

[0055] Optionally, in this embodiment, the joint features obtained through exponential mapping can be used to characterize the correspondence between joint position and velocity magnitude. Therefore, the joint velocity features can be calculated from the joint features, for example, by calculating the joint velocity using the following formula:

[0056] V t =X t ′-X t-1 ′,t>1

[0057] Among them, V t X represents the velocity characteristics of the joints at the current moment. t ′ represents the key feature at the current moment, X t-1 ′ represents the key feature of the previous time step.

[0058] Optionally, in this embodiment, position encoding can be performed for each time moment using the following formula:

[0059]

[0060]

[0061] Where, d p It is the embedding dimension, where i is the index of the feature.

[0062] As an optional embodiment, predicting the target object's pose at the target time based on the target action features, candidate object poses, and a second time step includes:

[0063] The target action features, the candidate object pose, and the second time step are input into the target pose prediction network, wherein the target pose prediction network is used to calculate the target action features, the candidate object pose, and the second time step based on ordinary differential equations to obtain the target object pose.

[0064] Obtain the target object pose output by the target pose prediction network.

[0065] Optionally, in this embodiment, the candidate object pose can be directly input into the target pose prediction network, or the candidate object pose can be feature extracted to obtain object pose features whose feature dimensions match the feature dimensions required by the target pose prediction network, and the extracted object pose features can be input into the target pose prediction network.

[0066] Optionally, in this embodiment, the target pose prediction network can be used to directly output the pose of the target object, or the target pose prediction network can be used to output the pose features of the target object at the target time, and then perform feature transformation on the pose features of the object to obtain the pose of the target object.

[0067] Figure 4 This is a schematic diagram of an optional target pose prediction network structure according to an embodiment of this application, such as... Figure 4 As shown, the pose prediction network can be an ordinary differential equation RNN decoder, which includes a gated recurrent unit (GRN). The GRN is used to predict the target object's pose at the target time based on the ordinary differential equation. As shown in the figure, the hidden state h of the target object during the target time period is output by the feature generation network. N (Target pose features), and then based on the hidden state h N , t J Object pose X at time (candidate time)j (Candidate pose) prediction t J+1 The pose of the target object X at time X j+1 .

[0068] As an optional embodiment, inputting the target action features, the candidate object pose, and the second time step into the target pose prediction network includes:

[0069] The target action features, the candidate object pose, and the second time step are input into the second gated recurrent unit to obtain the third object pose features of the target object at the target time. The target pose prediction network includes the second gated recurrent unit, which is used to calculate the target action features, the candidate object pose, and the second time step based on ordinary differential equations to obtain the third object pose features. The third object pose features are used to characterize the motion state of the target object at the target time.

[0070] The pose features of the third object are transformed to obtain the pose of the target object.

[0071] Optionally, in this embodiment, the operation of feature transformation on the object pose features may be, but is not limited to, first finding the object joint features in the object pose features, and then performing reverse processing on the joint features according to the principle of obtaining the joint features by exponential mapping of the joint positions to obtain the object joint positions, wherein the object pose includes the object joint positions.

[0072] As an optional embodiment, the target feature generation network is generated by the following method:

[0073] Construct an initial pose prediction model, wherein the initial pose prediction model includes an initial feature generation network and an initial pose prediction network connected in sequence;

[0074] The initial pose prediction model is trained using object pose samples labeled with the pose of the third object at the time to be predicted to obtain a target pose prediction model. The target pose prediction model includes a target feature generation network and a target pose prediction network connected in sequence. The target feature generation network is used to detect the target action features based on the first object pose, the second object pose, and the first time step. The target pose prediction network is used to predict the target object pose based on the target action features, the candidate object pose, and the second time step.

[0075] Optionally, in this embodiment, the object pose sample can be obtained by labeling M object poses within a reference time period. For example, the M object poses within the reference time period can be sorted according to time sequence, with the first a object poses as the object poses at historical moments and the remaining Ma object poses as the object poses at the moment to be predicted.

[0076] Optionally, in this embodiment, the initial pose prediction model may be, but is not limited to, a recurrent neural network model, which has an encoding network (i.e., an initial feature generation network) and a decoding network (i.e., an initial pose prediction network). This solution does not limit this.

[0077] As an optional embodiment, training the initial pose prediction model using object pose samples labeled with the pose of the object to be predicted at the time to be predicted, to obtain the target pose prediction model, includes:

[0078] The object pose sample is input into the initial pose prediction model to obtain the fourth object pose at the time to be predicted, output by the initial pose prediction model.

[0079] The quaternion loss value of the initial pose prediction model is calculated using the pose of the third object and the pose of the four objects.

[0080] The model parameters of the initial attitude prediction model are adjusted based on the quaternion loss value until the initial attitude prediction model converges, thereby obtaining the target attitude prediction model.

[0081] Optionally, in this embodiment, the loss value of the initial pose prediction model can also be based on a loss function constructed from the features output by a feature generation layer of the initial pose prediction model. For example, the loss value can be calculated by combining the object pose features output by the object pose feature network layer of the initial pose prediction model for predicting the object pose at the time to be predicted with the object pose features to be predicted (object pose features obtained by feature extraction of the object pose to be predicted) (such as quaternion loss value calculation) to obtain the loss value.

[0082] Optionally, in this embodiment, there may be, but is not limited to, a corresponding relationship between the quaternion loss value and the model parameter adjustment value. The model parameter is adjusted according to the model parameter by finding the model parameter adjustment value corresponding to the loss value from the corresponding relationship.

[0083] Figure 5 This is an optional model training diagram according to an embodiment of this application, such as... Figure 5As shown, the initial pose prediction model includes an initial feature generation network and an initial pose prediction network connected in sequence. During model training, features are extracted by inputting object pose samples into the initial feature generation network. The output of the initial feature generation network is then input into the initial pose prediction network to obtain the prediction result output by the initial pose prediction network. The prediction result output by the initial pose prediction network and the third object pose at the time to be predicted, labeled by the sample, are then used to calculate the loss value of the initial pose prediction model. The model parameters of the initial feature generation network and the initial pose prediction network are then modified based on the loss value until the model of the initial pose prediction network converges, thus obtaining the target pose prediction network.

[0084] Figure 6 This is an optional model training flowchart according to an embodiment of this application, used to train an initial pose prediction model. The model structure of the initial feature generation network included in the initial pose prediction model is similar to... Figure 3 The model structure shown is consistent with that of the initial pose prediction network. Figure 4 The model structure shown is consistent, such as Figure 6 As shown, it includes at least the following steps:

[0085] S601, firstly, read all the 3D skeletal point data in the human motion pose dataset, and divide the preset human motion sequence samples into historical sequences. and future sequences The human skeleton of all sequences is decomposed into joints and embedded as features for representation, as follows: Assuming the input human action sequence is... T represents the number of timing parameters, and U represents the number of key points. This represents the sampling time point for each frame. To achieve better feature extraction, the joint angles of the human motion sequence are exponentially mapped to obtain...

[0086] S602: Calculate the velocity information based on the joint position information of the human body sequence, as follows:

[0087] V t =X t ′-X t-1 ′,t>1

[0088] V0 = 0,

[0089] This provides the velocity information for the current sequence.

[0090] S603: Perform positional encoding on the sequence, specifically as follows: For each time point ts in TS, the positional encoding can be represented as:

[0091]

[0092]

[0093] d p It is the embedding dimension, where i is the index.

[0094] S604: Concatenate the position information X', velocity information V, and position encoding information p to obtain the input feature X.

[0095] S605: Input the input feature X” into the differential equation RNN encoder to extract features, specifically as follows: set the initial state. Given the previous state h t-1 ,have:

[0096]

[0097] h t =GRUCell(h′) t ,x″ t-1 ),

[0098] Where h i The solution to ′ is a differential equation The numerical solution employs a fourth-order Runge-Kutta solution, with absolute and relative error tolerances of 1e-5 and 1e-3, respectively. t Indicates the current state, h′ t This indicates the current state. GRUCell is a gated loop unit, as detailed below:

[0099] r t =σ(W r ·[h t-1 ,x t-1 ]),

[0100] z t =σ(W z ·[h t-1 ,x t-1 ]),

[0101]

[0102]

[0103] Where σ is the sigmoid function, tanh is the hyperbolic tangent function, and r t To reset the door, z t To update the gate, loop through the above process, traversing from time point ts1 to ts... t-1 Time point.

[0104] S606: The final state h obtained from the last step t-1 and x″ T The input is fed into the differential equation RNN decoder, as follows, assuming the time series to be predicted is... Given the previous state h t-1 and input features x″ t-1 ,have:

[0105]

[0106] h t =GRUCell(h′) t ,x″ t-1 ),

[0107] vp t =W o ·h t ,

[0108] The obtained vp is the predicted velocity information for the next frame. The position information for the next frame can be obtained by decoding the position information of the previous frame and the velocity information of the current frame.

[0109] x t ′=x t ′ -1 +vp t ,

[0110] x″ t =x t ⊙vp t ⊙p t ,

[0111] Where, p t The ⊙ symbol represents the frame position encoding information for prediction, and the ⊙ symbol represents the concatenation operation.

[0112] S607: During the training phase, the actual output sequence is... This invention employs quaternion transformation and uses quaternion loss to supervise the model. Specifically, it assumes e... v For the exponential mapping of key v, the true pose at a specific time point t is equal to the predicted pose. For simplification, use e instead of e. v Quaternion transformation can be expressed as:

[0113]

[0114] Where q is the quaternion corresponding to the joint rotation, and ||e||2 is the L2 modulus of vector e.

[0115] During backpropagation, the differential of q with respect to e is a 4x3 Jacobian matrix:

[0116]

[0117] Where I3 is a 3x3 identity matrix, and:

[0118]

[0119]

[0120] in, It is the normalized e. It is a cross product operation.

[0121] During training, the final loss function can be written as:

[0122]

[0123] Where q represents a four-element transformation, y T+j It is the true pose value, x′ T+j This is the predicted attitude value, and ||·||1 represents the absolute error calculation.

[0124] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0125] This application also provides an object pose prediction device. It should be noted that the object pose prediction device of this application can be used to execute the object pose prediction method provided in this application. The object pose prediction device provided in this application is described below.

[0126] Figure 7 This is a schematic diagram of an object pose prediction device according to an embodiment of this application. Figure 7As shown, the device includes: an acquisition module 72, used to acquire N reference object postures of the target object within a target time period, wherein the reference object postures are used to characterize the action form of the target object at the corresponding time, and N is an integer greater than 1; a detection module 74, used to detect target action features of the target object based on a first object posture, a second object posture, and a first time step, wherein the target action features are used to characterize the continuous change trajectory of the target object's action form within the target time period, the N reference object postures include the first object posture and the second object posture, and the first time step is used to indicate the time interval between the target object changing from the first object posture to the second object posture; and a prediction module 76, used to predict the target object posture of the target object at a target time based on the target action features, candidate object postures, and the second time step, wherein the N reference object postures include the candidate object postures, the target time is located after the target time period, the second time step is used to indicate the time interval between the candidate time corresponding to the candidate object posture and the target time, and the candidate time is the previous time of the target time.

[0127] Optionally, the detection module includes: a first input unit, used to input the first object pose, the second object pose, and the first time step into a target feature generation network, wherein the target feature generation network is used to extract features from the first object pose and the second object pose at the first time step based on ordinary differential equations to obtain the target action features and output them; and a first acquisition unit, used to acquire the target action features output by the target feature generation network.

[0128] Optionally, the first input unit is configured to: extract features from the first object's pose to obtain first object pose features, and extract features from the second object's pose to obtain second object pose features, wherein the first object pose features characterize the motion state of the target object at a first moment corresponding to the first object pose, and the second object pose features characterize the motion state of the target object at a second moment corresponding to the second object pose, and the first time step is the time interval between the first moment and the second moment; input the first object pose features, the second object pose features, and the first time step to a first gated recurrent unit, wherein the target feature generation network includes the first gated recurrent unit, and the first gated recurrent unit is configured to extract features from the first object pose features, the second object pose features, and the first time step based on ordinary differential equations to obtain hidden state features of the target object within a target time period, and the target action features include the hidden state features.

[0129] Optionally, the first input unit is configured to: perform exponential mapping on the position of a first joint of the target object at the first time moment to obtain a first joint feature, wherein the first object posture includes the first joint position; determine a first joint velocity feature of the joint at the first time moment based on the first joint feature, wherein the first joint velocity feature is used to characterize the movement speed value and velocity direction of the joint at the first time moment; perform position encoding on the temporal position of the first time moment within the target time period to obtain a first position feature; perform feature concatenation on the first joint feature, the first joint velocity feature, and the first position feature to obtain a first object posture feature; perform exponential mapping on the position of a second joint of the target object at the second time moment to obtain a second joint feature, wherein the second object posture includes the second joint position; determine a second joint velocity feature of the joint at the second time moment based on the second joint feature, wherein the second joint velocity feature is used to characterize the movement speed value and velocity direction of the joint at the second time moment; perform position encoding on the temporal position of the second time moment within the target time period to obtain a second position feature; and perform feature concatenation on the second joint feature, the second joint velocity feature, and the second position feature to obtain a second object posture feature.

[0130] Optionally, the prediction module includes: a second input unit, used to input the target action features, the candidate object pose, and the second time step into a target pose prediction network, wherein the target pose prediction network is used to calculate the target action features, the candidate object pose, and the second time step based on ordinary differential equations to obtain the target object pose; and a second acquisition unit, used to acquire the target object pose output by the target pose prediction network.

[0131] Optionally, the second input unit is configured to: input the target action features, the candidate object pose, and the second time step into the second gated recurrent unit to obtain the third object pose features of the target object at the target time, wherein the target pose prediction network includes the second gated recurrent unit, which is configured to calculate the target action features, the candidate object pose, and the second time step based on ordinary differential equations to obtain the third object pose features, which are used to characterize the motion state of the target object at the target time; and perform feature transformation on the third object pose features to obtain the target object pose.

[0132] Optionally, the apparatus further includes: a construction module for constructing an initial pose prediction model, wherein the initial pose prediction model includes an initial feature generation network and an initial pose prediction network connected in sequence; and a training module for training the initial pose prediction model using object pose samples labeled with the poses of a third object at the time to be predicted, to obtain a target pose prediction model, wherein the target pose prediction model includes the target feature generation network and the target pose prediction network connected in sequence, the target feature generation network being used to detect the target action features based on the first object pose, the second object pose, and the first time step, and the target pose prediction network being used to predict the target object pose based on the target action features, the candidate object pose, and the second time step.

[0133] Optionally, the training module includes: an input unit for inputting the object pose sample into the initial pose prediction model to obtain the fourth object pose at the time to be predicted output by the initial pose prediction model; a calculation unit for calculating the quaternion loss value of the initial pose prediction model using the third object pose and the four object poses; and an adjustment unit for adjusting the model parameters of the initial pose prediction model according to the quaternion loss value until the initial pose prediction model converges to obtain the target pose prediction model.

[0134] The object pose prediction device provided in this application embodiment acquires N reference object poses of a target object within a target time period, wherein the reference object poses characterize the action form of the target object at the corresponding moment, and N is an integer greater than 1; detects target action features of the target object based on a first object pose, a second object pose, and a first time step, wherein the target action features characterize the continuous change trajectory of the target object's action form within the target time period, the N reference object poses include the first object pose and the second object pose, and the first time step indicates the time interval between the change of the target object from the first object pose to the second object pose; and predicts the target object pose of the target object at a target moment based on the target action features, candidate object poses, and the second time step, wherein the N reference object poses include the candidate object poses, the target moment is located after the target time period, and the second time step indicates the time interval between the candidate moment corresponding to the candidate object pose and the target moment, the candidate moment being the previous moment of the target moment. This solves the problem of low prediction efficiency of object pose in related technologies, thereby improving the prediction efficiency of object pose. The object pose prediction device includes a processor and a memory. The aforementioned units are all stored in the memory as program units, and the processor executes the aforementioned program units stored in the memory to realize the corresponding functions.

[0135] The processor contains a kernel, which retrieves the corresponding program unit from memory. One or more kernels can be configured, and object pose prediction is performed by adjusting kernel parameters.

[0136] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0137] This invention provides a computer-readable storage medium storing a program that, when executed by a processor, implements a method for predicting the pose of an object.

[0138] This invention provides a processor for running a program, wherein the program executes a method for predicting the pose of an object.

[0139] Figure 8 This application provides a schematic diagram of an electronic device according to an embodiment of the present application, such as... Figure 8As shown, this embodiment of the invention provides an electronic device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs the following steps: acquiring N reference object postures of a target object within a target time period, wherein the reference object postures are used to characterize the action form of the target object at the corresponding time, and N is an integer greater than 1; detecting target action features of the target object based on a first object posture, a second object posture, and a first time step, wherein the target action features are used to characterize the continuous change trajectory of the target object's action form within the target time period, the N reference object postures include the first object posture and the second object posture, and the first time step is used to indicate the time interval between the target object changing from the first object posture to the second object posture; predicting the target object posture of the target object at a target time based on the target action features, candidate object postures, and the second time step, wherein the N reference object postures include the candidate object postures, the target time is after the target time period, the second time step is used to indicate the time interval between the candidate time corresponding to the candidate object posture and the target time, and the candidate time is the time before the target time.

[0140] Optionally, detecting the target action features of the target object based on the first object pose, the second object pose, and the first time step includes: inputting the first object pose, the second object pose, and the first time step into a target feature generation network, wherein the target feature generation network is used to extract features from the first object pose and the second object pose at the first time step based on ordinary differential equations to obtain and output the target action features; and obtaining the target action features output by the target feature generation network.

[0141] Optionally, inputting the first object pose, the second object pose, and the first time step into the target feature generation network includes: extracting features from the first object pose to obtain first object pose features, and extracting features from the second object pose to obtain second object pose features, wherein the first object pose features are used to characterize the motion state of the target object at a first moment corresponding to the first object pose, the second object pose features are used to characterize the motion state of the target object at a second moment corresponding to the second object pose, and the first time step is the time interval between the first moment and the second moment; inputting the first object pose features, the second object pose features, and the first time step into a first gated recurrent unit, wherein the target feature generation network includes the first gated recurrent unit, the first gated recurrent unit is used to extract features from the first object pose features, the second object pose features, and the first time step based on ordinary differential equations to obtain hidden state features of the target object within a target time period, and the target action features include the hidden state features.

[0142] Optionally, feature extraction is performed on the first object's pose to obtain first object pose features, and feature extraction is performed on the second object's pose to obtain second object pose features, including: performing an exponential mapping on the first joint position of the target object at the first time moment to obtain first joint features, wherein the first object pose includes the first joint position; determining the first joint velocity features of the joint at the first time moment based on the first joint features, wherein the first joint velocity features are used to characterize the movement velocity value and velocity direction of the joint at the first time moment; performing position encoding on the temporal position of the first time moment within the target time period to obtain first position features; and combining the first joint features and the first joint velocity features... The first object pose feature is obtained by concatenating the first position feature with the second object position feature; the second joint position of the target object at the second time moment is exponentially mapped to obtain the second joint feature, wherein the second object pose includes the second joint position; the second joint velocity feature of the joint at the second time moment is determined based on the second joint feature, wherein the second joint velocity feature is used to characterize the movement velocity value and velocity direction of the joint at the second time moment; the temporal position of the second time moment within the target time period is positionally encoded to obtain the second position feature; the second joint feature, the second joint velocity feature and the second position feature are concatenated to obtain the second object pose feature.

[0143] Optionally, predicting the target object's pose at the target time based on the target action features, candidate object poses, and the second time step includes: inputting the target action features, candidate object poses, and the second time step into a target pose prediction network, wherein the target pose prediction network is used to calculate the target action features, candidate object poses, and the second time step based on ordinary differential equations to obtain the target object pose; and obtaining the target object pose output by the target pose prediction network.

[0144] Optionally, inputting the target action features, the candidate object pose, and the second time step into the target pose prediction network includes: inputting the target action features, the candidate object pose, and the second time step into a second gated recurrent unit to obtain a third object pose feature of the target object at the target time, wherein the target pose prediction network includes the second gated recurrent unit, which is used to calculate the target action features, the candidate object pose, and the second time step based on ordinary differential equations to obtain the third object pose feature, which is used to characterize the motion state of the target object at the target time; and performing feature transformation on the third object pose feature to obtain the target object pose.

[0145] Optionally, the target feature generation network is generated by the following method: constructing an initial pose prediction model, wherein the initial pose prediction model includes an initial feature generation network and an initial pose prediction network connected in sequence; training the initial pose prediction model using object pose samples of a third object pose labeled with the time to be predicted to obtain a target pose prediction model, wherein the target pose prediction model includes the target feature generation network and the target pose prediction network connected in sequence, the target feature generation network is used to detect the target action features based on the first object pose, the second object pose, and the first time step, and the target pose prediction network is used to predict the target object pose based on the target action features, the candidate object pose, and the second time step.

[0146] Optionally, training the initial pose prediction model using object pose samples labeled with the object pose at the time to be predicted to obtain the target pose prediction model includes: inputting the object pose samples into the initial pose prediction model to obtain the fourth object pose at the time to be predicted output by the initial pose prediction model; calculating the quaternion loss value of the initial pose prediction model using the third object pose and the fourth object pose; and adjusting the model parameters of the initial pose prediction model according to the quaternion loss value until the initial pose prediction model converges to obtain the target pose prediction model.

[0147] The devices mentioned in this article can be servers, PCs, tablets, mobile phones, etc.

[0148] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program having the following method steps: acquiring N reference object postures of a target object within a target time period, wherein the reference object postures are used to characterize the action form of the target object at the corresponding time, and N is an integer greater than 1; detecting target action features of the target object based on a first object posture, a second object posture, and a first time step, wherein the target action features are used to characterize the continuous change trajectory of the target object's action form within the target time period, the N reference object postures include the first object posture and the second object posture, and the first time step is used to indicate the time interval between the target object changing from the first object posture to the second object posture; predicting the target object posture of the target object at a target time based on the target action features, candidate object postures, and the second time step, wherein the N reference object postures include the candidate object postures, the target time is located after the target time period, the second time step is used to indicate the time interval between the candidate time corresponding to the candidate object posture and the target time, and the candidate time is the time before the target time.

[0149] Optionally, detecting the target action features of the target object based on the first object pose, the second object pose, and the first time step includes: inputting the first object pose, the second object pose, and the first time step into a target feature generation network, wherein the target feature generation network is used to extract features from the first object pose and the second object pose at the first time step based on ordinary differential equations to obtain and output the target action features; and obtaining the target action features output by the target feature generation network.

[0150] Optionally, inputting the first object pose, the second object pose, and the first time step into the target feature generation network includes: extracting features from the first object pose to obtain first object pose features, and extracting features from the second object pose to obtain second object pose features, wherein the first object pose features are used to characterize the motion state of the target object at a first moment corresponding to the first object pose, the second object pose features are used to characterize the motion state of the target object at a second moment corresponding to the second object pose, and the first time step is the time interval between the first moment and the second moment; inputting the first object pose features, the second object pose features, and the first time step into a first gated recurrent unit, wherein the target feature generation network includes the first gated recurrent unit, the first gated recurrent unit is used to extract features from the first object pose features, the second object pose features, and the first time step based on ordinary differential equations to obtain hidden state features of the target object within a target time period, and the target action features include the hidden state features.

[0151] Optionally, feature extraction is performed on the first object's pose to obtain first object pose features, and feature extraction is performed on the second object's pose to obtain second object pose features, including: performing an exponential mapping on the first joint position of the target object at the first time moment to obtain first joint features, wherein the first object pose includes the first joint position; determining the first joint velocity features of the joint at the first time moment based on the first joint features, wherein the first joint velocity features are used to characterize the movement velocity value and velocity direction of the joint at the first time moment; performing position encoding on the temporal position of the first time moment within the target time period to obtain first position features; and combining the first joint features and the first joint velocity features... The first object pose feature is obtained by concatenating the first position feature with the second object position feature; the second joint position of the target object at the second time moment is exponentially mapped to obtain the second joint feature, wherein the second object pose includes the second joint position; the second joint velocity feature of the joint at the second time moment is determined based on the second joint feature, wherein the second joint velocity feature is used to characterize the movement velocity value and velocity direction of the joint at the second time moment; the temporal position of the second time moment within the target time period is positionally encoded to obtain the second position feature; the second joint feature, the second joint velocity feature and the second position feature are concatenated to obtain the second object pose feature.

[0152] Optionally, predicting the target object's pose at the target time based on the target action features, candidate object poses, and the second time step includes: inputting the target action features, candidate object poses, and the second time step into a target pose prediction network, wherein the target pose prediction network is used to calculate the target action features, candidate object poses, and the second time step based on ordinary differential equations to obtain the target object pose; and obtaining the target object pose output by the target pose prediction network.

[0153] Optionally, inputting the target action features, the candidate object pose, and the second time step into the target pose prediction network includes: inputting the target action features, the candidate object pose, and the second time step into a second gated recurrent unit to obtain a third object pose feature of the target object at the target time, wherein the target pose prediction network includes the second gated recurrent unit, which is used to calculate the target action features, the candidate object pose, and the second time step based on ordinary differential equations to obtain the third object pose feature, which is used to characterize the motion state of the target object at the target time; and performing feature transformation on the third object pose feature to obtain the target object pose.

[0154] Optionally, the target feature generation network is generated by the following method: constructing an initial pose prediction model, wherein the initial pose prediction model includes an initial feature generation network and an initial pose prediction network connected in sequence; training the initial pose prediction model using object pose samples of a third object pose labeled with the time to be predicted to obtain a target pose prediction model, wherein the target pose prediction model includes the target feature generation network and the target pose prediction network connected in sequence, the target feature generation network is used to detect the target action features based on the first object pose, the second object pose, and the first time step, and the target pose prediction network is used to predict the target object pose based on the target action features, the candidate object pose, and the second time step.

[0155] Optionally, training the initial pose prediction model using object pose samples labeled with the object pose at the time to be predicted to obtain the target pose prediction model includes: inputting the object pose samples into the initial pose prediction model to obtain the fourth object pose at the time to be predicted output by the initial pose prediction model; calculating the quaternion loss value of the initial pose prediction model using the third object pose and the fourth object pose; and adjusting the model parameters of the initial pose prediction model according to the quaternion loss value until the initial pose prediction model converges to obtain the target pose prediction model.

[0156] All information obtained in this disclosure (including but not limited to object poses) is information and data authorized by the user or fully authorized by all parties. The collection, use and processing of such data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and a corresponding operation entry point is provided for the user to choose to authorize or refuse.

[0157] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0158] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0159] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0160] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0161] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0162] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0163] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0164] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0165] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0166] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method of predicting a pose of an object, the method comprising: The method comprises the following steps: obtaining N reference object poses of a target object in a target time period, wherein the reference object poses are used to represent the action form of the target object at corresponding time points, and N is an integer greater than 1; detecting a target action feature of the target object according to a first object pose, a second object pose and a first time step, wherein the target action feature is used to represent the continuous transformation trajectory of the action form of the target object in the target time period, the N reference object poses include the first object pose and the second object pose, and the first time step is used to indicate the time interval from the first object pose to the second object pose of the target object; predicting a target object pose of the target object at a target time point according to the target action feature, a candidate object pose and a second time step, wherein the N reference object poses include the candidate object pose, the target time point is located after the target time period, the second time step is used to indicate the time interval between the candidate time point corresponding to the candidate object pose and the target time point, and the candidate time point is the last time point of the target time point.

2. The method of claim 1, wherein, The method comprises the following steps: inputting the first object pose, the second object pose and the first time step into a target feature generation network, wherein the target feature generation network is used to perform feature extraction on the first object pose and the second object pose based on an ordinary differential equation for the first time step, obtain the target action feature and output the target action feature; obtaining the target action feature output by the target feature generation network.

3. The method of claim 2, wherein, The method comprises the following steps: performing feature extraction on the first object pose to obtain a first object pose feature, and performing feature extraction on the second object pose to obtain a second object pose feature, wherein the first object pose feature is used to represent the motion state of the target object at a first time point corresponding to the first object pose, the second object pose feature is used to represent the motion state of the target object at a second time point corresponding to the second object pose, and the first time step is the time interval between the first time point and the second time point; inputting the first object pose feature, the second object pose feature and the first time step into a first gated recurrent unit, wherein the target feature generation network includes the first gated recurrent unit, the first gated recurrent unit is used to perform feature extraction on the first object pose feature, the second object pose feature and the first time step based on an ordinary differential equation to obtain a hidden state feature of the target object in a target time period, and the target action feature includes the hidden state feature.

4. The method of claim 3, wherein, performing feature extraction on the first object pose to obtain a first object pose feature, and performing feature extraction on the second object pose to obtain a second object pose feature, wherein the first object pose feature is used to represent the motion state of the target object at a first time point corresponding to the first object pose, the second object pose feature is used to represent the motion state of the target object at a second time point corresponding to the second object pose, and the first time step is the time interval between the first time point and the second time point. mapping the first joint position of the target object at the first time to obtain a first joint feature, wherein the first object pose includes the first joint position; determining a first joint velocity feature of the joint at the first time according to the first joint feature, wherein the first joint velocity feature is used to represent a moving speed value and a speed direction of the joint at the first time; performing position encoding on a time sequence position of the target time period at the first time to obtain a first position feature; and performing feature splicing on the first joint feature, the first joint velocity feature, and the first position feature to obtain the first object pose feature; mapping the second joint position of the target object at the second time to obtain a second joint feature, wherein the second object pose includes the second joint position; determining a second joint velocity feature of the joint at the second time according to the second joint feature, wherein the second joint velocity feature is used to represent a moving speed value and a speed direction of the joint at the second time; performing position encoding on a time sequence position of the target time period at the second time to obtain a second position feature; and performing feature splicing on the second joint feature, the second joint velocity feature, and the second position feature to obtain the second object pose feature.

5. The method of claim 1, wherein, The target object pose of the target object at the target time is predicted according to the target action feature, the candidate object pose, and the second time step, including: inputting the target action feature, the candidate object pose, and the second time step into a target pose prediction network, wherein the target pose prediction network is used to calculate the target action feature, the candidate object pose, and the second time step based on an ordinary differential equation to obtain the target object pose; obtaining the target object pose output by the target pose prediction network.

6. The method of claim 5, wherein, The target action feature, the candidate object pose, and the second time step are input into a target pose prediction network, including: inputting the target action feature, the candidate object pose, and the second time step into a second gated recurrent unit to obtain a third object pose feature of the target object at the target time, wherein the target pose prediction network includes the second gated recurrent unit, the second gated recurrent unit is used to calculate the target action feature, the candidate object pose, and the second time step based on an ordinary differential equation to obtain the third object pose feature, and the third object pose feature is used to represent a motion state of the target object at the target time; performing feature conversion on the third object pose feature to obtain the target object pose.

7. The method of claim 2, wherein, The target feature generation network is generated by the following method: constructing an initial pose prediction model, wherein the initial pose prediction model includes an initial feature generation network and an initial pose prediction network connected in sequence; The initial pose prediction model is trained using an object pose sample labeled with a third object pose at a to-be-predicted moment, to obtain a target pose prediction model, wherein the target pose prediction model comprises a target feature generation network and a target pose prediction network connected in sequence, the target feature generation network is configured to detect the target motion feature according to the first object pose, the second object pose and the first time step, and the target pose prediction network is configured to predict the target object pose at the target moment according to the target motion feature, the candidate object pose and the second time step.

8. The method of claim 7, wherein, The initial pose prediction model is trained using an object pose sample labeled with a third object pose at a to-be-predicted moment, to obtain a target pose prediction model, wherein the target pose prediction model comprises a target feature generation network and a target pose prediction network connected in sequence, the target feature generation network is configured to detect the target motion feature according to the first object pose, the second object pose and the first time step, and the target pose prediction network is configured to predict the target object pose at the target moment according to the target motion feature, the candidate object pose and the second time step. The initial pose prediction model is trained using an object pose sample labeled with a third object pose at a to-be-predicted moment, to obtain a target pose prediction model, wherein the target pose prediction model comprises a target feature generation network and a target pose prediction network connected in sequence, the target feature generation network is configured to detect the target motion feature according to the first object pose, the second object pose and the first time step, and the target pose prediction network is configured to predict the target object pose at the target moment according to the target motion feature, the candidate object pose and the second time step. The initial pose prediction model is trained using an object pose sample labeled with a third object pose at a to-be-predicted moment, to obtain a target pose prediction model, wherein the target pose prediction model comprises a target feature generation network and a target pose prediction network connected in sequence, the target feature generation network is configured to detect the target motion feature according to the first object pose, the second object pose and the first time step, and the target pose prediction network is configured to predict the target object pose at the target moment according to the target motion feature, the candidate object pose and the second time step. The initial pose prediction model is trained using an object pose sample labeled with a third object pose at a to-be-predicted moment, to obtain a target pose prediction model, wherein the target pose prediction model comprises a target feature generation network and a target pose prediction network connected in sequence, the target feature generation network is configured to detect the target motion feature according to the first object pose, the second object pose and the first time step, and the target pose prediction network is configured to predict the target object pose at the target moment according to the target motion feature, the candidate object pose and the second time step.

9. An object pose prediction device characterized by comprising: The processor is configured to run a program, and the program is configured to perform the object pose prediction method in any one of claims 1 to 8 when the program is running. The processor is configured to run a program, and the program is configured to perform the object pose prediction method in any one of claims 1 to 8 when the program is running. The processor is configured to run a program, and the program is configured to perform the object pose prediction method in any one of claims 1 to 8 when the program is running. ​ 10. A processor, comprising: ​ 11. An electronic device, comprising: ​

Images