Position and posture prediction method, position and posture prediction device, storage medium and electronic device

Abstract

The present disclosure provides a posture prediction method, device, computer-readable storage medium and electronic device, which relate to the field of navigation and positioning technology. The posture prediction method comprises: acquiring first inertial data; processing the first inertial data using a pre-trained posture prediction model to obtain posture data; determining second inertial data based on the first inertial data and the posture data; processing the second inertial data using a pre-trained position prediction model to obtain position data. The present disclosure improves the accuracy of posture prediction.

Description

Technical Field

[0001] The present disclosure relates to the field of navigation and positioning technology, and in particular to a posture prediction method, a posture prediction device, a computer-readable storage medium, and an electronic device. Background Art

[0002] With the popularization of informatization, more and more services based on positioning information are being provided. For example, by real-time positioning and tracking of pedestrians, services such as motion trajectory mapping, indoor navigation, and augmented reality can be provided.

[0003] In the related art, the accuracy of positioning and tracking is too dependent on the accuracy of the sensor, which makes the positioning result easily affected by sensor errors.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to those skilled in the art. Summary of the invention

[0005] The present disclosure provides a posture prediction method, a posture prediction device, a computer-readable storage medium and an electronic device to improve the accuracy of positioning at least to a certain extent.

[0006] According to a first aspect of the present disclosure, a posture prediction method is provided, comprising: acquiring first inertial data; processing the first inertial data using a pre-trained posture prediction model to obtain first posture data; determining second inertial data based on the first inertial data and the first posture data; and processing the second inertial data using a pre-trained position prediction model to obtain position data.

[0007] According to a second aspect of the present disclosure, a posture prediction device is provided, including: a data acquisition module, configured to acquire first inertial data; a posture prediction module, configured to process the first inertial data using a pre-trained posture prediction model to obtain first posture data; a data optimization module, configured to determine second inertial data based on the first inertial data and the first posture data; and a position prediction module, configured to process the second inertial data using a pre-trained position prediction model to obtain position data.

[0008] According to a third aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the posture prediction method of the first aspect and its possible implementation methods are implemented.

[0009] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the posture prediction method of the above-mentioned first aspect and its possible implementation method by executing the executable instructions.

[0010] The technical solution disclosed in this disclosure has the following beneficial effects:

[0011] In the posture prediction method, the first inertial data is processed by the posture prediction model to predict the posture data, and the first inertial data and the first posture data are optimized to obtain the second inertial data, and the second inertial data is processed by the position prediction model to predict the position data. On the one hand, the posture prediction and the position prediction are decomposed into two relatively independent processes, and the posture prediction model and the position prediction model respectively perform corresponding processing, which is conducive to improving the quality of single process processing, improving the accuracy of posture data and position data, and realizing relatively lightweight posture prediction models and position prediction models. On the other hand, the position prediction is achieved by processing the optimized second inertial data, which can reduce the error influence of the inertial sensor and further improve the accuracy of the position data. On the other hand, based on the generalization ability of the posture prediction model and the position prediction model, the limitation of the present solution on the application scenario can be reduced, and it can be applied to the posture prediction of mobile terminals in almost any state.

[0012] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and for those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0014] Figure 1 A schematic diagram showing a system architecture in this exemplary embodiment;

[0015] Figure 2 A flowchart of a posture prediction method in this exemplary embodiment is shown;

[0016] Figure 3 A schematic diagram showing a posture prediction model in this exemplary embodiment;

[0017] Figure 4 A sub-flow chart of a posture prediction method in this exemplary embodiment is shown;

[0018] Figure 5 A schematic diagram showing posture prediction in this exemplary embodiment is shown;

[0019] Figure 6 A sub-flow chart of a posture prediction method in this exemplary embodiment is shown;

[0020] Figure 7 A sub-flow chart of a posture prediction method in this exemplary embodiment is shown;

[0021] Figure 8 A schematic diagram showing forward propagation and back propagation of model training in this exemplary embodiment;

[0022] Fig. 9 A schematic diagram of the structure of a posture prediction device in this exemplary embodiment is shown;

[0023] Fig.10 A schematic structural diagram of an electronic device in this exemplary embodiment is shown. DETAILED DESCRIPTION

[0024] Exemplary embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, the exemplary embodiments can be implemented in various forms and should not be construed as being limited to the examples set forth herein. These embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concept of the exemplary embodiments will be fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, the known technical solutions are not shown or described in detail to avoid obscuring the various aspects of the present disclosure.

[0025] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0026] In one solution of the related technology, pedestrians are positioned and tracked by gait detection. This solution has certain requirements on the state of the mobile terminal. For example, pedestrians are required to fix the mobile terminal on their arms or legs, and the pedestrian's posture is calculated by sensing the swing of their arms or legs during movement. It can be seen that this solution has great limitations on application scenarios. For example, when the user puts the mobile terminal in a pocket or backpack, the solution cannot be implemented. In addition, this solution is also difficult to apply to the positioning and tracking of non-pedestrian targets such as unmanned vehicles and drones.

[0027] In view of the above problems, the exemplary embodiments of the present disclosure first provide a posture prediction method for predicting the posture of a mobile terminal, which is generally equivalent to the posture of a target (such as a user) holding or installing the mobile terminal. Figure 1 The system architecture and application scenarios of the operating environment of this exemplary embodiment are exemplarily described.

[0028] Figure 1 A schematic diagram of a system architecture is shown, and the system architecture 100 may include a mobile terminal 110 and a computing device 120. The mobile terminal 110 may be a smart phone, a tablet computer, a wearable device, an unmanned vehicle, a drone, etc. The mobile terminal 110 includes at least an inertial sensor for sensing inertial data of the mobile terminal 110. The computing device 120 may be a terminal computer or a server providing positioning-related services. The mobile terminal 110 and the computing device 120 may be connected via a wired or wireless communication link to exchange data.

[0029] In one embodiment, the mobile terminal 110 may send the inertial data to the computing device 120. For example, when the user holds the mobile terminal 110 for exercise, the mobile terminal 110 sends the inertial data to the computing device 120 in real time. The computing device 120 obtains the posture data and position data of the mobile terminal 110 by executing the above-mentioned posture prediction method, and may return the result to the mobile terminal 110. Therefore, the execution subject of the above-mentioned posture prediction method may be the computing device 120.

[0030] In one implementation, this exemplary implementation can also be implemented independently based on the mobile terminal 110. For example, after the mobile terminal 110 collects inertial data through the built-in inertial sensor, the mobile terminal 110 executes the above-mentioned posture prediction method to obtain the posture data and position data of the mobile terminal 110. Therefore, the execution subject of the above-mentioned posture prediction method can also be the mobile terminal 110.

[0031] Combine the following Figure 2 The posture prediction method in this exemplary embodiment is described. Figure 2 An exemplary process of the posture prediction method is shown, which may include:

[0032] Step S210, acquiring first inertial data;

[0033] Step S220, using a pre-trained posture prediction model to process the first inertial data to obtain first posture data;

[0034] Step S230, determining second inertial data according to the first inertial data and the first posture data;

[0035] Step S240: Process the second inertial data using a pre-trained position prediction model to obtain position data.

[0036] In the above-mentioned posture prediction method, the first inertial data is processed by the posture prediction model to predict the posture data, and the first inertial data and the first posture data are optimized to obtain the second inertial data, and then the second inertial data is processed by the position prediction model to predict the position data. On the one hand, the posture prediction and position prediction are decomposed into two relatively independent processes, and the posture prediction model and the position prediction model respectively perform corresponding processing, which is conducive to improving the quality of single process processing, improving the accuracy of posture data and position data, and realizing relatively lightweight posture prediction models and position prediction models. On the other hand, the position prediction is achieved by processing the optimized second inertial data, which can reduce the error influence of the inertial sensor and further improve the accuracy of the position data. On the other hand, based on the generalization ability of the posture prediction model and the position prediction model, the limitation of the present solution on the application scenario can be reduced, and it can be applied to the posture prediction of mobile terminals in almost any state.

[0037] Below Figure 2 Each step in the procedure is described in detail.

[0038] In step S210 , first inertial data is acquired.

[0039] The first inertial data may be raw data output by the inertial sensor, i.e., inertial data without further processing. In this exemplary embodiment, the inertial sensor may include one or more of an accelerometer, a gyroscope, and a magnetometer, wherein the accelerometer may output acceleration data (Acc data), the gyroscope may output angular velocity data (Gyro data), and the magnetometer may output magnetometer data (Mag data). Depending on the type of inertial sensor configured by the mobile terminal 110, the first inertial data may include one or more of the first acceleration data, the first angular velocity data, and the first magnetometer data.

[0040] In one implementation, the first inertial data at different times may be acquired according to the sensing frequency of the inertial sensor. For example, the sensing frequency of the inertial sensor is 50 Hz, that is, the period is 20 ms, which means that the inertial sensor collects the first inertial data every 20 ms.

[0041] The sensing frequencies of different types of inertial sensors may be the same or different. In one embodiment, when the sensing frequencies of at least two inertial sensors are different, the frequencies of the output data of the different types of inertial sensors may be the same by upsampling or downsampling the data of at least one of the inertial sensors.

[0042] There may be a time error between different types of inertial sensors. For example, an accelerometer outputs acceleration data at time t, and a gyroscope outputs angular velocity data at time t. Due to the response delay of the sensor itself, the delay in data transmission, and other factors, the acceleration data at time t and the angular velocity data actually correspond to different data sensing times. In one embodiment, the time error between different types of inertial sensors can be corrected in advance, for example, the time error between the accelerometer and the gyroscope can be calibrated in combination with other or external sensors, and the time error can be minimized as much as possible through compensation.

[0043] Continue to refer Figure 2 In step S220, the first inertial data is processed using a pre-trained posture prediction model to obtain first posture data.

[0044] The posture data is used to indicate the orientation state of the mobile terminal 110. The first posture data is the posture data output by the posture prediction model, and the posture data obtained after optimization below is the second posture data, and "first" and "second" are used to distinguish between the two. The present disclosure does not limit the form and specific content of the first posture data. For example, the first posture data can be an absolute posture in a certain coordinate system, or a relative posture relative to a certain reference posture, etc. Exemplarily, the first posture data can include any one or more of posture quaternion, Euler angle, and rotation matrix.

[0045] The posture prediction model is used to predict posture data based on the input first inertial data. In this exemplary embodiment, the posture prediction model can be trained to enable it to output accurate first posture data. The training process is referred to the content of the model training method section below, which is not repeated here.

[0046] When the first inertial data acquired in step S210 includes multiple sensor data, the first inertial data input into the posture prediction model in step S220 may be any one or more of them. For example, the first acceleration data, the first angular velocity data, and the first magnetometer data are acquired in step S210, and in step S220, the first acceleration data, the first angular velocity data, and the first magnetometer data may all be input into the posture prediction model, or only part of them may be input into the posture prediction model.

[0047] In one embodiment, the first inertial data at time t can be processed using a posture prediction model to obtain the first posture data at time t. In this exemplary embodiment, time 0 can represent the starting time (such as the time when the positioning tracking service is turned on), time 1, time 2, ..., time t, etc. represent the sampling time of the inertial sensor, and the interval between two adjacent sampling times can be represented as Δt, Δt can be the above-mentioned 20ms; t can represent any sampling time, such as the current time.

[0048] In one embodiment, the first inertial data from time i to time t can be processed using a posture prediction model to at least obtain the first posture data at time t. Wherein, i is a positive integer less than t. From time i to time t, the inertial sensor collects t-i+1 first inertial data to form a first inertial data sequence of length t-i+1. Exemplarily, the first inertial data sequence is input into the posture prediction model. After being processed by the model, the first posture data at time t can be output, and the first posture data at any one or more of i, i+1, ..., t-1 can also be output.

[0049] The posture prediction model can be an LSTM (Long Short-Term Memory) network. Figure 3 The processing of the posture prediction model is illustrated as follows. From time 1 to time t, there are t sampling moments in total, corresponding to the collection of t groups of first inertial data, where the first acceleration data is a1, a2, ..., a t , the first angular velocity data is w1, w2, ..., w t , the first magnetometer data is B1, B2, ..., B t t groups of first inertial data are input into LSTM, and processed by t LSTM cells and fully connected layers respectively, and the first posture data of each sampling moment is output, which is represented as q1, q2, ..., q t .

[0050] It should be understood that other neural networks may also be used to implement the posture prediction model, such as a general RNN (Recurrent Neural Network), a GRU (Gated Recurrent Unit), and the like.

[0051] Continue to refer Figure 2 In step S230, second inertial data is determined according to the first inertial data and the first posture data.

[0052] The first inertial data includes data related to the posture, such as the first angular velocity data, etc. Since there are usually errors in the first inertial data, and there may also be errors in the first posture data, the first inertial data and the first posture data are not completely consistent in essence. For example, the first inertial data and the first posture data are converted into data of the same form, such as posture quaternion, and the two are not exactly the same. Based on the errors between the two, data optimization can be performed to obtain the second inertial data. It can be seen that the second inertial data is the inertial data obtained by further processing on the basis of the first inertial data and the first posture data, rather than the raw data collected by the inertial sensor, and the second inertial data is used for subsequent position prediction.

[0053] In addition, the first posture data may lack translation information, so the information is not sufficient for position prediction. In step S230, the translation information in the first inertial data can be combined with the first posture data to obtain second inertial data including both translation information and posture information.

[0054] The second inertial data may be the same as or different from the first inertial data in type and form. In one embodiment, the second inertial data may include one or more of second acceleration data, second angular velocity data, and second magnetometer data. Exemplarily, the first inertial data includes first acceleration data, first angular velocity data, and first magnetometer data, and the second inertial data includes second acceleration data and second angular velocity data, but does not include magnetometer data.

[0055] In one implementation, the first inertial data and the first attitude data may be converted into data of the same form, such as attitude quaternion, and then the two may be fused, such as averaging or weighted averaging, to obtain the second inertial data.

[0056] In one embodiment, the posture prediction method may further include the following steps:

[0057] When the first inertial data is processed using the pre-trained posture prediction model, the uncertainty (or confidence, which is the opposite concept of uncertainty) of the first posture data is also obtained.

[0058] That is to say, the posture prediction model can output the first posture data and its uncertainty at the same time. Since the first posture data is prediction data, it has a certain uncertainty, and the uncertainty is used to indicate the degree of the uncertainty. In addition, the weight of the first inertial data and the first posture data when they are fused can also be determined according to the uncertainty. Generally, the higher the uncertainty (or the lower the confidence), the less credible the first posture data is, and the lower the weight of the first posture data when fused.

[0059] In one embodiment, reference Figure 4 As shown, the above-mentioned determination of the second inertial data according to the first inertial data and the first posture data may include the following steps S410 and S420:

[0060] Step S410, performing Kalman filtering based on the first inertial data and the first posture data;

[0061] Step S420: determining second inertial data according to the result of Kalman filtering.

[0062] Among them, either the first inertial data or the first posture data can be used as state data, and the other can be used as observation data, and the optimal estimation of the posture can be performed through Kalman filtering. The posture data obtained after Kalman filtering is recorded as the second posture data, that is, the result of Kalman filtering includes the second posture data.

[0063] The following two examples illustrate the Kalman filter:

[0064] Example 1: The Kalman filtering based on the first inertial data and the attitude data may include the following steps:

[0065] The first posture data is used as state data, the first inertial data is used as observation data, and the second posture data is obtained through Kalman filtering.

[0066] A state equation may be established for the first posture data, for example, by fitting the first posture data at multiple sampling moments to obtain the state equation. Data related to the posture (such as the first angular velocity data) is extracted from the first inertial data as observation data, and the posture estimation is optimized by Kalman filtering to obtain the second posture data.

[0067] Example 2: The Kalman filtering based on the first inertial data and the attitude data may include the following steps:

[0068] The first inertial data is used as the state data, the first posture data is used as the observation data, and the second posture data is obtained through Kalman filtering.

[0069] Among them, the data related to the posture (such as the first angular velocity data) can be extracted from the first inertial data, which can represent the posture change between two adjacent sampling moments, thereby establishing a state equation. The first posture data is used as the observation data, and the posture estimation is optimized through Kalman filtering to obtain the second posture data.

[0070] In one embodiment, when the uncertainty of the first posture data is obtained, the first inertial data is used as the state data, the first posture data is used as the observation data, and the second posture data is obtained by Kalman filtering, which may include the following steps:

[0071] Determining an observation data covariance matrix according to the uncertainty of the first posture data;

[0072] The first inertial data is used as the state data, the first posture data is used as the observation data, the observation data covariance matrix is ​​substituted and an extended Kalman filter is performed to obtain the second posture data.

[0073] EKF (Extended Kalman Filter) is a Kalman filter algorithm that can be used for nonlinear state estimation. In the EKF calculation process, the covariance matrix of multiple related parameters is involved. In this exemplary embodiment, the covariance matrix of the observed data is determined according to the uncertainty of the first posture data. As for other covariance matrices, they can be calculated based on the data at adjacent sampling moments, or preset values ​​can be used. The first inertial data is determined as the state data, the first posture data is determined as the observation data, the covariance matrix of the related parameters is substituted into the EKF algorithm formula, and the second posture data is calculated.

[0074] The calculation process of EKF is described in detail below. It should be noted that, since the first posture data is used as the observation data in this exemplary embodiment, the first posture data is essentially the data predicted by the posture prediction model, not the data directly observed. Therefore, in the following calculation process, the observation equation is simplified to the observation data itself, that is, the observation equation g(X t ,Ω ob )=X t (X t represents the prior value of the state data at time t, Ω ob represents the observation noise), and thereby simplifies the related observation matrix, observation noise control matrix, etc. into the unit matrix.

[0075] First, the uncertainty of the first posture data is expressed as a k-dimensional vector, denoted as s, where k is a positive integer and its value can be set according to experience or actual conditions, such as 10. The observed data covariance matrix R is calculated based on the uncertainty of the first posture data. t , when the first attitude data is attitude quaternion, R tIt is a 4*4 symmetric matrix, as shown below:

[0076]

[0077] in,

[0078] Then, based on the first angular velocity data in the first inertial data, the state equation is established as follows:

[0079] X t =X t-1 +dΔθX t-1 (2)

[0080]

[0081] Among them, X t represents the prior value of the state data at time t. X in formula (2) t-1 You can use the priori value at time t-1 or the a posteriori value. The a posteriori value is preferred. In this case, formula (2) is: t =X′ t-1 +dΔθX′ t-1 ; d represents the state update coefficient, and its value can be set according to experience or actual conditions. For example, the value of d can be determined in the range of 0 to 1, such as 1 / 2; Δθ represents the integral value calculated according to the first angular velocity data, Δθ x , Δθ y , Δθ z are the values ​​of Δθ in different axial directions respectively.

[0082] Next, calculate X t About X t-1 The derivative matrix A, i.e. the state transfer matrix, can be the Jacobian matrix of the state equation for the posterior value of the state data at time t-1 (i.e. the second posture data at time t-1). And establish the state prediction covariance estimate as shown below:

[0083]

[0084] in, represents the posterior state covariance matrix at time t-1, represents the prior state covariance matrix at time t. Q represents the preset noise covariance matrix. Can be a given initial value, then after extrapolation at each sampling moment, we can get

[0085] Then calculate the Kalman gain K t, since the observation matrix and the observation noise control matrix are simplified to the unit matrix, these two matrices are omitted when calculating the Kalman gain, as shown below:

[0086]

[0087] Finally, update the state data and the corresponding covariance matrix as follows:

[0088] X′ t =X t +K t (q t -X t ) (6)

[0089]

[0090] Among them, q t is the first posture data (i.e., observation data) at time t, X′ t is the posterior value of the state data at time t, that is, the second posture data at time t obtained through Kalman filtering. The posterior state covariance matrix at time t can be used to calculate the prior state covariance matrix at the next sampling time (i.e., time t+1).

[0091] After the second posture data is obtained through Kalman filtering, data required by the position prediction model, namely, the second inertial data, can be generated according to the second posture data and the first inertial data.

[0092] In one implementation, the determining of the second inertial data according to the result of Kalman filtering may include the following steps:

[0093] The first inertial data is converted into a preset coordinate system using the second posture data to obtain second inertial data.

[0094] The preset coordinate system may be a coordinate system where the second posture data is located (usually equivalent to the coordinate system where the first posture data is located), or any other coordinate system used as a reference. The first inertial data is usually data collected based on the coordinate system of the mobile terminal 110. Through the conversion of the coordinate system, the first inertial data and the second posture data (or the first posture data) are unified into the same coordinate system, so as to facilitate processing and representation of posture data in the same coordinate system.

[0095] In one embodiment, the first inertial data can be first converted into the same data form as the second posture data. For example, when the second posture data is a posture quaternion, the first inertial data can also be converted into a posture quaternion, such as converting the first inertial data into a posture quaternion based on the northeast sky coordinate system (or other geographic coordinate system), and recording the first inertial data after conversion into the posture data as the third posture data. Then, the third posture data is converted into a coordinate system through the second posture data. Finally, the third posture data after the coordinate system conversion is further converted according to the data form required by the position prediction model, such as being converted into acceleration data, angular velocity data, etc., to obtain the second inertial data.

[0096] In one embodiment, the first inertial data can be converted to a preset coordinate system using the second posture data, and the second posture data and the first inertial data are fused in the preset coordinate system to obtain the second inertial data. For example, the first acceleration data and the first angular velocity data in the first inertial data are obtained, and the first angular velocity data is converted into a posture quaternion, which is recorded as the third posture data; then the conversion parameters of the coordinate system are calculated according to the second posture data and the third posture data; then the first acceleration data is converted to the preset coordinate system according to the conversion parameters, and is fused with the second posture data to obtain the second inertial data including translation information and posture information.

[0097] In one embodiment, in step S220, the first acceleration data, the first angular velocity data, and the first magnetometer data can be processed using a pre-trained posture prediction model to obtain the first posture data. That is, the data of the three inertial sensors can all be input into the posture prediction model to improve the accuracy of posture prediction. In step S230, the second acceleration data and the second angular velocity data can be determined based on the first acceleration data, the first angular velocity data and the posture data. That is, the magnetometer data may not be used in the position prediction, so when performing posture data optimization and position prediction, only the acceleration data and the angular velocity data are applicable, thereby improving the calculation efficiency.

[0098] In one embodiment, second posture data can be determined based on the first angular velocity data and the first posture data, and the first acceleration data and the first angular velocity data can be converted to a preset coordinate system using the second posture data to obtain second inertial data, wherein the second inertial data includes second acceleration data and second angular velocity data.

[0099] Continue to refer Figure 2 In step S240, the second inertial data is processed using a pre-trained position prediction model to obtain position data.

[0100] The position data is used to represent the absolute position or relative position of the mobile terminal 110, and generally includes coordinates in three directions, namely, x, y, and z (ie, offsets), but the present disclosure does not limit this. For example, the position data may also be represented in the form of spherical coordinates.

[0101] The position prediction model is used to predict the position data according to the input second inertial data. In this exemplary embodiment, the position prediction model can be trained to enable it to output accurate position data. The training process is referred to the content of the model training method section below, which is not repeated here.

[0102] The position prediction model can adopt the same or similar structure as the posture prediction model. For example, the posture prediction model and the position prediction model can both adopt Figure 3 The LSTM network structure is the same as that of the previous one, but the parameters of the two models are different. The position prediction model can also adopt a structure different from that of the posture prediction model. For example, the position prediction model can adopt a ResNet (residual network) structure.

[0103] In one implementation, the second inertial data at time t may be processed using a position prediction model to obtain position data at time t.

[0104] In one embodiment, the second inertial data from time i to time t can be processed using a position prediction model to at least obtain the position data at time t. For example, the second inertial data sequence from time i to time t is input into the position prediction model, and after being processed by the model, the position data at time t can be output, and the position data at any one or more times among i, i+1, ..., t-1 can also be output; the position data at time t can represent the relative displacement at time t relative to time t-1.

[0105] After obtaining the position data, the first posture data (or the second posture data) and the position data can be output as a posture prediction result.

[0106] In one embodiment, the position data obtained in step S240 is recorded as the first position data. Kalman filtering can also be performed based on the first inertial data (or the second inertial data) and the first position data, the second position data is determined according to the result of Kalman filtering, and the second position data is output as the final position prediction result. The first inertial data (or the second inertial data) can be used as state data and the first position data can be used as observation data, or the first position data can be used as state data and the first inertial data (or the second inertial data) can be used as observation data. The process of Kalman filtering can refer to the content of Kalman filtering for attitude estimation above, which will not be repeated here.

[0107] Figure 5The schematic flow of the posture prediction method is shown. Taking time t as an example, the first inertial data a at time t is obtained. t 、w t , B t ; t 、w t , B t Input the posture prediction model to obtain the first posture data q at time t t and its uncertainty s t ; q t 、s t 、w t Input the extended Kalman filter to perform optimal attitude estimation and output the second attitude data X' t ; Using X' t A t 、w t Convert to X' t In the preset coordinate system, the second inertial data a' is obtained t 、w' t ; a' t 、w' t Input the location prediction model to obtain the location data Z at time t t ; Finally, the output can be X' t With Z t , as the pose prediction result at time t.

[0108] The exemplary embodiments of the present disclosure also provide a method for training a posture prediction model and a position prediction model. Figure 6 As shown, the posture prediction method may further include the following steps S610 to S650:

[0109] Step S610, obtaining first inertial sample data and position annotation data corresponding to the first inertial sample data;

[0110] Step S620, inputting the first inertia sample data into the posture prediction model to obtain first posture sample data;

[0111] Step S630, determining second inertia sample data according to the first inertia sample data and the first sample posture data;

[0112] Step S640, inputting the second inertial sample data into the position prediction model to obtain position sample data;

[0113] Step S650, based on the position sample data and the position annotation data, update the parameters of the position prediction model and the posture prediction model.

[0114] Among them, the first inertia sample data, the second inertia sample data, and the first posture sample data have the same meanings as the above-mentioned first inertia data, the second inertia data, and the first posture data, respectively. The difference is that the first inertia sample data, the second inertia sample data, and the first posture sample data are data used for model training, and are therefore sample data.

[0115] In step S610, the first inertial sample data can be acquired by an inertial sensor, and the position annotation data can be acquired by other means. The position annotation data is the ground truth in the model training. For example, the position of the target can be determined by wireless signal positioning, satellite positioning, visual positioning, etc. to obtain the position annotation data.

[0116] The implementation of steps S620 to S640 is substantially the same as the implementation of steps S220 to S240 described above, and will not be described in detail here.

[0117] After obtaining the position sample data, the loss function value can be calculated based on the deviation between the position sample data and the position annotation data, and then the position prediction model and the attitude prediction model can be back-propagated and updated according to the loss function value to realize the training of the position prediction model and the attitude prediction model.

[0118] In one embodiment, the above steps S610 to S650 may be performed in Figure 2 is executed before the step of to complete the training of the posture prediction model and the position prediction model, so as to Figure 2 In the step, the trained posture prediction model and position prediction model can be used to perform posture prediction.

[0119] In the above Figure 6 In the step, position annotation data is used to realize end-to-end training of the posture prediction model and the position prediction model, without the need to obtain a large amount of posture annotation data, thereby solving the problem that the posture data is difficult to annotate, which is conducive to training high-quality posture prediction models and position prediction models and improving the accuracy of posture prediction.

[0120] The loss function of the position sample data and the position annotation data is recorded as the first loss function. Any form of loss such as L1, L2, etc. can be used. Taking L1 loss as an example, the first loss function is as follows:

[0121]

[0122] Among them, a ij Represents the position sample data at the jth sampling time in the i-th group of samples, y ijRepresents the corresponding position annotation data; N1 represents a total of N1 groups of samples, and a group of samples includes data collected during one movement of the target; T1 represents that each group of samples collects data at T1 sampling moments.

[0123] In one embodiment, reference Figure 7 As shown, before the first inertial sample data is input into the posture prediction model to obtain the posture sample data, the posture prediction method may further include the following steps S710 to S730:

[0124] Step S710, obtaining third inertial sample data and posture annotation data corresponding to the third inertial sample data;

[0125] Step S720, inputting the third inertia sample data into the posture prediction model to obtain second posture sample data;

[0126] Step S730: updating the parameters of the posture prediction model based on the second posture sample data and the posture annotation data.

[0127] Among them, the third inertial sample data may be the same as or different from the above-mentioned first inertial sample data. The third inertial sample data may be obtained by collecting the inertial sensor, and the posture annotation data may be obtained by other means. The posture annotation data is the true value of the posture in the model training. Exemplarily, the posture annotation data may be obtained by a posture calculation method or system built into the mobile terminal 110, such as posture annotation by VIO (Visual Inertial Odometry). Relatively speaking, the annotation difficulty of posture data is higher than that of position data, so a small amount of third inertial sample data and corresponding posture annotation data may be obtained. For example, the third inertial sample data may be a subset of the first inertial sample data, and the inertial sample data in the subset has corresponding position annotation data and posture annotation data, and the inertial sample data outside the subset only has corresponding position annotation data.

[0128] The third inertial sample data is input into the posture prediction model to obtain the second posture sample data, and then the loss function value is calculated based on the deviation between the second posture sample data and the posture annotation data, and the back propagation parameters of the posture prediction model are updated according to the loss function value. That is, the posture prediction model is trained separately using the third inertial sample data and the posture annotation data. The loss function of the second posture sample data and the posture annotation data is recorded as the second loss function, and any form of loss such as L1, L2, etc. can be used. Taking L1 loss as an example, the second loss function is as follows:

[0129]

[0130] Among them, b ijrepresents the second posture sample data at the jth sampling time in the i-th group of samples, h ij Represents the corresponding posture annotation data; N2 represents a total of N2 groups of samples; T2 represents that each group of samples collects data at T2 sampling times.

[0131] The above process of using the third inertial sample data and the posture annotation data to train the posture prediction model separately can be regarded as pre-training of the posture prediction model, so that the posture prediction model first learns certain posture prediction information. Then, the first inertial sample data and the position annotation data are used to jointly train the position prediction model and the posture prediction model, which is conducive to the rapid convergence of the model and improves the training efficiency.

[0132] In one embodiment, the posture prediction method may further include the following steps:

[0133] When the third inertial sample data is input into the attitude prediction model, an uncertainty sample value of the second attitude sample data is also obtained;

[0134] The uncertainty annotation value of the second posture sample data is determined according to the second posture sample data and the posture annotation data.

[0135] Correspondingly, in step S730, the parameters of the posture prediction model may be updated based on the second posture sample data and the posture annotation data, as well as the uncertainty sample value and the uncertainty annotation value of the second posture sample data.

[0136] When constructing a posture prediction model, dual output channels can be set to output the posture prediction result and the uncertainty of the posture prediction result respectively. The uncertainty sample value is the uncertainty of the second posture sample data output by the posture prediction model. The uncertainty annotation value is the true value corresponding to the uncertainty sample value. The uncertainty annotation value can be determined according to the deviation between the second posture sample data and the posture annotation data, as shown below:

[0137]

[0138] The loss function value is calculated based on the deviation between the uncertainty sample value and the uncertainty annotation value of the second posture sample data. The loss function can be recorded as the third loss function. Any form of loss such as L1 and L2 can be used. Taking L1 loss as an example, the third loss function is as follows:

[0139]

[0140] Among them, c ij represents the uncertainty sample value of the second posture sample data at the jth sampling time in the i-th group of samples, v ijIndicates the corresponding uncertainty annotation value. For uncertainty, the number of its samples is the same as the number of samples of the second posture sample data, that is, there are N2 groups of samples, and each group of samples collects data at T2 sampling moments.

[0141] In one embodiment, the second loss function and the third loss function can be combined to train the posture prediction model from two aspects: posture prediction and uncertainty prediction. For example, the second loss function and the third loss function can be weighted to obtain a comprehensive loss function of the posture prediction model, and the posture prediction model can be back-propagated and parameter updated according to the comprehensive loss function value.

[0142] In one implementation, the determining of the second inertial sample data according to the first inertial sample data and the first sample posture data may include the following steps:

[0143] Kalman filtering is performed based on the first inertial sample data and the first sample attitude data to obtain second inertial sample data.

[0144] The process of performing Kalman filtering on the first inertial sample data and the first sample attitude data can be referred to Figure 4 The contents of the part are not repeated here. Therefore, the processing of the attitude prediction model, the Kalman filter, and the processing of the position prediction model are three serial processes, and the three processes are regarded as the global processing process of forward propagation. In step S650, the parameters of the position prediction model, the parameters of the Kalman filter, and the parameters of the attitude prediction model can be updated based on the position sample data and the position annotation data. In other words, the back propagation parameter update can be performed based on the global processing process, that is, the parameters of the two models and the Kalman filter are updated in the order of back propagation such as the position prediction model, the Kalman filter, and the attitude prediction model. The parameters of the Kalman filter include the covariance matrix, the parameter matrix, the noise, etc. of the relevant parameters. After calculating the loss function value, the update amount of one or more parameters in the Kalman filter is calculated in a gradient descent manner, and the parameters of the Kalman filter are updated accordingly, which can improve the accuracy of the Kalman filter and realize the end-to-end training of the position prediction model and the attitude prediction model.

[0145] In one embodiment, the posture prediction method may further include the following steps:

[0146] When the first inertia sample data is input into the posture prediction model, an uncertainty sample value of the first posture sample data is also obtained.

[0147] For example, the posture prediction model is set to a dual-channel structure, and is trained based on the second loss function and the third loss function, so that the posture prediction model has the ability of posture prediction and uncertainty prediction. In this way, when the first inertial sample data is input into the posture prediction model, the posture prediction model can simultaneously output the first posture sample data and the uncertainty sample value of the first posture sample data.

[0148] Furthermore, the Kalman filtering based on the first inertial sample data and the first sample attitude data to obtain the second inertial sample data may include the following steps:

[0149] Determine the sample observation data covariance matrix according to the uncertainty sample value of the first posture sample data;

[0150] The first inertial sample data is used as the state data, the first attitude sample data is used as the observation data, the sample observation data covariance matrix is ​​substituted and extended Kalman filtering is performed to obtain the second inertial sample data.

[0151] The calculation process of the EKF can refer to the above-mentioned content of performing EKF on the first inertial data and the first attitude data. In this way, the calculation method of the Kalman filter is consistent in the model training process and the model application process, so that the trained model can achieve more accurate posture prediction in the application process. In addition, the sample observation data covariance matrix is ​​used as a variable related to the sample. When updating the parameters of the Kalman filter through back propagation, the sample observation data covariance matrix may not be updated, thereby improving training efficiency.

[0152] Figure 8The schematic diagram of forward propagation and back propagation of model training is shown. The first inertial sample data and the third inertial sample data are output by the accelerometer, gyroscope, and magnetometer. The third inertial sample data is input into the posture prediction model to obtain the second posture sample data. The second loss function value is calculated according to the deviation between the second posture sample data and the posture annotation data, and the parameters of the posture prediction model are updated through back propagation, which can be regarded as the pre-training process of the posture prediction model. Input the first inertial sample data into the attitude prediction model to obtain the first attitude sample data; perform extended Kalman filtering on the first attitude sample data and the angular velocity data in the first inertial sample data, and output the optimized attitude sample data; use the optimized attitude sample data to perform coordinate system conversion on the acceleration data and the angular velocity data in the first inertial sample data to obtain the second inertial sample data; input the second inertial sample data into the position prediction model to obtain the position sample data; calculate the first loss function value according to the deviation between the position sample data and the position annotation data, and then update the parameters of the position prediction model, the extended Kalman filter, and the attitude prediction model through the position prediction model, coordinate system conversion (adjustable parameters are usually not used in the calculation of the coordinate system conversion, and only propagation calculation is performed in the back propagation path), extended Kalman filtering, and the back propagation path of the attitude prediction model to realize model training.

[0153] Figure 2 The data flow when performing pose prediction can also be referred to Figure 8 The forward propagation path in .

[0154] The exemplary embodiment of the present disclosure also provides a posture prediction device. Fig. 9 As shown, the posture prediction device 900 may include:

[0155] The data acquisition module 910 is configured to acquire first inertial data;

[0156] The posture prediction module 920 is configured to process the first inertial data using a pre-trained posture prediction model to obtain first posture data;

[0157] The data optimization module 930 is configured to determine the second inertial data according to the first inertial data and the first posture data;

[0158] The position prediction module 940 is configured to process the second inertial data using a pre-trained position prediction model to obtain position data.

[0159] In one implementation, the determining of the second inertial data based on the first inertial data and the first posture data may include:

[0160] Performing Kalman filtering based on the first inertial data and the first posture data;

[0161] The second inertial data is determined according to the result of the Kalman filter.

[0162] In one implementation, the result of the Kalman filtering includes the second posture data; and the Kalman filtering based on the first inertial data and the posture data includes:

[0163] The first inertial data is used as the state data, the first posture data is used as the observation data, and the second posture data is obtained through Kalman filtering.

[0164] In one embodiment, the posture prediction module 920 is further configured to:

[0165] When the first inertial data is processed using the pre-trained attitude prediction model, the uncertainty of the first attitude data is also obtained;

[0166] The first inertial data is used as the state data, the first posture data is used as the observation data, and the second posture data is obtained by Kalman filtering, including:

[0167] Determining an observation data covariance matrix according to the uncertainty of the first posture data;

[0168] The first inertial data is used as the state data, the first posture data is used as the observation data, the observation data covariance matrix is ​​substituted and an extended Kalman filter is performed to obtain the second posture data.

[0169] In one implementation, the determining of the second inertial data according to the result of Kalman filtering includes:

[0170] The first inertial data is converted into a preset coordinate system using the second posture data to obtain second inertial data.

[0171] In one embodiment, the inertial data includes at least one of acceleration data, angular velocity data, and magnetometer data;

[0172] The above-mentioned processing of the first inertial data using the pre-trained posture prediction model to obtain the first posture data includes:

[0173] Processing the first acceleration data, the first angular velocity data, and the first magnetometer data using a pre-trained posture prediction model to obtain first posture data;

[0174] The above-mentioned determining the second inertial data according to the first inertial data and the first posture data includes:

[0175] The second acceleration data and the second angular velocity data are determined according to the first acceleration data, the first angular velocity data and the first posture data.

[0176] In one embodiment, the posture prediction device may further include a model training module, which may include the following submodules:

[0177] A sample data acquisition submodule is configured to acquire the first inertial sample data and the position annotation data corresponding to the first inertial sample data;

[0178] The posture sample prediction submodule is configured to input the first inertia sample data into the posture prediction model to obtain the first posture sample data;

[0179] A sample data optimization submodule, configured to determine second inertia sample data according to the first inertia sample data and the first sample posture data;

[0180] A position sample prediction submodule is configured to input the second inertial sample data into the position prediction model to obtain position sample data;

[0181] The parameter updating submodule is configured to update the parameters of the position prediction model and the posture prediction model based on the position sample data and the position annotation data.

[0182] In one embodiment, before inputting the first inertial sample data into the posture prediction model to obtain the first posture sample data, the sample data acquisition submodule is further configured to obtain the third inertial sample data and the posture annotation data corresponding to the third inertial sample data; the posture sample prediction submodule is further configured to input the third inertial sample data into the posture prediction model to obtain the second posture sample data; the parameter updating submodule is further configured to update the parameters of the posture prediction model based on the second posture sample data and the posture annotation data.

[0183] In one implementation, the posture sample prediction submodule is further configured as follows:

[0184] When the third inertial sample data is input into the attitude prediction model, an uncertainty sample value of the second attitude sample data is also obtained;

[0185] Determine an uncertainty annotation value of the second posture sample data according to the second posture sample data and the posture annotation data;

[0186] The above-mentioned updating of the parameters of the posture prediction model based on the second posture sample data and the posture annotation data includes:

[0187] Based on the second posture sample data and the posture annotation data, as well as the uncertainty sample value and the uncertainty annotation value of the second posture sample data, the parameters of the posture prediction model are updated.

[0188] In one embodiment, the determining of the second inertial sample data according to the first inertial sample data and the first sample posture data includes:

[0189] Performing Kalman filtering based on the first inertial sample data and the first sample attitude data to obtain second inertial sample data;

[0190] The above-mentioned updating of the parameters of the position prediction model and the posture prediction model based on the position sample data and the position annotation data includes:

[0191] Based on the position sample data and the position annotation data, the parameters of the position prediction model, the parameters of the Kalman filter and the parameters of the attitude prediction model are updated.

[0192] In one implementation, the posture sample prediction submodule is further configured as follows:

[0193] When the first inertia sample data is input into the attitude prediction model, an uncertainty sample value of the first attitude sample data is also obtained;

[0194] The Kalman filtering is performed based on the first inertial sample data and the first sample attitude data to obtain the second inertial sample data, including:

[0195] Determine the sample observation data covariance matrix according to the uncertainty sample value of the first posture sample data;

[0196] The first inertial sample data is used as the state data, the first attitude sample data is used as the observation data, the sample observation data covariance matrix is ​​substituted and extended Kalman filtering is performed to obtain the second inertial sample data.

[0197] The specific details of each part of the above-mentioned device have been described in detail in the implementation method of the method part. The undisclosed details can be found in the implementation method of the method part, so they will not be repeated here.

[0198] The exemplary embodiments of the present disclosure also provide a computer-readable storage medium, which can be implemented in the form of a program product, which includes a program code, and when the program product is run on an electronic device, the program code is used to cause the electronic device to perform the steps described in the above "Exemplary Method" section of this specification according to various exemplary embodiments of the present disclosure. In an optional embodiment, the program product can be implemented as a portable compact disk read-only memory (CD-ROM) and includes program code, and can be run on an electronic device, such as a personal computer. However, the program product of the present disclosure is not limited to this, and in this document, the readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, device or device.

[0199] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0200] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0201] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0202] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0203] The exemplary embodiment of the present disclosure further provides an electronic device, which may be, for example, the mobile terminal 110 or the computing device 120 described above. The electronic device may include a processor and a memory. The memory stores executable instructions of the processor, such as program code. The processor executes the pose prediction method or the model training method for pose prediction in the exemplary embodiment by executing the executable instructions, such as executing Figure 2 , Figure 4 , Figure 6 , Figure 7 Any one or more steps in .

[0204] Below Fig.10 The mobile terminal 1000 in FIG. 1 is taken as an example to exemplify the structure of the electronic device. It should be understood by those skilled in the art that, in addition to the components specifically used for mobile purposes, Fig.10 The construction in can also be applied to fixed type equipment.

[0205] like Fig.10 As shown, the mobile terminal 1000 may specifically include: a processor 1010 , a memory 1020 , a bus 1030 , a communication module 1040 , an antenna 1050 , a power module 1060 and a sensor module 1070 .

[0206] The processor 1010 may include one or more processing units, for example: the processor 1010 may include an AP (Application Processor), a modem processor, a GPU (Graphics Processing Unit), an ISP (Image Signal Processor), a controller, an encoder, a decoder, a DSP (Digital Signal Processor), a baseband processor and / or an NPU (Neural-Network Processing Unit), etc. The posture prediction method or the model training method for posture prediction in this exemplary embodiment may be executed by one or more of the AP, GPU, DSP, and NPU. For example, in the posture prediction method, the NPU may load the parameters of the posture prediction model and the position prediction model and execute the algorithm instructions related to the model, and the AP may obtain the first inertial data and output the posture prediction result.

[0207] The processor 1010 may be connected to the memory 1020 or other components through a bus 1030 .

[0208] The memory 1020 may be used to store computer executable program codes, which may include instructions. The processor 1010 executes various functional applications and data processing of the mobile terminal 1000 by running the instructions stored in the memory 1020. The memory 1020 may also store application data, such as images, videos, and other files.

[0209] The communication function of the mobile terminal 1000 can be implemented through the communication module 1040 and the antenna 1050, the modulation and demodulation processor, the baseband processor, etc. The antenna 1050 is used to transmit and receive electromagnetic wave signals. The communication module 1040 can provide 3G, 4G, 5G and other mobile communication solutions applied to the mobile terminal 1000, or wireless communication solutions such as wireless LAN, Bluetooth, and near field communication.

[0210] The power module 1060 is used to implement power management functions, such as charging the battery, powering the device, monitoring the battery status, etc.

[0211] The sensor module 1070 may include one or more sensors for implementing corresponding sensing functions. In this exemplary embodiment, the sensor module 1070 may include an inertial sensor, such as an accelerometer 1071, a gyroscope 1072, and a magnetometer 1073, which are respectively used to sense and collect acceleration data, angular velocity data, and magnetometer data. These inertial data can be transmitted to the processor 1010, and the processor 1010 processes them by executing the posture prediction method in this exemplary embodiment to obtain a posture prediction result.

[0212] In addition, the mobile terminal 1100 may also include other components such as a display screen, an audio module, and a camera module, which are not limited in the present disclosure.

[0213] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the exemplary embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0214] It will be appreciated by those skilled in the art that various aspects of the present disclosure may be implemented as a system, method or program product. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which may be collectively referred to herein as a "circuit", "module" or "system". Those skilled in the art will readily think of other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any variations, uses or adaptive changes of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and implementation are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0215] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A posture prediction method, characterized in that: include: Acquiring first inertial data; Processing the first inertial data using a pre-trained posture prediction model to obtain first posture data; determining second inertial data according to the first inertial data and the first posture data; Processing the second inertial data using a pre-trained position prediction model to obtain position data; Processing the second inertial data using a pre-trained position prediction model to obtain position data includes: processing the second inertial data at time t using a pre-trained position prediction model to obtain the position data at time t; Alternatively, a pre-trained position prediction model is used to process a sequence of second inertial data from time i to time t to obtain position data at any one or more of time t, time i, time i+1, ..., time t-1; the position data at time t represents the relative displacement at time t relative to time t-1.

2. The method according to claim 1, characterized in that The determining the second inertial data according to the first inertial data and the first posture data comprises: Performing Kalman filtering based on the first inertial data and the first posture data; The second inertial data is determined according to a result of Kalman filtering.

3. The method according to claim 2, characterized in that The result of the Kalman filtering includes second posture data; and performing the Kalman filtering based on the first inertial data and the posture data includes: The first inertial data is used as state data, the first posture data is used as observation data, and second posture data is obtained through Kalman filtering.

4. The method according to claim 3, characterized in that The method further comprises: When the first inertial data is processed using the pre-trained posture prediction model, the uncertainty of the first posture data is also obtained; The method of using the first inertial data as state data and the first posture data as observation data and obtaining second posture data through Kalman filtering includes: Determining an observation data covariance matrix according to the uncertainty of the first posture data; The first inertial data is used as state data, the first posture data is used as observation data, the observation data covariance matrix is ​​substituted and extended Kalman filtering is performed to obtain the second posture data.

5. The method according to claim 3, characterized in that: The determining the second inertial data according to the result of Kalman filtering includes: The first inertial data is converted into a preset coordinate system using the second posture data to obtain the second inertial data.

6. The method according to claim 1, characterized in that The inertial data includes at least one of acceleration data, angular velocity data, and magnetometer data; The step of processing the first inertial data using a pre-trained posture prediction model to obtain first posture data includes: Processing the first acceleration data, the first angular velocity data, and the first magnetometer data using a pre-trained posture prediction model to obtain the first posture data; The determining the second inertial data according to the first inertial data and the first posture data comprises: Second acceleration data and second angular velocity data are determined according to the first acceleration data, the first angular velocity data and the first posture data.

7. The method according to claim 1, characterized in that The method further comprises: Acquire first inertial sample data and position annotation data corresponding to the first inertial sample data; Inputting the first inertial sample data into a posture prediction model to obtain first posture sample data; Determining second inertia sample data according to the first inertia sample data and the first posture sample data; Inputting the second inertial sample data into a position prediction model to obtain position sample data; Based on the position sample data and the position annotation data, the parameters of the position prediction model and the posture prediction model are updated.

8. A posture prediction device, characterized in that: include: A data acquisition module, configured to acquire first inertial data; a posture prediction module, configured to process the first inertial data using a pre-trained posture prediction model to obtain first posture data; a data optimization module, configured to determine second inertial data according to the first inertial data and the first posture data; The position prediction module is configured to process the second inertial data using a pre-trained position prediction model to obtain position data; the processing of the second inertial data using a pre-trained position prediction model to obtain position data is configured to: process the second inertial data at time t using a pre-trained position prediction model to obtain the position data at time t; or, process a sequence of second inertial data from time i to time t using a pre-trained position prediction model to obtain position data at any one or more of time t, time i, time i+1, ..., time t-1; the position data at time t represents the relative displacement of time t relative to time t-1.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 7 by executing the executable instructions.

Images