Pose determination method and device and electronic equipment

Abstract

The embodiment of the invention discloses a pose determination method and device and electronic equipment. The method comprises the following steps: performing pre-integration on inertial data, and respectively inputting pre-integration results obtained by pre-integration into a plurality of filters so as to obtain poses respectively output by the plurality of filters; based on the inertial data and a target neural network model, obtaining probabilities of the electronic device corresponding to the plurality of motion modes; obtaining candidate motion modes according to the probabilities of the plurality of motion modes; according to the candidate motion mode and an evaluation result, determining an optimal filter, the evaluation result being a result of evaluating the plurality of filters based on the candidate motion mode; and determining the pose output by the optimal filter as a target pose. In this way, the accuracy of pose determination can be improved, and then the accuracy of estimating the pedestrian position is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and more specifically, to a posture determination method, device and electronic device. Background Art

[0002] Pedestrian Dead Reckoning (PDR) is an autonomous positioning technology that estimates the position of pedestrians using inertial sensors such as accelerometers, gyroscopes, and magnetometers. However, the accuracy of related PDR methods needs to be improved. Summary of the invention

[0003] In view of the above problems, the present application proposes a posture determination method, device and electronic device to improve the above problems.

[0004] In a first aspect, the present application provides a posture determination method for an electronic device, the method comprising: pre-integrating inertial data, and inputting the pre-integration results obtained by the pre-integration into multiple filters respectively to obtain the posture output by each of the multiple filters; based on the inertial data and the target neural network model, obtaining the probability that the electronic device corresponds to multiple motion modes; obtaining candidate motion modes according to the probabilities of the multiple motion modes; determining an optimal filter according to the candidate motion modes and an evaluation result, wherein the evaluation result is a result of evaluating the multiple filters based on the candidate motion modes; and determining the posture output by the optimal filter as the target posture.

[0005] In a second aspect, the present application provides a posture determination device that runs on an electronic device, the device comprising: a pre-integration unit, used to pre-integrate inertial data, and input the pre-integration results obtained by the pre-integration into multiple filters respectively to obtain the posture output by each of the multiple filters; a motion mode acquisition unit, used to obtain the probability that the electronic device corresponds to multiple motion modes based on the inertial data and the target neural network model; a candidate motion mode acquisition unit, used to obtain candidate motion modes according to the probabilities of the multiple motion modes; a filter determination unit, used to determine the optimal filter based on the candidate motion modes and an evaluation result, wherein the evaluation result is the result of evaluating the multiple filters based on the candidate motion modes; a posture determination unit, used to determine the posture output by the optimal filter as the target posture.

[0006] In a third aspect, the present application provides an electronic device, which includes at least a processor and a memory; one or more programs are stored in the memory and configured to be executed by the processor to implement the above method.

[0007] In a fourth aspect, the present application provides a computer-readable storage medium having program code stored therein, wherein the above method is executed when the program code is executed by a processor.

[0008] The present application provides a method, device and electronic device for determining a posture, in which the inertial data can be pre-integrated first, and the pre-integration results obtained by the pre-integration are respectively input into multiple filters to obtain the postures output by each of the multiple filters, and then, based on the inertial data and the target neural network model, the probability of the electronic device corresponding to multiple motion modes is obtained, and then the candidate motion mode is obtained, so that the optimal filter can be determined according to the candidate motion mode and the evaluation result, so as to determine the posture output by the optimal filter as the target posture. Thus, through the above-mentioned method, under the condition of introducing the idea of ​​multi-hypothesis tracking, the probabilities of multiple motion modes can be obtained to estimate the state of multiple possible motion modes respectively, and then the correctness of each filter can be comprehensively judged through the candidate motion mode determined based on the probability of multiple motion modes, and the posture corresponding to the output of the filter with the highest degree of compliance (optimal filter) is used as the target posture, thereby improving the accuracy of posture determination, and then improving the accuracy of estimating the position of the pedestrian. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0010] Figure 1 A schematic diagram showing an application scenario of the posture determination method proposed in an embodiment of the present application;

[0011] Figure 2 A schematic diagram showing another application scenario of the posture determination method proposed in an embodiment of the present application;

[0012] Figure 3 A flow chart of a method for determining a posture proposed in an embodiment of the present application is shown;

[0013] Figure 4 A flow chart of a method for determining a posture according to another embodiment of the present application is shown;

[0014] Figure 5 A flow chart of a method for determining a posture according to another embodiment of the present application is shown;

[0015] Figure 6A structural block diagram of another posture determination device proposed in an embodiment of the present application is shown;

[0016] Figure 7 A structural block diagram of an electronic device for executing a posture determination method according to an embodiment of the present application is shown;

[0017] Figure 8 It is a storage unit of an embodiment of the present application for storing or carrying program code for implementing the posture determination method according to an embodiment of the present application. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0019] Pedestrian Dead Reckoning (PDR) is an advanced autonomous positioning technology that can continuously track and estimate the position, speed and direction of pedestrians without the need for external signal sources (such as GPS). This technology relies on a series of precision inertial sensors, mainly including accelerometers, gyroscopes and magnetometers, which work together to capture the motion characteristics of pedestrians.

[0020] However, although PDR technology has demonstrated its unique advantages in many scenarios, such as indoor positioning, urban canyons and other areas where GPS signals are restricted, its accuracy still faces many challenges and needs to be further improved.

[0021] Therefore, after the inventor discovered the above problems in the study, he proposed a posture determination method, device and electronic device in the present application that can improve the above problems. In this method, the inertial data can be pre-integrated first, and the pre-integration results obtained by the pre-integration are respectively input into multiple filters to obtain the postures output by each of the multiple filters, and then, based on the inertial data and the target neural network model, the probability of the electronic device corresponding to multiple motion modes is obtained, and then the candidate motion mode is obtained, so as to determine the optimal filter according to the candidate motion mode and the evaluation result, so as to determine the posture output by the optimal filter as the target posture. Thus, through the above method, under the condition of introducing the idea of ​​multi-hypothesis tracking, by obtaining the probability of multiple motion modes, the state estimation of multiple possible motion modes can be performed respectively, and then the correctness of each filter is comprehensively judged by the candidate motion mode determined based on the probability of multiple motion modes, and the posture corresponding to the output of the filter with the highest degree of compliance (optimal filter) is used as the target posture, thereby improving the accuracy of posture determination, and then improving the accuracy of estimating the position of the pedestrian.

[0022] Before further describing the embodiments of the present application in detail, an application environment involved in the embodiments of the present application is introduced.

[0023] The following first introduces the application scenarios involved in the embodiments of the present application.

[0024] In an embodiment of the present application, the provided posture determination method can be performed by an electronic device. In this manner, all steps in the posture determination method provided in the embodiment of the present application can be performed by the electronic device. For example, Figure 1 As shown, when all steps in the posture determination method provided in the embodiment of the present application can be executed by an electronic device, all steps can be executed by the processor of the electronic device 100.

[0025] Furthermore, the posture determination method provided in the embodiment of the present application can also be executed by a server. Correspondingly, in this mode executed by the server, the server can start executing the steps in the posture determination method provided in the embodiment of the present application in response to a trigger instruction. Among them, the trigger instruction can be sent by an electronic device used by the user, or it can be triggered locally by the server in response to some automation events.

[0026] In addition, the posture determination method provided in the embodiment of the present application can also be performed by the electronic device and the server in collaboration. In this way, some steps in the posture determination method provided in the embodiment of the present application are performed by the electronic device, while other steps are performed by the server. For example, Figure 2As shown, the electronic device 100 can perform the posture determination method including: pre-integrating the inertial data, and inputting the pre-integration results obtained by the pre-integration into multiple filters respectively to obtain the postures output by each of the multiple filters, and then transmitting the postures output by each of the multiple filters to the server 200, and then the server 200 performs the subsequent steps. Optionally, after obtaining the target posture, the server 200 can locate the electronic device 100 based on the determined target posture. Alternatively, the target posture can also be returned to the electronic device 100, and the electronic device 100 can perform positioning.

[0027] It should be noted that in this method of collaborative execution by the electronic device and the server, the steps respectively executed by the electronic device and the server are not limited to the methods introduced in the above examples. In actual applications, the steps respectively executed by the electronic device and the server can be dynamically adjusted according to actual conditions.

[0028] It should be noted that the electronic device 100 is Figure 1 and Figure 2 In addition to the smartphone shown in , it can also be a tablet computer, a smart watch, an intelligent voice assistant and other devices. Server 200 can be an independent physical server, or it can be a server cluster or a distributed system composed of multiple physical servers. Among them, in the case where the posture determination method provided in the embodiment of the present application is executed by a server cluster or a distributed system composed of multiple physical servers, different steps in the posture determination method can be executed by different physical servers respectively, or can be executed in a distributed manner by a server built based on a distributed system.

[0029] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0030] See also Figure 3 , an embodiment of the present application provides a method for determining a posture, the method comprising:

[0031] S110: Pre-integrate the inertial data, and input the pre-integration results obtained by the pre-integration into a plurality of filters respectively, so as to obtain the postures output by the plurality of filters respectively.

[0032] In an embodiment of the present application, the inertial data is mainly derived from inertial sensors (such as accelerometers and gyroscopes, etc.) in the electronic device used as the positioning target. Among them, the inertial sensor can continuously collect inertial data (for example, acceleration and angular velocity data) at a certain frequency (such as hundreds to thousands of hertz). In addition to including the dynamic motion information of the electronic device, these inertial data may also be accompanied by noise and deviation. Therefore, before pre-integrating the inertial data, these inertial data can also be filtered (such as low-pass filtering) and bias corrected (by calibration or online estimation) to reduce the impact of noise and deviation on subsequent processing.

[0033] The main purpose of pre-integration is to integrate the IMU (Inertial Measurement Unit) measurement value (inertial data) between two key frames into the constraint of relative motion, rather than directly integrating it into the world coordinate system. The advantage of this is that when the back-end optimization algorithm adjusts the pose (position and attitude) of the key frame, it does not need to re-integrate the IMU measurement value, which greatly reduces the amount of calculation.

[0034] Among them, pre-integration of inertial data may include rotation pre-integration, velocity pre-integration and position pre-integration. Among them, rotation pre-integration can be understood as representing the relative rotation of the IMU between two time points by quaternion or rotation matrix (or, it can be understood as posture change). This usually involves the integration of gyroscope measurements and considering the gyroscope bias. Velocity pre-integration can be understood as integrating the change in velocity based on the accelerometer measurement and the estimated rotation. Similarly, the accelerometer bias needs to be considered. Position pre-integration can be understood as further integrating the change in position through the velocity and the estimated rotation. In this way, the pre-integration result can include the data obtained after rotation pre-integration, velocity pre-integration and position pre-integration. As a way, in an embodiment of the present application, pre-integration of inertial data can be performed based on the following formula, which is:

[0035] ΔR i =Пexp SO3 ((ω t+i -b g )Δt)

[0036]

[0037]

[0038] As shown in the above formula, ΔR i represents the change in posture, Δv i represents the speed change, Δt i It represents the change of position.t+i , a t+i Indicates the angular velocity and acceleration measured by the IMU. g , b a represents the gyroscope and accelerometer bias; Δt represents the time difference between epochs.

[0039] In an embodiment of the present application, the posture outputted by each of the multiple filters may be different. For example, the data obtained by pre-integration is then input into multiple filters, each of which may be based on different assumptions, models or optimization objectives. Multiple filters can perform posture prediction based on the input data, respectively, and output the predicted posture. Among them, each filter independently calculates the posture estimate of the system (electronic device) based on the input pre-integration result and possible external observations. These estimates may vary depending on the filter. Furthermore, the difference in filter output also reflects the ability of each algorithm (the way the filter obtains the output posture) to handle uncertainty, noise and model errors.

[0040] In the embodiment of the present application, after obtaining the pre-integration result, in addition to enabling each filter to perform state prediction (predicted posture) based on the pre-integration result, the error propagation of each filter can also be calculated based on the pre-integration result. Exemplarily, the formula for state prediction can be shown as follows:

[0041]

[0042]

[0043]

[0044] Among them, in the formula, represents the attitude, velocity and position relative to the navigation coordinate system; g represents the gravitational acceleration vector; ΔT i Indicates the total time length of the pre-integration interval.

[0045] Furthermore, the state transition can also be calculated for each filter covariance matrix based on the same principle.

[0046] S120: Based on the inertial data and the target neural network model, obtain the probability that the electronic device corresponds to multiple motion modes.

[0047] After obtaining the inertial data, the inertial data can be input into the target neural network model so as to output the probabilities of multiple motion modes through the target neural network model. The multiple motion modes represent the motion modes that the electronic device may currently be in, and correspondingly, the probability of each motion mode represents the probability of the electronic device being in each motion mode. In other words, for a motion mode with a greater probability, the probability that the electronic device is in the motion mode is greater.

[0048] It should be noted that the target neural network model is a pre-trained neural network model. Before model training, training data can be obtained first.

[0049] First, data collection can be performed. During the data collection process, a device equipped with an inertial measurement unit (for example, a smartphone, a wearable device, or a professional sports tracker, etc.) can be selected first, and then data can be collected in a variety of different sports scenarios through the device equipped with the inertial measurement unit. The various sports scenarios include but are not limited to walking, running, cycling, standing still, going up and down stairs, driving, etc., to ensure that each sports mode has a sufficient sample size to improve the generalization ability of the model.

[0050] For the collected data, a data preprocessing process can be performed first. In the preprocessing process, data filtering can be performed first. Among them, data filtering includes using low-pass filters, Kalman filters and other technologies to remove high-frequency noise in the data and improve data quality. For the data after data filtering, feature extraction can be performed again. In the process of feature extraction, useful features can be extracted from the data after data filtering, such as the mean, variance, spectral features of acceleration, and the angular velocity change rate of the gyroscope. These features should be able to reflect the characteristics of different motion modes. For the extracted features, standardization or normalization can be performed to eliminate the dimensional differences between different features and improve the efficiency and stability of model training. After the aforementioned data preprocessing process, training data can be obtained.

[0051] In an embodiment of the present application, the neural network model to be trained may be a convolutional neural network, a recurrent neural network, and variants thereof, as well as a hybrid model.

[0052] The training data obtained above can be divided into a training set, a validation set and a test set. The training set data is used to train the neural network model to be trained, and the model parameters are adjusted through the back propagation algorithm to minimize the prediction error. At the same time, the validation set is used to monitor the training process to prevent overfitting.

[0053] After the above operations, the target neural network model can be trained.

[0054] It should be noted that in the embodiment of the present application, the input of the target neural network model can be not only the inertial data itself, but also the pre-integration result obtained after pre-integration of the inertial data. In this case, the target neural network model can obtain the probability that the electronic device corresponds to multiple motion modes based on the pre-integration result. It should be noted that when the input of the target neural network model can be the pre-integration result, in the process of training the model to be trained, the acquired training data also needs to be pre-integrated before being input into the neural network model to be trained.

[0055] In an embodiment of the present application, multiple motion modes may include static scenes, uniform linear motion, acceleration / deceleration motion, rotational motion, complex curve motion, etc. For example, the multiple motion modes determined may include uniform motion, acceleration motion, and other modes. Among them, when the electronic device is in the uniform motion mode, the user may place the electronic device in a trouser pocket or bag. When the electronic device is in the acceleration motion mode, the user may hold the electronic device in his hand (the electronic device will swing with the hand) and walk.

[0056] S130: Obtain candidate motion patterns according to the probabilities of the multiple motion patterns.

[0057] After the probabilities of the multiple motion patterns are obtained, the candidate motion patterns can be determined based on the magnitude relationship of the probabilities of the multiple motion patterns. The candidate motion patterns can be understood as motion patterns used to evaluate the multiple filters.

[0058] As a method, the motion mode with the highest corresponding probability among the multiple motion modes can be used as a candidate motion mode, and the motion mode that meets the corresponding probability condition among the multiple motion modes can be used as a candidate motion mode. Optionally, the probability condition can be shown as the following formula:

[0059]

[0060] Among them, p i represents the probability of the motion mode to be determined, p max represents the probability of the motion mode with the highest probability, λ ratio and λ th Indicates the decision threshold.

[0061] Exemplarily, the multiple motion modes output by the target neural network model include motion mode S1, motion mode S2 and motion mode S3. Among them, the probability of motion mode S1 is P1, the probability of motion mode S2 is P2, and the probability of motion mode S3 is P3. Among them, if P1 is the largest, motion mode S1 can be directly used as a candidate motion mode. If the probability P2 of motion mode S2 satisfies the above-mentioned probability condition, and the probability P3 of motion mode S3 does not satisfy the above-mentioned probability condition, motion mode S2 can also be determined as a candidate motion mode, and the obtained candidate motion modes can include motion mode S1 and motion mode S2.

[0062] Exemplarily, the multiple motion modes output by the target neural network model include motion mode S1, motion mode S2 and motion mode S3, wherein the probability of motion mode S1 is P4, the probability of motion mode S2 is P5, and the probability of motion mode S3 is P6, wherein if P4 is the largest, motion mode S1 can be directly used as a candidate motion mode. If the probability P5 of motion mode S2 does not meet the above probability conditions, and the probability P6 of motion mode S3 does not meet the above probability conditions, then the candidate motion modes finally determined only include motion mode S1.

[0063] S140: Determine an optimal filter according to the candidate motion pattern and an evaluation result, wherein the evaluation result is a result of evaluating the multiple filters based on the candidate motion pattern.

[0064] As can be seen from the above-mentioned process of determining the candidate motion mode, the candidate motion mode is the motion mode that the electronic device is currently in with a greater probability. Then, multiple filters are evaluated through the candidate motion mode, and the evaluation results obtained can characterize the degree of matching between the multiple filters and the candidate motion mode, and then the optimal filter determined based on the evaluation result can be understood as the filter that is currently most suitable for determining the posture of the electronic device.

[0065] S150: Determine the posture output by the optimal filter as the target posture.

[0066] After the target posture is obtained, the electronic device can be positioned based on the target posture to determine the position of the electronic device, and then determine the position of the user carrying the electronic device.

[0067] The present embodiment provides a method for determining a posture, in which the inertial data can be pre-integrated first, and the pre-integration results obtained by the pre-integration are respectively input into multiple filters to obtain the postures output by each of the multiple filters, and then, based on the inertial data and the target neural network model, the probability of the electronic device corresponding to multiple motion modes is obtained, and then the candidate motion mode is obtained, so that the optimal filter can be determined according to the candidate motion mode and the evaluation result, so as to determine the posture output by the optimal filter as the target posture. Thus, through the above method, under the condition of introducing the idea of ​​multi-hypothesis tracking, by obtaining the probabilities of multiple motion modes, the state estimation of multiple possible motion modes can be performed respectively, and then the correctness of each filter is comprehensively judged through the candidate motion mode determined based on the probability of multiple motion modes, and the posture corresponding to the output of the filter with the highest degree of compliance (optimal filter) is used as the target posture, thereby improving the accuracy of posture determination, and then improving the accuracy of pedestrian position estimation.

[0068] See also Figure 4 , an embodiment of the present application provides a method for determining a posture, the method comprising:

[0069] S210: Pre-integrate the inertial data, and input the pre-integration results obtained by the pre-integration into multiple filters respectively to obtain the postures output by each of the multiple filters.

[0070] S220: Based on the inertial data and the target neural network model, obtain the probability that the electronic device corresponds to multiple motion modes.

[0071] S230: Obtain candidate motion patterns according to the probabilities of the multiple motion patterns.

[0072] It should be noted that the motion mode of the electronic device may be different in different situations. Correspondingly, the probabilities of the multiple motion modes output by each of the multiple filters may also be different in different situations. Furthermore, the number of determined candidate motion modes may also be different in different situations. In an embodiment of the present application, when the number of determined candidate motion modes is different, the corresponding method for evaluating the multiple filters may also be different, and the method for determining the optimal filter may also be different.

[0073] S240: If the number of the currently determined candidate motion mode is one, the multiple filters are evaluated based on the currently determined candidate motion mode, and an optimal filter is determined from the multiple filters according to the evaluation result.

[0074] As a method, evaluating the multiple filters based on the currently determined candidate motion mode, and determining the optimal filter from the multiple filters according to the evaluation results may include: obtaining the observation equation of the corresponding motion law constraint based on the candidate motion mode, and calculating the observed value of the motion law constraint and the covariance corresponding to the observed value based on the observation equation and the filter; obtaining the matching degree of each of the multiple filters based on the observation equation, the observed value and the covariance corresponding to the observed value; and determining the optimal filter based on the matching degree. Exemplarily, when there is only one candidate motion mode determined, the multiple filters can be evaluated based on the following formula, which is:

[0075] η=(h(X t )-y t ) T R -1 (h(X t )-y t )

[0076] In the above formula, h(·) represents the observation equation constrained by the motion law of the carrier calculated by the current motion mode, and y t Represents the observed value corresponding to the motion law constraint, and R represents the covariance corresponding to the observed value. η represents the degree of conformity between the system state and the constraint information, and then, the filter with the smallest η can be selected as the optimal sub-filter.

[0077] Among them, the typical motion law constraint is the average velocity constraint, and the observation equation can be:

[0078]

[0079] In the above formula, Represents the velocity in the navigation system. t is the pedestrian movement speed estimated based on the step length.

[0080] S250: If there are multiple candidate motion modes currently determined, constraint information is calculated based on the motion modes marked by the multiple filters, so that the multiple filters are measured and updated based on the constraint information to obtain information for updating the multiple filters; based on the information for updating the multiple filters and the motion modes marked by the multiple filters, scores of the multiple filters are calculated respectively, so as to select the filter with the highest score as the optimal filter.

[0081] Optionally, the constraint information (such as speed, displacement, posture, trajectory, etc.) based on the carrier motion law can be calculated according to the motion mode marked by the filter. Then, the constraint information is used to measure and update each filter separately. In addition, the new information of the current round of measurement update of each filter can be recorded, where the calculation formula for the update can be:

[0082] γ=(h(X t )-y t ) T (HPH T +R) -1 (h(X t )-y t )

[0083] As an approach, the method of calculating the scores of each of the multiple filters based on the information updated by the multiple filters and the motion patterns marked by the multiple filters may include: calculating a first score based on the information updated by the multiple filters; calculating a second score based on the probability of the motion patterns marked by the multiple filters; multiplying the first score by a first weight to obtain a first weighted score; multiplying the second score by a second weight to obtain a second weighted score; and adding the first weighted score to the second weighted score to obtain the scores of each of the multiple filters.

[0084] Exemplarily, the formula for calculating the scores of the multiple filters can be as follows, where the score of the i-th filter can be expressed as:

[0085] κ i =ω γ γ i +ω p ·p i

[0086] In the above formula, ω γ and ω p Represents the corresponding weight. Among them, ω γ It can be understood as the first weight mentioned above, ω p It can be understood as the second weight mentioned above. i Characterizes the updated information of the i-th filter. i Characterize the labeled motion pattern of the i-th filter.

[0087] In the embodiment of the present application, the motion mode marked by the filter can be understood as a newly added candidate motion mode among multiple candidate operation modes, or can be understood as a motion mode with the highest probability among the candidate motion modes. Alternatively, it can be understood as a motion mode with the highest matching degree with the filter among the candidate motion modes.

[0088] Optionally, the first weight and the second weight are updated based on an optimal filter and a minimized cost function.

[0089] Among them, the formula for minimizing the cost function is:

[0090]

[0091] According to the minimization cost function, ω can be obtained γ and ω p Corresponding gradient Δ γ and Δ p , updated according to the following formula

[0092] ω γ =ω γ +αΔ γ

[0093] ω p =ω p +αΔ p

[0094] Where α represents the step size factor, which can be configured in advance.

[0095] S260: Determine the posture output by the optimal filter as the target posture.

[0096] This embodiment provides a method for determining a posture, in which the correctness of each filter is comprehensively judged through a candidate motion mode determined based on the probability of multiple motion modes, and the posture corresponding to the output of the filter with the highest degree of compliance (optimal filter) is used as the target posture, thereby improving the accuracy of posture determination, and then improving the accuracy of estimating the position of the pedestrian. In addition, in this embodiment, according to the number of determined candidate motion modes, there can be different ways to determine the optimal filter, thereby further improving the flexibility of the solution.

[0097] like Figure 5 As shown, Figure 5 A process of a posture determination method provided in an embodiment of the present application is shown.

[0098] in, Figure 5 The ACC (Acceleration) shown in the figure represents the acceleration data collected by the accelerometer of the electronic device. The GYR (mechanical gyro) represents the gyroscope data collected by the gyroscope of the electronic device. Among them, ACC and GYR together constitute the inertial data introduced in the previous content. Figure 5 The candidate modes shown in actually refer to candidate motion modes.

[0099] It should be noted that the electronic device will continuously judge its own posture and then continuously position itself. That is to say, the electronic device will acquire the posture multiple times in the entire positioning process (from the beginning of positioning to the end of positioning). Among them, in the process of acquiring the posture, the inertial data acquired in the acquisition process can be input into the target neural network model to obtain the probabilities of multiple motion modes output by the target neural network model. Among them, the multiple motion modes output by the target neural network model each time can be the same, but the corresponding probabilities for multiple motion modes each time may be different.

[0100] Furthermore, the candidate motion modes obtained each time may be different. If the candidate motion modes obtained include multiple motion modes, but there are multiple filters currently maintained (filters in the filter library), the filters other than the optimal filter finally determined may be deleted.

[0101] Correspondingly, in the case of filter deletion, there may be multiple candidate motion modes, but only one filter is maintained (for example, other filters may have been deleted in the process of acquiring postures in the previous few times). In order to deal with the above situation, in the current acquisition process, after acquiring the candidate motion mode, it is possible to first detect whether the candidate motion mode acquired this time is the same as the candidate motion mode acquired in the last posture acquisition process. If they are the same, the optimal filter is determined directly based on the currently maintained filter. If they are not the same (that is, there are newly added candidate motion modes), the optimal filter determined last time can be added to the currently maintained filter, and then, based on the currently maintained filter (to which the optimal filter determined last time has been added), the optimal filter is determined.

[0102] See also Figure 6 , a posture determination device 300 provided in an embodiment of the present application runs on an electronic device, and the device 300 includes:

[0103] The pre-integration unit 310 is used to pre-integrate the inertial data, and input the pre-integration results obtained by the pre-integration into multiple filters respectively to obtain the postures output by each of the multiple filters.

[0104] The motion mode acquisition unit 320 is used to obtain the probability that the electronic device corresponds to multiple motion modes based on the inertial data and the target neural network model.

[0105] As a method, the motion mode acquisition unit 320 is specifically used to take the motion mode with the largest corresponding probability among the multiple motion modes as the candidate motion mode, and take the motion mode that meets the corresponding probability condition among the multiple motion modes as the candidate motion mode.

[0106] The candidate motion mode acquisition unit 330 is configured to obtain a candidate motion mode according to the probabilities of the multiple motion modes.

[0107] The filter determination unit 340 is configured to determine an optimal filter according to the candidate motion patterns and an evaluation result, wherein the evaluation result is a result of evaluating the multiple filters based on the candidate motion patterns.

[0108] The posture determination unit 350 is used to determine the posture output by the optimal filter as the target posture.

[0109] As a method, the filter determination unit 340 is specifically configured to evaluate the multiple filters based on the currently determined candidate motion mode if the number of the currently determined candidate motion mode is one, and determine the optimal filter from the multiple filters according to the evaluation result.

[0110] Optionally, the filter determination unit 340 is specifically used to obtain the observation equation of the corresponding motion law constraint based on the candidate motion mode, and based on the observation equation and the filter, calculate the observation value of the motion law constraint and the covariance corresponding to the observation value; based on the observation equation, the observation value and the covariance corresponding to the observation value, obtain the matching degree of each of the multiple filters; based on the matching degree, determine the optimal filter.

[0111] As a method, the filter determination unit 340 is specifically used to calculate constraint information based on the motion modes marked by the multiple filters if there are multiple candidate motion modes currently determined, so as to measure and update the multiple filters based on the constraint information to obtain information on the updates of the multiple filters; and calculate the scores of the multiple filters based on the information on the updates of the multiple filters and the motion modes marked by the multiple filters, so as to select the filter with the highest score as the optimal filter.

[0112] Optionally, the filter determination unit 340 is specifically used to calculate a first score based on information updated by multiple filters; calculate a second score based on the probability of motion patterns marked by multiple filters; multiply the first score by a first weight to obtain a first weighted score; multiply the second score by a second weight to obtain a second weighted score; and add the first weighted score to the second weighted score to obtain scores for each of the multiple filters.

[0113] Optionally, the filter determination unit 340 is further configured to update the first weight and the second weight based on an optimal filter and a minimized cost function.

[0114] The present embodiment provides a posture determination device, through which the above-mentioned device can make state estimation for multiple possible motion modes respectively by obtaining the probabilities of multiple motion modes under the condition of introducing the idea of ​​multi-hypothesis tracking, and then make a comprehensive judgment on the correctness of each filter through the candidate motion mode determined based on the probabilities of multiple motion modes, and take the posture outputted by the filter with the highest degree of compliance (optimal filter) as the target posture, thereby improving the accuracy of posture determination and further improving the accuracy of pedestrian position estimation.

[0115] It should be noted that the device embodiment in the present application corresponds to the aforementioned method embodiment. The specific principles in the device embodiment can be found in the contents of the aforementioned method embodiment and will not be repeated here.

[0116] The following will be combined Figure 7 An electronic device provided by the present application is described.

[0117] See also Figure 7 Based on the above-mentioned posture determination method and device, the embodiment of the present application also provides another electronic device 100 that can execute the above-mentioned posture determination method. The electronic device 100 includes one or more (only one is shown in the figure) processors 102, a memory 104 and a network module 106 coupled to each other. Among them, the memory 104 stores a program that can execute the content of the above-mentioned embodiment, and the processor 102 can execute the program stored in the memory 104.

[0118] Among them, the processor 102 may include one or more processing cores. The processor 102 uses various interfaces and lines to connect various parts of the entire electronic device 100, and executes various functions and processes data of the electronic device 100 by running or executing instructions, programs, code sets or instruction sets stored in the memory 104, and calling data stored in the memory 104. Optionally, the processor 102 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 102 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 102, but may be implemented separately through a communication chip.

[0119] The memory 104 may include a random access memory (RAM) or a read-only memory (ROM). The memory 104 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 104 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The data storage area may also store data (such as a phone book, audio and video data, chat record data) created by the terminal 100 during use.

[0120] The network module 106 is used to receive and send electromagnetic waves, realize the mutual conversion between electromagnetic waves and electrical signals, so as to communicate with a communication network or other devices, such as communicating with an audio playback device. The network module 106 may include various existing circuit elements for performing these functions, such as antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, user identity modules (SIM) cards, memories, etc. The network module 106 may communicate with various networks such as the Internet, corporate intranets, wireless networks, or communicate with other devices via wireless networks. The above-mentioned wireless networks may include cellular telephone networks, wireless local area networks, or metropolitan area networks. For example, the network module 106 may exchange information with a base station.

[0121] Please refer to Figure 8 , which shows a structural block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable medium 800 stores program codes, which can be called by a processor to execute the method described in the above method embodiment.

[0122] The computer readable storage medium 800 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer readable storage medium 800 includes a non-transitory computer-readable storage medium. The computer readable storage medium 800 has storage space for program code 810 that performs any method step in the above method. These program codes can be read from or written to one or more computer program products. The program code 810 can be compressed, for example, in an appropriate form.

[0123] In summary, the present application provides a method, device and electronic device for determining a posture, in which the inertial data can be pre-integrated first, and the pre-integration results obtained by the pre-integration are respectively input into multiple filters to obtain the postures output by each of the multiple filters, and then, based on the inertial data and the target neural network model, the probability of the electronic device corresponding to multiple motion modes is obtained, and then the candidate motion mode is obtained, so as to determine the optimal filter according to the candidate motion mode and the evaluation result, so as to determine the posture output by the optimal filter as the target posture. Thus, through the above method, by introducing the idea of ​​multi-hypothesis tracking, by obtaining the probabilities of multiple motion modes, the state estimation of multiple possible motion modes can be performed respectively, and then the correctness of each filter is comprehensively judged through the candidate motion mode determined based on the probability of multiple motion modes, and the posture corresponding to the output of the filter with the highest degree of compliance (optimal filter) is used as the target posture, thereby improving the accuracy of posture determination, and then improving the accuracy of pedestrian position estimation.

[0124] Specifically, in the embodiment of the present application, in order to solve the problem of inaccurate judgment of complex motion patterns of pedestrians, the idea of ​​multi-hypothesis tracking is introduced to perform state estimation for multiple possible motion patterns. The correctness of each filter is comprehensively judged based on the degree of compliance with the motion constraint model, and the corresponding pose of the filter with the highest degree of compliance is output. At the same time, the motion mode judgment parameters are trained based on historical data to improve the applicability of the model.

[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for determining a posture, characterized in that: Applied to electronic equipment, the method comprises: Pre-integrating the inertial data, and inputting the pre-integration results obtained by the pre-integration into a plurality of filters respectively, so as to obtain the postures outputted by the plurality of filters respectively; Based on the inertial data and the target neural network model, obtaining probabilities that the electronic device corresponds to multiple motion modes; Obtaining a candidate motion mode according to the probabilities of the multiple motion modes; Determining an optimal filter according to the candidate motion mode and an evaluation result, wherein the evaluation result is a result of evaluating the multiple filters based on the candidate motion mode; The pose output by the optimal filter is determined as the target pose.

2. The method according to claim 1, characterized in that The step of determining an optimal filter according to the candidate motion modes and the evaluation results includes: If the number of the currently determined candidate motion mode is one, the multiple filters are evaluated based on the currently determined candidate motion mode, and an optimal filter is determined from the multiple filters according to the evaluation result.

3. The method according to claim 2, characterized in that The step of evaluating the plurality of filters based on the currently determined candidate motion mode and determining an optimal filter from the plurality of filters according to the evaluation result comprises: Based on the candidate motion mode, an observation equation of the corresponding motion law constraint is obtained, and based on the observation equation and the filter, an observation value of the motion law constraint and a covariance corresponding to the observation value are calculated; Based on the observation equation, the observation value and the covariance corresponding to the observation value, obtaining the matching degree of each of the multiple filters; Based on the degree of matching, an optimal filter is determined.

4. The method according to claim 1, characterized in that: The step of determining an optimal filter according to the candidate motion modes and the evaluation results includes: If there are multiple candidate motion modes currently determined, calculate constraint information according to the motion modes marked by the multiple filters, so as to measure and update the multiple filters based on the constraint information to obtain information for updating the multiple filters; According to the information about the updates of the multiple filters and the motion patterns marked by the multiple filters, scores of the multiple filters are calculated to select the filter with the highest score as the optimal filter.

5. The method according to claim 4, characterized in that The step of calculating scores of the plurality of filters according to the information updated by the plurality of filters and the motion modes marked by the plurality of filters comprises: Calculate a first score based on the updated information of the multiple filters; A second score is calculated based on the probability of the motion patterns marked by the plurality of filters; Multiplying the first score by a first weight to obtain a first weighted score; multiplying the second score by a second weight to obtain a second weighted score; The first weight score and the second weight score are added to obtain scores of each of the plurality of filters.

6. The method according to claim 5, characterized in that The method also includes: The first weight and the second weight are updated based on the optimal filter and the minimized cost function.

7. The method according to claim 1, characterized in that The obtaining of candidate motion modes according to the probabilities of the multiple motion modes includes: The motion pattern with the greatest corresponding probability among the multiple motion patterns is taken as a candidate motion pattern, and the motion pattern that meets the corresponding probability condition among the multiple motion patterns is taken as a candidate motion pattern.

8. A posture determination device, characterized in that: Running on an electronic device, the device comprises: A pre-integration unit, used to pre-integrate the inertial data, and input the pre-integration results obtained by the pre-integration into a plurality of filters respectively, so as to obtain the postures output by the plurality of filters respectively; A motion mode acquisition unit, configured to obtain, based on the inertial data and the target neural network model, the probability that the electronic device corresponds to a plurality of motion modes; a candidate motion mode acquisition unit, configured to obtain a candidate motion mode according to the probabilities of the plurality of motion modes; a filter determination unit, configured to determine an optimal filter according to the candidate motion mode and an evaluation result, wherein the evaluation result is a result of evaluating the multiple filters based on the candidate motion mode; The posture determination unit is used to determine the posture output by the optimal filter as the target posture.

9. An electronic device, characterized in that: The method comprises a processor and a memory; one or more programs are stored in the memory and configured to be executed by the processor to implement any one of the methods of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, wherein when the program code is executed by a processor, the method according to any one of claims 1 to 7 is executed.

Images