Adaptive pedestrian dead reckoning method and device based on gait recognition, electronic equipment, storage medium and program product

Abstract

The invention relates to a self-adaptive pedestrian dead reckoning method and device based on gait recognition, electronic equipment, a storage medium and a program product. The method comprises the steps of obtaining first waist motion feature data of a target person; processing the first waist motion feature data through a pre-trained long-short-term memory recurrent neural network to obtain a first gait prediction result of the target person; according to the first gait prediction result and the first waist motion feature data, step length estimation is carried out on the target person, and a step length estimation result of the target person is obtained; and updating the position information of the target person according to the initial position information of the target person, the course angle estimation result of the target person and the step length estimation result.

Description

Technical Field

[0001] The present disclosure relates to the field of indoor positioning technology, and in particular to a pedestrian dead reckoning method, a pedestrian dead reckoning device, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] Pedestrian Dead Reckoning (PDR) uses data from multiple sensors such as accelerometers, magnetometers, and gyroscopes to estimate the movement of pedestrians, and then combines the initial position information of pedestrians to calculate the real-time position of pedestrians through data fusion and filtering algorithms. The specific principles of traditional PDR involve four key steps: step detection, step length estimation, heading angle estimation, and position update. After the system detects the pedestrian's steps through the step detection algorithm, it estimates the step length based on the pedestrian's human kinematics and dynamic characteristics during the step, and estimates the pedestrian's current posture based on the gyroscope readings, and finally completes the real-time dead reckoning of the pedestrian's position.

[0003] Although the pedestrian dead reckoning technology is different from the traditional strapdown inertial navigation system (SINS) and will not cause the accelerometer error to directly accumulate into the system and cause system failure, due to the high nonlinearity of human motion, the error in estimating the stride length of a person at each gait becomes one of the main error sources of the pedestrian dead reckoning system and is also the biggest challenge faced by the pedestrian dead reckoning technology. Summary of the invention

[0004] The present disclosure provides a pedestrian dead reckoning technology solution.

[0005] According to one aspect of the present disclosure, a pedestrian dead reckoning method is provided, comprising:

[0006] Obtaining first waist motion characteristic data of the target person;

[0007] Processing the first waist movement feature data by a pre-trained long short-term memory recurrent neural network to obtain a first step prediction result of the target person;

[0008] According to the first step prediction result and the first waist movement characteristic data, the stride length of the target person is estimated to obtain a stride length estimation result of the target person;

[0009] The position information of the target person is updated according to the initial position information of the target person, the heading angle estimation result of the target person and the step length estimation result.

[0010] In a possible implementation, the step of obtaining the first waist motion characteristic data of the target person includes:

[0011] Collecting first waist motion data of the target person through a waist-mounted inertial measurement unit;

[0012] Based on the first waist motion data, first waist motion feature data of the target person is extracted.

[0013] In a possible implementation, the first waist motion data includes a first waist acceleration and a first waist angular velocity;

[0014] The extracting the first waist motion feature data of the target person based on the first waist motion data includes:

[0015] Performing continuous wavelet transform based on the first waist acceleration to obtain first step feature data of the target person;

[0016] Based on the first waist acceleration, the first waist angular velocity and the first step characteristic data, first waist movement characteristic data of the target person is obtained.

[0017] In a possible implementation, after collecting the first waist motion data of the target person by using the waist-mounted inertial measurement unit and before processing the first waist motion feature data by using the pre-trained long short-term memory recurrent neural network, the method further includes:

[0018] Based on a sliding window with a preset length and a preset overlap rate, data extraction is performed on the first waist motion data, and secondary sampling is performed at intervals of a preset number of groups.

[0019] In a possible implementation, before the first waist movement feature data is processed by the pre-trained long short-term memory recurrent neural network, the method further includes:

[0020] Acquire a training data set, wherein the training data set includes second waist motion data of a plurality of persons;

[0021] extracting second waist motion feature data based on the second waist motion data;

[0022] The long short-term memory recursive neural network is trained based on the second waist movement feature data.

[0023] In a possible implementation, the first gait prediction result includes the probability that the target person has multiple preset gaits;

[0024] The plurality of preset gaits include at least some of the following: standing, walking, walking backward, running, going up stairs, going down stairs, falling down, and jumping.

[0025] In a possible implementation, performing step length estimation on the target person according to the first step prediction result and the first waist movement feature data to obtain the step length estimation result of the target person includes:

[0026] Determining a target step length estimation model from at least two adaptive step length estimation models according to the first step prediction result;

[0027] The target stride length estimation model is used to estimate the stride length of the target person based on the first waist motion feature data to obtain a stride length estimation result of the target person.

[0028] In a possible implementation manner, before determining the target step length estimation model from at least two adaptive step length estimation models according to the first step prediction result, the method further includes:

[0029] Use an optical motion capture system and a waist-mounted inertial measurement unit to collect motion data of people at different gaits, and extract the kinematic and dynamic characteristics of people;

[0030] Based on the kinematic characteristics and the dynamic characteristics, an adaptive step length estimation model corresponding to different gaits is constructed.

[0031] In a possible implementation, determining a target step length estimation model from at least two adaptive step length estimation models according to the first step prediction result includes:

[0032] In response to the first step prediction result indicating that the current gait of the target person is clear, a target step length estimation model is determined from at least two adaptive step length estimation models according to the first step prediction result.

[0033] In a possible implementation, the at least two adaptive step length estimation models include at least some of the following types:

[0034] Statistical regression model, sports physiological model, direct integration model based on center of mass acceleration.

[0035] In a possible implementation, performing step length estimation on the target person according to the first step prediction result and the first waist movement feature data to obtain the step length estimation result of the target person includes:

[0036] In response to the first step prediction result indicating that the current gait of the target person is blurred, the step length of the target person is estimated based on the first waist motion feature data using a multi-gait universal step length estimation model to obtain a step length estimation result of the target person.

[0037] In a possible implementation, before estimating the stride length of the target person based on the first waist motion feature data using the multi-gait universal stride length estimation model, the method further includes:

[0038] Using an optical motion capture system and a waist-mounted inertial measurement unit to collect motion data of a person at different gaits, and extracting specified features related to step length based on the motion data of the person at different gaits;

[0039] Analyzing the correlation coefficient between the specified feature and the step length;

[0040] According to the correlation coefficient, a multi-gait universal stride length estimation model is constructed.

[0041] In a possible implementation, the specified features include at least part of the following:

[0042] Cadence, maximum value of waist acceleration modulus, minimum value of waist acceleration modulus, standard deviation of waist acceleration.

[0043] According to one aspect of the present disclosure, there is provided a pedestrian dead reckoning device, comprising:

[0044] An acquisition module, used for acquiring first waist motion characteristic data of a target person;

[0045] A processing module, used for processing the first waist movement feature data through a pre-trained long short-term memory recursive neural network to obtain a first step prediction result of the target person;

[0046] a step length estimation module, used to estimate the step length of the target person according to the first step prediction result and the first waist movement characteristic data, and obtain a step length estimation result of the target person;

[0047] An updating module is used to update the position information of the target person according to the initial position information of the target person, the heading angle estimation result of the target person and the step length estimation result.

[0048] In a possible implementation, the obtaining module is used to:

[0049] Collecting first waist motion data of the target person through a waist-mounted inertial measurement unit;

[0050] Based on the first waist motion data, first waist motion feature data of the target person is extracted.

[0051] In a possible implementation, the first waist motion data includes a first waist acceleration and a first waist angular velocity;

[0052] The acquisition module is used for:

[0053] Performing continuous wavelet transform based on the first waist acceleration to obtain first step feature data of the target person;

[0054] Based on the first waist acceleration, the first waist angular velocity and the first step characteristic data, first waist movement characteristic data of the target person is obtained.

[0055] In a possible implementation manner, the device further includes:

[0056] The first extraction module is used to extract the first waist motion data based on a sliding window with a preset length and a preset overlap rate, and perform secondary sampling at intervals of a preset number of groups.

[0057] In a possible implementation manner, the device further includes:

[0058] An acquisition module, used for acquiring a training data set, wherein the training data set includes second waist motion data of a plurality of persons;

[0059] A second extraction module, used for extracting second waist motion feature data based on the second waist motion data;

[0060] A training module is used to train the long short-term memory recursive neural network based on the second waist movement feature data.

[0061] In a possible implementation, the first gait prediction result includes the probability that the target person has multiple preset gaits;

[0062] The plurality of preset gaits include at least some of the following: standing, walking, walking backward, running, going up stairs, going down stairs, falling down, and jumping.

[0063] In a possible implementation, the step length estimation module is used to:

[0064] Determining a target step length estimation model from at least two adaptive step length estimation models according to the first step prediction result;

[0065] The target stride length estimation model is used to estimate the stride length of the target person based on the first waist motion feature data to obtain a stride length estimation result of the target person.

[0066] In a possible implementation manner, the device further includes:

[0067] The third extraction module is used to collect motion data of the personnel at different gaits by using an optical motion capture system and a waist-mounted inertial measurement unit, and to extract kinematic and dynamic characteristics of the personnel;

[0068] The first building module is used to build an adaptive step length estimation model corresponding to different gaits based on the kinematic characteristics and the dynamic characteristics.

[0069] In a possible implementation, the step length estimation module is used to:

[0070] In response to the first step prediction result indicating that the current gait of the target person is clear, a target step length estimation model is determined from at least two adaptive step length estimation models according to the first step prediction result.

[0071] In a possible implementation, the at least two adaptive step length estimation models include at least some of the following types:

[0072] Statistical regression model, sports physiological model, direct integration model based on center of mass acceleration.

[0073] In a possible implementation, the step length estimation module is used to:

[0074] In response to the first step prediction result indicating that the current gait of the target person is blurred, the step length of the target person is estimated based on the first waist motion feature data using a multi-gait universal step length estimation model to obtain a step length estimation result of the target person.

[0075] In a possible implementation manner, the device further includes:

[0076] a fourth extraction module, for collecting motion data of the person at different gaits using an optical motion capture system and a waist-mounted inertial measurement unit, and extracting specified features related to step length based on the motion data of the person at different gaits;

[0077] An analysis module, used for analyzing the correlation coefficient between the specified feature and the step length;

[0078] The second building module is used to build a multi-gait universal step length estimation model according to the correlation coefficient.

[0079] In a possible implementation, the specified features include at least part of the following:

[0080] Cadence, maximum value of waist acceleration modulus, minimum value of waist acceleration modulus, standard deviation of waist acceleration.

[0081] According to one aspect of the present disclosure, an electronic device is provided, comprising: one or more processors; a memory for storing executable instructions; wherein the one or more processors are configured to call the executable instructions stored in the memory to execute the above method.

[0082] According to one aspect of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor.

[0083] According to one aspect of the present disclosure, a computer program product is provided, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in an electronic device, a processor in the electronic device executes the above method.

[0084] In the disclosed embodiment, by obtaining the first waist motion characteristic data of the target person, the first waist motion characteristic data is processed by a pre-trained long short-term memory recurrent neural network to obtain the first step prediction result of the target person, and the step length of the target person is estimated according to the first step prediction result and the first waist motion characteristic data to obtain the step length estimation result of the target person, and the position information of the target person is updated according to the initial position information of the target person, the heading angle estimation result of the target person, and the step length estimation result, thereby realizing complex action pedestrian dead reckoning based on gait recognition of the person. In the disclosed embodiment, relying only on a single inertial measurement unit, it is possible to achieve high recognition rate estimation of multiple gaits, which can improve the accuracy of gait estimation.

[0085] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

[0086] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used to illustrate the technical solutions of the present disclosure together with the specification.

[0088] Figure 1 A flowchart of a pedestrian dead reckoning method provided by an embodiment of the present disclosure is shown.

[0089] Figure 2 A schematic diagram showing a sliding window in a pedestrian dead reckoning method provided by an embodiment of the present disclosure.

[0090] Figure 3 A schematic diagram showing a step detection process in a pedestrian dead reckoning method provided in an embodiment of the present disclosure.

[0091] Figure 4 A schematic diagram of the structure of a long short-term memory recursive neural network in a pedestrian dead reckoning method provided by an embodiment of the present disclosure is shown.

[0092] Figure 5 A calibration and analysis flow chart of the adaptive step length estimation method provided by an embodiment of the present disclosure is shown.

[0093] Figure 6 A flowchart of analysis and calibration of a multi-gait universal stride length estimation model provided by an embodiment of the present disclosure is shown.

[0094] Figure 7 A structural diagram of an adaptive pedestrian dead reckoning system based on gait recognition provided by an embodiment of the present disclosure is shown.

[0095] Figure 8 The figure shows the difference in positioning accuracy between the pedestrian dead reckoning system provided by the embodiment of the present disclosure and the traditional pedestrian dead reckoning system in a complex environment.

[0096] Fig. 9 A block diagram of a pedestrian dead reckoning device provided by an embodiment of the present disclosure is shown.

[0097] Fig.10 A block diagram of an electronic device 1900 provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0098] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0099] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0100] The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

[0101] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0102] To achieve pedestrian dead reckoning in complex environments, we must first determine the person's current gait and perform step length estimation and adaptation based on the current gait. The following methods are proposed in related technologies to achieve gait determination and step length estimation and adaptation:

[0103] For the task of completing the current gait recognition of a person based on an inertial measurement unit (IMU), if multiple IMUs are installed on multiple parts of the person to work together, the motion recognition rate based on the hidden Markov model algorithm can reach 92.13%; however, if only a single IMU is used to complete the gait recognition task, using methods such as the nearest neighbor algorithm, support vector machine classifier or decision tree classifier, the recognition rate can only reach about 85%.

[0104] Regarding the estimation method of personnel step length, the relevant technology has established the following three types of step length estimation models:

[0105] The first type of model is to establish a step length estimation model through the feature quantity related to the step length, and fit the statistical regression model of its parameters. Among this type, the model proposed by Shin et al. is the most classic. This model estimates the step length of a person based on the variance of the person's step frequency and waist acceleration. The expression is as follows:

[0106] SL=a·f+b·v+c

[0107] Among them, f is the pedestrian's step length, v is the variance of the waist acceleration modulus, and (a, b, c) are the parameters to be fitted, which need to be calibrated and set in advance. However, under different gaits, the correlation between features such as step frequency and step length will change, which leads to certain differences in the parameters to be fitted. Therefore, the model has poor robustness and can often only be applied to a single gait.

[0108] The second type of model is the motion physiological model method based on human motion characteristics. This type of model is based on the human motion model and estimates the stride length of a person through the dynamic characteristics of various parts of the human body. For example, the Weinberg model and Kim model based on the human inverted tree model are expressed as follows:

[0109] Weinberg Model:

[0110]

[0111] Kim Model:

[0112]

[0113] Among them, K1 and K2 are unknown coefficients related to the height and weight of the person, which need to be calibrated and set in advance. Because human motion is too complex, such models are often based on some heuristic assumptions, so they only perform well in individual gaits.

[0114] The third type of model is a direct integration model based on the quadratic integration of the pedestrian's center of mass acceleration. The most representative model of this type is the Kose model based on the optimal filtering forward and reverse integration method proposed by Kose. However, this type of model must face the problem of accelerometer measurement error accumulation. In addition to sensor fusion and Kalman filtering solutions, heuristic virtual observations are often used as corrections. Therefore, this type of model also has strong application scenario limitations.

[0115] In summary, the related technologies have the following problems:

[0116] First, the gait recognition method based on a single inertial measurement unit has a low recognition rate and low accuracy;

[0117] Second, the step-size estimation method is not robust enough to handle complex situations.

[0118] Third, the lack of an adaptive step length estimation strategy for human gait makes it difficult to put the pedestrian dead reckoning system in related technologies into practical application in complex environments.

[0119] In order to solve the technical problems similar to those described above, the embodiments of the present disclosure provide a pedestrian dead reckoning method, which obtains the first waist motion feature data of the target person, processes the first waist motion feature data through a pre-trained long short-term memory recurrent neural network, obtains the first step prediction result of the target person, estimates the step length of the target person according to the first step prediction result and the first waist motion feature data, obtains the step length estimation result of the target person, and updates the position information of the target person according to the initial position information of the target person, the heading angle estimation result of the target person, and the step length estimation result, thereby realizing pedestrian dead reckoning of complex actions based on gait recognition of the person. In the embodiments of the present disclosure, high recognition rate estimation of multiple gaits can be achieved by relying only on a single inertial measurement unit, which can improve the accuracy of gait estimation.

[0120] The pedestrian dead reckoning method provided by the embodiment of the present disclosure is described in detail below with reference to the accompanying drawings.

[0121] Figure 1A flow chart of a pedestrian dead reckoning method provided by an embodiment of the present disclosure is shown. In one possible implementation, the execution subject of the pedestrian dead reckoning method may be a pedestrian dead reckoning apparatus. For example, the pedestrian dead reckoning method may be executed by a terminal device or a server or other electronic device. The terminal device may be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device or a wearable device. In some possible implementations, the pedestrian dead reckoning method may be implemented by a processor calling computer-readable instructions stored in a memory. For example Figure 1 As shown, the pedestrian dead reckoning method includes steps S11 to S14.

[0122] In step S11, first waist motion characteristic data of the target person is obtained.

[0123] In step S12, the first waist movement feature data is processed by a pre-trained long short-term memory recurrent neural network to obtain a first step prediction result of the target person.

[0124] In step S13, the stride length of the target person is estimated according to the first step prediction result and the first waist movement characteristic data to obtain the stride length estimation result of the target person.

[0125] In step S14, the position information of the target person is updated according to the initial position information of the target person, the heading angle estimation result of the target person, and the step length estimation result.

[0126] In the embodiment of the present disclosure, the target person may represent an individual tracked and located in the pedestrian dead reckoning system. The first waist motion feature data may represent waist motion feature data of the target person.

[0127] In a possible implementation, the waist motion feature data may include dynamic feature data and kinematic feature data. In this implementation, by estimating the gait of the target person based on the differences in waist motion and dynamic features when the person performs various complex movements, the recognition rate and accuracy of gait estimation can be improved.

[0128] In a possible implementation, the waist motion characteristic data includes at least part of the following: waist acceleration, waist angular velocity, and step characteristic data.

[0129] In other possible implementations, the waist motion characteristic data may also include other types of dynamic characteristic data and / or kinematic characteristic data.

[0130] In a possible implementation, obtaining the first waist motion characteristic data of the target person includes: collecting the first waist motion characteristic data of the target person through a waist-mounted inertial measurement unit; and extracting the first waist motion characteristic data of the target person based on the first waist motion data.

[0131] In this implementation, the first waist motion data of the target person can be collected by a waist-mounted inertial measurement unit (IMU). The first waist motion data can represent the waist motion data of the target person. Based on the analysis of the first waist motion data, the first waist motion feature data of the target person can be extracted.

[0132] In this implementation, only a single waist-mounted inertial measurement unit may be employed.

[0133] In a possible implementation, after collecting the first waist motion data of the target person through the waist-mounted inertial measurement unit, and before processing the first waist motion feature data through the pre-trained long short-term memory recursive neural network, the method further includes: extracting data from the first waist motion data based on a sliding window with a preset length and a preset overlap rate, and performing secondary sampling at intervals of a preset number of groups.

[0134] In one example, the preset length may be 2 seconds, the preset overlap rate may be 50%, and the preset number of groups may be 8. In this example, a sliding window with a length of 2 seconds and an overlap rate of 50% may be selected to extract the first waist motion data (first waist acceleration and first waist angular velocity) of the target person in the sliding window, and perform secondary sampling at intervals of 8 groups of data.

[0135] Figure 2 A schematic diagram of a sliding window in a pedestrian dead reckoning method provided by an embodiment of the present disclosure is shown. Figure 2 In the example shown, the sliding window length is 2 seconds and the overlap is 50%.

[0136] Of course, those skilled in the art can flexibly set the preset length, preset overlap rate and preset number of groups according to the actual application scenario requirements, and no limitation is made here.

[0137] In a possible implementation, the first waist motion data includes a first waist acceleration and a first waist angular velocity; extracting the first waist motion feature data of the target person based on the first waist motion data includes: performing a continuous wavelet transform based on the first waist acceleration to obtain the first step feature data of the target person; obtaining the first waist motion feature data of the target person based on the first waist acceleration, the first waist angular velocity and the first step feature data.

[0138] In this implementation, by performing continuous wavelet transform (CWT) based on the first waist acceleration, the time-frequency characteristics of the first waist acceleration can be obtained, so that the first step feature data of the target person can be extracted based on the time-frequency characteristics.

[0139] Figure 3 FIG. 1 is a schematic diagram showing a step detection process in a pedestrian dead reckoning method provided by an embodiment of the present disclosure. Figure 3 As shown, first, the L2 norm of the vector of the first waist acceleration can be calculated to avoid the influence of the installation angle of the waist-mounted inertial measurement unit on the step detection result. Subsequently, a low-pass filter with a cutoff frequency of 4 Hz is used to reduce the high-frequency error caused by the hardware. The modulus of the filtered first waist acceleration is used as the input value of the continuous wavelet analysis, and the Daubechies wavelet is used as the mother wave. The signal is decomposed by the expansion wavelet and the translation wavelet to analyze the time-frequency characteristics of the signal and make a judgment on the step event.

[0140] A wavelet is a short waveform that can be named And with the neighborhood of t=0 as the center, it satisfies:

[0141]

[0142] Assume ψ is a real wavelet, then the wavelet transform corresponding to the function f can be expressed as:

[0143]

[0144] Among them, the variable u represents the time conversion variable, which is used to control the wavelet to scan the entire time window; the variable s represents the scale of the extended wavelet to analyze the different frequency characteristics of the signal at a specific time. This operation obtains the time-frequency characteristics of the function f(t) by analyzing the convolution of the wavelet with a scale of s and the function f(t) at time u. The function f(t) represents a low-pass filtered version of the modulus of the first waist acceleration. The higher the correlation between the function f(t) and the appropriate extended wavelet, the higher the convolution value obtained. The moment when the step action occurs (that is, when the modulus of the first waist acceleration reaches the local maximum) is considered to be the moment with the highest correlation between the specific force modulus and the wavelet. After analyzing the time-frequency characteristics of the modulus of the first waist acceleration and obtaining the step time node, the raw data collected by the waist-mounted inertial measurement unit can be segmented according to the step time node and the first step feature data can be extracted.

[0145] In the disclosed embodiment, the first waist motion feature data can be input into a pre-trained long short-term memory recursive neural network (Long Short Term Memory, LSTM), and the first step prediction result of the target person is output through the long short-term memory recursive neural network. LSTM is a modified recurrent neural network that is widely used in the modeling and processing of sequence data. Compared with the ordinary RNN (Recurrent Neural Network) model, LSTM introduces a gating mechanism to control the flow of information to solve the long-term dependency problem of traditional RNN. The working principle of LSTM is based on three important gates: input gate, forget gate and output gate. These gates control the input information and previous memory information through learning, solving the problem that traditional RNN is difficult to process long sequence data.

[0146] In a possible implementation, the first waist acceleration, the first waist angular velocity and the first step feature data may be combined and input into a long short-term memory recurrent neural network for gait prediction.

[0147] In the disclosed embodiment, the first step prediction result may represent the result obtained by analyzing and processing the first waist motion feature data of the target person using the long short-term memory recurrent neural network. The first step prediction result may be used to guide subsequent step length estimation and pedestrian dead reckoning.

[0148] In one possible implementation, the first gait prediction result includes the probability that the target person belongs to multiple preset gaits; wherein the multiple preset gaits include at least part of the following: standing, walking, backward, running, going upstairs, going downstairs, falling down, and jumping.

[0149] The LSTM network is trained to recognize and distinguish different human gaits, such as standing, walking, walking backward, running, going upstairs, going downstairs, falling, jumping, etc. After the LSTM network analyzes the collected data, it outputs the probability of each preset gait, which represents the confidence of the model's prediction that the target person may currently be in various gaits.

[0150] For example, the first step prediction results include: standing: 0.1, walking: 0.5, backing up: 0.05, running: 0.2, going upstairs: 0.05, going downstairs: 0.03, falling down: 0.02, jumping: 0.1. In this example, the LSTM network predicts that the target person is in the "walking" state with the highest probability of 0.5. This means that according to the judgment of the LSTM network, the target person is most likely currently performing the "walking" action. In practical applications, a threshold value, such as 0.7, can be set to determine the credibility of the gait prediction results. If the predicted probability of a certain gait exceeds this threshold, the prediction can be considered credible. In the above example, the predicted probability of no gait exceeds 0.7, so further processing can be performed, such as removing abnormal points through methods such as mean filtering, threshold judgment, and continuity judgment, and finally determining the gait of the target person.

[0151] Of course, in other possible implementations, the types of gaits that can be detected and recognized by the pedestrian dead reckoning system may be more or less, which is not limited here.

[0152] In a possible implementation, before the first waist motion feature data is processed by the pre-trained long short-term memory recurrent neural network, the method further includes: acquiring a training data set, wherein the training data set includes second waist motion data of multiple persons; extracting second waist motion feature data based on the second waist motion data; and training the long short-term memory recurrent neural network based on the second waist motion feature data.

[0153] As an example of this implementation, a long short-term memory recursive neural network can be pre-trained for eight preset gaits: standing, walking, backward, running, going upstairs, going downstairs, falling, and jumping. Figure 4 FIG. 1 is a schematic diagram showing the structure of a long short-term memory recursive neural network in a pedestrian dead reckoning method provided by an embodiment of the present disclosure. Figure 4 As shown, the long short-term memory recurrent neural network may include an LSTM unit and two fully connected layers.

[0154] When setting the training data set for the LSTM recurrent neural network, the eight gaits of standing, walking, backing up, running, going upstairs, going downstairs, falling down, and jumping can be one-hot encoded to obtain supervision data. For example, each gait can be represented by an 8-bit binary vector. Standing: [1,0,0,0,0,0,0,0], Walking: [0,1,0,0,0,0,0,0], Backing up: [0,0,1,0,0,0,0,0], Running: [0,0,0,1,0,0,0,0], Going upstairs: [0,0,0,0,1,0,0,0], Going downstairs: [0,0,0,0,0,1,0,0], Falling down: [0,0,0,0,0,0,1,0], Jumping: [0,0,0,0,0,0,0,1]. When training a LSTM recurrent neural network, these one-hot encoded vectors can be used as labels for training data.

[0155] The second waist motion data may include a second waist acceleration and a second waist angular velocity, wherein the second waist acceleration may be three-axis acceleration data, and the second waist angular velocity may be three-axis angular velocity data.

[0156] Similar to the above, continuous wavelet transform can be performed based on the second waist acceleration to obtain the second step feature data of the person in the training data set. The second waist motion feature data of the person in the training data set can be obtained based on the second waist acceleration, the second waist angular velocity and the second step feature data. Data extraction can be performed on the second waist acceleration, the second waist angular velocity and the second step feature data based on a sliding window with a length of 2 seconds and an overlap rate of 50%, and sub-sampling is performed at intervals of 8 groups of data, and the data of 25 sampling cycles is used as a sequence sample to input the long short-term memory recurrent neural network to obtain the gait prediction probability through the long short-term memory recurrent neural network. According to the gait prediction probability and one-hot encoding, the parameters of the long short-term memory recurrent neural network can be updated.

[0157] In a possible implementation, after the first waist motion feature data is processed by the pre-trained long short-term memory recurrent neural network to obtain the first step prediction result of the target person, the method further includes: performing abnormal point elimination processing based on the first step prediction result to obtain a corrected first step prediction result.

[0158] Among them, the outlier removal process may include at least some of the processing methods such as mean filtering, threshold judgment, continuity judgment, etc., which are not limited here.

[0159] Among them, mean filtering can be used to reduce random noise in the data and smooth the prediction results. For example, in gait recognition, three groups of gait prediction probabilities can be taken in time sequence and the average of these three groups of gait prediction probabilities can be calculated to reduce the fluctuation and uncertainty of a single prediction.

[0160] Threshold judgment can determine the credibility of the prediction results and filter out low-confidence predictions. After mean filtering, a preset threshold (e.g., 0.7) can be applied to each gait prediction probability. If the maximum probability value in a gait prediction probability is less than this threshold, it can be considered that the gait prediction probability is not reliable enough and may be misclassified, so its result can be regarded as "other gaits". Threshold judgment helps to exclude predictions that the model is not sure about, thereby improving the overall gait recognition accuracy.

[0161] Continuity judgment helps improve the stability and logical consistency of the prediction results. After the threshold judgment, the continuity of the prediction can be further checked. For example, you can check whether the prediction output results of three groups in the time series are consistent. If the three groups of results are the same, then the prediction is considered stable and reliable and is finally output. This helps avoid misjudgments caused by accidental and unstable prediction results.

[0162] In a possible implementation, the first step prediction result or the corrected first step prediction result of the target person may be recorded and output in a document or other form.

[0163] In a possible implementation, the step length of the target person is estimated based on the first step prediction result and the first waist motion characteristic data to obtain the step length estimation result of the target person, including: determining a target step length estimation model from at least two adaptive step length estimation models based on the first step prediction result; and estimating the step length of the target person based on the first waist motion characteristic data through the target step length estimation model to obtain the step length estimation result of the target person.

[0164] In this implementation, the optimal step length estimation model (i.e., the target step length estimation model) can be selected from at least two predefined adaptive step length estimation models according to the first step prediction result of the target person, and the step length of the target person can be estimated. The selection process of the adaptive step length estimation model can be based on the gait with the highest probability in the gait prediction result, or can be a comprehensive evaluation based on multiple gait prediction results.

[0165] After the target stride length estimation model is determined, the target stride length estimation model may be used to estimate the stride length of the target person based on the first waist motion feature data.

[0166] In this implementation, each adaptive step length estimation model is more adaptable to complex environments and complex actions. By estimating the step length of the target person using the target step length estimation model that matches the first step prediction result, a more accurate step length estimation result can be obtained.

[0167] In one possible implementation, determining a target stride length estimation model from at least two adaptive stride length estimation models based on the first step prediction result includes: in response to the first step prediction result indicating that the current gait of the target person is clear, determining a target stride length estimation model from at least two adaptive stride length estimation models based on the first step prediction result.

[0168] Among them, the current gait is clear, which means that the LSTM network is very confident in the gait prediction results of the target person, that is, the probability of one or several gaits in the prediction probability output by the LSTM network is significantly higher than other gaits, which shows that the LSTM network is very certain about the judgment of the current gait.

[0169] Based on this clear first step prediction result, the most suitable model (i.e., the target step length estimation model) can be selected from at least two pre-defined adaptive step length estimation models. For example, if the LSTM network predicts that the target person is walking, a step length estimation model specifically for walking gait can be selected; if the LSTM network predicts that the target person is running, a step length estimation model specifically for running gait can be selected.

[0170] In this implementation, when the first step prediction result indicates that the current gait of the target person is clear, an adaptive stride length estimation model corresponding to the current gait of the target person is selected to estimate the stride length of the target person, thereby improving the accuracy of the stride length estimation of the target person.

[0171] In a possible implementation, the at least two adaptive step length estimation models include at least some of the following types: a statistical regression model, a motion physiological model, and a direct integration model based on center of mass acceleration.

[0172] Among them, the statistical regression model is a model that uses statistical methods to study the relationship between variables and use it for prediction. In step length estimation, the statistical regression model can collect various features related to step length (such as cadence, gait cycle, acceleration peak, etc.) and use these features as independent variables to predict step length. The statistical regression model can establish a regression equation between features and step length through historical data. Specific methods may include linear regression, multivariate regression analysis, etc. The statistical regression model attempts to find the feature combination that best explains the change in step length. The statistical regression model is relatively simple, easy to understand and implement. If the features are selected properly, it can provide fairly accurate predictions.

[0173] Movement physiological models can predict step length based on the principles of human kinematics and dynamics. Movement physiological models are usually more complex, and they take into account factors such as human structure, joint angles, and muscle strength. For example, the Inverted Pendulum Model can be used to simplify human motion and estimate step length by simulating the physics of human motion. Movement physiological models may involve more complex physical and biological equations. Movement physiological models can provide a deeper understanding of step length estimation, taking into account the complexity of human motion.

[0174] The direct integration model based on center of mass acceleration can estimate the step length by directly integrating the center of mass acceleration data. The direct integration model based on center of mass acceleration can use the acceleration data measured by the accelerometer to obtain velocity and displacement by integration. The acceleration data can first be filtered to remove noise, then integrated once to obtain velocity, and then integrated again to obtain displacement (i.e., step length). The direct integration model based on center of mass acceleration can use various filtering and correction techniques to deal with drift problems during the integration process. The direct integration model based on center of mass acceleration can directly utilize sensor data without the need for complex knowledge of human motion.

[0175] For example, the statistical regression model may include the Zhu model, the Shin model, etc., and the sports physiological model may include the Weinberg model, the Mikov model, the Kim model, etc.

[0176] In a possible implementation, the adaptive step length estimation model may include at least part of the following: a Weinberg model, a Mikov model, a Zhu model, a Kim model, and a Shin model.

[0177] Among them, the Weinberg model has been widely used in the field of pedestrian navigation due to its accuracy in dynamic step length estimation and low computational complexity. The model is based on the biomechanical hypothesis that when pedestrians walk, the supporting leg will remain straight, so that the movement trajectory of the waist is similar to an inverted pendulum. Although the Weinberg model was originally designed for walking and walking backwards, considering the similarity between pedestrian movement characteristics and walking states in other gaits such as running, going up and down stairs, the disclosed embodiment can use it as an adaptive step length estimation model. The expression of the Weinberg model is as follows:

[0178]

[0179] Among them, A max With A min are the maximum and minimum values ​​of the specific force modulus in each step respectively, and K1 is the parameter that needs to be calibrated.

[0180] The Mikov model takes into account the duration of each step on the basis of the Weinberg model and adds it to the parameters to be calibrated in the Weinberg expression. The expression is as follows:

[0181]

[0182] Among them, A max With A min are the maximum and minimum values ​​of the specific force modulus in each step, t step is the duration of the step, and K2 is the parameter that needs to be calibrated.

[0183] The Zhu model replaces the parameters to be calibrated in the Weinberg model with a linear regression model of cadence and acceleration variance, and its expression is as follows:

[0184]

[0185] Among them, f, v are the variance of step frequency and acceleration, r lf and r lv They are the Pearson correlation coefficients between stride length and stride frequency, and between stride length and acceleration variance.

[0186] The step length estimation of Kim's model is based only on the average value of the acceleration modulus in the step, and its expression is as follows:

[0187]

[0188] The Shin model is a priori statistical regression model that assumes that the step length is linearly related to the pedestrian's step frequency and the variance of the acceleration signal. Its expression is as follows:

[0189] SL5=a·f+b·v+c

[0190] Among them, f and v are the variances of the step frequency and acceleration modulus, respectively, and a, b, and c are the coefficients that need to be calibrated.

[0191] In a possible implementation, before determining the target stride length estimation model from at least two adaptive stride length estimation models based on the first step prediction result, the method further includes: using an optical motion capture system and a waist-mounted inertial measurement unit to collect motion data of a person in different gaits, and extracting kinematic characteristics and dynamic characteristics of the person; and constructing adaptive stride length estimation models corresponding to different gaits based on the kinematic characteristics and the dynamic characteristics.

[0192] Figure 5 FIG. 2 shows a calibration and analysis flow chart of the adaptive step length estimation method provided by the embodiment of the present disclosure. Figure 5As shown, according to the real-time position of the person measured by the optical motion capture system and the dynamic characteristics of the person's waist measured by the waist-mounted inertial measurement unit, the calibration of multiple step length estimation models under various gaits can be completed by the least squares method, and the step length estimation accuracy and error distribution of each step length estimation model under different gaits can be analyzed, so as to determine the type and parameters of the adaptive step length estimation model corresponding to various gaits.

[0193] In a possible implementation, the step length of the target person is estimated based on the first step prediction result and the first waist motion characteristic data to obtain the step length estimation result of the target person, including: in response to the first step prediction result indicating that the current gait of the target person is blurred, the step length of the target person is estimated based on the first waist motion characteristic data through a multi-gait general step length estimation model to obtain the step length estimation result of the target person.

[0194] Among them, the first step prediction result indicates that the current gait of the target person is ambiguous, which means that the LSTM network is not very sure or clear about the gait prediction result of the target person. In other words, the LSTM network does not give a high probability for any specific gait (such as walking, running, etc.), which may be because the target person is performing mixed actions of different gaits, or the characteristics of the action are not obvious enough, making it difficult for the LSTM network to make a clear judgment.

[0195] In this case, a multi-gait general stride length estimation model can be used to estimate the stride length of the target person. The multi-gait general model considers multiple gaits. It does not rely on the accurate prediction of a specific gait, but estimates the stride length by comprehensively considering multiple possible gaits.

[0196] The multi-gait universal stride length estimation method provided by this implementation is suitable for environments with variable gaits or complex environments, such as in crowded public places, or when people perform multiple actions (such as fast walking, short running, sudden stops, etc.). When the gait prediction results are ambiguous, the multi-gait universal stride length estimation model is used to adapt to this situation to ensure that effective stride length estimation can still be performed when it is difficult to accurately determine the specific gait. This implementation improves the robustness of the system in the face of uncertainty.

[0197] In a possible implementation, before estimating the stride length of the target person based on the first waist motion feature data using a multi-gait universal stride length estimation model, the method further includes: using an optical motion capture system and a waist-mounted inertial measurement unit to collect motion data of the person in different gaits, and extracting specified features related to the stride length based on the motion data of the person in different gaits; analyzing the correlation coefficient between the specified features and the stride length; and constructing a multi-gait universal stride length estimation model based on the correlation coefficient.

[0198] As an example of this implementation, the specified characteristics include at least part of the following: step frequency, maximum value of waist acceleration modulus, minimum value of waist acceleration modulus, and standard deviation of waist acceleration.

[0199] Figure 6 The analysis and calibration flow chart of the multi-gait universal stride length estimation model provided by the embodiment of the present disclosure is shown. In this implementation, the kinematic characteristics of personnel concerned by the traditional statistical regression model and the dynamic characteristics of personnel concerned by the sports physiological model can be comprehensively tested and analyzed to obtain the multi-gait universal stride length estimation model.

[0200] In terms of data collection and organization, a data sample library of waist movement and dynamic characteristics when a person completes complex movements can be established based on the real-time position of the person measured by the optical motion capture system and the dynamic characteristics of the waist of the person measured by the IMU. For example, the optical motion capture system can be used to measure the movement of key parts such as the ankle joint, knee joint, and shoulder joint during complex movements; on this basis, the Hill ternary model can be used to model human muscles and perform inverse kinematics to obtain a general three-dimensional human motion physiological model; and based on the general three-dimensional human motion physiological model, the kinematic characteristics of the person's step length, step frequency, and step speed can be extracted. In addition, while performing optical motion capture, the output value of the 6-axis IMU installed on the waist of the person can be recorded and used as the input value of the step detection algorithm. The original data is segmented and the step dynamic characteristics are extracted through the step detection algorithm based on continuous wavelet analysis. Based on the measurement results, the step length l, step frequency f, the difference ΔA between the maximum and minimum values ​​of the waist acceleration modulus in the step interval, and the standard deviation v of the waist acceleration modulus can be calculated and organized. A Etc. feature quantities.

[0201] In order to determine the weight of each feature in the multi-gait universal stride length estimation model, we can first analyze the relationship between each feature and the stride length l. Calculate the Pearson correlation coefficient r between the stride length l and the stride frequency f lf , the expression is as follows:

[0202]

[0203] in is the average step frequency, l i is the true value of the distance of each step, is the average step length. Similarly, calculate the Pearson correlation coefficient r of the remaining feature quantities and step length lΔA and Furthermore, for the four step characteristics considered in this implementation (cadence, maximum waist acceleration modulus, minimum waist acceleration modulus, waist acceleration standard deviation), in order to analyze the correlation between any one of the characteristics and the step length, while excluding the influence of the other characteristics, the second-order partial correlation coefficient between each characteristic and the step length can be calculated. The expression of the second-order partial correlation coefficient between step length and cadence is as follows:

[0204]

[0205] Similarly, the second-order partial correlation coefficients between the remaining feature quantities are solved. lf·ΔA and is the first-order partial correlation coefficient, and its expression is as follows:

[0206]

[0207] The expressions of the first-order partial correlation coefficients between the other feature quantities are similar and will not be repeated here.

[0208] In summary, after analyzing and calculating the second-order partial correlation coefficient between the stride length of the personnel and the other feature quantities, this implementation method can divide the considered features into two items according to their characteristics, and construct a multi-gait universal stride length estimation model as follows:

[0209]

[0210] in, is the estimated value of the person's step length, (a, b, c) are the parameters to be calibrated, f is the person's step frequency, ΔA is the difference between the maximum and minimum values ​​of the waist acceleration modulus, v A is the standard deviation of waist acceleration modulus. Finally, based on the multi-gait universal step length estimation model and the constructed data sample library, the least squares method is used to optimally estimate the calibration parameters (a, b, c).

[0211] In this implementation, by analyzing the big data samples of human waist kinematics and dynamics characteristics, calculating the partial correlation coefficient between the person's step length and waist motion characteristics, a multi-gait universal step length estimation model with stronger adaptability to complex movements is designed and proposed. The multi-gait universal step length estimation model considers the correlation between the step frequency, the maximum value of the waist acceleration modulus upon landing, the standard deviation of acceleration and the step length, and calculates the partial correlation coefficient between the step length and the waist motion characteristics. Therefore, it more comprehensively considers the relationship between the person's motion characteristics and the step length, and significantly improves the adaptability of the traditional step length estimation model to different people and complex environments. In addition, the multi-gait universal step length estimation model has the advantages of low system complexity and strong adaptability to complex environments.

[0212] In the disclosed embodiment, after obtaining the step length estimation result, the initial position information of the target person, the step length estimation result and the heading angle estimation result of the target person can be combined to update the position information of the target person through data fusion and filtering algorithms.

[0213] In a possible implementation, the heading angle of the target person can be updated by using a strategy of adding a single sample to the previous sample in the equivalent rotation vector method according to the angular velocity measured by the gyroscope. The principle is as follows:

[0214]

[0215] Among them, Δθ1 is the angular increment in the new angular velocity update cycle, which is obtained by multiplying the updated angular velocity w1 by the difference between the angular velocity and the previous angular velocity timestamp by Δt. Δθ0 is the angular increment in the last angular velocity update cycle. Combining Δθ1 and Δθ0, the equivalent rotation vector at that moment can be obtained, and the heading angle estimation result of the target person can be obtained.

[0216] The pedestrian dead reckoning method provided in the embodiments of the present disclosure can be applied to technical fields such as indoor positioning, pedestrian dead reckoning, and personnel step length estimation based on human kinematics and dynamics, and is not limited here.

[0217] The pedestrian dead reckoning method provided by the embodiment of the present disclosure is described below through a specific application scenario. Figure 7 FIG. 2 shows a structural diagram of an adaptive pedestrian dead reckoning system based on gait recognition provided by an embodiment of the present disclosure. Figure 7 As shown, the first waist motion data of the target person can be collected by a single waist-mounted inertial measurement unit, wherein the first waist motion data includes a first waist acceleration and a first waist angular velocity, the first waist acceleration can be three-axis acceleration data, and the first waist angular velocity can be three-axis angular velocity data. The waist-mounted inertial measurement unit can include an accelerometer and a gyroscope.

[0218] 1. Gait Detection

[0219] By performing continuous wavelet transform (CWT) based on the first waist acceleration, the time-frequency characteristics of the first waist acceleration can be obtained, so that the first step characteristic data of the target person can be extracted based on the time-frequency characteristics. The first waist motion characteristic data of the target person can be obtained based on the first waist acceleration, the first waist angular velocity and the first step characteristic data.

[0220] The first waist motion feature data can be processed by a motion state (i.e., gait) detection and recognition algorithm to obtain the first step prediction result of the target person. For example, the first waist acceleration, the first waist angular velocity, and the first step feature data can be combined and input into a pre-trained long short-term memory recursive neural network for gait prediction, and the first step prediction result of the target person is output through the long short-term memory recursive neural network. The first step prediction result may include the probability that the target person belongs to a plurality of preset gaits; wherein the plurality of preset gaits may include: standing, walking, backward, running, going upstairs, going downstairs, falling, and jumping.

[0221] After the first waist motion feature data is processed by the long short-term memory recurrent neural network to obtain the first posture prediction result of the target person, an abnormal point elimination process can be performed based on the first posture prediction result to obtain a corrected first posture prediction result. The abnormal point elimination process can include mean filtering, threshold judgment and continuity judgment.

[0222] 2. Step Length Estimation

[0223] The first step prediction result of the correction can be responded to to indicate that the current gait of the target person is clear. According to the first step prediction result, the target step length estimation model is determined from at least two adaptive step length estimation models, and the step length estimation of the target person is performed on the target person through the target step length estimation model to obtain the step length estimation result of the target person. Wherein, the current gait is clear, which means that the LSTM network is very confident in the gait prediction result of the target person, that is, the probability of one or more gaits in the prediction probability output by the LSTM network is significantly higher than other gaits, which shows that the LSTM network is very certain about the judgment of the current gait. Based on this clear first step prediction result, a most suitable model (i.e., the target step length estimation model) can be selected from at least two predefined adaptive step length estimation models. Wherein, the at least two adaptive step length estimation models include at least some of the following types: statistical regression model, motion physiological model, direct integration model based on centroid acceleration. For example, if the LSTM network predicts that the target person is walking, a step length estimation model specifically for walking gait can be selected; if the LSTM network predicts that the target person is running, a step length estimation model for running gait can be selected.

[0224] The target person's current gait can be estimated in response to the corrected first step prediction result indicating that the target person's current gait is fuzzy, and the target person is estimated by a multi-gait universal step length estimation model to obtain the target person's step length estimation result. Wherein, the first step prediction result indicates that the target person's current gait is fuzzy, which means that the LSTM network is not very certain or clear about the gait prediction result of the target person. In other words, the LSTM network does not give a high probability for any specific gait (such as walking, running, etc.), which may be because the target person is performing a mixed action of different gaits, or the characteristics of the action are not obvious enough, causing the LSTM network to be difficult to make a clear judgment. In this case, a multi-gait universal step length estimation model can be used to estimate the stride of the target person. The multi-gait universal model considers multiple gaits, which does not rely on the accurate prediction of a specific gait, but estimates the stride length by comprehensively considering multiple possible gaits.

[0225] 3. Adaptive Pedestrian Dead Reckoning

[0226] A personnel heading estimation algorithm may be used to determine the heading angle estimation result of the target personnel.

[0227] The position information of the target person may be updated according to the initial position information of the target person, the heading angle estimation result of the target person, and the step length estimation result.

[0228] Figure 8 The figure shows the difference in positioning accuracy between the pedestrian dead reckoning system provided by the embodiment of the present disclosure and the traditional pedestrian dead reckoning system in a complex environment. Figure 8 In the figure, GA-PDR represents the pedestrian dead reckoning system provided by the embodiment of the present disclosure, RTK represents the real-time kinematic system, and Classic-PDR represents the traditional pedestrian dead reckoning system. Figure 8 As shown, in the face of a complex environment involving actions such as walking, running, jumping, going upstairs, going downstairs, falling, and backing up, the pedestrian dead reckoning system provided by the embodiments of the present disclosure has better positioning accuracy and robustness than the traditional pedestrian dead reckoning system, and has broader application prospects.

[0229] The pedestrian dead reckoning method provided by the embodiments of the present disclosure can independently detect and adapt to complex movements while providing high-precision indoor positioning services when people are walking, thereby significantly improving the positioning accuracy and robustness of the pedestrian dead reckoning system in complex environments.

[0230] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, the present disclosure will not repeat them. It can be understood by those skilled in the art that in the above-mentioned method of the specific implementation method, the specific execution order of each step should be determined according to its function and possible internal logic.

[0231] In addition, the present disclosure also provides a pedestrian dead reckoning device, an electronic device, a computer-readable storage medium, and a computer program product, all of which can be used to implement any pedestrian dead reckoning method provided by the present disclosure. The corresponding technical solutions and technical effects can be found in the corresponding records of the method part and will not be repeated here.

[0232] Fig. 9 FIG. 2 is a block diagram of a pedestrian dead reckoning device provided by an embodiment of the present disclosure. Fig. 9 As shown, the pedestrian dead reckoning device includes:

[0233] An acquisition module 21 is used to obtain first waist motion characteristic data of a target person;

[0234] A processing module 22, configured to process the first waist motion feature data through a pre-trained long short-term memory recursive neural network to obtain a first step prediction result of the target person;

[0235] A step length estimation module 23, configured to estimate the step length of the target person according to the first step prediction result and the first waist movement characteristic data, and obtain a step length estimation result of the target person;

[0236] The updating module 24 is used to update the position information of the target person according to the initial position information of the target person, the heading angle estimation result of the target person and the step length estimation result.

[0237] In a possible implementation, the obtaining module 21 is used to:

[0238] Collecting first waist motion data of the target person through a waist-mounted inertial measurement unit;

[0239] Based on the first waist motion data, first waist motion feature data of the target person is extracted.

[0240] In a possible implementation, the first waist motion data includes a first waist acceleration and a first waist angular velocity;

[0241] The obtaining module 21 is used for:

[0242] Performing continuous wavelet transform based on the first waist acceleration to obtain first step feature data of the target person;

[0243] Based on the first waist acceleration, the first waist angular velocity and the first step characteristic data, first waist movement characteristic data of the target person is obtained.

[0244] In a possible implementation manner, the device further includes:

[0245] The first extraction module is used to extract the first waist motion data based on a sliding window with a preset length and a preset overlap rate, and perform secondary sampling at intervals of a preset number of groups.

[0246] In a possible implementation manner, the device further includes:

[0247] An acquisition module, used for acquiring a training data set, wherein the training data set includes second waist motion data of a plurality of persons;

[0248] A second extraction module, used for extracting second waist motion feature data based on the second waist motion data;

[0249] A training module is used to train the long short-term memory recursive neural network based on the second waist movement feature data.

[0250] In a possible implementation, the first gait prediction result includes the probability that the target person has multiple preset gaits;

[0251] The plurality of preset gaits include at least some of the following: standing, walking, walking backward, running, going up stairs, going down stairs, falling down, and jumping.

[0252] In a possible implementation, the step length estimation module 23 is used to:

[0253] Determining a target step length estimation model from at least two adaptive step length estimation models according to the first step prediction result;

[0254] The target stride length estimation model is used to estimate the stride length of the target person based on the first waist motion feature data to obtain a stride length estimation result of the target person.

[0255] In a possible implementation manner, the device further includes:

[0256] The third extraction module is used to collect motion data of the personnel at different gaits by using an optical motion capture system and a waist-mounted inertial measurement unit, and to extract kinematic and dynamic characteristics of the personnel;

[0257] The first building module is used to build an adaptive step length estimation model corresponding to different gaits based on the kinematic characteristics and the dynamic characteristics.

[0258] In a possible implementation, the step length estimation module 23 is used to:

[0259] In response to the first step prediction result indicating that the current gait of the target person is clear, a target step length estimation model is determined from at least two adaptive step length estimation models according to the first step prediction result.

[0260] In a possible implementation, the at least two adaptive step length estimation models include at least some of the following types:

[0261] Statistical regression model, sports physiological model, direct integration model based on center of mass acceleration.

[0262] In a possible implementation, the step length estimation module 23 is used to:

[0263] In response to the first step prediction result indicating that the current gait of the target person is blurred, the step length of the target person is estimated based on the first waist motion feature data using a multi-gait universal step length estimation model to obtain a step length estimation result of the target person.

[0264] In a possible implementation manner, the device further includes:

[0265] a fourth extraction module, for collecting motion data of the person at different gaits using an optical motion capture system and a waist-mounted inertial measurement unit, and extracting specified features related to step length based on the motion data of the person at different gaits;

[0266] An analysis module, used for analyzing the correlation coefficient between the specified feature and the step length;

[0267] The second building module is used to build a multi-gait universal step length estimation model according to the correlation coefficient.

[0268] In a possible implementation, the specified features include at least part of the following:

[0269] Cadence, maximum value of waist acceleration modulus, minimum value of waist acceleration modulus, standard deviation of waist acceleration.

[0270] In the disclosed embodiment, by obtaining the first waist motion characteristic data of the target person, the first waist motion characteristic data is processed by a pre-trained long short-term memory recurrent neural network to obtain the first step prediction result of the target person, and the step length of the target person is estimated according to the first step prediction result and the first waist motion characteristic data to obtain the step length estimation result of the target person, and the position information of the target person is updated according to the initial position information of the target person, the heading angle estimation result of the target person, and the step length estimation result, thereby realizing complex action pedestrian dead reckoning based on gait recognition of the person. In the disclosed embodiment, relying only on a single inertial measurement unit, it is possible to achieve high recognition rate estimation of multiple gaits, which can improve the accuracy of gait estimation.

[0271] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. Its specific implementation and technical effects can refer to the description of the above method embodiments. For the sake of brevity, they will not be repeated here.

[0272] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium may be a non-volatile computer-readable storage medium, or may be a volatile computer-readable storage medium.

[0273] The embodiment of the present disclosure further provides a computer program, including a computer-readable code. When the computer-readable code is executed in an electronic device, a processor in the electronic device executes the above method.

[0274] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in an electronic device, a processor in the electronic device executes the above method.

[0275] An embodiment of the present disclosure also provides an electronic device, comprising: one or more processors; and a memory for storing executable instructions; wherein the one or more processors are configured to call the executable instructions stored in the memory to execute the above method.

[0276] The electronic device may be provided as a terminal, a server, or a device in other forms.

[0277] Fig.10 1 is a block diagram of an electronic device 1900 provided in an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server or a terminal. Fig.10, the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0278] The electronic device 1900 may also include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output interface 1958 (I / O interface). The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as a Microsoft Server operating system (Windows Server 2003). TM ), a graphical user interface operating system launched by Apple (MacOS X TM ), a multi-user, multi-process computer operating system (Unix TM ), a free and open source Unix-like operating system (Linux TM ), an open source Unix-like operating system (FreeBSD TM ) or similar.

[0279] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.

[0280] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0281] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0282] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0283] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0284] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0285] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0286] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0287] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0288] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK) and the like.

[0289] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0290] If the technical solution of the embodiments of the present disclosure involves personal information, the product using the technical solution of the embodiments of the present disclosure has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of the embodiments of the present disclosure involves sensitive personal information, the product using the technical solution of the embodiments of the present disclosure has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the requirement of "explicit consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to collect his or her personal information; or on the device for processing personal information, when the personal information processing rules are notified by obvious signs / information, the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0291] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A pedestrian dead reckoning method, characterized in that: include: Obtaining first waist motion characteristic data of the target person; Processing the first waist movement feature data by a pre-trained long short-term memory recurrent neural network to obtain a first step prediction result of the target person; According to the first step prediction result and the first waist movement characteristic data, the stride length of the target person is estimated to obtain a stride length estimation result of the target person; The position information of the target person is updated according to the initial position information of the target person, the heading angle estimation result of the target person and the step length estimation result.

2. The method according to claim 1, characterized in that The step of obtaining the first waist motion characteristic data of the target person includes: Collecting first waist motion data of the target person through a waist-mounted inertial measurement unit; Based on the first waist motion data, first waist motion feature data of the target person is extracted.

3. The method according to claim 2, characterized in that The first waist motion data includes a first waist acceleration and a first waist angular velocity; The extracting the first waist motion feature data of the target person based on the first waist motion data includes: Performing continuous wavelet transform based on the first waist acceleration to obtain first step feature data of the target person; Based on the first waist acceleration, the first waist angular velocity and the first step characteristic data, first waist movement characteristic data of the target person is obtained.

4. The method according to claim 2, characterized in that After collecting the first waist motion data of the target person by the waist-mounted inertial measurement unit, and before processing the first waist motion feature data by the pre-trained long short-term memory recursive neural network, the method further includes: Based on a sliding window with a preset length and a preset overlap rate, data extraction is performed on the first waist motion data, and secondary sampling is performed at intervals of a preset number of groups.

5. The method according to any one of claims 1 to 4, characterized in that: Before the first waist movement feature data is processed by the pre-trained long short-term memory recurrent neural network, the method further includes: Acquire a training data set, wherein the training data set includes second waist motion data of a plurality of persons; extracting second waist motion feature data based on the second waist motion data; The long short-term memory recursive neural network is trained based on the second waist movement feature data.

6. The method according to claim 1, characterized in that The first step prediction result includes the probability that the target person has multiple preset gaits; The plurality of preset gaits include at least some of the following: standing, walking, walking backward, running, going up stairs, going down stairs, falling down, and jumping.

7. The method according to claim 1, characterized in that The step estimating the stride length of the target person according to the first step prediction result and the first waist movement characteristic data to obtain the step length estimation result of the target person includes: Determining a target step length estimation model from at least two adaptive step length estimation models according to the first step prediction result; The target stride length estimation model is used to estimate the stride length of the target person based on the first waist motion feature data to obtain a stride length estimation result of the target person.

8. The method according to claim 7, characterized in that Before determining the target step length estimation model from at least two adaptive step length estimation models according to the first step prediction result, the method further includes: Use an optical motion capture system and a waist-mounted inertial measurement unit to collect motion data of people at different gaits, and extract the kinematic and dynamic characteristics of people; Based on the kinematic characteristics and the dynamic characteristics, an adaptive step length estimation model corresponding to different gaits is constructed.

9. The method according to claim 7, characterized in that: Determining a target step length estimation model from at least two adaptive step length estimation models according to the first step prediction result includes: In response to the first step prediction result indicating that the current gait of the target person is clear, a target step length estimation model is determined from at least two adaptive step length estimation models according to the first step prediction result.

10. The method according to any one of claims 7 to 9, characterized in that The at least two adaptive step-size estimation models include at least some of the following types: Statistical regression model, sports physiological model, direct integration model based on center of mass acceleration.

11. The method according to claim 7, characterized in that The step estimating the stride length of the target person according to the first step prediction result and the first waist movement characteristic data to obtain the step length estimation result of the target person includes: In response to the first step prediction result indicating that the current gait of the target person is blurred, the step length of the target person is estimated based on the first waist motion feature data using a multi-gait universal step length estimation model to obtain a step length estimation result of the target person.

12. The method according to claim 11, characterized in that Before estimating the stride length of the target person based on the first waist motion feature data using the multi-gait universal stride length estimation model, the method further includes: Using an optical motion capture system and a waist-mounted inertial measurement unit to collect motion data of a person at different gaits, and extracting specified features related to step length based on the motion data of the person at different gaits; Analyzing the correlation coefficient between the specified feature and the step length; According to the correlation coefficient, a multi-gait universal stride length estimation model is constructed.

13. The method according to claim 12, characterized in that The specified features include at least some of the following: Cadence, maximum value of waist acceleration modulus, minimum value of waist acceleration modulus, standard deviation of waist acceleration.

14. A pedestrian dead reckoning device, characterized in that: include: An acquisition module, used for acquiring first waist motion characteristic data of a target person; A processing module, used for processing the first waist movement feature data through a pre-trained long short-term memory recursive neural network to obtain a first step prediction result of the target person; a step length estimation module, used to estimate the step length of the target person according to the first step prediction result and the first waist movement characteristic data, and obtain a step length estimation result of the target person; An updating module is used to update the position information of the target person according to the initial position information of the target person, the heading angle estimation result of the target person and the step length estimation result.

15. An electronic device, characterized in that: include: one or more processors; a memory for storing executable instructions; The one or more processors are configured to call the executable instructions stored in the memory to execute the method described in any one of claims 1 to 13.

16. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 13 is implemented.

17. A computer program product comprising computer readable code, or a non-volatile computer readable storage medium carrying computer readable code, characterized in that: When the computer readable code is executed in an electronic device, a processor in the electronic device executes the method according to any one of claims 1 to 13.

Images