Pedestrian positioning method based on fusion of millimeter wave radar and inertial measurement unit

Abstract

The application provides a pedestrian positioning method fusing a millimeter wave radar and an inertial measurement unit, and relates to the technical field of indoor positioning. The method comprises the following steps: determining a target gait according to target motion data of a pedestrian, wherein the target motion data comprises target acceleration, target angular velocity and target speed; the target acceleration and the target angular velocity are determined according to inertial measurement data output by an inertial measurement unit; the target speed is determined according to radar point cloud data output by a millimeter wave radar; determining a current position of the pedestrian according to the target gait, wherein the current position of the pedestrian is determined according to a target step length parameter corresponding to the target gait. The method can identify the gait with close motion states by introducing radar speed information, combining acceleration and angular velocity for gait recognition, thereby determining corresponding step length estimation parameters, so as to improve the accuracy of pedestrian positioning.

Description

Technical Field

[0001] This invention relates to the field of indoor positioning technology, and in particular to a pedestrian positioning method that integrates millimeter-wave radar and inertial measurement unit. Background Technology

[0002] Pedestrian Dead Reckoning (PDR) utilizes sensor information such as accelerometers, gyroscopes, and magnetometers to calculate the number of steps, stride length, and direction of travel for a pedestrian. It then updates the pedestrian's position based on the stride length and direction of each step. A typical PDR algorithm includes stride detection, stride length estimation, heading update, and position update. Current technologies, after detecting a pedestrian's strides, usually determine the gait based on the pedestrian's acceleration and body parameters, and then estimate the stride length based on the gait.

[0003] Because pedestrians' movement states are complex and change frequently, and pedestrian body parameters vary from person to person, existing pedestrian positioning technologies have difficulty distinguishing gaits with similar movement states. Summary of the Invention

[0004] This invention provides a pedestrian localization method that integrates millimeter-wave radar and inertial measurement unit to address the shortcomings of existing pedestrian localization methods that struggle to distinguish gaits with similar motion states. By introducing radar velocity information and combining acceleration and angular velocity for gait recognition, it is possible to identify gaits with similar motion states and thus determine the corresponding step length estimation parameters, thereby improving the accuracy of pedestrian localization.

[0005] This invention provides a pedestrian localization method that integrates millimeter-wave radar and inertial measurement unit, comprising the following steps:

[0006] Based on the pedestrian's target motion data, the target gait is determined. The target motion data includes target acceleration, target angular velocity, and target velocity. The target acceleration and target angular velocity are determined based on inertial measurement data output by the inertial measurement unit, and the target velocity is determined based on radar point cloud data output by the millimeter-wave radar. The timestamp of the target velocity corresponds to the timestamps of the target angular velocity and the target acceleration. The pedestrian carries the millimeter-wave radar and the inertial measurement unit.

[0007] The pedestrian's current position is determined based on the target gait, which is determined according to the target stride length parameter corresponding to the target gait.

[0008] According to the pedestrian localization method fused with millimeter-wave radar and inertial measurement unit provided by the present invention, before determining the target gait based on the pedestrian's target motion data, the method further includes:

[0009] The initial radar velocity is determined based on the radar point cloud data. The initial radar velocity is the radar's own velocity calculated from the Doppler velocity and normalized distance of each radar point cloud in the radar point cloud data.

[0010] The initial radar velocity is segmented into cubic Hermitian interpolation to obtain the target velocity. The timestamp of the target velocity is aligned with the timestamps of the target acceleration and the target angular velocity. The cubic Hermitian interpolation is used to construct a cubic polynomial between each adjacent interpolation node, and each interpolation node corresponds to the time point of the initial radar velocity acquisition.

[0011] According to the present invention, a pedestrian positioning method integrating millimeter-wave radar and inertial measurement unit is provided, wherein determining the initial radar velocity based on the radar point cloud data includes:

[0012] The inner point set acquisition step is repeatedly performed on the current frame point cloud data in the radar point cloud data to obtain multiple inner point sets corresponding to the current frame. The inner point set acquisition step includes: determining the initial radar velocity based on the target point cloud data in the current frame point cloud data, wherein the target point cloud data is the data of a randomly selected number of target radar point clouds; determining the inner point set corresponding to the current frame based on the initial radar velocity, wherein the inner points in the inner point set are point clouds obtained by filtering based on the velocity deviation between other point clouds and the initial radar velocity, and a preset deviation threshold, wherein the other point clouds are point clouds in the current frame point cloud data other than the target point cloud;

[0013] The initial radar velocity is determined based on the plurality of interior point sets.

[0014] According to the present invention, a pedestrian localization method integrating millimeter-wave radar and inertial measurement unit is provided, wherein the current frame point cloud data is obtained by filtering the radar point cloud data based on target features, and the target features include at least one of signal-to-noise ratio, distance, and angle.

[0015] According to the present invention, a pedestrian localization method integrating millimeter-wave radar and inertial measurement unit is provided, wherein determining the initial radar velocity based on the plurality of interior point sets includes:

[0016] The reference radar velocity is determined based on the target interior point set in the plurality of interior point sets, wherein the target interior point set is the interior point set with the largest number of point clouds included in the plurality of interior point sets.

[0017] The initial radar velocity is obtained by optimizing the reference radar velocity by establishing the optimization parameters of the angle error and velocity error of the radar measurement.

[0018] According to the pedestrian localization method fused with millimeter-wave radar and inertial measurement unit provided by the present invention, before determining the target gait based on the pedestrian's target motion data, the method further includes:

[0019] The inertial measurement data is windowed to obtain windowed inertial data;

[0020] Gaussian filtering is applied to the windowed inertial data to obtain the target acceleration and the target angular velocity.

[0021] According to the present invention, a pedestrian localization method integrating millimeter-wave radar and inertial measurement unit is provided, wherein determining the target gait based on the pedestrian's target motion data includes:

[0022] The target motion data is input into the trained gait recognition network to obtain multiple sets of gait probability values. The trained gait recognition network is obtained by training an initial gait recognition network with motion data samples. The motion data samples include filtered acceleration, filtered angular velocity, interpolated radar velocity, and marked gait labels.

[0023] The target gait is obtained by performing mean filtering, threshold judgment, and continuity judgment on three temporally continuous gait probability values ​​from the multiple sets of gait probability values.

[0024] According to the present invention, a pedestrian localization method integrating millimeter-wave radar and inertial measurement unit is provided, wherein determining the current position of the pedestrian based on the target gait includes:

[0025] Determine whether the target gait matches the preset gait;

[0026] If the target gait matches the preset gait, obtain the calibration step length parameters, and determine the target step length parameters corresponding to the step length model as the calibration step length parameters;

[0027] If the target gait does not match the preset gait, the target step length parameter corresponding to the step length model is determined as the current step length parameter;

[0028] The current position is determined based on the target step size parameter.

[0029] According to the present invention, a pedestrian localization method integrating millimeter-wave radar and inertial measurement unit is provided, wherein obtaining calibration step size parameters includes:

[0030] Using the initial radar velocity as an observation, the velocity integrated in the inertial measurement data is corrected by an extended Kalman filter based on the error state to obtain the actual distance.

[0031] The calculated trajectory is obtained based on the target angular velocity, the target acceleration, and the preset fixed step size parameters;

[0032] The calibration step length parameters are determined based on the actual distance traveled and the calculated trajectory.

[0033] According to the present invention, a pedestrian localization method integrating millimeter-wave radar and inertial measurement unit is provided, wherein the target step length parameters corresponding to the determined step length model are calibration step length parameters, including:

[0034] Obtain the parameter change values ​​of the calibrated step length parameters and the step length parameters corresponding to the target gait, and compare the parameter change values ​​with a preset change value threshold;

[0035] If the change value of the parameter is less than the change value threshold, the calibrated step size parameter and the step size parameter corresponding to the target gait are smoothed to obtain the target step size parameter.

[0036] If the change value of the parameter is greater than or equal to the change value threshold, the step length parameter corresponding to the target gait is updated to the calibrated step length parameter to obtain the target step length parameter.

[0037] According to the present invention, a pedestrian localization method integrating millimeter-wave radar and inertial measurement unit is provided, wherein determining the current position based on the target step length parameter includes:

[0038] Obtain a first peak value and a minimum peak value, wherein the first peak value is the first peak value of the target acceleration, and the minimum peak value is the minimum distance between two consecutive peak values ​​of the target acceleration;

[0039] The first peak value and the minimum peak value are input into the current step size model to obtain the step size estimate, wherein the current step size model adopts the target step size parameter;

[0040] The current position is determined based on the pedestrian's direction of travel and the estimated step length.

[0041] The present invention also provides a pedestrian positioning device that integrates millimeter-wave radar and inertial measurement unit, comprising the following modules:

[0042] A gait determination module is used to determine the target gait based on the pedestrian's target motion data. The target motion data includes target acceleration, target angular velocity, and target velocity. The target acceleration and target angular velocity are determined based on inertial measurement data output by an inertial measurement unit, and the target velocity is determined based on radar point cloud data output by a millimeter-wave radar. The timestamp of the target velocity corresponds to the timestamps of the target angular velocity and the target acceleration. The pedestrian carries the millimeter-wave radar and the inertial measurement unit.

[0043] The position determination module is used to determine the current position of the pedestrian based on the target gait, wherein the current position of the pedestrian is determined based on the target step length parameter corresponding to the target gait.

[0044] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a pedestrian positioning method that fuses millimeter-wave radar and an inertial measurement unit as described above.

[0045] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a pedestrian positioning method that fuses millimeter-wave radar and inertial measurement unit as described above.

[0046] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements a pedestrian positioning method that fuses millimeter-wave radar and inertial measurement unit as described above.

[0047] The pedestrian positioning method, device, electronic device, and storage medium provided by this invention, by introducing radar velocity information and combining acceleration and angular velocity for gait recognition, can identify gaits with similar motion states, thereby determining the corresponding step length estimation parameters to improve the accuracy of pedestrian positioning. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0049] Figure 1 This is one of the flowcharts illustrating the pedestrian positioning method provided by the present invention.

[0050] Figure 2 This is the second flowchart of the pedestrian positioning method provided by the present invention.

[0051] Figure 3 This is a flowchart illustrating the method for obtaining initial radar velocity provided by the present invention.

[0052] Figure 4 This is a flowchart illustrating the method for determining the target gait provided by the present invention.

[0053] Figure 5 This is a flowchart of the millimeter-wave radar-assisted pedestrian motion state detection and recognition algorithm provided by the present invention.

[0054] Figure 6 This is an example diagram of the long short-term memory recurrent neural network structure provided by the present invention.

[0055] Figure 7 This is one of the flowcharts illustrating the method for determining the current position provided by the present invention.

[0056] Figure 8 This is a flowchart of the online calibration method for pedestrian gait models provided by the present invention.

[0057] Figure 9 This is a schematic diagram of the trajectory output of the radar inertial odometer provided by the present invention.

[0058] Figure 10 This is the second flowchart illustrating the method for determining the current position provided by the present invention.

[0059] Figure 11 This is a schematic diagram of a pedestrian navigation system that integrates millimeter-wave radar and inertial measurement unit provided by the present invention.

[0060] Figure 12 This is a schematic diagram of the hardware design of the millimeter-wave radar and inertial measurement unit fusion system provided by the present invention.

[0061] Figure 13 This is a schematic diagram of the pedestrian positioning device provided in an embodiment of the present invention.

[0062] Figure 14 A schematic diagram of the physical structure of the electronic device provided by the present invention. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0064] Pedestrian Dead Reckoning (PDR) utilizes sensor information such as accelerometers, gyroscopes, and magnetometers to calculate the number of steps, stride length, and direction of travel for a pedestrian. It then updates the pedestrian's position based on the stride length and direction of each step. A typical PDR algorithm includes stride detection, stride length estimation, heading update, and position update. Current technologies, after detecting a pedestrian's strides, usually determine the gait based on the pedestrian's acceleration and body parameters, and then estimate the stride length based on the gait.

[0065] Because pedestrians' movement states are complex and change frequently, and pedestrian body parameters vary from person to person, existing pedestrian positioning technologies have difficulty distinguishing gaits with similar movement states.

[0066] In view of this, embodiments of the present invention provide a pedestrian localization method integrating millimeter-wave radar and inertial measurement unit (IMU). Based on the pedestrian's target motion data, the method determines the target gait, including target acceleration, target angular velocity, and target velocity. The target acceleration and target angular velocity are determined based on inertial measurement data output by the IMU, and the target velocity is determined based on radar point cloud data output by the millimeter-wave radar. Based on the target gait, the method determines the pedestrian's current position, which is determined based on the target step length parameter corresponding to the target gait. This method can identify gaits with similar motion states, thereby determining the corresponding step length estimation parameters to improve the accuracy of pedestrian localization.

[0067] The technical solutions of the present invention will now be described with reference to the accompanying drawings in the embodiments of the present invention.

[0068] Figure 1 This is one of the flowcharts illustrating the pedestrian positioning method provided by the present invention. The pedestrian positioning method, which integrates millimeter-wave radar and an inertial measurement unit, can be applied to electronic devices, which can be various types of devices with information processing capabilities. For example, the electronic device may include a personal computer, laptop, handheld computer, or server; the electronic device may also be a mobile terminal, such as a mobile phone, in-vehicle computer, tablet computer, or projector. Figure 1 As shown, the method may include the following steps 101 to 102:

[0069] Step 101: Determine the target gait based on the pedestrian's target motion data. The target motion data includes target acceleration, target angular velocity, and target velocity. The target acceleration and target angular velocity are determined based on inertial measurement data output by the inertial measurement unit, and the target velocity is determined based on radar point cloud data output by the millimeter-wave radar. The timestamp of the target velocity corresponds to the timestamps of the target angular velocity and the target acceleration.

[0070] It should be noted that the target acceleration and target angular velocity are determined based on inertial measurement data output by the inertial measurement unit (IMU), which can be obtained through the IMU. The target velocity is determined based on radar point cloud data output by the millimeter-wave radar, which can be obtained through the millimeter-wave radar unit. The timestamps for the target acceleration, target angular velocity, and target velocity correspond and can be used for gait recognition.

[0071] There are many ways to determine the target gait based on the pedestrian's target motion data. For example, it can be calculated by a gait recognition algorithm or obtained by a gait recognition model. This invention does not limit the method of determining the target gait based on the pedestrian's target motion data.

[0072] Step 102: Determine the current position of the pedestrian based on the target gait. The current position of the pedestrian is determined based on the target step length parameter corresponding to the target gait.

[0073] It should be noted that after determining the target gait, the current step length can be calculated based on the target gait, and the pedestrian's current position can be determined based on the current step length. Alternatively, after determining the target gait, the current position can be determined by using a model that determines the current position. This invention does not limit the method of determining the pedestrian's current position based on the target gait.

[0074] This invention addresses the problem of existing technologies' inability to distinguish gaits with similar motion characteristics. It proposes an indoor method for pedestrian motion state detection and identification based on the fusion of millimeter-wave radar data and inertial measurement unit data. The target gait can be used for pedestrian localization, and the localization results can be used for pedestrian navigation. This method can identify gaits with similar motion states, thereby determining the corresponding step length estimation parameters to improve the accuracy of pedestrian localization.

[0075] Figure 2 This is the second flowchart illustrating the pedestrian positioning method provided by this invention. Figure 2 As shown, the pedestrian localization method fused with millimeter-wave radar and inertial measurement unit may include the following steps 201 to 204:

[0076] Step 201: Determine the initial radar velocity based on the radar point cloud data. The initial radar velocity is the radar's own velocity calculated from the Doppler velocity and normalized distance of each radar point cloud in the radar point cloud data.

[0077] It should be noted that the initial radar velocity can be determined based on the radar point cloud data using methods such as Radar Ego Velocity Estimation (REVE). This invention does not limit the method of determining the initial radar velocity based on the radar point cloud data.

[0078] Step 202: Perform piecewise cubic Hermitian interpolation on the initial radar velocity to obtain the target velocity. The timestamp of the target velocity is aligned with the timestamps of the target acceleration and the target angular velocity. The cubic Hermitian interpolation is used to construct a cubic polynomial between each adjacent interpolation node. Each interpolation node corresponds to the time point of the initial radar velocity acquisition.

[0079] It should be noted that the angular velocity and acceleration information can be the inertial data output from the inertial sensor, with a frequency of 100Hz. The velocity information can be obtained through the Radar Ego Velocity Estimation (REVE) algorithm. This algorithm can obtain the velocity of the millimeter-wave radar itself based on the radar data from the millimeter-wave radar sensor, with a frequency of 5Hz. The velocity information is then aligned with the angular velocity and acceleration information according to the timestamp using a piecewise cubic Hermite interpolation method.

[0080] Step 203: Determine the target gait based on the pedestrian's target motion data. The target motion data includes target acceleration, target angular velocity, and target velocity. The target acceleration and target angular velocity are determined based on inertial measurement data output by the inertial measurement unit, and the target velocity is determined based on radar point cloud data output by the millimeter-wave radar. The timestamp of the target velocity corresponds to the timestamps of the target angular velocity and the target acceleration.

[0081] The description of step 203 can be found in the description of step 101 in the foregoing embodiments.

[0082] Step 204: Determine the current position of the pedestrian based on the target gait. The current position of the pedestrian is determined based on the target step length parameter corresponding to the target gait.

[0083] The description of step 204 can be found in the description of step 102 in the foregoing embodiments.

[0084] This invention proposes a millimeter-wave radar-assisted method for pedestrian motion state detection and recognition. It determines the target's gait by combining the radar's own velocity estimation results after piecewise cubic Hermite interpolation with inertial measurement results to form target motion data. This improves the accuracy of acquiring the target's gait.

[0085] In some embodiments, step 201, determining the initial radar velocity based on the radar point cloud data, may include: repeatedly performing the inner point set acquisition step multiple times on the current frame point cloud data in the radar point cloud data to obtain multiple inner point sets corresponding to the current frame. The inner point set acquisition step includes: determining the initial radar velocity based on the target point cloud data in the current frame point cloud data, wherein the target point cloud data is data of a randomly selected number of target radar point clouds; determining the inner point set corresponding to the current frame based on the initial radar velocity, wherein the inner points in the inner point set are point clouds filtered based on the velocity deviation between other point clouds and the initial radar velocity, and a preset deviation threshold, wherein the other point clouds are point clouds in the current frame point cloud data other than the target point cloud; and determining the initial radar velocity based on the multiple inner point sets.

[0086] Furthermore, determining the initial radar velocity based on the plurality of interior point sets may include: determining a reference radar velocity based on a target interior point set within the plurality of interior point sets, wherein the target interior point set is the interior point set with the largest number of point clouds included in the plurality of interior point sets; and optimizing the reference radar velocity by establishing optimization parameters for the angle error and velocity error of radar measurements to obtain the initial radar velocity.

[0087] Furthermore, the current frame point cloud data is obtained by filtering the radar point cloud data based on target features, which include at least one of signal-to-noise ratio, distance, and angle.

[0088] Understandably, using the above method to obtain the initial radar velocity can improve the accuracy of obtaining the target's gait.

[0089] Figure 3 This is a flowchart illustrating the method for obtaining initial radar velocity provided by the present invention. Figure 3 As shown, the target velocity can be obtained using the REVE algorithm. The algorithm process includes: firstly, processing the raw data output of the millimeter-wave radar development board's point cloud; then, filtering each point cloud in a single frame of data based on its signal-to-noise ratio, range, and angle. For example, four point clouds are randomly selected from the filtered point clouds, and their Doppler velocities are used to determine the velocity. Distance from normalization The optimal radar speed is obtained using the least squares method. .

[0090] , .

[0091] in, In the point cloud representing radar measurements Doppler velocity value; In the point cloud Dot at Distance in direction , Similarly; Indicating the calculated point cloud Point speed value, , Similarly.

[0092] Calculate the velocity deviation of other points in the point cloud, i.e.:

[0093] .

[0094] A threshold for deviation is set, and the point cloud data that passes through it forms a random set of interior points. The above steps are repeated 34 times in this implementation case. The set of interior points with the largest number of interior points is extracted, and the radar velocity is calculated from this set using the least squares method. The quantities to be optimized for radar measurement angle error and velocity error are established:

[0095] .

[0096] in, , , The definition is the same as above. For point The true value of the azimuth angle, For point The azimuth angle measurement value. The velocity of this formula has been obtained through the above steps. The quantities to be optimized are N X values. These N summation quantities can be optimized separately using methods such as Newton-Gauss method and Levenberg-Marquardt. After updating the true values ​​of the N azimuth angles, the velocity is then... , Optimization is performed to obtain the final radar velocity estimate, i.e., the target velocity.

[0097] In some embodiments, before determining the target gait based on the pedestrian's target motion data, the method may further include: windowing the inertial measurement data to obtain windowed inertial data; and performing Gaussian filtering on the windowed inertial data to obtain the target acceleration and the target angular velocity.

[0098] Understandably, using the above methods to obtain target acceleration and target angular velocity can improve the accuracy of obtaining target gait.

[0099] Figure 4 This is a flowchart illustrating the method for determining the target gait provided by the present invention. Figure 4 As shown, determining the target gait based on the pedestrian's target motion data may include:

[0100] Step 301: Input the target motion data into the trained gait recognition network to obtain multiple sets of gait probability values. The trained gait recognition network is obtained by training the initial gait recognition network with motion data samples. The motion data samples include filtered acceleration, filtered angular velocity, interpolated radar velocity, and marked gait labels.

[0101] It should be noted that the gait recognition network can use a Long Short-Term Memory (LSTM) recurrent neural network. The network input consists of three-axis angular velocity, three-axis acceleration, and three-axis velocity information. These three angular velocity, acceleration, and velocity information are combined into nine-dimensional data as training data for the LSTM network. In gait design, the gait patterns that significantly influence the stride length model parameters are selected. The millimeter-wave radar-assisted pedestrian motion state detection and recognition algorithm uses the LSTM network as its core. First, the raw inertial data is windowed and then Gaussian filtered. The filtered data is then combined with the millimeter-wave radar velocity information and input into the network for prediction.

[0102] The target motion data can be acquired using a millimeter-wave radar-inertial sensor, which integrates millimeter-wave radar and an inertial sensor. The target motion data output by the millimeter-wave radar-inertial sensor can include three-axis angular velocity, three-axis acceleration, and three-axis velocity information. The trained gait recognition network identifies the pedestrian's motion state based on their three-axis acceleration, three-axis angular velocity, and three-axis velocity. Acceleration and angular velocity are obtained through the inertial measurement unit within the millimeter-wave radar-inertial sensor, while velocity is obtained through the radar unit within the same sensor.

[0103] This invention can train a Long Short Term Memory (LSTM) recurrent neural network using the radar's own velocity estimation results after piecewise cubic Hermite interpolation and inertial measurement results, and use this network as the core to determine the pedestrian's motion state.

[0104] Step 302: Perform mean filtering, threshold judgment, and continuity judgment on the three gait probability values ​​that are temporally continuous among the multiple sets of gait probability values ​​to obtain the target gait.

[0105] It should be noted that the network outputs the discrimination probability for each gait, which can be used to remove outliers. Specific operations include mean filtering, thresholding, and continuity testing. Finally, the results are written to an Excel spreadsheet and plotted.

[0106] Figure 5This is a flowchart of the millimeter-wave radar-assisted pedestrian motion state detection and recognition algorithm provided by this invention. Figure 5 As shown, after obtaining the radar's own velocity estimate, i.e., the initial radar velocity, the millimeter-wave radar-assisted pedestrian motion state detection and recognition algorithm windows the 100Hz inertial measurement unit with a window length of 100. Then, a Gaussian filter with sigma=1.0 is applied to the data within the window to complete the raw processing of the inertial data. The output frequency of the initial radar velocity data is limited by the point cloud output frequency of the millimeter-wave radar; in this implementation, it is set to 5Hz. The velocity frequency is aligned with the inertial data based on timestamps using a piecewise cubic Hermite interpolation method. Within a window, there are 100*9 data points, including filtered inertial data (three-axis acceleration and three-axis angular velocity) and interpolated radar velocity (three-axis radar own velocity). 50*9 radar-inertial data points are taken from every other row and packaged together, with the gait of each data point manually labeled in a CSV file. The ground truth gait value and the 50*9 radar-inertial data points serve as inputs for neural network training.

[0107] Figure 6 This is an example diagram of the long short-term memory recurrent neural network structure provided by the present invention. Figure 6 As shown, the training and test datasets for the network are randomly and non-repeatingly obtained from the collected radar-inertial data. To prevent overfitting, training is terminated when the network's prediction accuracy on the test set improves by less than 0.1 in two consecutive rounds. In the outlier removal section, the gait probabilities of three consecutive time-series sets are mean-filtered. After filtering, the data is thresholded, and predictions with a maximum probability less than 0.7 are considered as other gait outputs. The data after thresholding undergoes continuity testing, and the data with the same prediction output across the three consecutive time-series sets are finally output. In this section, the removed data is considered as gaits other than the specified gaits. The probabilities of each gait are recorded in chronological order in an .excel file for easy integration with other navigation algorithms. The gait recognition results and probability distributions for each time period are also output as graphs. In this implementation, four gaits are labeled: small steps, large steps, backward steps, and running.

[0108] This invention provides a gait recognition network training method and a millimeter-wave radar-assisted pedestrian motion state detection and recognition algorithm. The algorithm incorporates radar velocity information and utilizes an LSTM network to accurately identify similar gaits of pedestrians, such as small steps, large steps, running, and backward walking. The algorithm outputs at a frequency of 1Hz, achieving real-time output in pedestrian navigation scenarios. Furthermore, due to the inclusion of velocity information, this method can accurately identify various gaits that are difficult to recognize using traditional inertial information, such as going up and down stairs, demonstrating promising application prospects in the field of gait recognition.

[0109] Figure 7 This is one of the flowcharts illustrating the method for determining the current position provided by this invention. For example... Figure 7 As shown, determining the pedestrian's current position based on the target gait may include:

[0110] Step 401: Determine whether the target gait matches the preset gait;

[0111] Step 402: If the target gait matches the preset gait, obtain the calibration step length parameter and determine the target step length parameter corresponding to the step length model as the calibration step length parameter; if the target gait does not match the preset gait, determine the target step length parameter corresponding to the step length model as the current step length parameter.

[0112] Step 403: Determine the current position based on the target step size parameter.

[0113] It should be noted that existing step size models lack generalization ability and require extensive step size model parameter calibration. The calibration method provided by this invention can be implemented entirely online. It outputs the pedestrian's motion state through a millimeter-wave radar-assisted pedestrian motion state detection and recognition algorithm, and uses a threshold to determine whether the gait is standard. If the threshold is not met, the calibrated parameters will not be updated; if the threshold is met, the calibrated step size parameters are obtained.

[0114] Understandably, the above method allows for parameter recalibration when necessary, improving the efficiency of obtaining the current location of pedestrians.

[0115] In some embodiments, obtaining the calibration step size parameter may include: using the initial radar velocity as an observation, correcting the velocity integral in the inertial measurement data based on the extended Kalman filter of the error state to obtain the actual distance; obtaining the calculated trajectory based on the target angular velocity, the target acceleration, and the preset fixed step size parameter; and determining the calibration step size parameter based on the actual distance and the calculated trajectory.

[0116] It should be noted that this invention proposes a generalizable online calibration method for pedestrian gait models assisted by millimeter-wave radar. The method uses millimeter-wave radar inertial odometry (RIO) to calibrate, optimize, and update the step length parameters of different gait states and different pedestrians online, while storing historical records for parameter optimization.

[0117] Among them, an online calibration method for step length model parameters is based on radar inertial odometry implemented with millimeter-wave radar and inertial measurement unit. The radar inertial odometry estimates its own velocity based on millimeter-wave radar point cloud information using the REVE algorithm, and fuses radar velocity information with inertial information using error state Kalman filter (ESKF) to realize radar inertial odometry (RIO). The pedestrian step length parameters are calibrated based on the trajectory output of RIO combined with step detection results.

[0118] In some embodiments, determining the target step length parameter corresponding to the step length model as a calibration step length parameter may include: obtaining the parameter change value of the calibration step length parameter and the step length parameter corresponding to the target gait, and comparing the parameter change value with a preset change value threshold; if the parameter change value is less than the change value threshold, smoothing the calibration step length parameter and the step length parameter corresponding to the target gait to obtain the target step length parameter; if the parameter change value is greater than or equal to the change value threshold, updating the step length parameter corresponding to the target gait to the calibration step length parameter to obtain the target step length parameter.

[0119] It should be noted that the entire calibration process can be performed online. During the movement of a pedestrian wearing a millimeter-wave radar-inertial sensor, the motion state detection and recognition algorithm identifies their motion state and determines whether to update the corresponding gait stride parameters based on the recognition results and calibration quality verification results. Each updated calibration result is compared with historical results. If the individuals are different, the stride parameters are updated directly; if the individuals are the same, smoothing is considered before updating.

[0120] This application provides a high-precision indoor positioning service for pedestrian multigait and presents a generalizable online gait model calibration method, which significantly improves the accuracy of pedestrian multigait motion navigation while greatly simplifying the calibration steps.

[0121] It is understood that the pedestrian gait model calibration method in this invention is simple, requiring only a small millimeter-wave radar-inertial sensor module. The entire calibration process is completed online, eliminating the need for additional calibration steps by the pedestrian and enabling rapid updates to various gait parameters. This calibration process considers historical calibration results; by comparing the updated parameters with historical parameters, it allows for the gradual optimization of gait parameters for the same individual and direct updates of gait parameters for different individuals. The online calibration method proposed in this invention significantly simplifies the system calibration process and enables rapid deployment.

[0122] Figure 8This is a flowchart of the online calibration method for pedestrian gait models provided by this invention. Figure 8 As shown, an online calibration method for step-size model parameters is based on a radar inertial odometry system implemented using millimeter-wave radar and an inertial measurement unit (IMU). The radar IMU uses the results of the REVE algorithm as the observation, and corrects the velocity integrated by the IMU using an error-state extended Kalman filter (ESKF). By updating the inertial data, the velocity in the navigation system can be obtained. The radar's own velocity in a radar system can be obtained using Euler's equations. Using the REVE algorithm, the estimated radar velocity is obtained. The difference between the two is the residual of the radar velocity estimate:

[0123] .

[0124] Linearizing the velocity residuals yields the Jacobian matrix for the velocity measurement:

[0125] ,

[0126]

[0127]

[0128] .

[0129] in, This is the linearized velocity measurement matrix. For attitude measurement matrix, This is a barometric measurement matrix. This is the transpose of the transformation matrix from the carrier coordinate system to the radar coordinate system. This is the transpose of the transformation matrix from the navigation coordinate system to the vehicle coordinate system. For speed estimation under navigation system, This is the translation vector from the carrier coordinate system to the radar coordinate system.

[0130] The velocity measurement is updated to remove outliers based on the Mahalanobis distance, which is calculated as follows:

[0131]

[0132] .

[0133] in, , Same as the definition above, The variance of radar velocity measurements. For the filter at time Predicted time The covariance matrix.

[0134] A threshold is set based on the chi-square distribution for Mahalanobis distance. A judgment is made, and values ​​below a threshold are considered usable. The Kalman filter ESKF based on the error state is then updated based on this residual.

[0135] Figure 9 This is a schematic diagram of the trajectory output of the radar inertial odometer provided by the present invention. For example... Figure 9 The image shows the pedestrian trajectory output by a radar inertial odometry system. Taking the calibration of the Weinburg model as an example, the odometry output trajectory is taken as the actual distance, and the K value in the model is fixed to a specific value, such as K=0.3. The distance output solely based on the inertial data is... The odometer output distance is The parameters obtained from the calibration are:

[0136] .

[0137] The calibration method proposed in this invention is implemented entirely online. It outputs the pedestrian's motion state through a millimeter-wave radar-assisted pedestrian motion state detection and recognition algorithm, and uses a threshold to determine whether the gait is standard. If the threshold is not met, the calibration parameters are not updated. After passing the threshold, the calibration result K is compared with the historical parameters of the corresponding gait. If the difference is large, it is considered that a new user is wearing the module, and the new calibration result is directly updated and used, achieving user-defined step length parameter selection. If the difference is small, it is considered that the same user is wearing the module, and the current calibration result is smoothed with historical results, such as by mean filtering, to gradually optimize the step length parameters for the same user. This online calibration method overcomes the shortcomings of existing technologies that are difficult to apply to different pedestrians and different gaits. Since the step length parameters of different users vary greatly, and the step length parameters of the same user also differ each time they use the device, this online step length calibration method allows the step length parameters of the same user to be adjusted online according to changes in the usage scenario, and allows for rapid switching of step length parameters between different users.

[0138] Figure 10 This is a second schematic flowchart of the method for determining the current position provided by the present invention. The step of determining the current position based on the target step size parameter may include:

[0139] Step 501: Obtain the first peak value and the minimum peak value, wherein the first peak value is the first peak value of the target acceleration, and the minimum peak value is the minimum distance between two consecutive peak values ​​of the target acceleration.

[0140] Step 502: Input the first peak value and the minimum peak value into the current step size model to obtain the step size estimate. The current step size model adopts the target step size parameter.

[0141] Step 503: Determine the current position based on the pedestrian's direction of travel and the estimated step length.

[0142] It should be noted that this invention addresses the problem of high-precision dead reckoning for pedestrians under various motion states, which is difficult to achieve with existing technologies. It proposes a pedestrian navigation method that integrates indoor millimeter-wave radar and inertial measurement unit (PDR). This method discloses a fusion algorithm design for millimeter-wave radar and PDR. This fusion navigation algorithm uses the PDR algorithm as its core, supplemented by a millimeter-wave radar-assisted pedestrian motion state detection and recognition algorithm to determine the pedestrian's gait. Combined with the step length model parameters calibrated by the radar inertial odometry, the pedestrian's dead reckoning is determined.

[0143] Figure 11 This is a schematic diagram of the pedestrian navigation system integrating millimeter-wave radar and inertial measurement unit provided by the present invention. Figure 11 As shown, the point cloud output by the millimeter-wave radar module is solved using the REVE algorithm to provide an estimate of the radar's own velocity. This estimate is then combined with the angular velocity and acceleration output by the IMU to produce the original data. Within the MCU processor, a millimeter-wave radar-assisted pedestrian motion state detection and recognition algorithm is used to determine the gait. Based on the determination results, specified gait step length parameters are calibrated online.

[0144] The IMU's acceleration information is used for gait determination. The specific steps are as follows: Two consecutive peak values ​​are found using a peak-finding algorithm, considered as one step for the pedestrian. The minimum value within this interval is taken as the minimum acceleration value for the step, and the first peak value is taken as the maximum acceleration value for the step. These values ​​are then substituted into the step size model. The IMU's angular velocity information is used for mechanical energy attitude updating. In this implementation, the attitude is updated using a single sample + previous cycle method, the principle of which is as follows:

[0145] .

[0146] in, The angle increment within the new angular velocity update cycle is used. Multiply by the difference between the current angular velocity and the timestamp of the previous angular velocity. get. The angular increment within the previous angular velocity update cycle, combined with This yields the equivalent rotation vector at that moment, allowing for the update of the pedestrian's pose. Combining the pedestrian's direction of travel and stride length, the pedestrian's position can then be updated.

[0147] .

[0148] in, for Current position facing north for Current position facing north , Similarly; for Heading angle at any time This is the step size for prediction.

[0149] It is understood that this invention discloses a fusion algorithm design for millimeter-wave radar and PDR, and designs a typical system. This algorithm design includes a millimeter-wave radar-assisted motion state detection and recognition algorithm and a PDR algorithm. By combining gait recognition results and online gait parameter calibration results, it improves the step length estimation in traditional PDR, achieving higher-precision navigation and positioning for pedestrians.

[0150] Furthermore, the pedestrian motion state detection and recognition method and PDR algorithm fused with millimeter-wave radar provided by this invention, combined with the online calibration results of the millimeter-wave radar-assisted generalizable pedestrian gait model, realizes a pedestrian positioning method that integrates millimeter-wave radar and inertial measurement unit, which can be used in pedestrian navigation methods.

[0151] Furthermore, this invention proposes a pedestrian navigation method that integrates indoor millimeter-wave radar and inertial measurement unit and designs a typical system. In terms of hardware, a millimeter-wave radar-inertial sensor module is designed. The integrated navigation system greatly improves the step length estimation accuracy in pedestrian dead reckoning and realizes high-precision dead reckoning function under multi-gait motion conditions of pedestrians of different heights, weights, ages and genders.

[0152] For example, in a typical hardware system design, a millimeter-wave radar-inertial sensor module is implemented. The module integrates the millimeter-wave radar and the inertial measurement unit onto a single circuit board, along with a heat sink and tooling. It also includes a geomagnetic module, a barometer, and a UWB module for subsequent development.

[0153] Figure 12 This is a schematic diagram of the hardware design of the millimeter-wave radar and inertial measurement unit fusion system provided by this invention. Figure 12 As shown, the hardware implementation scheme provided by this invention includes a UWB module, a millimeter-wave radar module, a geomagnetic module, a barometer module, and an IMU module. The millimeter-wave radar-inertial measurement module can be deployed at the waist of a pedestrian and can adapt to various pedestrian movements.

[0154] The pedestrian localization method provided by this invention can accurately locate pedestrians under various motion states, and different motion state recognition tags can be designed to meet the application requirements of different scenarios. This invention designs a typical system and provides a hardware implementation method reference for a millimeter-wave radar-inertial sensor module.

[0155] Based on the foregoing embodiments, this invention provides a pedestrian positioning device that integrates millimeter-wave radar and inertial measurement unit. The modules and units included in the device can be implemented by a processor; of course, they can also be implemented by specific logic circuits. In the implementation process, the processor can be a central processing unit (CPU), microprocessor (MPU), digital signal processor (DSP), or field programmable gate array (FPGA), etc.

[0156] The pedestrian positioning device provided by the present invention is described below. The pedestrian positioning device described below and the pedestrian positioning method described above can be referred to in correspondence.

[0157] Figure 13 This is a schematic diagram of the pedestrian positioning device provided in an embodiment of the present invention. Figure 13 As shown, the device 600 includes a gait determination module 601 and a position determination module 602, wherein:

[0158] The gait determination module 601 is used to determine the target gait based on the target motion data of the pedestrian. The target motion data includes target acceleration, target angular velocity, and target velocity. The target acceleration and target angular velocity are determined based on inertial measurement data output by the inertial measurement unit, and the target velocity is determined based on radar point cloud data output by the millimeter-wave radar. The timestamp of the target velocity corresponds to the timestamp of the target angular velocity and the target acceleration. The pedestrian carries the millimeter-wave radar and the inertial measurement unit.

[0159] The position determination module 602 is used to determine the current position of the pedestrian based on the target gait, wherein the current position of the pedestrian is determined based on the target step length parameter corresponding to the target gait.

[0160] In some embodiments, the apparatus further includes:

[0161] An initial velocity acquisition module is used to determine an initial radar velocity based on the radar point cloud data. The initial radar velocity is the radar's own velocity calculated based on the Doppler velocity and normalized distance of each radar point cloud in the radar point cloud data.

[0162] The target velocity acquisition module is used to perform piecewise cubic Hermitian interpolation on the initial radar velocity to obtain the target velocity. The timestamp of the target velocity is aligned with the timestamps of the target acceleration and the target angular velocity. The cubic Hermitian interpolation is used to construct a cubic polynomial between each adjacent interpolation node, and each interpolation node corresponds to the time point of the initial radar velocity acquisition.

[0163] In some embodiments, the initial velocity acquisition module is specifically used to: repeatedly perform the inner point set acquisition step multiple times on the current frame point cloud data in the radar point cloud data to obtain multiple inner point sets corresponding to the current frame. The inner point set acquisition step includes: determining an initial radar velocity based on the target point cloud data in the current frame point cloud data, wherein the target point cloud data is data of a randomly selected number of target radar point clouds; determining the inner point set corresponding to the current frame based on the initial radar velocity, wherein the inner points in the inner point set are point clouds filtered based on the velocity deviation between other point clouds and the initial radar velocity, and a preset deviation threshold, wherein the other point clouds are point clouds in the current frame point cloud data other than the target point cloud; and determining the initial radar velocity based on the multiple inner point sets.

[0164] In some embodiments, the current frame point cloud data is obtained by filtering the radar point cloud data based on target features, wherein the target features include at least one of signal-to-noise ratio, distance, and angle.

[0165] In some embodiments, the initial velocity acquisition module is further configured to: determine a reference radar velocity based on a target interior point set in the plurality of interior point sets, wherein the target interior point set is the interior point set with the largest number of point clouds included in the plurality of interior point sets; and optimize the reference radar velocity by establishing optimization parameters for the angle error and velocity error of radar measurements to obtain the initial radar velocity.

[0166] In some embodiments, the apparatus further includes:

[0167] Other data acquisition modules are used to window the inertial measurement data to obtain windowed inertial data; and to perform Gaussian filtering on the windowed inertial data to obtain the target acceleration and the target angular velocity.

[0168] In some embodiments, the gait determination module 601 includes:

[0169] The probability acquisition unit is used to input the target motion data into the trained gait recognition network to obtain multiple sets of gait probability values. The trained gait recognition network is obtained by training an initial gait recognition network with motion data samples. The motion data samples include filtered acceleration, filtered angular velocity, interpolated radar velocity, and marked gait labels.

[0170] The gait acquisition unit is used to perform mean filtering, threshold judgment, and continuity judgment on three temporally continuous gait probability values ​​from the multiple sets of gait probability values ​​to obtain the target gait.

[0171] In some embodiments, the position determination module 602 includes:

[0172] A gait determination unit is used to determine whether the target gait matches a preset gait.

[0173] The parameter determination unit is used to obtain the calibration step length parameter and determine the target step length parameter corresponding to the step length model as the calibration step length parameter if the target gait matches the preset gait; and to determine the target step length parameter corresponding to the step length model as the current step length parameter if the target gait does not match the preset gait.

[0174] The position determination unit is used to determine the current position based on the target step size parameter.

[0175] In some embodiments, the parameter determination unit is specifically used to: take the initial radar velocity as an observation, correct the velocity integral in the inertial measurement data based on the extended Kalman filter of the error state to obtain the actual distance; obtain the calculated trajectory based on the target angular velocity, the target acceleration and the preset fixed step size parameter; and determine the calibration step size parameter based on the actual distance and the calculated trajectory.

[0176] In some embodiments, the parameter determination unit is further configured to: obtain the parameter change value of the calibrated step length parameter and the step length parameter corresponding to the target gait, and compare the parameter change value with a preset change value threshold; if the parameter change value is less than the change value threshold, smooth the calibrated step length parameter and the step length parameter corresponding to the target gait to obtain the target step length parameter; if the parameter change value is greater than or equal to the change value threshold, update the step length parameter corresponding to the target gait to the calibrated step length parameter to obtain the target step length parameter.

[0177] In some embodiments, the position determination unit is specifically used to: obtain a first peak value and a minimum peak value, wherein the first peak value is the first peak value of the target acceleration and the minimum peak value is the minimum distance between two consecutive peak values ​​of the target acceleration; input the first peak value and the minimum peak value into a current step length model to obtain a step length estimate, wherein the current step length model adopts the target step length parameters; and determine the current position based on the pedestrian's forward direction and the step length estimate.

[0178] In this embodiment of the invention, it is possible to identify gaits with similar motion states, thereby determining the corresponding step length estimation parameters to improve the accuracy of pedestrian positioning.

[0179] Figure 14 This is a schematic diagram of the physical structure of the electronic device provided by the present invention. For example... Figure 14 As shown, the electronic device 700 may include a processor 701, a communications interface 702, a memory 703, and a communication bus 704. The processor 701, communications interface 702, and memory 703 communicate with each other via the communication bus 704. The processor 701 can call logical instructions in the memory 703 to execute a pedestrian positioning method. This method includes: determining a target gait based on the pedestrian's target motion data, wherein the target motion data includes target acceleration, target angular velocity, and target velocity, the target acceleration and target angular velocity being determined based on inertial measurement data output by an inertial measurement unit, and the target velocity being determined based on radar point cloud data output by a millimeter-wave radar, the timestamp of the target velocity corresponding to the timestamps of the target angular velocity and the target acceleration, and the pedestrian carrying the millimeter-wave radar and the inertial measurement unit; and determining the pedestrian's current position based on the target gait, wherein the pedestrian's current position is determined based on the target step length parameter corresponding to the target gait.

[0180] Furthermore, the logical instructions in the aforementioned memory 703 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0181] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the pedestrian positioning method provided by the above methods. The method includes: determining a target gait based on the pedestrian's target motion data, wherein the target motion data includes target acceleration, target angular velocity, and target velocity, wherein the target acceleration and the target angular velocity are determined based on inertial measurement data output by an inertial measurement unit, and the target velocity is determined based on radar point cloud data output by a millimeter-wave radar, wherein the timestamp of the target velocity corresponds to the timestamps of the target angular velocity and the target acceleration, and the pedestrian carries the millimeter-wave radar and the inertial measurement unit; and determining the pedestrian's current position based on the target gait, wherein the pedestrian's current position is determined based on the target step length parameter corresponding to the target gait.

[0182] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)).

[0183] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, is implemented to perform the pedestrian positioning method provided by the methods described above. The method includes: determining a target gait based on target motion data of a pedestrian, wherein the target motion data includes target acceleration, target angular velocity, and target velocity, the target acceleration and target angular velocity being determined based on inertial measurement data output by an inertial measurement unit, and the target velocity being determined based on radar point cloud data output by a millimeter-wave radar, wherein the timestamp of the target velocity corresponds to the timestamps of the target angular velocity and the target acceleration, and the pedestrian carrying the millimeter-wave radar and the inertial measurement unit; and determining the current position of the pedestrian based on the target gait, wherein the current position of the pedestrian is determined based on a target step length parameter corresponding to the target gait.

[0184] The aforementioned computer-readable storage medium may be any combination of one or more computer-readable media. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0185] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including—but not limited to—electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0186] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wire, optical fiber, radio frequency (RF), etc., or any suitable combination thereof.

[0187] Computer program code for performing the operations described herein can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as "C" or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0188] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0189] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0190] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A pedestrian positioning method based on fusion of millimeter wave radar and inertial measurement unit, characterized in that, include: Based on the pedestrian's target motion data, the target gait is determined. The target motion data includes target acceleration, target angular velocity, and target velocity. The target acceleration and target angular velocity are determined based on inertial measurement data output by the inertial measurement unit, and the target velocity is determined based on radar point cloud data output by the millimeter-wave radar. The timestamp of the target velocity corresponds to the timestamps of the target angular velocity and the target acceleration. The pedestrian carries the millimeter-wave radar and the inertial measurement unit. The pedestrian's current position is determined based on the target gait, and the pedestrian's current position is determined based on the target stride length parameter corresponding to the target gait. Before determining the target gait based on the pedestrian's target motion data, the method further includes: The initial radar velocity is determined based on the radar point cloud data. The initial radar velocity is the radar's own velocity calculated from the Doppler velocity and normalized distance of each radar point cloud in the radar point cloud data. The initial radar velocity is subjected to piecewise cubic Hermitian interpolation to obtain the target velocity. The timestamp of the target velocity is aligned with the timestamps of the target acceleration and the target angular velocity. The cubic Hermitian interpolation is used to construct a cubic polynomial between each adjacent interpolation node, and each interpolation node corresponds to the time point of the initial radar velocity acquisition. Determining the initial radar velocity based on the radar point cloud data includes: The inner point set acquisition step is repeatedly performed on the current frame point cloud data in the radar point cloud data to obtain multiple inner point sets corresponding to the current frame. The inner point set acquisition step includes: determining the initial radar velocity based on the target point cloud data in the current frame point cloud data, wherein the target point cloud data is the data of a randomly selected number of target radar point clouds; determining the inner point set corresponding to the current frame based on the initial radar velocity, wherein the inner points in the inner point set are point clouds obtained by filtering based on the velocity deviation between other point clouds and the initial radar velocity, and a preset deviation threshold, wherein the other point clouds are point clouds in the current frame point cloud data other than the target point cloud; The initial radar velocity is determined based on the plurality of interior point sets.

2. The pedestrian positioning method fusing millimeter-wave radar and inertial measurement unit according to claim 1, characterized in that, The current frame point cloud data is obtained by filtering the radar point cloud data based on target features, which include at least one of signal-to-noise ratio, distance, and angle.

3. The method of claim 1, wherein, Determining the initial radar velocity based on the plurality of interior point sets includes: The reference radar velocity is determined based on the target interior point set in the plurality of interior point sets, wherein the target interior point set is the interior point set with the largest number of point clouds included in the plurality of interior point sets. The initial radar velocity is obtained by optimizing the reference radar velocity by establishing the optimization parameters of the angle error and velocity error of the radar measurement.

4. The method of claim 1, wherein, Before determining the target gait based on the pedestrian's target motion data, the method further includes: The inertial measurement data is windowed to obtain windowed inertial data; Gaussian filtering is applied to the windowed inertial data to obtain the target acceleration and the target angular velocity.

5. The method of claim 1, wherein, The step of determining the target gait based on the pedestrian's target movement data includes: The target motion data is input into the trained gait recognition network to obtain multiple sets of gait probability values. The trained gait recognition network is obtained by training an initial gait recognition network with motion data samples. The motion data samples include filtered acceleration, filtered angular velocity, interpolated radar velocity, and marked gait labels. The target gait is obtained by performing mean filtering, threshold judgment, and continuity judgment on three temporally continuous gait probability values ​​from the multiple sets of gait probability values.

6. The method of claim 1, wherein, Determining the pedestrian's current position based on the target gait includes: Determine whether the target gait matches the preset gait; If the target gait matches the preset gait, obtain the calibration step length parameters, and determine the target step length parameters corresponding to the step length model as the calibration step length parameters; If the target gait does not match the preset gait, the target step length parameter corresponding to the step length model is determined as the current step length parameter; The current position is determined based on the target step size parameter.

7. The method of claim 6, wherein, The process of obtaining the calibration step size parameters includes: Using the initial radar velocity as an observation, the velocity integrated in the inertial measurement data is corrected by an extended Kalman filter based on the error state to obtain the actual distance. The calculated trajectory is obtained based on the target angular velocity, the target acceleration, and the preset fixed step size parameters; The calibration step length parameters are determined based on the actual distance traveled and the calculated trajectory.

8. The method of claim 6, wherein, The target step size parameters corresponding to the determined step size model are calibration step size parameters, including: Obtain the parameter change values ​​of the calibrated step length parameters and the step length parameters corresponding to the target gait, and compare the parameter change values ​​with a preset change value threshold; If the change value of the parameter is less than the change value threshold, the calibrated step size parameter and the step size parameter corresponding to the target gait are smoothed to obtain the target step size parameter. If the change value of the parameter is greater than or equal to the change value threshold, the step length parameter corresponding to the target gait is updated to the calibrated step length parameter to obtain the target step length parameter.

9. The method of claim 6, wherein, Determining the current position based on the target step size parameter includes: Obtain a first peak value and a minimum peak value, wherein the first peak value is the first peak value of the target acceleration, and the minimum peak value is the minimum distance between two consecutive peak values ​​of the target acceleration; The first peak value and the minimum peak value are input into the current step size model to obtain the step size estimate, wherein the current step size model adopts the target step size parameter; The current position is determined based on the pedestrian's direction of travel and the estimated step length. 10.A pedestrian positioning device fusing millimeter wave radar and inertial measurement unit, characterized in that, include: A gait determination module is used to determine the target gait based on the pedestrian's target motion data. The target motion data includes target acceleration, target angular velocity, and target velocity. The target acceleration and target angular velocity are determined based on inertial measurement data output by an inertial measurement unit, and the target velocity is determined based on radar point cloud data output by a millimeter-wave radar. The timestamp of the target velocity corresponds to the timestamps of the target angular velocity and the target acceleration. The pedestrian carries the millimeter-wave radar and the inertial measurement unit. The position determination module is used to determine the current position of the pedestrian based on the target gait, wherein the current position of the pedestrian is determined based on the target step length parameter corresponding to the target gait. An initial velocity acquisition module is used to determine an initial radar velocity based on the radar point cloud data. The initial radar velocity is the radar's own velocity calculated based on the Doppler velocity and normalized distance of each radar point cloud in the radar point cloud data. The target velocity acquisition module is used to perform piecewise cubic Hermitian interpolation on the initial radar velocity to obtain the target velocity. The timestamp of the target velocity is aligned with the timestamps of the target acceleration and the target angular velocity. The cubic Hermitian interpolation is used to construct a cubic polynomial between each adjacent interpolation node, and each interpolation node corresponds to the time point of the initial radar velocity acquisition. The initial velocity acquisition module is specifically used to: repeatedly perform the inner point set acquisition step multiple times on the current frame point cloud data in the radar point cloud data to obtain multiple inner point sets corresponding to the current frame. The inner point set acquisition step includes: determining the initial radar velocity based on the target point cloud data in the current frame point cloud data, wherein the target point cloud data is data of a randomly selected number of target radar point clouds; determining the inner point set corresponding to the current frame based on the initial radar velocity, wherein the inner points in the inner point set are point clouds selected based on the velocity deviation between other point clouds and the initial radar velocity, and a preset deviation threshold, wherein the other point clouds are point clouds in the current frame point cloud data other than the target point cloud; and determining the initial radar velocity based on the multiple inner point sets.

11. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the pedestrian positioning method that integrates millimeter-wave radar and inertial measurement unit as described in any one of claims 1 to 9.

12. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by the processor, it implements the pedestrian positioning method that integrates millimeter-wave radar and inertial measurement unit as described in any one of claims 1 to 9.

13. A computer program product comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the pedestrian positioning method that integrates millimeter-wave radar and inertial measurement unit as described in any one of claims 1 to 9.

Images