Pedestrian positioning method based on fusion of millimeter wave radar and inertial measurement unit

Abstract

The invention provides a millimeter wave radar and inertial measurement unit fused pedestrian positioning method, and relates to the technical field of indoor positioning, the method comprises the following steps: determining a target gait according to target motion data of a pedestrian, the target motion data comprising a target acceleration, a target angular velocity and a target velocity, the target acceleration and the target angular velocity are determined according to inertial measurement data output by an inertial measurement unit, and the target velocity is determined according to radar point cloud data output by a millimeter wave radar; according to the target gait, the current position of the pedestrian is determined, and the current position of the pedestrian is determined according to a target step length parameter corresponding to the target gait. According to the method, the gait recognition is performed by introducing the radar speed information and combining the acceleration and the angular velocity, so that the gait close to the motion state can be recognized, the corresponding step length estimation parameter is determined, and the accuracy of pedestrian positioning is improved.

Description

Technical Field

[0001] The present invention relates to the field of indoor positioning technology, and in particular to a pedestrian positioning method integrating a millimeter-wave radar and an inertial measurement unit. Background Art

[0002] Pedestrian Dead Reckoning (PDR) uses information from sensors such as accelerometers, gyroscopes, and magnetometers to calculate the number of steps, step length, and direction of movement of pedestrians, and updates the position of pedestrians based on the length and direction of each step. PDR algorithms usually include step detection, step length estimation, heading update, and position update. After detecting the steps of pedestrians, existing technologies usually determine the gait based on the pedestrian's acceleration and body parameters, and then estimate the step length based on the gait.

[0003] Since the motion state of pedestrians is complex and changes frequently when they walk, and the physical parameters of pedestrians vary from person to person, it is difficult for existing pedestrian positioning technologies to distinguish gaits with similar motion states. Summary of the invention

[0004] The present invention provides a pedestrian positioning method integrating millimeter-wave radar and inertial measurement unit, so as to solve the defect that pedestrian positioning in the prior art is difficult to distinguish gaits with similar motion states. By introducing radar speed information and combining acceleration and angular velocity for gait recognition, gaits with similar motion states can be identified, thereby determining the corresponding step length estimation parameters, so as to improve the accuracy of pedestrian positioning.

[0005] The present invention provides a pedestrian positioning method integrating millimeter wave radar and inertial measurement unit, comprising the following steps: Determine a target gait according to target motion data of the pedestrian, the target motion data including a target acceleration, a target angular velocity and a target speed, the target acceleration and the target angular velocity are determined according to inertial measurement data output by an inertial measurement unit, the target speed is determined according to radar point cloud data output by a millimeter-wave radar, a timestamp of the target speed corresponds to a timestamp of the target angular velocity and the target acceleration, and the pedestrian carries the millimeter-wave radar and the inertial measurement unit; The current position of the pedestrian is determined according to the target gait, and the current position of the pedestrian is determined according to a target step length parameter corresponding to the target gait.

[0006] According to a pedestrian positioning method fused with a millimeter wave radar and an inertial measurement unit provided by the present invention, before determining the target gait according to the target motion data of the pedestrian, the method further includes: Determine an initial radar speed according to the radar point cloud data, wherein the initial radar speed is a radar speed obtained by calculating the Doppler speed and normalized distance of each radar point cloud in the radar point cloud data; The initial radar velocity is subjected to piecewise cubic Hermitian interpolation to obtain the target velocity, wherein a timestamp of the target velocity is aligned with a timestamp of the target acceleration and a timestamp of the target angular velocity, and the cubic Hermitian interpolation is used to construct a cubic polynomial between each adjacent interpolation node, and each interpolation node corresponds to a time point of the initial radar data collection.

[0007] According to a pedestrian positioning method for integrating millimeter wave radar and inertial measurement unit provided by the present invention, the initial radar speed is determined according to the radar point cloud data, including: Repeating the inner point set acquisition step for multiple times on the current frame point cloud data in the radar point cloud data to obtain multiple inner point sets corresponding to the current frame, the inner point set acquisition step comprising: determining an initial radar speed according to target point cloud data in the current frame point cloud data, the target point cloud data is data of radar point clouds of a randomly selected number of targets, determining an inner point set corresponding to the current frame according to the initial radar speed, the inner points in the inner point set are point clouds obtained by screening according to speed deviations between other point clouds and the initial radar speed and a preset deviation threshold, and the other point clouds are point clouds other than the target point cloud in the current frame point cloud data; The initial radar velocity is determined based on the plurality of interior point sets.

[0008] According to a pedestrian positioning method integrating millimeter-wave radar and inertial measurement unit provided by the present invention, the current frame point cloud data is obtained by screening the radar point cloud data according to target features, and the target features include at least one of signal-to-noise ratio, distance, and angle.

[0009] According to a pedestrian positioning method for integrating a millimeter wave radar and an inertial measurement unit provided by the present invention, the determining of the initial radar velocity according to the multiple interior point sets includes: determining a reference radar speed according to a target inlier point set among the plurality of inlier point sets, the target inlier point set being an inlier point set with the largest number of point clouds included in the plurality of inlier point sets; The reference radar speed is optimized by establishing the amount to be optimized of the angle error and the speed error measured by the radar, so as to obtain the initial radar speed.

[0010] According to a pedestrian positioning method fused with a millimeter wave radar and an inertial measurement unit provided by the present invention, before determining the target gait according to the target motion data of the pedestrian, the method further includes: Windowing the inertial measurement data to obtain windowed inertial data; Gaussian filtering is performed on the windowed inertial data to obtain the target acceleration and the target angular velocity.

[0011] According to a pedestrian positioning method provided by the present invention by integrating a millimeter wave radar with an inertial measurement unit, the method of determining a target gait according to target motion data of a pedestrian includes: Inputting the target motion data into a trained gait recognition network to obtain multiple groups of gait probability values, wherein the trained gait recognition network is obtained by training an initial gait recognition network through motion data samples, wherein the motion data samples include filtered acceleration, filtered angular velocity, interpolated radar velocity, and marked gait labels; Mean filtering, threshold judgment, and continuity judgment are performed on three groups of gait probability values ​​that are continuous in time sequence among the multiple groups of gait probability values ​​to obtain the target gait.

[0012] According to a pedestrian positioning method provided by the present invention by integrating a millimeter wave radar and an inertial measurement unit, determining the current position of the pedestrian according to the target gait includes: Determining whether the target gait matches a preset gait; If the target gait matches the preset gait, a calibration step length parameter is obtained, and the target step length parameter corresponding to the step length model is determined as the calibration step length parameter; If the target gait does not match the preset gait, determining the target step length parameter corresponding to the step length model as the current step length parameter; The current position is determined according to the target step size parameter.

[0013] According to a pedestrian positioning method for integrating millimeter wave radar and inertial measurement unit provided by the present invention, the step of obtaining the calibration step length parameter includes: The initial radar velocity is used as the observed quantity, and the extended Kalman filter based on the error state corrects the integrated velocity in the inertial measurement data to obtain the actual distance. Obtaining a calculated trajectory according to the target angular velocity, the target acceleration and a preset fixed step length parameter; The calibration step length parameter is determined according to the actual distance and the estimated trajectory.

[0014] According to a pedestrian positioning method for integrating millimeter wave radar and inertial measurement unit provided by the present invention, the target step length parameter corresponding to the step length model is determined as a calibration step length parameter, including: Obtaining a parameter change value of the calibrated step length parameter and the step length parameter corresponding to the target gait, and comparing the parameter change value with a preset change value threshold; If the parameter change value is less than the change value threshold, smoothing the calibrated step length parameter and the step length parameter corresponding to the target gait to obtain the target step length parameter; If the parameter change value is greater than or equal to the change value threshold, the step length parameter corresponding to the target gait is updated to the calibrated step length parameter to obtain the target step length parameter.

[0015] According to a pedestrian positioning method for integrating millimeter wave radar and inertial measurement unit provided by the present invention, determining the current position according to the target step length parameter includes: Obtaining a first peak value and a minimum peak value, wherein the first peak value is the first peak value of the target acceleration, and the minimum peak value is the minimum distance between two consecutive peak values ​​of the target acceleration; Inputting the first peak value and the minimum peak value into a current step length model to obtain a step length estimation value, wherein the current step length model adopts the target step length parameter; The current position is determined according to the pedestrian's moving direction and the estimated step length.

[0016] The present invention also provides a pedestrian positioning device integrating millimeter wave radar and inertial measurement unit, comprising the following modules: a gait determination module, configured to determine a target gait according to target motion data of a pedestrian, wherein the target motion data includes a target acceleration, a target angular velocity, and a target speed, wherein the target acceleration and the target angular velocity are determined according to inertial measurement data output by an inertial measurement unit, wherein the target speed is determined according to radar point cloud data output by a millimeter-wave radar, wherein a timestamp of the target speed corresponds to a timestamp of the target angular velocity and the target acceleration, and wherein the pedestrian carries the millimeter-wave radar and the inertial measurement unit; The position determination module is used to determine the current position of the pedestrian according to the target gait, and the current position of the pedestrian is determined according to the target step length parameter corresponding to the target gait.

[0017] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, a pedestrian positioning method integrating a millimeter-wave radar and an inertial measurement unit as described above is implemented.

[0018] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a pedestrian positioning method integrating a millimeter-wave radar and an inertial measurement unit as described in any one of the above.

[0019] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements a pedestrian positioning method that integrates a millimeter-wave radar and an inertial measurement unit as described in any of the above.

[0020] The pedestrian positioning method, device, electronic device and storage medium provided by the present invention can identify gaits with similar motion states by introducing radar speed information and combining acceleration and angular velocity for gait recognition, thereby determining corresponding step length estimation parameters to improve the accuracy of pedestrian positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 This is one of the flow charts of the pedestrian positioning method provided by the present invention.

[0023] Figure 2 This is the second flow chart of the pedestrian positioning method provided by the present invention.

[0024] Figure 3 It is a flow chart of the method for obtaining initial radar velocity provided by the present invention.

[0025] Figure 4 It is a flow chart of the method for determining the target gait provided by the present invention.

[0026] Figure 5 It is a flow chart of the millimeter wave radar-assisted pedestrian motion state detection and recognition algorithm provided by the present invention.

[0027] Figure 6 This is an example diagram of the long short-term memory recursive neural network structure provided by the present invention.

[0028] Figure 7 This is one of the flow charts of the method for determining the current position provided by the present invention.

[0029] Figure 8 It is a flow chart of the pedestrian gait model online calibration method provided by the present invention.

[0030] Fig. 9 It is a schematic diagram of the trajectory output of the radar inertial odometer provided by the present invention.

[0031] Fig.10 This is the second flow chart of the method for determining the current position provided by the present invention.

[0032] Fig.11 It is a schematic diagram of a pedestrian navigation system integrating a millimeter-wave radar and an inertial measurement unit provided by the present invention.

[0033] Fig.12 It is a hardware design schematic diagram of the millimeter wave radar and inertial measurement unit fusion system provided by the present invention.

[0034] Fig.13 A schematic diagram of the structure of a pedestrian positioning device provided in an embodiment of the present invention.

[0035] Fig.14 A schematic diagram of the physical structure of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] Pedestrian Dead Reckoning (PDR) uses information from sensors such as accelerometers, gyroscopes, and magnetometers to calculate the number of steps, step length, and direction of movement of pedestrians, and updates the position of pedestrians based on the length and direction of each step. PDR algorithms usually include step detection, step length estimation, heading update, and position update. After detecting the steps of pedestrians, existing technologies usually determine the gait based on the pedestrian's acceleration and body parameters, and then estimate the step length based on the gait.

[0038] Since the motion state of pedestrians is complex and changes frequently when they walk, and the physical parameters of pedestrians vary from person to person, it is difficult for existing pedestrian positioning technologies to distinguish gaits with similar motion states.

[0039] In view of this, an embodiment of the present invention provides a pedestrian positioning method that integrates millimeter-wave radar and inertial measurement unit, and determines the target gait according to the target motion data of the pedestrian, wherein the target motion data includes target acceleration, target angular velocity and target speed, wherein the target acceleration and target angular velocity are determined according to the inertial measurement data output by the inertial measurement unit, and the target speed is determined according to the radar point cloud data output by the millimeter-wave radar. According to the target gait, the current position of the pedestrian is determined, and the current position of the pedestrian is determined according to the target step length parameter corresponding to the target gait. The method can identify gaits with similar motion states, thereby determining the corresponding step length estimation parameters to improve the accuracy of pedestrian positioning.

[0040] The technical solutions in the embodiments of the present invention will be described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0041] Figure 1 It is one of the flow charts of the pedestrian positioning method provided by the present invention. The pedestrian positioning method integrating the millimeter wave radar and the inertial measurement unit can be applied to electronic devices, which can be various types of devices with information processing capabilities during implementation. For example, the electronic device can include a personal computer, a laptop computer, a PDA or a server, etc.; the electronic device can also be a mobile terminal, for example, the mobile terminal can include a mobile phone, a car computer, a tablet computer or a projector, etc. Figure 1 As shown, the method may include the following steps 101 to 102: Step 101: Determine a target gait based on target motion data of the pedestrian, the target motion data including target acceleration, target angular velocity and target speed, the target acceleration and the target angular velocity are determined based on inertial measurement data output by an inertial measurement unit, the target speed is determined based on radar point cloud data output by a millimeter-wave radar, and a timestamp of the target speed corresponds to a timestamp of the target angular velocity and the target acceleration.

[0042] It should be noted that the target acceleration and the target angular velocity are determined based on the inertial measurement data output by the inertial measurement unit, and the inertial measurement data can be obtained by the inertial measurement unit. The target speed is determined based on the radar point cloud data output by the millimeter wave radar, and the radar point cloud data can be obtained by the millimeter wave radar unit. Among them, the timestamps of the target acceleration, target angular velocity and target speed correspond to each other and can be used for gait recognition.

[0043] There are many ways to determine the target gait based on the pedestrian's target motion data, for example, it can be calculated through a gait recognition algorithm or obtained through a gait recognition model. The present invention does not limit the method of determining the target gait based on the pedestrian's target motion data.

[0044] Step 102: Determine the current position of the pedestrian according to the target gait, wherein the current position of the pedestrian is determined according to a target step length parameter corresponding to the target gait.

[0045] It should be noted that after determining the target gait, the current step length can be calculated according to the target gait, and the current position of the pedestrian can be determined according to the current step length. Alternatively, after determining the target gait, the current step length can be input into a position determination model to determine the current position. The present invention does not limit the method of determining the current position of the pedestrian according to the target gait.

[0046] In view of the problem that the existing technology is difficult to distinguish gaits with similar motion characteristics, the present invention proposes a method for detecting and identifying pedestrian motion states based on the fusion of millimeter-wave radar data and inertial measurement unit data indoors. The target gait can be used for pedestrian positioning, and the positioning result can be used for pedestrian navigation. The method can identify gaits with similar motion states, thereby determining the corresponding step length estimation parameters to improve the accuracy of pedestrian positioning.

[0047] Figure 2 FIG. 2 is a flow chart of the pedestrian positioning method provided by the present invention. Figure 2 As shown, the pedestrian positioning method integrating the millimeter wave radar and the inertial measurement unit may include the following steps 201 to 204: Step 201: determining an initial radar speed according to the radar point cloud data, wherein the initial radar speed is a radar speed obtained by calculating the Doppler speed and the normalized distance of each radar point cloud in the radar point cloud data.

[0048] It should be noted that the initial radar velocity may be determined based on the radar point cloud data by using a radar ego velocity estimation algorithm (REVE) or other methods. The present invention does not limit the method for determining the initial radar velocity based on the radar point cloud data.

[0049] Step 202: performing piecewise cubic Hermitian interpolation on the initial radar velocity to obtain the target velocity, wherein the timestamp of the target velocity is aligned with the timestamps of the target acceleration and the target angular velocity, and the cubic Hermitian interpolation is used to construct a cubic polynomial between each adjacent interpolation node, and each interpolation node corresponds to a time point of the initial radar data collection.

[0050] It should be noted that the angular velocity and acceleration information can be the inertial data output of the inertial sensor, and its frequency is 100Hz. The velocity information can be obtained by the Radar Ego Velocity Estimation (REVE) algorithm, which can obtain the millimeter-wave radar's own velocity based on the radar data of the millimeter-wave radar sensor, and its frequency is 5Hz. The velocity information is aligned with the angular velocity and acceleration information according to the timestamp through the piecewise cubic Hermite interpolation method.

[0051] Step 203: Determine a target gait based on the pedestrian's target motion data, where the target motion data includes a target acceleration, a target angular velocity, and a target speed. The target acceleration and the target angular velocity are determined based on the inertial measurement data output by an inertial measurement unit. The target speed is determined based on the radar point cloud data output by a millimeter-wave radar. The timestamp of the target speed corresponds to the timestamp of the target angular velocity and the target acceleration.

[0052] The description of step 203 may refer to the description of step 101 in the above embodiment.

[0053] Step 204: Determine the current position of the pedestrian according to the target gait, wherein the current position of the pedestrian is determined according to a target step length parameter corresponding to the target gait.

[0054] The description of step 204 may refer to the description of step 102 in the above embodiment.

[0055] The present invention proposes a method for detecting and identifying pedestrian motion states assisted by millimeter-wave radar, which determines the target gait by forming target motion data based on the radar's own velocity estimation results after piecewise cubic Hermite interpolation and inertial measurement results, thereby improving the accuracy of obtaining the target gait.

[0056] In some embodiments, step 201 determines the initial radar speed according to the radar point cloud data, and may include: repeatedly performing multiple inner point set acquisition steps on the current frame point cloud data in the radar point cloud data to obtain multiple inner point sets corresponding to the current frame, the inner point set acquisition step including: determining the initial radar speed according to the target point cloud data in the current frame point cloud data, the target point cloud data is data of a randomly selected number of radar point clouds of the target, determining the inner point set corresponding to the current frame according to the initial radar speed, the inner points in the inner point set are point clouds obtained by screening according to the speed deviation between other point clouds and the initial radar speed and a preset deviation threshold, the other point clouds are point clouds in the current frame point cloud data other than the target point cloud; determining the initial radar speed according to the multiple inner point sets.

[0057] Furthermore, determining the initial radar speed according to the multiple interior point sets may include: determining a reference radar speed according to a target interior point set among the multiple interior point sets, the target interior point set being an interior point set including the largest number of point clouds among the multiple interior point sets; and optimizing the reference radar speed by establishing an amount to be optimized of an angle error and a speed error measured by the radar to obtain the initial radar speed.

[0058] Furthermore, the current frame point cloud data is obtained by screening the radar point cloud data according to target features, and the target features include at least one of a signal-to-noise ratio, a distance, and an angle.

[0059] It can be understood that using the above method to obtain the initial radar initial velocity can improve the accuracy of obtaining the target gait.

[0060] Figure 3 FIG. 1 is a flow chart of a method for obtaining initial radar velocity provided by the present invention. Figure 3 As shown in the figure, the target speed can be obtained by the REVE algorithm. The algorithm flow includes: firstly, the raw data of the point cloud output of the millimeter wave radar development board is processed, and each point cloud in the single frame data is screened according to the signal-to-noise ratio, distance, and angle of each radar point cloud. For example, 4 point clouds are randomly selected from the screened point clouds, and the Doppler velocity of the point clouds is calculated. The distance to the normalized , the optimal radar speed is obtained by using the least squares method .

[0061] , .

[0062] in, Represents the point cloud measured by radar Point Doppler velocity value; Indicates point cloud Point The distance of the direction, , Similarly; Indicates the calculated point cloud Point speed value, , Same reason.

[0063] Calculate the velocity deviation of other points in the point cloud, that is: .

[0064] Set the threshold for deviation, and the point cloud that passes through constitutes the random interior point set. Repeat the above steps, repeat 34 times in this implementation case, extract the interior point set with the largest number of interior points, and solve the radar speed in the interior point set according to the least squares method. Establish the amount to be optimized for the radar measurement angle error and speed error: .

[0065] in, , , The definition of is the same as above. For point The true value of the azimuth, For point The azimuth measurement value of this formula has been obtained through the above steps. The quantity to be optimized is N X. The N summed quantities can be optimized separately by Newton-Gauss method, Levenberg-Marquardt method, etc. After the N azimuth true values ​​are updated, the speed , Optimize and obtain the final radar self-speed estimation result, that is, the target speed.

[0066] In some embodiments, before determining the target gait based on the pedestrian's target motion data, the method may further include: windowing the inertial measurement data to obtain windowed inertial data; and Gaussian filtering the windowed inertial data to obtain the target acceleration and the target angular velocity.

[0067] It can be understood that using the above method to obtain the target acceleration and target angular velocity can improve the accuracy of obtaining the target gait.

[0068] Figure 4 FIG. 1 is a flow chart of a method for determining a target gait provided by the present invention. Figure 4 As shown, determining the target gait according to the target motion data of the pedestrian may include: Step 301: Input the target motion data into a trained gait recognition network to obtain multiple groups of gait probability values, wherein the trained gait recognition network is obtained by training an initial gait recognition network with motion data samples, and the motion data samples include filtered acceleration, filtered angular velocity, interpolated radar velocity, and marked gait labels.

[0069] It should be noted that the gait recognition network can use a long short-term memory recurrent neural network (LSTM), and the network input is three-axis angular velocity information, three-axis acceleration information, and three-axis speed information. The three-axis angular velocity information, three-axis acceleration information, and three-axis speed information are combined into nine-dimensional data as the training data for the LSTM network. In the design of gait, the gait that significantly affects the parameters of the step length model in the motion gait is selected. The millimeter-wave radar-assisted pedestrian motion state detection and recognition algorithm is based on the LSTM network. First, the original inertial data is windowed and Gaussian filtered. The filtered data is combined with the millimeter-wave radar speed information and input into the network for prediction.

[0070] Among them, the target motion data can be obtained through the millimeter wave radar-inertial sensor, which is a sensor that integrates the millimeter wave radar and the inertial sensor. The target motion data output by the millimeter wave radar-inertial sensor may include three-axis angular velocity information, three-axis acceleration information, and three-axis speed information. The trained gait recognition network identifies the motion state of the pedestrian based on his three-axis acceleration, three-axis angular velocity, and three-axis speed, where the acceleration and angular velocity can be obtained through the inertial measurement unit in the millimeter wave radar-inertial sensor, and the speed can be obtained through the radar unit in the millimeter wave radar-inertial sensor.

[0071] The present invention can train a long short-term memory recursive neural network (LSTM) network through the radar's own speed estimation result after piecewise cubic Hermite interpolation and the inertial measurement result, and judge the pedestrian's motion state with the network as the core.

[0072] Step 302: performing mean filtering, threshold judgment, and continuity judgment on three groups of gait probability values ​​that are continuous in time sequence among the multiple groups of gait probability values ​​to obtain the target gait.

[0073] It should be noted that the network outputs the discrimination probability of each gait, and outliers can be removed. The specific operations include mean filtering, threshold judgment and continuity judgment. Finally, the results are written into Excel tables and plotted.

[0074] Figure 5 This is a flow chart of the millimeter wave radar-assisted pedestrian motion state detection and recognition algorithm provided by the present invention. Figure 5As shown in the figure, after obtaining the radar's own speed estimation result, that is, the initial radar speed, the millimeter-wave radar-assisted pedestrian motion state detection and recognition algorithm adds a window to the 100Hz inertial measurement unit, and the window length is set to 100. Then, the data in the window is Gaussian filtered with sigma=1.0 to complete the original processing of the inertial data. For the initial radar speed data, its output frequency is limited by the point cloud output frequency of the millimeter-wave radar, which is set to 5Hz in this implementation case. The speed frequency and the inertial data are aligned according to the timestamp by the piecewise cubic Hermite interpolation method. In a window, there are 100*9 data including the filtered inertial data (three-axis acceleration, three-axis angular velocity) and the interpolated radar speed (three-axis radar own speed). 50*9 radar-inertial data are taken every other line for packaging, and the gait to which each data belongs is manually marked in the csv file. The true value of the gait and the 50*9 radar-inertial data are used as the input for neural network training.

[0075] Figure 6 is an example diagram of the long short-term memory recursive neural network structure provided by the present invention. Figure 6 As shown, the training data set and test data set of the network are obtained randomly and non-repeatedly from the collected radar-inertial data. To prevent the network from overfitting, the training is terminated when the prediction accuracy of the test set in two consecutive rounds of the network is less than 0.1. In the part of removing abnormal points, the three groups of gait probabilities in the time series are mean filtered; the filtered data are judged by the threshold, and the predicted output with the maximum value of each gait probability less than 0.7 is regarded as other gait outputs; the data after the threshold judgment needs to be judged for continuity, and the data with the same prediction output results in the time series of three groups are finally output. In this part, the eliminated data is regarded as a gait other than the specified gait. The probability of each gait is recorded in the .excel file in time sequence, which is convenient for joint use with other navigation algorithms. At the same time, the gait recognition results of each time period and the probability distribution of each gait will be output by drawing. In this implementation case, four gaits are marked: small steps, large steps, backwards, and running.

[0076] The present invention provides a gait recognition network training method and a millimeter wave radar-assisted pedestrian motion state detection and recognition algorithm. Radar speed information is introduced into the pedestrian motion state detection and recognition algorithm, and the LSTM network is used to recognize the similar gaits of pedestrians, such as small steps, large steps, running, and backwards, with a certain accuracy. The algorithm output frequency is 1Hz, and real-time output is achieved in the scene of pedestrian navigation. And due to the introduction of speed information, this method can accurately recognize a variety of gaits that are difficult to recognize with traditional inertial information, such as going up and down stairs, and has good application prospects in the field of gait recognition.

[0077] Figure 7FIG. 1 is one of the flow charts of the method for determining the current position provided by the present invention. Figure 7 As shown, determining the current position of the pedestrian according to the target gait may include: Step 401: Determine whether the target gait matches a preset gait; Step 402: if the target gait matches the preset gait, obtain a calibration step length parameter, determine the target step length parameter corresponding to the step length model as the calibration step length parameter, and if the target gait does not match the preset gait, determine the target step length parameter corresponding to the step length model as the current step length parameter; Step 403: Determine the current position according to the target step length parameter.

[0078] It should be noted that the existing step length model is not generalizable and requires a lot of step length model parameter calibration work. The calibration method provided by the present invention can be implemented online throughout the process, outputting the pedestrian's motion state through the millimeter wave radar-assisted pedestrian motion state detection and recognition algorithm, and using the threshold to determine whether the gait is standard. If the threshold judgment is not passed, the calibrated parameters will not be updated. After the threshold judgment is passed, the calibrated step length parameters are obtained.

[0079] It is understandable that the above method can recalibrate parameters when necessary, thereby improving the efficiency of obtaining the current position of the pedestrian.

[0080] In some embodiments, obtaining the calibration step parameters may include: taking the initial radar velocity as an observed quantity, and correcting the integrated velocity in the inertial measurement data based on an extended Kalman filter of an error state to obtain an actual distance; obtaining an estimated trajectory according to the target angular velocity, the target acceleration, and a preset fixed step parameter; and determining the calibration step parameters according to the actual distance and the estimated trajectory.

[0081] It should be noted that the present invention proposes a generalizable millimeter-wave radar-assisted pedestrian gait model online calibration method, which uses millimeter-wave radar inertial odometry (RIO) to online calibrate, optimize and update the step length parameters of different gaits and pedestrians, and stores historical records for parameter optimization.

[0082] Among them, the step length model parameter online calibration method based on the radar inertial odometer realized by millimeter-wave radar and inertial measurement unit is used. The radar inertial odometer estimates its own speed based on the millimeter-wave radar point cloud information through the REVE algorithm, and uses the error state Kalman filter (ESKF) to fuse the radar speed information with the inertial information to realize the radar inertial odometer (RIO). The pedestrian step length parameters are calibrated based on the trajectory output of RIO combined with the step detection results.

[0083] In some embodiments, the target step length parameter corresponding to the determined step length model is a calibrated step length parameter, which may include: obtaining a parameter change value of the calibrated step length parameter and the step length parameter corresponding to the target gait, and comparing the parameter change value with a preset change value threshold; if the parameter change value is less than the change value threshold, smoothing the calibrated step length parameter and the step length parameter corresponding to the target gait to obtain the target step length parameter; if the parameter change value is greater than or equal to the change value threshold, updating the step length parameter corresponding to the target gait to the calibrated step length parameter to obtain the target step length parameter.

[0084] It should be noted that the entire calibration process can be an online operation. When a pedestrian is wearing a millimeter-wave radar-inertial sensor and moving, the motion state detection and recognition algorithm recognizes his or her motion state, and decides whether to update the step length parameters of the corresponding gait based on the recognition result and the calibration quality verification result. Each updated calibration result will be compared with the historical result. If it is a different individual, the step length parameter will be directly updated; if it is the same individual, it will be updated after smoothing.

[0085] This application provides a high-precision indoor positioning service for pedestrians with multiple gaits, and gives a generalizable online calibration method for gait models, which significantly improves the navigation accuracy of pedestrian multi-gait motion while greatly simplifying the calibration steps.

[0086] It is understandable that the pedestrian gait model calibration method in the present invention is simple and can be implemented only by relying on a small millimeter-wave radar-inertial sensor module. The entire calibration process is completed through an online method, and pedestrians do not need to perform additional calibration steps, and multiple gait parameters can be quickly updated. The calibration process takes into account the historical calibration results. By comparing the updated parameters with the historical parameters, the gait parameters of the same individual can be gradually optimized and the gait parameters of different individuals can be directly updated. The online calibration method proposed in the present invention greatly simplifies the calibration process of the system and can achieve rapid deployment.

[0087] Figure 8: is a flow chart of the pedestrian gait model online calibration method provided by the present invention. Figure 8 As shown in the figure, the step model parameter online calibration method based on the radar inertial odometer implemented by millimeter wave radar and inertial measurement unit. The radar inertial odometer uses the result of the REVE algorithm as the observation quantity, and corrects the speed of the inertial measurement unit integration through the error state extended Kalman filter (ESKF). By updating the inertial data, the speed under the navigation system can be obtained. , the radar speed under the radar system can be obtained through the Euler equation . Using the REVE algorithm, the estimated radar speed is obtained , the difference between the two is the residual of radar velocity estimation: .

[0088] The linearization of the velocity residual can obtain the Jacobian matrix of the velocity measurement: ,

[0089] .

[0090] in, is the linearized velocity measurement matrix, is the attitude measurement matrix, is the barometric measurement matrix, is the transpose of the transformation matrix from the carrier coordinate system to the radar coordinate system, is the transpose of the transformation matrix from the navigation coordinate system to the carrier coordinate system, is the velocity estimate in the navigation system, is the translation vector from the carrier coordinate system to the radar coordinate system.

[0091] The update of the velocity measurement is performed by removing outliers based on the Mahalanobis distance, which is calculated as:

[0092] .

[0093] in, , Same as the previous definition, is the variance of the radar velocity measurement, For the filter at time Prediction Moment The covariance matrix of .

[0094] The Mahalanobis distance is calculated by setting the threshold according to the chi-square distribution. A judgment is made, and a value below the threshold is considered to be a usable value, and the error-state-based Kalman filter ESKF is updated according to this residual.

[0095] Fig. 9 FIG. 1 is a schematic diagram of the trajectory output of the radar inertial odometer provided by the present invention. Fig. 9 As shown in the figure, it is the pedestrian trajectory output by the radar inertial odometer. Taking the calibration of the Weinburg model as an example, the odometer output trajectory is taken as the actual distance, and the K value in the model is fixed to a specific value, such as K=0.3. The distance output based only on the inertial data is , the odometer output distance is , then the calibration parameters are: .

[0096] The calibration method proposed in the present invention is implemented online throughout the process. The pedestrian motion state detection and recognition algorithm assisted by millimeter wave radar outputs the pedestrian's motion state, and the threshold is used to determine whether the gait is standard. If the threshold judgment is not passed, the calibration parameters will not be updated. After the threshold judgment is passed, the calibration result K will be compared with the historical parameters of the corresponding gait. If the difference between the two is large, it is considered that a new user is wearing the module, and the new calibration result is directly updated and used to realize the user-friendly step length parameter selection; if the difference between the two is small, it is considered that the same user is wearing the module, and the calibration result of this time and the historical result are smoothed, such as mean filtering, to realize the step length parameter of the same user. The design of this online calibration method can make up for the defect that the existing technology is difficult to apply to different pedestrians and different gaits. Since the step length parameters of different users are quite different, and the step length parameters of the same user are also different each time, the step length online calibration method designed in this way can allow the step length parameters of the same user to be adjusted online with the change of the usage scenario when using it, and the step length parameters of different users can be quickly switched.

[0097] Fig.10 This is a second flow chart of the method for determining the current position provided by the present invention. Determining the current position according to the target step size parameter may include: Step 501: Obtain a first peak value and a minimum peak value, wherein the first peak value is the first peak value of the target acceleration, and the minimum peak value is the minimum distance between two consecutive peak values ​​of the target acceleration.

[0098] Step 502: Input the first peak value and the minimum peak value into a current step length model to obtain a step length estimation value, and the current step length model adopts the target step length parameter.

[0099] Step 503: Determine the current position according to the pedestrian's moving direction and the estimated step length.

[0100] It should be noted that the present invention aims to solve the problem that the existing technology is difficult to achieve high-precision dead reckoning of pedestrians in various motion states, and proposes a pedestrian navigation method that integrates indoor millimeter-wave radar and inertial measurement unit. The method discloses a fusion algorithm design of millimeter-wave radar and PDR. The fusion navigation algorithm takes the PDR algorithm as the core, and uses the millimeter-wave radar-assisted pedestrian motion state detection and recognition algorithm to judge the pedestrian's gait, and combines the step model parameters calibrated by the radar inertial odometer to locate the pedestrian's dead position.

[0101] Fig.11 Schematic diagram of the pedestrian navigation system provided by the present invention that integrates the millimeter wave radar and the inertial measurement unit. Fig.11 As shown in the figure, the point cloud output by the millimeter-wave radar module is solved by the REVE algorithm to give the radar's own velocity estimation solution, and the angular velocity and acceleration output by the IMU are combined to realize the output of raw data. In the MCU processor, the millimeter-wave radar-assisted pedestrian motion state detection and recognition algorithm is used to distinguish the gait, and the specified gait step length parameters are calibrated online based on the results of the discrimination.

[0102] The acceleration information of IMU is used to judge the pace. The specific steps are as follows: find two consecutive peaks through the peak-finding algorithm, regard them as a step of the pedestrian, find the minimum value in this interval as the minimum acceleration of the step, take the first peak as the maximum acceleration of the step, and bring it into the step length model. The angular velocity information of IMU is used to update the mechanical energy posture. In this implementation, the single sample + previous cycle method is used to update the posture. The principle is as follows: .

[0103] in, The angular increment for the new angular velocity update period, using the updated angular velocity Multiply the difference between this angular velocity and the previous angular velocity timestamp get. is the angular increment in the last angular velocity update cycle, combined with The equivalent rotation vector at that moment can be obtained to complete the update of the pedestrian's posture. The pedestrian's position can be updated by combining the pedestrian's forward direction and step length: .

[0104] in, for North position at the moment, for North position at the moment, , Similarly; for The heading angle at the moment, is the prediction step size.

[0105] It is understandable that the present invention discloses a millimeter wave radar and PDR fusion algorithm design, and designs a typical system. The algorithm design includes a millimeter wave radar-assisted motion state detection and recognition algorithm and a PDR algorithm, and combines the gait recognition results and the gait parameter online calibration results to improve the step length estimation in the traditional PDR, thereby achieving higher-precision navigation and positioning of pedestrians.

[0106] In addition, the present invention provides a fusion millimeter-wave radar-assisted pedestrian motion state detection and recognition method and PDR algorithm, combined with the millimeter-wave radar-assisted generalized pedestrian gait model online calibration results, to realize a pedestrian positioning method that integrates millimeter-wave radar and inertial measurement unit, which can be used for pedestrian navigation methods.

[0107] In addition, the present invention also proposes a pedestrian navigation method integrating indoor millimeter-wave radar and inertial measurement unit and designs a typical system. A millimeter-wave radar-inertial sensor module is designed in hardware. The fusion navigation system greatly improves the step length estimation accuracy in pedestrian dead reckoning, and realizes high-precision dead reckoning function in multi-gait motion of pedestrians of different heights, weights, ages and genders.

[0108] For example, in the typical hardware system designed, a millimeter-wave radar-inertial sensor module is implemented. The module integrates the millimeter-wave radar and the inertial measurement unit on a circuit board, and is equipped with a heat sink and tooling, and is designed with a geomagnetic module, a barometer, a UWB module, etc. for subsequent development and use.

[0109] Fig.12 Schematic diagram of the hardware design of the millimeter wave radar and inertial measurement unit fusion system provided by the present invention. Fig.12 As shown, the hardware implementation scheme provided by the present invention includes a UWB module, a millimeter wave radar module, a geomagnetic module, a barometer module and an IMU module. The millimeter wave radar-inertial measurement module can be deployed at the waist of pedestrians and can adapt to various movements of pedestrians.

[0110] The pedestrian positioning method provided by the present invention can accurately position pedestrians in various motion states, and different motion state identification tags can be designed to meet the application requirements of different scenarios. A typical system is designed in the present invention, and a millimeter wave radar-inertial sensor module is designed to provide a reference for the hardware implementation method.

[0111] Based on the foregoing embodiments, an embodiment of the present invention provides a pedestrian positioning device integrating a millimeter-wave radar and an inertial measurement unit. Each module included in the device and each unit included in each module can be implemented by a processor; of course, it can also be implemented by a specific logic circuit; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP) or a field programmable gate array (FPGA), etc.

[0112] The pedestrian positioning device provided by the present invention is described below. The pedestrian positioning device described below and the pedestrian positioning method described above can be referred to each other.

[0113] Fig.13 Schematic diagram of the structure of the pedestrian positioning device provided by the embodiment of the present invention. Fig.13 As shown, the device 600 includes a gait determination module 601 and a position determination module 602, wherein: a gait determination module 601, configured to determine a target gait according to target motion data of a pedestrian, wherein the target motion data includes a target acceleration, a target angular velocity, and a target speed, wherein the target acceleration and the target angular velocity are determined according to inertial measurement data output by an inertial measurement unit, and the target speed is determined according to radar point cloud data output by a millimeter-wave radar, and the timestamp of the target speed corresponds to the timestamp of the target angular velocity and the target acceleration, and the pedestrian carries the millimeter-wave radar and the inertial measurement unit; The position determination module 602 is used to determine the current position of the pedestrian according to the target gait, and the current position of the pedestrian is determined according to the target step length parameter corresponding to the target gait.

[0114] In some embodiments, the apparatus further comprises: An initial speed acquisition module, used to determine an initial radar speed according to the radar point cloud data, wherein the initial radar speed is a radar speed obtained by solving a Doppler speed and a normalized distance of each radar point cloud in the radar point cloud data; A target velocity acquisition module is used to perform piecewise cubic Hermitian interpolation on the initial radar velocity to obtain the target velocity, wherein the timestamp of the target velocity is aligned with the timestamps of the target acceleration and the target angular velocity, and the cubic Hermitian interpolation is used to construct a cubic polynomial between each adjacent interpolation node, and each interpolation node corresponds to a time point of the initial radar data acquisition.

[0115] In some embodiments, the initial velocity acquisition module is specifically used to: repeatedly perform multiple inner point set acquisition steps on the current frame point cloud data in the radar point cloud data to obtain multiple inner point sets corresponding to the current frame, and the inner point set acquisition step includes: determining the initial radar velocity according to the target point cloud data in the current frame point cloud data, the target point cloud data is data of a randomly selected number of radar point clouds of the target, determining the inner point set corresponding to the current frame according to the initial radar velocity, the inner points in the inner point set are point clouds obtained by screening according to the velocity deviation between other point clouds and the initial radar velocity and a preset deviation threshold, the other point clouds are point clouds in the current frame point cloud data except the target point cloud; determining the initial radar velocity according to the multiple inner point sets.

[0116] In some embodiments, the current frame point cloud data is obtained by screening the radar point cloud data according to target features, and the target features include at least one of a signal-to-noise ratio, a distance, and an angle.

[0117] In some embodiments, the initial velocity acquisition module is further specifically used to: determine a reference radar velocity based on a target inner point set among the multiple inner point sets, wherein the target inner point set is an inner point set including the largest number of point clouds among the multiple inner point sets; and optimize the reference radar velocity by establishing an optimized amount of angular error and velocity error measured by the radar to obtain the initial radar velocity.

[0118] In some embodiments, the apparatus further comprises: The other data acquisition module is used to perform windowing on the inertial measurement data to obtain windowed inertial data; and perform Gaussian filtering on the windowed inertial data to obtain the target acceleration and the target angular velocity.

[0119] In some embodiments, the gait determination module 601 includes: a probability acquisition unit, configured to input the target motion data into a trained gait recognition network to obtain a plurality of groups of gait probability values, wherein the trained gait recognition network is obtained by training an initial gait recognition network through motion data samples, wherein the motion data samples include filtered acceleration, filtered angular velocity, interpolated radar velocity, and marked gait labels; The gait acquisition unit is used to perform mean filtering, threshold judgment, and continuity judgment on three groups of gait probability values ​​that are continuous in time sequence among the multiple groups of gait probability values ​​to obtain the target gait.

[0120] In some embodiments, the location determination module 602 includes: a gait determination unit, used to determine whether the target gait matches a preset gait; a parameter determination unit, configured to obtain a calibration step length parameter if the target gait matches the preset gait, and determine the target step length parameter corresponding to the step length model as the calibration step length parameter; and if the target gait does not match the preset gait, determine the target step length parameter corresponding to the step length model as the current step length parameter; A position determination unit is used to determine the current position according to the target step size parameter.

[0121] In some embodiments, the parameter determination unit is specifically used to: use the initial radar velocity as an observed quantity, and correct the integrated velocity in the inertial measurement data based on an extended Kalman filter of an error state to obtain an actual distance; obtain an estimated trajectory according to the target angular velocity, the target acceleration and a preset fixed step parameter; determine the calibration step parameter according to the actual distance and the estimated trajectory.

[0122] In some embodiments, the parameter determination unit is also specifically used to: obtain the parameter change value of the calibrated step length parameter and the step length parameter corresponding to the target gait, and compare the parameter change value with a preset change value threshold; if the parameter change value is less than the change value threshold, smooth the calibrated step length parameter and the step length parameter corresponding to the target gait to obtain the target step length parameter; if the parameter change value is greater than or equal to the change value threshold, update the step length parameter corresponding to the target gait to the calibrated step length parameter to obtain the target step length parameter.

[0123] In some embodiments, the position determination unit is specifically used to: obtain a first peak value and a minimum peak value, wherein the first peak value is the first peak value of the target acceleration, and the minimum peak value is the minimum distance between two consecutive target acceleration peak values; input the first peak value and the minimum peak value into a current step length model to obtain a step length estimation value, wherein the current step length model adopts the target step length parameter; and determine the current position according to the pedestrian's forward direction and the step length estimation value.

[0124] In the embodiment of the present invention, gaits with similar motion states can be identified, thereby determining corresponding step length estimation parameters to improve the accuracy of pedestrian positioning.

[0125] Fig.14 The physical structure diagram of the electronic device provided by the present invention is shown in FIG. Fig.14As shown, the electronic device 700 may include: a processor 701, a communication interface 702, a memory 703 and a communication bus 704, wherein the processor 701, the communication interface 702 and the memory 703 communicate with each other through the communication bus 704. The processor 701 may call the logic instructions in the memory 703 to execute the pedestrian positioning method, the method comprising: determining the target gait according to the target motion data of the pedestrian, the target motion data comprising the target acceleration, the target angular velocity and the target speed, the target acceleration and the target angular velocity are determined according to the inertial measurement data output by the inertial measurement unit, the target speed is determined according to the radar point cloud data output by the millimeter wave radar, the timestamp of the target speed corresponds to the timestamp of the target angular velocity and the target acceleration, the pedestrian carries the millimeter wave radar and the inertial measurement unit; determining the current position of the pedestrian according to the target gait, the current position of the pedestrian is determined according to the target step length parameter corresponding to the target gait.

[0126] In addition, the logic instructions in the above-mentioned memory 703 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk.

[0127] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the pedestrian positioning method provided by the above-mentioned methods, the method including: determining a target gait based on the target motion data of the pedestrian, the target motion data including a target acceleration, a target angular velocity and a target speed, the target acceleration and the target angular velocity are determined based on the inertial measurement data output by an inertial measurement unit, the target speed is determined based on the radar point cloud data output by a millimeter-wave radar, the timestamp of the target speed corresponds to the timestamp of the target angular velocity and the target acceleration, and the pedestrian carries the millimeter-wave radar and the inertial measurement unit; determining the current position of the pedestrian based on the target gait, the current position of the pedestrian is determined based on the target step length parameter corresponding to the target gait.

[0128] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0129] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the pedestrian positioning method provided by the above-mentioned methods, the method comprising: determining a target gait based on target motion data of the pedestrian, the target motion data comprising a target acceleration, a target angular velocity and a target speed, the target acceleration and the target angular velocity being determined based on inertial measurement data output by an inertial measurement unit, the target speed being determined based on radar point cloud data output by a millimeter-wave radar, the timestamp of the target speed corresponding to the timestamp of the target angular velocity and the target acceleration, the pedestrian carrying the millimeter-wave radar and the inertial measurement unit; determining a current position of the pedestrian based on the target gait, the current position of the pedestrian being determined based on a target step length parameter corresponding to the target gait.

[0130] The above-mentioned computer-readable storage medium can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM) or flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0131] Computer-readable signal media may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0132] The program code embodied on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wire, optical cable, radio frequency (RF), etc., or any suitable combination of the foregoing.

[0133] Computer program code for performing the operations of the present specification may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0134] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0135] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0136] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A pedestrian positioning method integrating millimeter wave radar and inertial measurement unit, characterized in that: include: Determine a target gait according to target motion data of the pedestrian, the target motion data including a target acceleration, a target angular velocity and a target speed, the target acceleration and the target angular velocity are determined according to inertial measurement data output by an inertial measurement unit, the target speed is determined according to radar point cloud data output by a millimeter-wave radar, a timestamp of the target speed corresponds to a timestamp of the target angular velocity and the target acceleration, and the pedestrian carries the millimeter-wave radar and the inertial measurement unit; According to the target gait, the current position of the pedestrian is determined, and the current position of the pedestrian is determined according to the target step length parameter corresponding to the target gait.

2. The pedestrian positioning method integrating millimeter wave radar and inertial measurement unit according to claim 1 is characterized in that: Before determining the target gait according to the target motion data of the pedestrian, the method further includes: Determine an initial radar speed according to the radar point cloud data, wherein the initial radar speed is a radar speed obtained by calculating the Doppler speed and normalized distance of each radar point cloud in the radar point cloud data; The initial radar velocity is subjected to piecewise cubic Hermitian interpolation to obtain the target velocity, wherein a timestamp of the target velocity is aligned with a timestamp of the target acceleration and a timestamp of the target angular velocity, and the cubic Hermitian interpolation is used to construct a cubic polynomial between each adjacent interpolation node, and each interpolation node corresponds to a time point of the initial radar data collection.

3. The pedestrian positioning method integrating millimeter wave radar and inertial measurement unit according to claim 2 is characterized in that: The determining the initial radar speed according to the radar point cloud data comprises: Repeating the inner point set acquisition step for multiple times on the current frame point cloud data in the radar point cloud data to obtain multiple inner point sets corresponding to the current frame, the inner point set acquisition step comprising: determining an initial radar speed according to target point cloud data in the current frame point cloud data, the target point cloud data is data of radar point clouds of a randomly selected number of targets, determining an inner point set corresponding to the current frame according to the initial radar speed, the inner points in the inner point set are point clouds obtained by screening according to speed deviations between other point clouds and the initial radar speed and a preset deviation threshold, and the other point clouds are point clouds other than the target point cloud in the current frame point cloud data; The initial radar velocity is determined based on the plurality of interior point sets.

4. The pedestrian positioning method integrating millimeter wave radar and inertial measurement unit according to claim 3 is characterized in that: The current frame point cloud data is obtained by screening the radar point cloud data according to target features, and the target features include at least one of a signal-to-noise ratio, a distance, and an angle.

5. The pedestrian positioning method integrating millimeter wave radar and inertial measurement unit according to claim 3 is characterized in that: The determining the initial radar velocity according to the plurality of interior point sets comprises: determining a reference radar speed according to a target inlier set among the plurality of inlier sets, wherein the target inlier set is an inlier set having the largest number of point clouds included in the plurality of inlier sets; The reference radar speed is optimized by establishing the amount to be optimized of the angle error and the speed error measured by the radar, so as to obtain the initial radar speed.

6. The pedestrian positioning method integrating millimeter wave radar and inertial measurement unit according to claim 1, characterized in that: Before determining the target gait according to the target motion data of the pedestrian, the method further includes: Windowing the inertial measurement data to obtain windowed inertial data; Gaussian filtering is performed on the windowed inertial data to obtain the target acceleration and the target angular velocity.

7. The pedestrian positioning method integrating millimeter wave radar and inertial measurement unit according to claim 1, characterized in that: Determining the target gait according to the target motion data of the pedestrian includes: Inputting the target motion data into a trained gait recognition network to obtain multiple groups of gait probability values, wherein the trained gait recognition network is obtained by training an initial gait recognition network through motion data samples, wherein the motion data samples include filtered acceleration, filtered angular velocity, interpolated radar velocity, and marked gait labels; Mean filtering, threshold judgment, and continuity judgment are performed on three groups of gait probability values ​​that are continuous in time sequence among the multiple groups of gait probability values ​​to obtain the target gait.

8. The pedestrian positioning method integrating millimeter wave radar and inertial measurement unit according to claim 1, characterized in that: Determining the current position of the pedestrian according to the target gait includes: Determining whether the target gait matches a preset gait; If the target gait matches the preset gait, a calibration step length parameter is obtained, and the target step length parameter corresponding to the step length model is determined as the calibration step length parameter; If the target gait does not match the preset gait, determining the target step length parameter corresponding to the step length model as the current step length parameter; The current position is determined according to the target step size parameter.

9. The pedestrian positioning method integrating millimeter wave radar and inertial measurement unit according to claim 8, characterized in that: The step of obtaining the calibration step parameters includes: The initial radar velocity is used as the observed quantity, and the extended Kalman filter based on the error state corrects the integrated velocity in the inertial measurement data to obtain the actual distance. Obtaining a calculated trajectory according to the target angular velocity, the target acceleration and a preset fixed step length parameter; The calibration step length parameter is determined according to the actual distance and the estimated trajectory.

10. The pedestrian positioning method integrating millimeter wave radar and inertial measurement unit according to claim 8, characterized in that: The step length model is determined to correspond to the target step length parameter as a calibration step length parameter, including: Obtaining a parameter change value of the calibrated step length parameter and the step length parameter corresponding to the target gait, and comparing the parameter change value with a preset change value threshold; If the parameter change value is less than the change value threshold, smoothing the calibrated step length parameter and the step length parameter corresponding to the target gait to obtain the target step length parameter; If the parameter change value is greater than or equal to the change value threshold, the step length parameter corresponding to the target gait is updated to the calibrated step length parameter to obtain the target step length parameter.

11. The pedestrian positioning method integrating millimeter wave radar and inertial measurement unit according to claim 8, characterized in that: The determining the current position according to the target step length parameter includes: Obtaining a first peak value and a minimum peak value, wherein the first peak value is the first peak value of the target acceleration, and the minimum peak value is the minimum distance between two consecutive peak values ​​of the target acceleration; Inputting the first peak value and the minimum peak value into a current step length model to obtain a step length estimation value, wherein the current step length model adopts the target step length parameter; The current position is determined according to the pedestrian's moving direction and the estimated step length.

12. A pedestrian positioning device integrating millimeter wave radar and inertial measurement unit, characterized in that: include: a gait determination module, configured to determine a target gait according to target motion data of a pedestrian, wherein the target motion data includes a target acceleration, a target angular velocity, and a target speed, wherein the target acceleration and the target angular velocity are determined according to inertial measurement data output by an inertial measurement unit, wherein the target speed is determined according to radar point cloud data output by a millimeter-wave radar, wherein a timestamp of the target speed corresponds to a timestamp of the target angular velocity and the target acceleration, and wherein the pedestrian carries the millimeter-wave radar and the inertial measurement unit; The position determination module is used to determine the current position of the pedestrian according to the target gait, and the current position of the pedestrian is determined according to the target step length parameter corresponding to the target gait.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the pedestrian positioning method integrating the millimeter-wave radar and the inertial measurement unit as claimed in any one of claims 1 to 11 is implemented.

14. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the pedestrian positioning method integrating the millimeter-wave radar and the inertial measurement unit as claimed in any one of claims 1 to 11 is implemented.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the pedestrian positioning method integrating the millimeter-wave radar and the inertial measurement unit as claimed in any one of claims 1 to 11 is implemented.

Images