IMU (Inertial Measurement Unit) data missing complementation method, device and equipment based on space-time fusion modeling

Abstract

The invention relates to the technical field of IMU (Inertial Measurement Unit) data processing, solves the problem that missing data points in IMU data cannot be effectively complemented in the prior art, and provides an IMU data missing complementing method, device and equipment based on space-time fusion modeling. The method comprises the following steps: acquiring original IMU data to be subjected to missing completion processing; performing missing detection and marking on the original IMU data to obtain missing data points in the original IMU data; and performing missing completion processing on the missing data points by using a pre-established space-time fusion model to obtain target IMU data after missing completion processing. According to the method, the missing points of the IMU data are accurately complemented.

Description

Technical Field

[0001] This invention relates to the field of IMU data processing technology, and in particular to a method, apparatus and device for IMU data missing completion based on spatiotemporal fusion modeling. Background Technology

[0002] Inertial measurement units (IMUs) are widely used in aerospace, robotics, smartphones, and wearable devices, providing crucial motion and attitude information. However, due to sensor malfunctions, environmental interference, or data transmission issues, IMU data is often missing, preventing the system from accurately acquiring the device's motion state and impacting subsequent attitude estimation, navigation, and control decisions. Therefore, timely and effective handling of missing IMU data not only improves data reliability and integrity but also ensures the real-time performance and accuracy of related applications, thereby enhancing the overall system performance and security.

[0003] Existing technologies face several technical challenges in IMU data missing completion based on spatiotemporal fusion modeling, primarily including insufficient accuracy of prediction models and the complexity of spatial domain correlation modeling. Many traditional prediction methods, such as linear interpolation or simple autoregressive models, often fail to effectively capture the dynamic characteristics of the data, resulting in inaccurate missing data completion. Furthermore, existing spatial domain correlation models often fail to adequately consider the nonlinear relationships and time delay effects between sensors when processing data from multiple sensors, leading to biased estimation results. Moreover, when IMU data loss is severe, existing algorithms may fail to recover critical motion information in a timely manner, thus affecting the system's real-time performance and reliability. Therefore, there is an urgent need to develop more advanced and accurate IMU data missing completion methods based on spatiotemporal fusion modeling to improve the estimation capability for complex motion states.

[0004] Therefore, how to effectively complete missing data points in IMU data is an urgent problem to be solved. Summary of the Invention

[0005] In view of this, the present invention provides a method, apparatus and device for IMU data missing completion based on spatiotemporal fusion modeling, in order to solve the problem that the prior art cannot effectively complete missing data points in IMU data.

[0006] The technical solution adopted in this invention is: In a first aspect, the present invention provides a method for IMU data missing completion based on spatiotemporal fusion modeling, the method comprising: Obtain the original IMU data to be filled in; Missing data points in the original IMU data are obtained by performing missing detection and marking on the original IMU data; Using a pre-established spatiotemporal fusion model, the missing data points are filled in to obtain the target IMU data after the missing data points are filled in.

[0007] Preferably, the step of performing missing data detection and labeling on the original IMU data to obtain missing data points in the original IMU data includes: Based on the timestamp sequence corresponding to each data point in the original IMU data, the sampling time interval between adjacent data points is calculated to obtain the actual time interval between adjacent data points; The actual time interval is compared with a preset time interval threshold. If the actual time interval is the same as the time interval threshold, then the adjacent data points are normal data points. If the actual time interval is not the same as the time interval threshold, then the adjacent data points are abnormal data points; Based on the first timestamp corresponding to the normal data point and the second timestamp corresponding to the abnormal data point, the original IMU data is subjected to missing data detection and marking to obtain the missing data point.

[0008] Preferably, the step of performing missing data detection and marking on the original IMU data based on the first timestamp corresponding to the normal data points and the second timestamp corresponding to the abnormal data points to obtain the missing data points includes: Based on the first timestamp, the first time interval between normal adjacent data points is calculated; Based on the second timestamp, the second time interval between adjacent abnormal data points is calculated; Based on the first time interval and the second time interval, the target timestamp corresponding to the missing data point is calculated; Based on the target timestamp, missing data points in the original IMU data are detected and marked to obtain the missing data points.

[0009] Preferably, the step of using a pre-established spatiotemporal fusion model to perform missing data point completion processing to obtain the target IMU data after missing data completion processing includes: Based on the target timestamp, the continuous valid data points in the original IMU data before the missing data point are processed to obtain a time-domain autoregressive model for missing data completion processing; Based on the physical relationship between the accelerometer and gyroscope in the IMU corresponding to the original IMU data, correlation modeling is performed on the sensor data of the accelerometer and gyroscope to obtain a spatial domain model for missing data completion processing. The spatiotemporal fusion model is determined based on the time-domain autoregressive model and the spatial domain model. Based on the spatiotemporal fusion model, the missing data points are filled in to obtain the target IMU data.

[0010] Preferably, the step of processing consecutive valid data points in the original IMU data before the missing data point according to the target timestamp to obtain a time-domain autoregressive model for missing data completion includes: Based on the target timestamp, consecutive valid data points preceding the missing data point are extracted to obtain a historical data sequence for modeling. Based on the historical data sequence, the parameters of the autoregressive model are calculated using the least squares method to obtain the autoregressive model coefficients; The time-domain autoregressive model is constructed based on the historical data sequence and the autoregressive model coefficients.

[0011] Preferably, the step of performing correlation modeling on the sensor data of the accelerometer and gyroscope based on the physical relationship between the accelerometer and gyroscope in the IMU corresponding to the original IMU data to obtain a spatial domain model for missing data completion includes: Based on the principles of rigid body kinematics, the linear acceleration of the accelerometer in the IMU is correlated with the angular velocity and angular acceleration of the gyroscope to obtain the physical relationship between acceleration and angular velocity and angular acceleration. Based on the angular velocity, the angular acceleration, and the preset position vector of the accelerometer relative to the rotation center, the theoretical acceleration value of the accelerometer is calculated to obtain the theoretical calculation data of the accelerometer. The difference between the theoretical calculation data and the actual measurement data of the accelerometer is calculated to obtain the accelerometer measurement error value; Based on the accelerometer measurement error, the accelerometer measurement data is corrected and modeled to obtain an error correction model for correcting the accelerometer data; Based on the theoretical calculation data and the error correction model, when accelerometer data is missing, the acceleration value of the missing data point is estimated to obtain a missing data completion method. Based on the physical relationship, the error correction model, and the missing data completion method, the sensor data from the accelerometer and gyroscope are comprehensively modeled to obtain the spatial domain model.

[0012] Preferably, the step of performing missing data point completion processing based on the spatiotemporal fusion model to obtain the target IMU data includes: Based on the time-domain autoregressive model, the missing data points are predicted and calculated to obtain the first completion result based on the time domain; Based on the spatial domain model, the missing data points are estimated and calculated to obtain a second completion result based on the spatial domain; The first completion result and the first and second completion results are weighted and fused to obtain fused completion data; Based on a preset error minimization criterion or dynamic weighting strategy, the fused and completed data is optimized to obtain completed data points; Based on the completed data points, the missing positions in the original IMU data are replaced to obtain the target IMU data.

[0013] Secondly, the present invention provides an IMU data missing completion device based on spatiotemporal fusion modeling, the device comprising: The data acquisition module is used to acquire the raw IMU data to be filled in. The missing data detection and marking module is used to perform missing data detection and marking on the original IMU data to obtain the missing data points in the original IMU data; The missing data completion module is used to perform missing data completion processing on the missing data points using a pre-established spatiotemporal fusion model, so as to obtain the target IMU data after missing data completion processing.

[0014] Thirdly, embodiments of the present invention also provide an electronic device, including: at least one processor, at least one memory, and computer program instructions stored in the memory, which, when executed by the processor, implement the method of the first aspect described above.

[0015] Fourthly, embodiments of the present invention also provide a storage medium storing computer program instructions thereon, which, when executed by a processor, implement the method of the first aspect described above.

[0016] In summary, the beneficial effects of the present invention are as follows: This invention provides a method, apparatus, and device for IMU data missing completion based on spatiotemporal fusion modeling. The method includes: acquiring the original IMU data to be completed; performing missing detection and marking on the original IMU data to obtain missing data points; and using a pre-established spatiotemporal fusion model to complete the missing data points, obtaining the target IMU data after missing completion. This invention first constructs an autoregressive model in the time domain, using continuous valid data points before the missing points for parameter estimation, thereby achieving trend prediction based on historical sequences. Simultaneously, it establishes a physical relationship model between the accelerometer and gyroscope in the spatial domain, calculating theoretical acceleration through rigid body kinematics equations and combining actual measurement values ​​for error correction, thereby obtaining a reasonable estimate in the case of sensor missingness. Then, the time domain prediction results and spatial domain estimation results are fused, and a weighted sum error optimization strategy is used to generate the final completed data points. The above design overcomes the limitations of traditional methods that rely on single interpolation or filtering and cannot take into account both temporal characteristics and the correlation between multiple sensors. It achieves accurate completion of missing IMU data points, effectively improves the continuity and reliability of data, and provides stable data support for applications such as navigation, attitude estimation and motion tracking. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments of the present invention will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, and these are all within the protection scope of the present invention.

[0018] Figure 1 This is a schematic diagram of the overall workflow of the IMU data missing completion method based on spatiotemporal fusion modeling in Embodiment 1 of the present invention; Figure 2 This is a structural block diagram of the IMU data missing completion device based on spatiotemporal fusion modeling in Embodiment 2 of the present invention; Figure 3 This is a schematic diagram of the electronic device in Embodiment 3 of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. In the description of the present invention, it should be understood that the terms "center," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, the element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. Where there is no conflict, embodiments of the present invention and the various features thereof can be combined with each other, all of which are within the scope of protection of the present invention.

[0020] Example 1 Please see Figure 1 Embodiment 1 of the present invention discloses a method for IMU data missing completion based on spatiotemporal fusion modeling, the method comprising: Obtain the original IMU data to be filled in; Specifically, raw IMU data refers to the original measurements and corresponding timestamps of the IMU sensors recorded in a time series, such as the linear acceleration of the accelerometer's three axes and the angular velocity of the gyroscope's three axes. Each record contains both a timestamp and the sensor measurement. The purpose is to prepare complete raw input data for missing data detection and subsequent data completion while maintaining the original time reference, so that subsequent algorithms can make accurate judgments and models based on the real time series. In implementation, the IMU device collects data at preset sampling intervals and transmits or stores the data to the processing end. The processing end parses the received data, standardizes the timestamps and unifies the units, performs basic validity checks to confirm the monotonicity and numerical range of the timestamps, and generates necessary metadata such as preset sampling interval thresholds, sensor coordinate system descriptions, and historical window lengths. At the same time, it buffers the historical valid samples required for missing data completion in memory or storage. By retaining the original timestamps and complete sensor measurement information and recording metadata, a consistent and traceable data foundation is provided for missing data detection, temporal modeling, and spatial modeling, reducing completion errors caused by inconsistencies in the initial processing.

[0021] Missing data points in the original IMU data are obtained by performing missing detection and marking on the original IMU data; Specifically, missing data detection and labeling refers to identifying points that lack valid measurements at the expected sampling time based on timestamps and measurement rules, and writing missing data markers into the data structure to clarify the missing locations. Missing data points are those marked as needing to be filled in. By accurately locating all locations requiring missing data completion and quantifying the scale of the missing data, it provides context for selecting windows in the temporal domain model, for spatial domain estimation, and for assigning weights to the spatiotemporal fusion strategy. In implementation, the time interval between adjacent timestamps is first calculated and compared with a preset sampling interval threshold. If the time interval is greater than the threshold, the number of missing data points in the middle is calculated, and the start and end times of the missing data are recorded. For cases with duplicates or extremely small intervals, deduplication or jitter correction is performed. Samples with invalid or out-of-bounds values ​​are also marked as missing, and the reason is recorded. Furthermore, multi-channel consistency checks are performed to identify single-channel or multi-channel simultaneous missing data. The detection results should output a time series table with a missing mask, and each missing segment should include metadata such as the length of the effective window before and after, the estimated number of missing data points, and the reason for the missing data. The technical features of this step are the joint judgment of time interval and numerical rationality, as well as cross-channel consistency check; its beneficial effect is to improve the accuracy and quantifiability of missing identification, and to provide a clear basis for subsequent use of appropriate prediction windows, determination of autoregression order and design of spatial domain estimation constraints, thereby reducing the risk of false completion and improving completion accuracy.

[0022] Using a pre-established spatiotemporal fusion model, the missing data points are filled in to obtain the target IMU data after the missing data points are filled in.

[0023] Specifically, the "spatiotemporal fusion model" refers to a completion framework jointly composed of a time-domain autoregressive model and a spatial domain model. The time-domain autoregressive model predicts trends based on continuous valid data points before the missing data points, while the spatial domain model calculates theoretical acceleration and establishes an error correction mechanism based on the physical relationship between accelerometers and gyroscopes. The two are used together for missing data completion. Its purpose is to take into account both the continuity of time series and the physical consistency of multiple sensors when completing missing data points, thereby generating target IMU data that is both continuous and conforms to physical constraints. In implementation, for each missing segment, the forward historical valid window is first read, and time-domain predicted values ​​are generated time-by-time according to the aforementioned time-domain autoregressive model. When available, the angular velocity of the gyroscope and the angular acceleration obtained from its difference are used, combined with the position parameters of the accelerometer relative to the rotation center, to calculate the theoretical value of the accelerometer. Based on the historical valid samples, the error between the theoretical value and the actual measured value is calculated, and an error correction function is established to obtain the spatial domain estimated value. Subsequently, the time-domain predicted value and the spatial domain estimated value are fused at each missing time. The fusion can adopt a weighted least squares criterion with adaptive weighting according to their respective estimated variances or a Bayesian joint estimation. A smooth transition is applied at the boundary of the completed segment to avoid abrupt changes. Finally, the fused completed value is written back to the missing position, and the confidence level and completion source of each point are recorded in the metadata. After completion, a physical consistency check can also be performed on the entire sequence. If the confidence level is low, alternative strategies such as retaining only the time-domain prediction or retaining the missing marker are adopted. The technical features of this step are the joint estimation of temporal prediction and spatial physical constraints, adaptive weight fusion, and boundary smoothing and confidence labeling. Its beneficial effects are that it significantly improves the accuracy and physical rationality of missing data completion, reduces phase deviation and amplitude error caused by single interpolation or single model, and enables the obtained target IMU data to directly support downstream applications such as navigation, attitude estimation and motion tracking, thereby improving the overall robustness and reliability of the system.

[0024] Preferably, the step of performing missing data detection and labeling on the original IMU data to obtain missing data points in the original IMU data includes: Based on the timestamp sequence corresponding to each data point in the original IMU data, the sampling time interval between adjacent data points is calculated to obtain the actual time interval between adjacent data points; Specifically, the timestamp sequence refers to the sampling time identifier carried by each raw IMU record, such as the time stamp recorded by the device. Its purpose is to restore the raw time series into benchmark data that can be used to determine sampling uniformity and detect missing data. In implementation, the received raw IMU data is first timestamped and the time base is unified, including identifying the time format, correcting clock offsets, or rearranging out-of-order records. Then, the time difference is calculated for each pair of adjacent data points in chronological order as the actual time interval. Abnormal zero-interval or duplicate timestamps are deduplicated or merged. If necessary, short-term jitter is smoothed using a sliding median filter to reduce the risk of misjudgment. The technical feature of this step is that it retains the original timestamps and clearly obtains the actual time interval for each pair of adjacent points as a direct basis for subsequent missing data determination. Its beneficial effect is that it provides a reliable time reference for accurately locating missing intervals and selecting appropriate modeling windows.

[0025] The actual time interval is compared with a preset time interval threshold. If the actual time interval is the same as the time interval threshold, then the adjacent data points are normal data points. Specifically, the preset time interval threshold refers to the expected sampling interval obtained based on the IMU device's nominal sampling rate or runtime statistics. Examples include the sampling interval given in the device manual or the median calculated from historical data. The purpose of this step is to identify adjacent points that conform to the expected sampling pattern as normal samples for subsequent modeling and correction. In implementation, each actual time interval is compared with the preset threshold. The judgment method can be absolute difference or relative tolerance: if the difference is within the pre-set tolerance range, they are considered the same, and the two points are marked as normal data points. Simultaneously, the timestamp of the normal point is recorded for historical window extraction during time-domain modeling. The technical feature of this step is maintaining robustness to minute clock jitter in the sensor through tolerance-based threshold comparison. Its beneficial effect is to accurately screen out highly reliable valid samples, ensuring that subsequent time-domain models such as autoregressive models are based on stable historical data for parameter estimation, reducing modeling errors caused by noise or minute clock drift.

[0026] If the actual time interval is not the same as the time interval threshold, then the adjacent data points are abnormal data points; Specifically, outlier data points include long time intervals caused by packet loss, extremely short time intervals caused by duplication or jitter, and invalid values ​​generated by acquisition errors. The purpose is to label all sampled pairs that may affect the accuracy of completion judgment and modeling so that different subsequent processing strategies can be adopted. In implementation, for each interval that does not meet the same criteria, the specific difference is recorded and classified: when the actual interval is significantly greater than a preset threshold, it is labeled as a missing anomaly and the number of missing values ​​is estimated; when the actual interval is significantly less than the threshold or zero, it is labeled as a duplication or jitter anomaly and deduplication or temporal correction is performed; when the corresponding measured value exceeds the physically reasonable range, it is labeled as a numerical anomaly and listed as unusable. To facilitate subsequent processing, metadata such as the anomaly type, the start and end times of the anomaly, and the length of the available windows before and after should also be stored in the anomaly record. The technical feature of this step is the typological labeling of anomalies and the storage of detailed metadata. Its beneficial effect is to provide a clear basis for subsequent missing value estimation, spatiotemporal fusion weight allocation, and anomaly correction, thereby reducing the false completion rate and enhancing the interpretability of the completion results.

[0027] Based on the first timestamp corresponding to the normal data point and the second timestamp corresponding to the abnormal data point, the original IMU data is subjected to missing data detection and marking to obtain the missing data point.

[0028] Specifically, by utilizing the previously labeled normal and abnormal time points to accurately locate missing intervals, the aim is to generate a clear list of missing data points and missing segment information for use in the completion process. In implementation, each adjacent pair is used as a unit. When the current point is normal and the next point is abnormal, the time difference between the two is calculated, and the number of intermediate lost points is estimated using integer logic based on a preset sampling interval threshold (e.g., dividing the time difference by the sampling interval and rounding down or to the nearest integer, then subtracting the existing number of points). Based on the estimation results, a set of missing timestamps arranged according to the desired sampling interval is constructed, and placeholder records with missing markers are inserted into the original sequence. Simultaneously, the start and end timestamps of the missing segment, the estimated number of missing points, the length of the preceding and following effective windows, and the determination of the cause of the missing segment are recorded. If multiple consecutive abnormalities are encountered, they should be merged into the same missing segment and estimated as a whole. When there is a systematic offset in the timestamps or clock jumps, global time base correction should be used to ensure accurate estimation. The technical feature of this step is to generate missing markers with structured meta-information and write the missing points into the sequence in the form of explicit timestamps. Its beneficial effect is to provide clear completion positions and contexts for time-domain autoregressive modeling and spatial-domain estimation, so that subsequent missing completion processing can selectively select historical windows, set the model order, and perform accurate interpolation, ultimately improving the continuity and reliability of the target IMU data after completion.

[0029] Preferably, the step of performing missing data detection and marking on the original IMU data based on the first timestamp corresponding to the normal data points and the second timestamp corresponding to the abnormal data points to obtain the missing data points includes: Based on the first timestamp, the first time interval between normal adjacent data points is calculated; Specifically, the first timestamp refers to the sampling time corresponding to the data point that has been determined to be normal. The purpose of this step is to obtain a representative first-order sampling interval as the benchmark for subsequent missing data determination and target timestamp calculation. In implementation, the original IMU data timestamps need to be format parsed and time base unified first, duplicate or non-monotonic records need to be removed, and median smoothing is used for short-term jitter to reduce the impact of noise. Then, the time difference is calculated for each pair of adjacent normal data points in chronological order and summarized into a first time interval sequence. Finally, the median or mode of the sequence can be used as the representative first time interval and saved as metadata. By retaining the original time information based on the identified normal points and generating a robust benchmark sampling interval, a reliable and jitter-resistant time scale is provided for missing data determination and target timestamp estimation, thereby reducing misjudgments caused by clock jitter or short-term anomalies.

[0030] Based on the second timestamp, the second time interval between adjacent abnormal data points is calculated; Specifically, the second timestamp refers to the sampling time corresponding to the data point that has been identified as an anomaly. The purpose of this step is to quantify the actual time difference between adjacent points of anomaly, thereby identifying potential missing intervals or other sampling anomaly types. In practice, the time difference is calculated and classified for each pair of adjacent records marked as anomaly. The classification items include long missing intervals that are significantly larger than the baseline interval, repetition / jitter intervals that are very small or zero, and time anomalies such as reverse order or jump. At the same time, the start and end times of each anomaly interval and the length of the effective samples before and after are recorded for subsequent processing. By performing fine classification and recording of meta-information for anomaly intervals, the anomaly type is clarified and the necessary basis is provided for subsequent estimation of the number of missing points and selection of completion strategies, thereby improving the pertinence and accuracy of the completion process.

[0031] Based on the first time interval and the second time interval, the target timestamp corresponding to the missing data point is calculated; Specifically, using a representative first time interval as the step size, the necessary sampling times are filled into the abnormal interval, thereby generating a set of clearly defined target timestamps for insertion of placeholders and subsequent completion calculations. In implementation, the previous normal timestamp of the abnormal interval is usually taken as the starting point, and candidate times are generated by accumulating them sequentially according to the first time interval as the step size. During the generation process, the tolerance strategy of the reference interval is referenced to adjust the last margin to take into account clock offset or residual error. For very large intervals, segment processing or correction of the step size can be combined with surrounding multi-channel information. Each generated time is a target timestamp, and the missing segment number to which the target timestamp belongs and its sequence number within the segment are recorded. The missing times are accurately calculated using a robust reference interval as the scale and the complete time element information is recorded, providing accurate time positions for subsequent insertion of placeholders in the sequence and application of time domain and spatial domain models for completion, ensuring the consistency and traceability of the completion results on the time axis.

[0032] Based on the target timestamp, missing data points in the original IMU data are detected and marked to obtain the missing data points.

[0033] Specifically, the target timestamp generated in the previous step is transformed into identifiable missing placeholder records in the data structure, facilitating subsequent autoregressive modeling, physical correlation estimation, and spatiotemporal fusion processing. In implementation, a series of placeholder records are inserted or mapped into the original IMU sequence. Each placeholder record contains metadata such as the target timestamp, missing identifier, missing segment number, estimated missing sequence number, effective window lengths before and after, initial confidence level, and missing cause. If necessary, adjacent small-interval missing segments are merged into the same missing segment, and segment-level metadata is updated. After marking, an IMU time series with a missing mask and structured metadata is output for subsequent temporal autoregressive prediction and spatial domain physical estimation. Structured placeholders are generated based on the target timestamp, and detailed metadata is saved, making the missing information machine-identifiable and quantifiable at the data level, thereby supporting accurate completion calculations, confidence assessment of completion results, and subsequent auditing and backtracking.

[0034] Preferably, the step of using a pre-established spatiotemporal fusion model to perform missing data point completion processing to obtain the target IMU data after missing data completion processing includes: Based on the target timestamp, the continuous valid data points in the original IMU data before the missing data point are processed to obtain a time-domain autoregressive model for missing data completion processing; Specifically, based on the target timestamp, time-series features are extracted using consecutive valid data points preceding the missing point, and a time-domain autoregressive model is established. Consecutive valid data points refer to raw IMU data that are not missing on the time axis and have passed validity checks; these data points reflect the true trend of sensor data changes. By setting the autoregressive order and using these historical data points, model parameters can be estimated, enabling the autoregressive model to capture the dynamic characteristics of the time series. The resulting time-domain autoregressive model can generate predicted values ​​when missing points occur, thus achieving completion in the time dimension.

[0035] Based on the physical relationship between the accelerometer and gyroscope in the IMU corresponding to the original IMU data, correlation modeling is performed on the sensor data of the accelerometer and gyroscope to obtain a spatial domain model for missing data completion processing. Specifically, the focus of this process is establishing the physical constraints between the sensors. Accelerometers in the IMU measure linear acceleration, while gyroscopes measure angular velocity. Using the principles of rigid body kinematics, the mathematical relationship between these two types of sensors can be derived. Specifically, using the angular velocity and angular acceleration of the gyroscope, the theoretical acceleration count can be calculated and compared with the actual measured values ​​to construct an error model. In this way, a spatial domain model can be formed that can estimate based on gyroscope data when accelerometer data is missing. The purpose of this model is to ensure that the completed data not only satisfies numerical continuity but also conforms to the physical laws governing the sensors.

[0036] The spatiotemporal fusion model is determined based on the time-domain autoregressive model and the spatial domain model. Specifically, the aforementioned time-domain autoregressive model and spatial domain model are combined to construct a spatiotemporal fusion model. The time-domain autoregressive model provides the predictive capability of time series data, while the spatial domain model provides the physical constraints between multiple sensors. The combination of the two allows the model to simultaneously consider temporal continuity and spatial consistency. In practical implementation, a comprehensive completion result can be determined through weighted fusion, error minimization, or adaptive weight allocation. The resulting spatiotemporal fusion model can both predict trends using historical data and correct deviations using the physical relationships between sensors, thereby enhancing the accuracy and robustness of the completion process.

[0037] Based on the spatiotemporal fusion model, the missing data points are filled in to obtain the target IMU data.

[0038] Specifically, a spatiotemporal fusion model is applied to complete the missing data points detected and labeled in the original IMU data. The model first uses the time-domain autoregressive component to predict the values ​​of the missing points, and simultaneously uses the spatial domain model to calculate estimated values ​​derived from the gyroscope. The two results are then fused to generate the final completed data points. These completed data points are inserted into the missing positions, replacing the original gaps and thus restoring a complete and continuous IMU data sequence. The resulting target IMU data not only maintains numerical smoothness and coherence but also physically conforms to the sensor characteristics of an inertial measurement unit, making it directly applicable to navigation, attitude estimation, and motion tracking applications, ensuring the data's practicality and reliability.

[0039] Preferably, the step of processing consecutive valid data points in the original IMU data before the missing data point according to the target timestamp to obtain a time-domain autoregressive model for missing data completion includes: Based on the target timestamp, consecutive valid data points preceding the missing data point are extracted to obtain a historical data sequence for modeling. Specifically, the target timestamp refers to the time identifier of the missing data point to be filled, and consecutive valid data points refer to a series of sampling records that have passed the missing data detection and validity test before that time and are temporally adjacent; the purpose of this step is to extract a historical window that can represent recent dynamic characteristics from the original IMU sequence, such as the first N valid data points. This provides training samples for subsequent time-domain modeling. The implementation first scans forward from the target timestamp as the dividing point, selecting a pre-set length or a number of effective samples determined based on information criteria. If necessary, time alignment is performed on multi-channel data, outliers are removed, or robust filtering is used to smooth minor fluctuations. Subsequently, the selected samples are homogenized and DC bias removed to improve modeling stability. Finally, a historical data sequence is output in chronological order for parameter estimation. Historical windows are defined based on the target timestamp, and preprocessing is performed on multi-channel data, outliers, and time-series consistency to ensure the representativeness and stability of the data used for modeling, thus laying a data foundation for accurate estimation of autoregressive model parameters.

[0040] Based on the historical data sequence, the parameters of the autoregressive model are calculated using the least squares method to obtain the autoregressive model coefficients; Specifically, an autoregressive model is first constructed based on historical data sequences, and its mathematical expression is as follows: Where, x t Data representing time t, a i The coefficients of the autoregressive model are ε. t This is the noise error term. Next, the coefficient a needs to be adjusted.i Estimate the parameters. Using the least squares method, the optimal parameters can be solved while minimizing the sum of squared overall prediction errors. The objective function is: Where T is the total number of data points. The purpose of this process is to use the statistical regularities of historical sequences to fit the model, so that the estimated a... i This reflects the dependencies between historical data points. The final result is a set of autoregressive model coefficients, which will play a central role in predicting missing data points.

[0041] The time-domain autoregressive model is constructed based on the historical data sequence and the autoregressive model coefficients.

[0042] Specifically, after obtaining the historical data sequence and model coefficients, a complete time-domain autoregressive model can be constructed. This model takes historical data points as input and uses the estimated parameters for prediction calculations. When the target timestamp corresponding to a missing point appears, the model calculates the predicted value based on the first NNN valid data points, i.e.: in, This represents the predicted value for the missing data. Through continuous iteration, the model can also predict multiple consecutive missing data points. Its advantage lies in the fact that the autoregressive model, based on the statistical regularities of time series data, can better maintain the smoothness and trend consistency of the data, thus providing reliable time-domain support for IMU data missing data completion.

[0043] Preferably, the step of performing correlation modeling on the sensor data of the accelerometer and gyroscope based on the physical relationship between the accelerometer and gyroscope in the IMU corresponding to the original IMU data to obtain a spatial domain model for missing data completion includes: Based on the principles of rigid body kinematics, the linear acceleration of the accelerometer in the IMU is correlated with the angular velocity and angular acceleration of the gyroscope to obtain the physical relationship between acceleration and angular velocity and angular acceleration. Specifically, the kinematics of a rigid body refers to the physical laws describing the acceleration of any point on a rigid body under translational and rotational motions; the position vector r refers to the installation offset vector of the accelerometer relative to the center of rotation; angular velocity ω and angular acceleration... These are the angular velocities directly measured by the gyroscope and their derivatives with respect to time (obtainable through angular velocity sequence differencing). The purpose of this step is to establish a quantitative relationship between the accelerometer and gyroscope observations at the physical level, clarifying how the Coriolis, centrifugal, and tangential components generated by rotation are superimposed on the accelerometer readings. This provides a theoretical basis for subsequent acceleration calculations using gyroscope information, quantization errors, and missing data correction. The implementation process is as follows: First, confirm the installation coordinate system of each sensor in the IMU and calibrate the position vector r of the accelerometer relative to the rotation center; perform time synchronization and low-pass filtering on the gyroscope's angular velocity sequence to suppress high-frequency noise; then calculate the angular acceleration through numerical differencing (e.g., center differencing) and using a sampling interval Δt. ; Calculate the acceleration components time-by-time according to the rigid body kinematics formulas in a unified coordinate system. The general form can be written as: Where a represents the total acceleration measured by the accelerometer, a lin The first term represents the linear translational acceleration of the rigid body containing the sensor; the second term is a centrifugal or centripetal related term; and the third term is the tangential term caused by angular acceleration. Depending on the specific working conditions (e.g., in cases of small displacement dominated by rotation), a can be... lin It is considered zero or estimated from other sources. During implementation, attention should be paid to coordinate consistency in cross product operations, smoothing or regularization to address noise amplification in difference calculations, and time base correction when sampling intervals are not constant. On one hand, this physical relationship provides a sensor-to-sensor mapping centered on physical constraints, making it feasible to extrapolate acceleration using gyroscope data; on the other hand, it provides a clear reference model for subsequent error modeling and missing value estimation, enabling the combination of empirical interpolation or pure statistical prediction with physical estimation, significantly improving the physical consistency and robustness of the completion results, and reducing phase or amplitude deviations caused by a single interpolation method.

[0044] Based on the angular velocity, the angular acceleration, and the preset position vector of the accelerometer relative to the rotation center, the theoretical acceleration value of the accelerometer is calculated to obtain the theoretical calculation data of the accelerometer. Specifically, angular velocity refers to the angular rate sequence directly output by the gyroscope, angular acceleration is the derivative of angular velocity with respect to time (obtainable by differencing the angular velocity sequence), and the position vector refers to the offset vector r of the accelerometer relative to the center of rotation in the sensor coordinate system. The purpose of this step is to calculate, at a physical level, the theoretical values ​​that the accelerometer should observe under given rotational conditions, thus providing a physical reference for subsequent error quantification and missing value estimation. The specific implementation process includes: first, performing time-series alignment and low-pass filtering on the gyroscope angular velocity sequence to suppress high-frequency noise; then, using numerical differentiation to obtain the angular acceleration sequence; next, calculating the eccentric and tangential terms generated by rotation in a unified coordinate system, i.e., calculating the theoretical acceleration time-by-time according to the rigid body kinematics formula, for example, expressed as: For the cross product operation, coordinate consistency must be ensured and sampling interval correction must be considered; if the linear translation acceleration a lin Not directly available, the data can be approximated using zero or estimated from other sensors / filters, depending on the scenario. Implementation details also include applying appropriate filtering or regularization before and after the difference to suppress noise amplification, performing coordinate transformation and clock synchronization in multi-sensor scenarios, and handling cases where the sampling interval is not constant during calculation. Mapping gyroscope information to theoretical accelerometer readings using physical formulas provides a rigid body dynamics-based reference sequence that can be used to quantify systematic errors in accelerometers, perform error correction, and serve as a reliable estimation basis when accelerometer data is missing, thereby significantly improving the physical rationality and accuracy of subsequent completion and correction.

[0045] The difference between the theoretical calculation data and the actual measurement data of the accelerometer is calculated to obtain the accelerometer measurement error value; Specifically, the theoretically calculated acceleration obtained in the second step is denoted as a. theo The measured value of the accelerometer is recorded as a. meas (Recorded separately along the three axes), the purpose of this step is to quantify the deviation between the two to reveal the systematic and random errors of the accelerometer, providing a direct basis for subsequent calibration modeling and confidence assessment. Implementation requires first completing time alignment and coordinate transformation to ensure a... theo With a measThe data is compared at the same sampling time and in the same coordinate system. Then, the residual sequence is calculated time-by-time along each axis, typically expressed as e(t) = ameas(t) − atheo(t) (calculated separately for each of the three axes). Statistical analysis and preprocessing are performed on this residual sequence, including outlier removal (based on thresholds or median absolute deviation), low-pass filtering or moving median smoothing to suppress high-frequency noise, and calculating the mean and variance of the residuals using a sliding window to capture time-varying bias and noise levels. Furthermore, the residual sequence can be further decomposed to distinguish between zero-bias components, scale factor errors, and random noise (e.g., using linear regression to estimate the scale factor and bias), and a confidence index for each residual point is calculated for subsequent fusion weighting. Based on the time-by-time difference calculation between theoretical and measured values, combined with robust statistical processing, the systematic bias and random fluctuations of the sensor can be quantitatively revealed and separated. This provides a reliable statistical basis for the construction of error correction models, bias compensation for missing point estimation, and weight allocation in spatiotemporal fusion, fundamentally improving the accuracy and robustness of completion and correction.

[0046] Based on the accelerometer measurement error, the accelerometer measurement data is corrected and modeled to obtain an error correction model for correcting the accelerometer data; Specifically, accelerometer measurement error refers to the residual sequence obtained by comparing the theoretical calculation data obtained in the second step with the actual measurement data of the accelerometer at each time step. These residuals can reflect systematic and random deviations such as zero-bias drift, scale factor error, inter-axis coupling, and environmental factors. The purpose of this step is to systematize these residuals into an estimable mathematical model or filter so that the accelerometer readings can be corrected to an output that is closer to the physical truth in both normal sampling and missing data completion scenarios. The implementation first performs time alignment, outlier removal, and low-pass filtering on the residual sequence to obtain robust error statistics. Then, an appropriate parameterization form is selected, such as a constant zero-bias model, a zero-bias scaling model, an extended linear model with temperature or environmental terms, or online estimation methods such as recursive least squares and Kalman filtering to fit these parameters. In multi-axis cases, coupling terms are simultaneously estimated and coordinate transformations are performed to ensure model consistency between the sensor and platform coordinate systems. Model parameters should support online updates to track sensor drift over time. Parameterized modeling based on physical reference residuals supports batch processing and online adaptive estimation, significantly reducing systematic bias and scale distortion of the accelerometer, and improving the consistency of the physical relationship between the accelerometer and gyroscope. This provides higher-confidence input for subsequent spatial domain estimation and spatiotemporal fusion. Based on the theoretical calculation data and the error correction model, when accelerometer data is missing, the acceleration value of the missing data point is estimated to obtain a missing data completion method. Specifically, the theoretical calculation data comes from the acceleration reference sequence derived from rigid body kinematics using gyroscope information. The error correction model comes from the parameterized corrector in the previous step. The purpose of this step is to provide a reliable alternative value by adding a correction term to the physical model when accelerometers are missing, thereby restoring data continuity. In implementation, for each time point to be filled, the theoretical acceleration at that moment is first calculated using the synchronized gyroscope angular velocity and angular acceleration. Then, the error correction model is applied to this theoretical value to compensate for systematic bias. At the same time, the estimated value is biased and confidence is assessed by combining historical residual statistics. In the case of continuous missing values, the estimated value is smoothed by time series or recursively pushed by short window to avoid abrupt changes. During the estimation process, it is calculated in parallel with the time domain prediction results so that the optimal solution can be selected through subsequent fusion. To enhance robustness, a degradation strategy should also be designed to revert to time domain interpolation only or retain the missing marker when the confidence is low or the gyroscope quality is insufficient. By using theoretical physical mapping as the main framework and applying dynamic corrections based on observation residuals, acceleration estimates that both comply with physical constraints and eliminate systematic biases are provided within the missing interval, thereby maximizing the restoration of the physical rationality and temporal continuity of the IMU sequence.

[0047] Based on the physical relationship, the error correction model, and the missing data completion method, the sensor data from the accelerometer and gyroscope are comprehensively modeled to obtain the spatial domain model.

[0048] Specifically, the various components of physical relationships, error correction, and missing data estimation are uniformly encapsulated into a runnable spatial domain model. The aim is to provide a comprehensive mapping across the entire time domain that can both correct normal observations and estimate missing data, thereby achieving spatial consistency and redundancy among sensors. In implementation, the physical mapping module, error correction module, and missing data estimation module are combined in a modular structure. Input / output interfaces are established to accept synchronized gyroscope data, correction parameters, and historical residual statistics. During runtime, the model outputs the corrected acceleration and corresponding confidence level. Internally, error propagation or covariance estimation can be used to measure uncertainty. If necessary, regression or Bayesian estimation methods are used to weightedly fuse estimation results from different sources and provide an online adaptive update mechanism. Furthermore, at the system integration level, unified standards must be ensured for handling coordinate transformations, inconsistent sampling intervals, and multi-axis coupling. By combining theoretical physical constraints with empirical corrections and missing estimates to form an integrated model, we can achieve unified correction and completion of accelerometer and gyroscope data, improve spatial consistency, reduce system errors introduced by completion, and enhance robustness in the event of sensor failure or interference, thereby providing reliable input for downstream navigation and attitude calculation.

[0049] Preferably, the step of performing missing data point completion processing based on the spatiotemporal fusion model to obtain the target IMU data includes: Based on the time-domain autoregressive model, the missing data points are predicted and calculated to obtain the first completion result based on the time domain; Specifically, the time-domain autoregressive model refers to a time series predictor obtained by estimating parameters from continuous valid data prior to the missing point. Its purpose is to recover the trend value at the missing point using historical dynamic information. In implementation, first-order or multi-order predicted values ​​are typically generated sequentially using a forward recursive approach. Single-point missing data is predicted once, while multiple consecutive missing data points employ a recursive backfilling strategy with monitoring of the recursion error. To reduce prediction drift, the maximum recursion step size can be limited, short-window re-estimation of model coefficients can be used, or a prediction variance estimate can be attached to the predicted values ​​to quantify uncertainty. The technical characteristic of this step is that it uses historical statistical characteristics for time series prediction, which has the beneficial effect of maintaining data smoothness and trend consistency, providing a primary reference for the temporal continuity of the completion process, and providing a credibility evaluation for subsequent fusion through prediction variance.

[0050] Based on the spatial domain model, the missing data points are estimated and calculated to obtain a second completion result based on the spatial domain; Specifically, the spatial domain model refers to an estimator based on the rigid body kinematics relationship between accelerometers and gyroscopes, combined with error correction. Its purpose is to provide physically reasonable alternative values ​​at missing moments using cross-sensor physical constraints. In implementation, the synchronized gyroscope angular velocity and its differential angular acceleration are used as inputs. The theoretical acceleration is calculated by combining the accelerometer's position parameters relative to the rotation center, and the error correction model is applied to the theoretical value to correct systematic biases. In multi-axis and multi-channel scenarios, coordinate transformation and axis coupling processing are required, and uncertainty assessments are provided for the estimation results. The technical characteristic of this step is that it uses physical equations as its core and combines them with observation error correction. The beneficial effect is that it can fill in missing values ​​while maintaining physical consistency, effectively suppressing non-physical biases that may be introduced by pure statistical predictions. It is particularly suitable for rotation-dominated scenarios or scenarios where multiple sensors are redundantly available.

[0051] The first completion result and the first and second completion results are weighted and fused to obtain fused completion data; Specifically, the purpose of this step is to organically combine trend prediction in the time domain with physical estimation in the spatial domain, thereby compensating for the shortcomings of a single source. In practice, this is typically achieved by weighting the two types of estimates based on their confidence levels or variances. The weights can be dynamically adjusted from pre-estimated prediction variances, historical residual statistics, or online performance evaluations. Fusion can be performed independently for each axis and each time point, or collaboratively within a covariance framework considering channel coupling. The fusion result should simultaneously output the estimated mean and a measure of uncertainty. The technical feature of this step is uncertainty-driven adaptive weighted fusion, which reduces the impact of single model errors on the final result. In most cases, it yields a more robust and accurate complete output than any single source, and provides more reliable initial values ​​and confidence information for subsequent optimization.

[0052] Based on a preset error minimization criterion or dynamic weighting strategy, the fused and completed data is optimized to obtain completed data points; Specifically, the purpose of this step is to further correct the completion result based on the initial fusion value by minimizing the residuals or maximizing the posterior probability to meet the requirements of temporal continuity, smoothness, and physical constraints. In implementation, an objective function containing time-domain residuals, spatial-domain residuals, and smoothing or regularization terms can be constructed and solved using least squares or Bayesian estimation methods. Alternatively, the fusion weights can be adjusted based on the observed residuals through iterative reweighting. If necessary, boundary smoothing constraints can be added during the optimization process to avoid jumps between the completed segment and the actual observation point. After optimization, the final value is output for each completed point, and its uncertainty estimate is updated. The technical feature of this step is to constrain and correct the fusion result using explicit optimization criteria. Its beneficial effects are to minimize the overall error of the joint estimation, balance time and spatial constraints, and prevent long-term shifts caused by the accumulation of local errors, making the final completed value more reliable both numerically and physically.

[0053] Based on the completed data points, the missing positions in the original IMU data are replaced to obtain the target IMU data.

[0054] Specifically, the optimized completed values ​​are written into the original sequence to form complete target IMU data usable downstream. In implementation, completed data points are inserted into or replace placeholder missing records in the original data sequence according to their corresponding target timestamps. Simultaneously, the source of the completed data, confidence level, fusion strategy, and optimization iteration information are recorded in the metadata. After insertion, a short-term consistency check should be performed, including a smoothness check with neighboring real observations and a residual check based on the physical model. If the check fails, alternative strategies are triggered or low confidence levels are marked. The technical feature of this step is the structured write-back of the completed results with complete audit information. Its beneficial effect is the generation of a continuous, physically consistent, and traceable set of target IMU data, allowing downstream modules such as navigation, attitude estimation, and motion analysis to directly use it and adopt different processing strategies based on confidence levels when needed, thereby improving the overall reliability and interpretability of the system.

[0055] Example 2 Please see Figure 2 Embodiment 2 of the present invention also provides an IMU data missing completion device based on spatiotemporal fusion modeling, the device comprising: The data acquisition module is used to acquire the raw IMU data to be filled in. The missing data detection and marking module is used to perform missing data detection and marking on the original IMU data to obtain the missing data points in the original IMU data; The missing data completion module is used to perform missing data completion processing on the missing data points using a pre-established spatiotemporal fusion model, so as to obtain the target IMU data after missing data completion processing.

[0056] Specifically, the IMU data missing completion device based on spatiotemporal fusion modeling provided in this embodiment of the invention is adopted. The device includes: a data acquisition module for acquiring the original IMU data to be missing and completed; a missing detection and marking module for detecting and marking the missing data in the original IMU data to obtain missing data points; and a missing completion module for using a pre-established spatiotemporal fusion model to complete the missing data points and obtain the target IMU data after missing completion. This device first constructs an autoregressive model in the time domain, using continuous valid data points before the missing points for parameter estimation, thereby achieving trend prediction based on historical sequences. Simultaneously, it establishes a physical relationship model between the accelerometer and gyroscope in the spatial domain, calculates theoretical acceleration using rigid body kinematics equations, and performs error correction based on actual measurements to obtain reasonable estimates in the case of sensor missingness. Then, the time domain prediction results and spatial domain estimation results are fused, and a weighted sum error optimization strategy is used to generate the final completed data points. The above design overcomes the limitations of traditional technologies that rely on single interpolation or filtering and cannot simultaneously consider temporal characteristics and the correlation between multiple sensors. It achieves accurate completion of missing IMU data points, effectively improves the continuity and reliability of data, and provides stable data support for applications such as navigation, attitude estimation, and motion tracking.

[0057] Example 3 In addition, combined Figure 1 The IMU data missing completion method based on spatiotemporal fusion modeling described in Embodiment 1 of the present invention can be implemented by an electronic device. Figure 3 A schematic diagram of the hardware structure of the electronic device provided in Embodiment 3 of the present invention is shown.

[0058] Electronic devices may include processors and memory storing computer program instructions.

[0059] Specifically, the processor may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement embodiments of the present invention.

[0060] The memory may include a large-capacity storage device for data or instructions. For example, and not limitingly, the memory may include a hard disk drive (HDD), a floppy disk drive, flash memory, an optical disk drive, a magneto-optical disk drive, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory may include removable or non-removable (or fixed) media. Where appropriate, the memory may be internal or external to a data processing device. In a particular embodiment, the memory is a non-volatile solid-state memory. In a particular embodiment, the memory includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0061] The processor reads and executes computer program instructions stored in memory to implement any of the spatiotemporal fusion modeling-based IMU data missing completion methods in the above embodiments.

[0062] In one example, the electronic device may also include a communication interface and a bus. For example, Figure 3 As shown, the processor, memory, and communication interface are connected via a bus and communicate with each other.

[0063] The communication interface is mainly used to enable communication between various modules, devices, units and / or equipment in the embodiments of the present invention.

[0064] A bus, including hardware, software, or both, couples components of the device together. For example, and not limitingly, a bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, a bus may include one or more buses. While specific buses are described and illustrated in embodiments of the invention, the invention contemplates any suitable bus or interconnect.

[0065] Example 4 Furthermore, in conjunction with the IMU data missing completion method based on spatiotemporal fusion modeling in Embodiment 1 above, Embodiment 4 of the present invention can also provide a computer-readable storage medium for implementation. This computer-readable storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the IMU data missing completion methods based on spatiotemporal fusion modeling in the above embodiments.

[0066] In summary, the embodiments of the present invention provide a method, apparatus, and device for IMU data missing completion based on spatiotemporal fusion modeling.

[0067] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0068] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the required tasks. The programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0069] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant locality, and corresponding operation entry points shall be provided for the user to choose to authorize or refuse.

[0070] It should also be noted that the exemplary embodiments mentioned in this invention describe methods or systems based on a series of steps or apparatus. However, this invention is not limited to the order of the steps described above; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0071] The above description is merely a specific embodiment of the present invention. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the protection scope of the present invention.

Claims

1. A method for completing missing IMU data based on spatiotemporal fusion modeling, characterized in that, The method includes: Obtain the original IMU data to be filled in; Missing data points in the original IMU data are obtained by performing missing detection and marking on the original IMU data; Using a pre-established spatiotemporal fusion model, the missing data points are filled in to obtain the target IMU data after the missing data points are filled in.

2. The IMU data missing completion method based on spatiotemporal fusion modeling according to claim 1, characterized in that, The step of performing missing data detection and labeling on the original IMU data to obtain missing data points in the original IMU data includes: Based on the timestamp sequence corresponding to each data point in the original IMU data, the sampling time interval between adjacent data points is calculated to obtain the actual time interval between adjacent data points; The actual time interval is compared with a preset time interval threshold. If the actual time interval is the same as the time interval threshold, then the adjacent data points are normal data points. If the actual time interval is not the same as the time interval threshold, then the adjacent data points are abnormal data points; Based on the first timestamp corresponding to the normal data point and the second timestamp corresponding to the abnormal data point, the original IMU data is subjected to missing data detection and marking to obtain the missing data point.

3. The IMU data missing completion method based on spatiotemporal fusion modeling according to claim 2, characterized in that, The missing data points are obtained by performing missing detection and marking on the original IMU data based on the first timestamp corresponding to the normal data points and the second timestamp corresponding to the abnormal data points, including: Based on the first timestamp, the first time interval between normal adjacent data points is calculated; Based on the second timestamp, the second time interval between adjacent abnormal data points is calculated; Based on the first time interval and the second time interval, the target timestamp corresponding to the missing data point is calculated; Based on the target timestamp, missing data points in the original IMU data are detected and marked to obtain the missing data points.

4. The IMU data missing completion method based on spatiotemporal fusion modeling according to claim 3, characterized in that, The missing data points are filled in using a pre-established spatiotemporal fusion model to obtain the target IMU data after missing data filling, including: Based on the target timestamp, the continuous valid data points in the original IMU data before the missing data point are processed to obtain a time-domain autoregressive model for missing data completion processing; Based on the physical relationship between the accelerometer and gyroscope in the IMU corresponding to the original IMU data, correlation modeling is performed on the sensor data of the accelerometer and gyroscope to obtain a spatial domain model for missing data completion processing. The spatiotemporal fusion model is determined based on the time-domain autoregressive model and the spatial domain model. Based on the spatiotemporal fusion model, the missing data points are filled in to obtain the target IMU data.

5. The IMU data missing completion method based on spatiotemporal fusion modeling according to claim 4, characterized in that, The step of processing consecutive valid data points in the original IMU data before the missing data point according to the target timestamp to obtain a time-domain autoregressive model for missing data completion includes: Based on the target timestamp, consecutive valid data points preceding the missing data point are extracted to obtain a historical data sequence for modeling; Based on the historical data sequence, the parameters of the autoregressive model are calculated using the least squares method to obtain the autoregressive model coefficients; The time-domain autoregressive model is constructed based on the historical data sequence and the autoregressive model coefficients.

6. The IMU data missing completion method based on spatiotemporal fusion modeling according to claim 4, characterized in that, The step of performing correlation modeling on the sensor data of the accelerometer and gyroscope based on the physical relationship between the accelerometer and gyroscope in the IMU corresponding to the original IMU data, to obtain a spatial domain model for missing data completion processing, includes: Based on the principles of rigid body kinematics, the linear acceleration of the accelerometer in the IMU is correlated with the angular velocity and angular acceleration of the gyroscope to obtain the physical relationship between acceleration and angular velocity and angular acceleration. Based on the angular velocity, the angular acceleration, and the preset position vector of the accelerometer relative to the rotation center, the theoretical acceleration value of the accelerometer is calculated to obtain the theoretical calculation data of the accelerometer. The difference between the theoretical calculation data and the actual measurement data of the accelerometer is calculated to obtain the accelerometer measurement error value; Based on the accelerometer measurement error, the accelerometer measurement data is corrected and modeled to obtain an error correction model for correcting the accelerometer data; Based on the theoretical calculation data and the error correction model, when accelerometer data is missing, the acceleration value of the missing data point is estimated to obtain a missing data completion method. Based on the physical relationship, the error correction model, and the missing data completion method, the sensor data from the accelerometer and gyroscope are comprehensively modeled to obtain the spatial domain model.

7. The IMU data missing completion method based on spatiotemporal fusion modeling according to claim 4, characterized in that, The step of performing missing data point completion processing based on the spatiotemporal fusion model to obtain the target IMU data includes: Based on the time-domain autoregressive model, the missing data points are predicted and calculated to obtain the first completion result based on the time domain; Based on the spatial domain model, the missing data points are estimated and calculated to obtain a second completion result based on the spatial domain; The first completion result and the first and second completion results are weighted and fused to obtain fused completion data; Based on a preset error minimization criterion or dynamic weighting strategy, the fused and completed data is optimized to obtain completed data points; Based on the completed data points, the missing positions in the original IMU data are replaced to obtain the target IMU data.

8. A device for completing missing IMU data based on spatiotemporal fusion modeling, characterized in that, The device includes: The data acquisition module is used to acquire the raw IMU data to be filled in. The missing data detection and marking module is used to perform missing data detection and marking on the original IMU data to obtain the missing data points in the original IMU data; The missing data completion module is used to perform missing data completion processing on the missing data points using a pre-established spatiotemporal fusion model, so as to obtain the target IMU data after missing data completion processing.

9. An electronic device, characterized in that, include: At least one processor, at least one memory, and computer program instructions stored in the memory, which, when executed by the processor, implement the method as described in any one of claims 1-7.

10. A storage medium storing computer program instructions thereon, characterized in that, The method as described in any one of claims 1-7 is implemented when the computer program instructions are executed by the processor.

Images