Multi-modal sensor embedded self-calibration system and real-time compensation method

Abstract

The invention relates to the technical field of multi-modal sensors, in particular to a multi-modal sensor embedded self-calibration system and a real-time compensation method, and the method comprises the following steps: S1, data acquisition: a multi-modal data acquisition mode is adopted, a multi-modal data acquisition module is responsible for acquiring original data of a camera, an inertial measurement unit and a microphone, and the original data of the camera, the inertial measurement unit and the microphone are acquired; the camera collects image data at the frequency of 30 Hz. The method has the advantages of comprehensive data acquisition, accurate data processing, comprehensive and deep feature extraction, scientific and flexible calibration parameters and efficient and continuous real-time compensation, and a multi-modal data fusion technology is adopted in the actual use process; a composite sensing system covering space positioning, dynamic sensing and acoustic analysis is constructed, and the multi-source heterogeneous data fusion scheme can comprehensively capture multi-dimensional features in a complex scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of multi-modal sensor, in particular to a multi-modal sensor embedded self-calibration system and real-time compensation method. BACKGROUND

[0002] Multi-modal sensor is an integrated sensing device, which breaks through the limitation of single sensor by fusing two or more physical quantities or environmental information such as vision (image), hearing (sound), touch (vibration / pressure), smell (gas composition), etc. The design inspiration comes from the human perception system - humans perceive the world through the coordinated work of multiple organs such as eyes, ears, and skin. Multi-modal sensor simulates this mechanism to achieve more comprehensive and three-dimensional environmental cognition.

[0003] The prior art is difficult to realize the synchronous collection of camera visual information, inertial measurement unit motion data and microphone audio signals, leading to difficulty in constructing a composite perception system, and being unable to comprehensively capture multi-dimensional features in a complex scene, especially in a dynamic environment, time and space information is easily dislocated, affecting the real-time and accuracy of comprehensive perception, the traditional system adopts a unified sampling frequency, and the working frequency is not dynamically adjusted according to the characteristics of the sensor, leading to insufficient data effectiveness or resource waste, it is difficult to balance data quality and system efficiency, the high-frequency noise of the inertial measurement unit acceleration data, and due to the lack of a dynamic noise reduction algorithm, it is difficult to suppress noise while retaining effective motion features, leading to insufficient data smoothness, affecting the accuracy of subsequent motion analysis, the dimensional differences of multi-modal data are not unified through feature space normalization, leading to unbalanced feature weights in cross-modal correlation analysis, low model training efficiency, and even algorithm convergence failure caused by data scale differences, in addition, the prior art relies on a single synchronization mechanism, and the time alignment error of multi-sensor data cannot be controlled within microseconds, especially in a high-speed dynamic scene, time and space inconsistency will lead to decision delay or misjudgment, the traditional system does not comprehensively analyze the sensitivity, nonlinearity, hysteresis effect and other dynamic response characteristics of the sensor, leading to insufficient accuracy of the digital twin model, deviation between the calibration parameters and the actual working condition, error accumulation in long-term operation, affecting the measurement stability, the calibration parameter generation algorithm does not fuse the real-time response curve of the sensor, the standard physical model and the environmental parameter compensation term, and lacks efficient optimization means, leading to poor generalization ability of the calibration result, and it is difficult to adapt to changes in temperature, humidity and other environmental changes, the existing system lacks an online performance evaluation mechanism, and the control parameters cannot be dynamically adjusted according to the system state change or environmental disturbance, leading to performance optimization lag, especially in long-term operation scenarios, the measurement accuracy gradually decreases, for the influence of temperature, humidity and other environmental factors on the measurement accuracy, the traditional technology does not establish a coupled mathematical model, relies on fixed compensation rules, and cannot intelligently predict the compensation amount, leading to a significant increase in measurement error in high-temperature and high-humidity environments, the compensation algorithm library lacks a special processing module, and the optimal strategy is not dynamically selected through real-time data stream analysis, leading to a mismatch between the compensation effect and the scene demand, and poor spatio-temporal consistency of the output data, the existing system does not use a rolling update mechanism to continuously optimize the compensation parameters, and the data buffer refresh lags, leading to a disconnection between the compensation parameters and the actual working condition in long-term operation, and the inability to form a closed-loop optimization, especially in dynamic environments. SUMMARY

[0004] The present application aims to provide a multi-modal sensor embedded self-calibration system and real-time compensation method, which has the advantages of comprehensive data collection, accurate data processing, comprehensive and in-depth feature extraction, scientific and flexible calibration parameters, and efficient and continuous real-time compensation, solving the problems raised in the above background art.

[0005] In order to achieve the above object, the present application provides the following technical solutions: a multi-modal sensor embedded self-calibration system and real-time compensation method, the method comprises the following steps: S1: data acquisition: using multi-modal data acquisition method, the multi-modal data acquisition module is responsible for collecting the original data of the camera, the inertial measurement unit and the microphone, the camera collects image data at a frequency of 30Hz, the inertial measurement unit collects acceleration and angular velocity data at a frequency of 100Hz.

[0006] Data preprocessing: the collected original data contains noise and interference, for acceleration data, Kalman filter is used to remove high-frequency noise, improve the smoothness and accuracy of data, at the same time, normalization processing can be carried out on the data, so that the data of different modalities are unified into the same scale range, which is convenient for subsequent analysis and comparison.

[0007] Data synchronization: the method of combining hardware synchronization and software alignment is adopted, the hardware synchronization uses GPS timestamp or network time protocol to unify the clocks of all sensors; the software alignment can align the data on the time axis by interpolation, and the data of low-frequency sensors is interpolated to the timestamp of high-frequency sensors.

[0008] S2: response feature extraction: the preprocessed data is analyzed by using the sensor response analysis module to extract the response features of the sensor: sensitivity, linearity, hysteresis, sensitivity reflects the sensitivity of the sensor to the measured change, linearity represents the linear relationship between the output and input of the sensor, and hysteresis describes the difference between the outputs of the sensor in the positive and negative strokes.

[0009] S3: calibration parameter generation: standard model establishment: the calibration parameter generation module generates calibration parameters for different modal sensors according to the results of sensor response analysis, combined with the preset standard model and environmental parameters, the standard model can be established based on the theoretical characteristics and experimental data of the sensor, which describes the relationship between the output and input of the sensor in the ideal case, and the environmental parameters include temperature, humidity, pressure and other factors, which will affect the performance of the sensor and need to be considered in the calibration process.

[0010] Adaptive algorithm application: the generation of calibration parameters adopts adaptive algorithm for dynamic adjustment, the adaptive algorithm can automatically adjust the calibration parameters according to the real-time performance of the sensor and environmental changes.

[0011] Environmental factor compensation: the mathematical model between environmental factors and sensor performance can be established to correct the calibration parameters according to the real-time measured environmental parameters, in the environment with large temperature change, the sensitivity can be adjusted through temperature compensation algorithm to ensure that the sensor can maintain high measurement accuracy under different temperature conditions.

[0012] S4: Real-time compensation: The real-time output data of the sensor is compensated according to the generated calibration parameters. For linear sensors, a simple linear transformation can be used for compensation. For nonlinear sensors, more complex nonlinear compensation algorithms are needed. The compensation process is carried out in real time in the embedded system to ensure the accuracy of the sensor output data.

[0013] Data update and processing: During real-time compensation, the output data of the sensor is continuously updated and compensated according to the latest calibration parameters. At the same time, further processing and analysis of the compensated data are needed, such as feature extraction and target recognition, to meet the needs of different application scenarios.

[0014] Further, as a preferred embodiment of the present application, in step S1, Kalman filter algorithm is used to remove high-frequency noise from acceleration data, which specifically includes establishing a state space model to describe the dynamic characteristics of the sensor, and recursively estimating the optimal state value through a prediction-update cycle, wherein the state transition matrix is determined according to the inherent frequency and damping ratio of the sensor.

[0015] Further, as a preferred embodiment of the present application, in step S1, hardware synchronization uses GPS second pulse signal to synchronize with local clock, and software alignment uses cubic spline interpolation algorithm to align the timestamp of camera image data to the 100Hz time reference of inertial measurement unit, ensuring that the time alignment error of multi-modal data is less than 5ms.

[0016] Further, as a preferred embodiment of the present application, in step S2, linearity is fitted by least squares method to input-output curve, and the ratio of maximum nonlinear error to range is calculated; hysteresis characteristics are obtained by forward and reverse stroke scanning to get output difference, and the ratio of maximum difference to range is taken as hysteresis error index.

[0017] Further, as a preferred embodiment of the present application, in step S3, the influence of environmental parameters is modeled by a polynomial regression model, the model coefficients are fitted by least squares method based on experimental data, and a cross-validation mechanism is introduced to ensure the generalization ability of the model within the temperature range of-20℃-80℃.

[0018] Further, as a preferred embodiment of the present application, in step S3, the adaptive algorithm uses recursive least squares method to update the calibration parameters through a sliding window, and the forgetting factor is set to 0.98.

[0019] Further, as a preferred embodiment of the present application, in step S3, humidity compensation uses lookup table method, a humidity-sensitivity correction coefficient table is established in advance, and the correction coefficient corresponding to the current humidity value is queried in real time to linearly compensate the sensor output.

[0020] Further, as a preferred of the present application, in the step S3, the calibration parameters are corrected according to the real-time measured environmental parameters by establishing a mathematical model between environmental factors and sensor performance, and in the environment with large temperature change, the sensitivity is adjusted through the temperature compensation algorithm.

[0021] Further, as a preferred of the present application, in the step S4, the sensor output data is continuously updated in the real-time compensation process, compensated according to the latest calibration parameters, and the compensated data is further processed and analyzed through feature extraction and target recognition.

[0022] Advantages, the technical scheme of the present application has the following technical effects: the present application has the advantages of comprehensive data acquisition, accurate data processing, comprehensive and in-depth feature extraction, scientific and flexible calibration parameters, and efficient and continuous real-time compensation. In actual use, the multi-modal data fusion technology: by synchronously collecting camera visual information, inertial measurement unit motion data and microphone audio signals, a composite perception system covering spatial positioning, dynamic perception and acoustic analysis is constructed. This multi-source heterogeneous data fusion scheme can comprehensively capture multi-dimensional features in complex scenes and meet the comprehensive perception needs in dynamic environments. At the data acquisition level, the system uses an asynchronous sampling strategy, and each sensor works independently at the optimal frequency according to its own characteristics, which not only ensures data validity but also optimizes system resource utilization.

[0023] For the high-frequency noise problem of the inertial measurement unit acceleration data, the system uses an improved Kalman filter algorithm for dynamic noise reduction processing, which significantly improves the data smoothness while retaining the effective motion characteristics. To solve the problem of dimension difference of multi-modal data, a normalization method based on feature space is used to construct a unified data representation space, which lays a foundation for subsequent cross-modal correlation analysis. In terms of time synchronization, a hybrid synchronization mechanism combining hardware trigger synchronization and software timestamp correction is used to control the time alignment error of multi-sensor data to the microsecond level, ensuring the spatiotemporal consistency.

[0024] The system establishes a complete sensor characteristic representation system, analyzes the dynamic response characteristics such as sensitivity, nonlinearity and hysteresis effect, and constructs a high-precision sensor digital twin model. The calibration parameter generation algorithm combines the sensor real-time response curve, the standard physical model and the environmental parameter compensation term, and uses the least squares support vector machine for parameter optimization, so that the calibration result is closer to the actual working condition. The adaptive adjustment module can automatically adjust the control parameters according to the system state change and environmental disturbance through the online performance evaluation mechanism, dynamically optimizing the system performance.

[0025] To address the influence of environmental factors on measurement accuracy, the research team established a temperature and humidity-performance coupling mathematical model, and used a radial basis function neural network to realize intelligent prediction of environmental compensation. The compensation algorithm library contains special processing modules for different sensors, and dynamically selects the optimal compensation strategy through real-time data stream analysis to ensure the spatio-temporal consistency of the output data. The system uses a rolling update mechanism to continuously optimize the compensation parameters and refresh the data buffer. Finally, it outputs high-precision perception data that has undergone multiple levels of verification, and can be flexibly adapted to the needs of different application scenarios such as autonomous driving and industrial monitoring, forming a closed-loop optimized intelligent perception system.

[0026] It should be understood that all combinations of the aforementioned concepts and additional concepts described in greater detail below can be seen as part of the subject matter of the present disclosure as long as such concepts are not mutually inconsistent. BRIEF DESCRIPTION OF DRAWINGS

[0027] The accompanying drawings are included to provide a further understanding of the application, and constitute a part of the specification, illustrate the application, and are used to explain the application without limiting the application. In the drawings: Figure 1 The flowchart of the present application. DETAILED DESCRIPTION

[0028] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. In order to better understand the technical content of the present application, specific embodiments are given and described in conjunction with the accompanying drawings as follows. In the present disclosure, aspects of the present application are described with reference to the accompanying drawings, which show many illustrative embodiments. It should be understood that the various concepts and embodiments introduced above, as well as those described in greater detail below, can be implemented in any of a number of ways. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0029] As shown in the accompanying Figure 1 The present embodiment provides a multi-modal sensor embedded self-calibration system and real-time compensation method, which comprises the following steps: S1: Data acquisition: multi-modal data acquisition method is adopted, and the multi-modal data acquisition module is responsible for collecting the original data of the camera, the inertial measurement unit and the microphone. The camera collects image data at a frequency of 30Hz, and the inertial measurement unit collects acceleration and angular velocity data at a frequency of 100Hz.

[0030] Data preprocessing: The collected raw data contains noise and interference. For acceleration data, Kalman filter is used to remove high-frequency noise, improve data smoothness and accuracy, and normalize the data to the same scale range, facilitating subsequent analysis and comparison.

[0031] Data synchronization: A combination of hardware synchronization and software alignment is used. Hardware synchronization uses GPS timestamps or network time protocol to synchronize all sensor clocks. Software alignment aligns data on the time axis by interpolation, and interpolates low-frequency sensor data to high-frequency sensor timestamps.

[0032] S2: Response feature extraction: The preprocessed data is analyzed by the sensor response analysis module to extract the response features of the sensor: sensitivity, linearity, and hysteresis. Sensitivity reflects the sensitivity of the sensor to the measured change, linearity represents the linear relationship between the sensor output and input, and hysteresis describes the difference in output between the forward and reverse strokes.

[0033] S3: Calibration parameter generation: Standard model establishment: The calibration parameter generation module generates calibration parameters for different modal sensors based on the results of sensor response analysis, combined with pre-set standard models and environmental parameters. The standard model can be established based on the theoretical characteristics and experimental data of the sensor, describing the relationship between the output and input of the sensor in ideal conditions. Environmental parameters include temperature, humidity, pressure, etc., which will affect the performance of the sensor and need to be considered in the calibration process.

[0034] Adaptive algorithm application: The generation of calibration parameters uses adaptive algorithms for dynamic adjustment, which can automatically adjust the calibration parameters according to the real-time performance of the sensor and environmental changes.

[0035] Environmental factor compensation: A mathematical model between environmental factors and sensor performance can be established to correct the calibration parameters based on real-time environmental parameters. In environments with large temperature changes, the sensitivity can be adjusted through temperature compensation algorithms to ensure that the sensor maintains high measurement accuracy under different temperature conditions.

[0036] S4: Real-time compensation: The real-time compensation execution module compensates the real-time output data of the sensor based on the generated calibration parameters. For linear sensors, simple linear transformation can be used for compensation; for nonlinear sensors, more complex nonlinear compensation algorithms are needed. The compensation process is performed in real time in the embedded system to ensure the accuracy of the sensor output data.

[0037] Data updating and processing: During real-time compensation, the output data of the sensor is continuously updated and compensated according to the latest calibration parameters, and the compensated data needs to be further processed and analyzed, such as feature extraction and target recognition, to meet the needs of different application scenarios.

[0038] Specifically, in step S1, Kalman filtering algorithm is used to remove high-frequency noise from acceleration data, which specifically includes establishing a state space model to describe the dynamic characteristics of the sensor, and recursively estimating the optimal state value through a prediction-update cycle, wherein the state transition matrix is determined according to the inherent frequency and damping ratio of the sensor.

[0039] In this embodiment: By establishing a state space model to describe the dynamic characteristics of the sensor and recursively estimating the optimal state value through a prediction-update cycle, the working state of the sensor can be accurately described. At the same time, the state transition matrix is determined according to the inherent frequency and damping ratio of the sensor, which conforms to the physical characteristics of the sensor itself, making the Kalman filtering algorithm more effective in removing high-frequency noise in acceleration data, improving data smoothness and accuracy, and providing a high-quality data foundation for subsequent analysis.

[0040] Specifically, in step S1, hardware synchronization uses GPS second pulse signal to synchronize with local clock, and software alignment uses cubic spline interpolation algorithm to align the timestamp of camera image data to the 100Hz time reference of the inertial measurement unit, ensuring that the time alignment error of multi-modal data is less than 5ms.

[0041] In this embodiment: Hardware synchronization uses GPS second pulse signal to synchronize with local clock, which takes advantage of the high precision and stability of GPS signal to ensure the uniformity of each sensor clock. Software alignment uses cubic spline interpolation algorithm to align the timestamp of camera image data to the 100Hz time reference of the inertial measurement unit, which ensures the smoothness and accuracy of data alignment on the time axis, making the time alignment error of multi-modal data less than 5ms, effectively solving the problem of time asynchronization of multi-source heterogeneous data and ensuring the spatiotemporal consistency.

[0042] Specifically, in step S2, linearity is fitted by least squares method to input-output curve, and the ratio of maximum nonlinear error to range is calculated; hysteresis characteristics are obtained by forward and reverse stroke scanning to get output difference, and the ratio of maximum difference to range is taken as hysteresis error index.

[0043] In this embodiment: the linearity is fitted by the least square method to the input-output curve, and the ratio of the maximum non-linear error to the range is calculated, which can accurately reflect the linear deviation between the sensor output and the input, the hysteresis characteristic is obtained by scanning the output difference in the forward and reverse strokes, and the ratio of the maximum difference to the range is taken as the hysteresis error index, which can accurately describe the difference in the output of the sensor in the forward and reverse strokes, and provide accurate basis for comprehensive and in-depth analysis of the sensor performance.

[0044] Specifically, in the step S3, the influence of the environmental parameter is modeled by a polynomial regression model, the model coefficients are fitted based on experimental data using the least square method, and a cross-validation mechanism is introduced to ensure the generalization ability of the model within the temperature range of -20℃-80℃.

[0045] In this embodiment: the influence of the environmental parameter is modeled by a polynomial regression model, which can better fit the nonlinear relationship between environmental factors and sensor performance, the model coefficients are fitted based on experimental data using the least square method to ensure the fitting degree of the model to the actual data, and a cross-validation mechanism is introduced to ensure that the model has good generalization ability within the temperature range of -20℃-80℃, and can accurately predict the changes in sensor performance under different environmental conditions.

[0046] Specifically, in the step S3, the adaptive algorithm uses the recursive least square method to update the calibration parameters through a sliding window, and the forgetting factor is set to 0.98.

[0047] In this embodiment: the adaptive algorithm uses the recursive least square method to update the calibration parameters through a sliding window, which can track the changes in sensor performance and environment in a timely manner, and the forgetting factor is set to 0.98, which not only retains the historical data information, but also focuses more on the influence of recent data, so that the calibration parameters can quickly respond to changes in system state, realize dynamic adaptive adjustment, and improve the adaptability and calibration accuracy of the system under different working conditions.

[0048] Specifically, in the step S3, the humidity compensation uses the look-up table method to pre-establish a humidity-sensitivity correction coefficient table, and real-time queries the correction coefficient corresponding to the current humidity value to linearly compensate the sensor output.

[0049] In this embodiment: the humidity compensation uses the look-up table method to pre-establish a humidity-sensitivity correction coefficient table, and the humidity value is corresponded to the correction coefficient, so that in the real-time compensation process, only the correction coefficient corresponding to the current humidity value needs to be queried to linearly compensate the sensor output, without the need for complex calculation process, and the compensation efficiency is high, which can quickly and effectively eliminate the influence of humidity on the sensor performance.

[0050] Specifically, in the step S3, the calibration parameters are corrected according to the real-time measured environmental parameters by establishing a mathematical model between environmental factors and sensor performance, and the sensitivity is adjusted by a temperature compensation algorithm in an environment with large temperature variation.

[0051] In the embodiment, the calibration parameters are corrected according to the real-time measured environmental parameters by establishing a mathematical model between environmental factors and sensor performance, and the temperature compensation algorithm can adjust the sensitivity of the sensor according to the real-time temperature value in an environment with large temperature variation, so that the sensor can maintain high measurement accuracy under different temperature conditions, and the adaptability and stability of the system to temperature variation are enhanced.

[0052] Specifically, in the step S4, the sensor output data is updated constantly in the real-time compensation process, compensated according to the latest calibration parameters, and the compensated data is further processed and analyzed through feature extraction and target recognition.

[0053] In the embodiment, the sensor output data is updated constantly in the real-time compensation process, and compensated according to the latest calibration parameters, so that the real-time and accuracy of the data are ensured, and the compensated data is further processed and analyzed through feature extraction and target recognition, so that useful information in the data can be mined, the system can provide targeted data analysis and processing results according to the requirements of different application scenarios, and the practicability and flexibility of the system are improved.

[0054] It should be noted that, in this paper, relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations.

[0055] Although the present application has been disclosed with reference to the preferred embodiments above, it is not intended to limit the present application. Those skilled in the art without departing from the spirit and scope of the present application can make various modifications and improvements. Therefore, the protection scope of the present application shall be subject to the scope defined by the claims.

Claims

1. A multimodal sensor embedded self-calibration system and real-time compensation method, characterized by: The method comprises the following steps: S1: Data acquisition: A multimodal data acquisition method is used. The multimodal data acquisition module is responsible for collecting raw data from the camera, inertial measurement unit, and microphone. The camera collects image data at a frequency of 30 Hz, and the inertial measurement unit collects acceleration and angular velocity data at a frequency of 100 Hz. Data preprocessing: The collected raw data contains noise and interference. For acceleration data, Kalman filtering is used to remove high-frequency noise, improve the smoothness and accuracy of the data. At the same time, the data can also be normalized to unify data from different modes into the same scale range to facilitate subsequent analysis and comparison; Data synchronization: A combination of hardware synchronization and software alignment is used. Hardware synchronization uses GPS timestamps or network time protocol to unify the clocks of all sensors. Software alignment aligns data on the time axis through interpolation, interpolating data from low-frequency sensors to the timestamps of high-frequency sensors. S2: Response Feature Extraction: The sensor response analysis module is used to analyze the preprocessed data and extract the sensor response characteristics: sensitivity, linearity, and hysteresis. Sensitivity reflects the sensor's sensitivity to changes in the measured value, linearity indicates the linear relationship between the sensor output and input, and hysteresis describes the difference in the sensor's output during forward and reverse travel. S3: Calibration parameter generation: Standard model establishment: The calibration parameter generation module generates calibration parameters for different modal sensors based on the results of sensor response analysis, combined with a preset standard model and environmental parameters. The standard model can be established based on the theoretical characteristics of the sensor and experimental data, and describes the relationship between the output and input of the sensor under ideal conditions. Environmental parameters include temperature, humidity, pressure, etc. These factors will affect the performance of the sensor and need to be considered during the calibration process. Adaptive algorithm application: The calibration parameters are generated using an adaptive algorithm for dynamic adjustment. The adaptive algorithm can automatically adjust the calibration parameters according to the real-time performance of the sensor and environmental changes; Environmental factor compensation: By establishing a mathematical model between environmental factors and sensor performance, the calibration parameters can be corrected according to the real-time measured environmental parameters. In environments with large temperature changes, the sensitivity can be adjusted through the temperature compensation algorithm to ensure that the sensor can maintain high measurement accuracy under different temperature conditions. S4: Real-time compensation: The real-time compensation execution module compensates the real-time output data of the sensor according to the generated calibration parameters. For linear sensors, simple linear transformation can be used for compensation; for nonlinear sensors, more complex nonlinear compensation algorithms are required. The compensation process is carried out in real time in the embedded system to ensure the accuracy of the sensor output data. Data updating and processing: During the real-time compensation process, the sensor output data is continuously updated and compensated according to the latest calibration parameters. At the same time, the compensated data needs to be further processed and analyzed, feature extraction, target recognition, etc. to meet the needs of different application scenarios.

2. The multimodal sensor embedded self-calibration system and real-time compensation method according to claim 1, characterized in that: In step S1, a Kalman filter algorithm is used to remove high-frequency noise from the acceleration data, specifically including establishing a state space model to describe the dynamic characteristics of the sensor, and recursively estimating the optimal state value through a prediction-update loop, wherein the state transfer matrix is ​​determined according to the natural frequency and damping ratio of the sensor.

3. The multimodal sensor embedded self-calibration system and real-time compensation method according to claim 1, characterized in that: In step S1, hardware synchronization uses the GPS second pulse signal to synchronize with the local clock, and software alignment uses the cubic spline interpolation algorithm to align the timestamp of the camera image data to the 100 Hz time base of the inertial measurement unit, ensuring that the multimodal data time alignment error is less than 5 ms.

4. The multimodal sensor embedded self-calibration system and real-time compensation method according to claim 1, characterized in that: In step S2, the linearity is measured by fitting the input-output curve using the least squares method to calculate the ratio of the maximum nonlinear error to the range; The hysteresis characteristic obtains the output difference by forward and reverse stroke scanning, and the ratio of the maximum difference to the range is taken as the hysteresis error index.

5. The multimodal sensor embedded self-calibration system and real-time compensation method according to claim 1, characterized in that: In step S3, the influence of environmental parameters is modeled using a polynomial regression model, the model coefficients are fitted using the least squares method based on experimental data, and a cross-validation mechanism is introduced to ensure the generalization ability of the model within the temperature range of -20°C to 80°C.

6. The multimodal sensor embedded self-calibration system and real-time compensation method according to claim 1, characterized in that: In step S3, the adaptive algorithm adopts recursive least squares (RLS) to update the calibration parameters through a sliding window, wherein the forgetting factor is set to 0.

98.

7. The multimodal sensor embedded self-calibration system and real-time compensation method according to claim 1, characterized in that: In step S3, humidity compensation adopts a table lookup method, a humidity-sensitivity correction coefficient table is pre-established, and the correction coefficient corresponding to the current humidity value is queried in real time to perform linear compensation on the sensor output.

8. The multimodal sensor embedded self-calibration system and real-time compensation method according to claim 1, characterized in that: In step S3, a mathematical model between environmental factors and sensor performance is established, and calibration parameters are corrected according to the environmental parameters measured in real time. In an environment with large temperature changes, the sensitivity is adjusted through a temperature compensation algorithm.

9. The multimodal sensor embedded self-calibration system and real-time compensation method according to claim 1, characterized in that: In step S4, the sensor output data is continuously updated during the real-time compensation process, compensated according to the latest calibration parameters, and the compensated data is further processed and analyzed for feature extraction and target recognition to meet the needs of different application scenarios.

Images