A method and system for HD projection sharpness correction based on inertial sensors
Patent Information
- Application Number
- CN202610907343.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-06-23
- Publication Date
- 2026-10-16
AI Technical Summary
[0004]现有HD系统对路面状态适应能力不足,但缺乏对路面自身物理状态(如沥青路面、水泥路面、砂石路面、湿滑路面等)的感知能力
[0032] (1) This invention eliminates geometric distortion of HD projection images by jointly correcting the vehicle body posture and the road surface. A basic geometric transformation matrix is constructed by combining the vehicle body pitch angle and roll angle to compensate for the translation, rotation and trapezoidal distortion of the projected virtual image caused by the vehicle body posture; then, the transformation matrix is dynamically corrected according to the correction intensity coefficients corresponding to different road surfaces to achieve adaptive adjustment of geometric correction and ensure that the position and shape of the projected virtual image are always accurately matched with the road surface.
Smart Images

Figure CN122933869A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of vehicle HD projection correction technology. Specifically, this invention relates to an HD projection sharpness correction method and system based on inertial sensors. Background Technology
[0002] With the rapid development of intelligent vehicle technology, enhanced HD projection has gradually become a mainstream configuration in automotive intelligent cockpits. Through HD projection, key information such as patterns and images can be projected as virtual images to locations such as below the front doors and in front of the car, and overlaid with the real road environment, significantly improving driving safety and interactive experience.
[0003] Chinese Patent 111442776B provides a system for correcting HD map images. The system is configured to: receive ground-view image data of map hotspots collected by multiple vehicles; convert the ground-view image data into bird's-eye view image data using homography; remove outliers from the bird's-eye view image data; apply an error reduction algorithm; after error reduction, stitch together a continuous sequence of images from the image data to generate a high-definition image, wherein each continuous image has been generated by converting the ground-view image data into bird's-eye view image data using homography, removing outliers from the bird's-eye view image data, and applying the error reduction algorithm; compare the stitched continuous image sequence with the high-definition image of the hotspot to determine modifications to the high-definition image; and provide the modified high-definition image data for navigation near map hotspots.
[0004] Existing HD systems are insufficient in adapting to road conditions, but lack the ability to perceive the physical state of the road surface itself (such as asphalt, cement, gravel, and slippery surfaces). When a vehicle travels on different road surfaces, the vibration response characteristics of the suspension system vary significantly, causing the projected virtual image to exhibit varying degrees of shaking, drifting, and blurring in the field of vision, severely affecting the visual experience of virtual-real fusion. Summary of the Invention
[0005] The present invention aims to provide a method and system for HD projection sharpness correction based on inertial sensors, so as to enhance the adaptability of vehicle HD systems to road conditions.
[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:
[0007] This invention provides a method for HD projection sharpness correction based on inertial sensors, comprising the following steps:
[0008] Step 1: Collect real-time data from the inertial sensors during vehicle operation. The real-time data includes three-axis acceleration data and three-axis angular velocity data, and simultaneously collect vehicle speed information.
[0009] Step 2: Obtain the vehicle's current body posture information;
[0010] Step 3: Preprocess and extract features from the real-time data acquired by the inertial sensor;
[0011] Step 4: Input the extracted feature vector into the pre-trained road surface state classification model to obtain the classification result of the current road surface state;
[0012] Step 5: Based on the classification results of road surface conditions and vehicle posture information, generate multi-dimensional projection correction parameters; the multi-dimensional projection correction parameters include geometric correction parameters, optical focusing parameters, and rendering optimization parameters;
[0013] Step Six: Perform real-time correction on the HD projection system using multi-dimensional projection correction parameters, adjust the geometric transformation matrix of the projected image, drive the optical focusing mechanism, and update the display parameters of the rendering engine.
[0014] In step three, the real-time data collected by the inertial sensor is segmented by a sliding window and bandpass filtered to remove sensor noise and low-frequency motion components of the vehicle itself; for the data in each time window, time-domain features and frequency-domain features are extracted to form a feature vector.
[0015] The time-domain features include the mean, variance, peak-to-peak value, kurtosis, skewness, and zero-crossing rate of the vertical acceleration; the frequency-domain features include the energy distribution, dominant frequency component, and spectral centroid of each preset frequency band obtained through fast Fourier transform.
[0016] In step four, the output types of the road surface condition classification model include at least: asphalt pavement, cement pavement, gravel pavement, wet and slippery pavement, and bumpy pavement. The road surface condition classification model is a classification network based on deep learning, and its training data includes feature vectors of inertial sensors collected under different road surface conditions and corresponding road surface condition labels.
[0017] Step five involves generating multi-dimensional projection correction parameters, including: querying a preset mapping table between road surface conditions and correction parameters; determining the geometric correction intensity coefficient, optical focus adjustment step size, and rendering refresh strategy based on the road surface condition classification results; calculating the basic geometric transformation matrix of the projected virtual image based on the pitch and roll angles in the vehicle body attitude information; using the geometric correction intensity coefficient to perform weighted correction on the basic geometric transformation matrix to obtain the final geometric correction parameters, and adjusting the optical focus parameters and rendering optimization parameters accordingly.
[0018] This invention provides a system for applying an HD projection sharpness correction method based on an inertial sensor, comprising:
[0019] The inertial sensor data acquisition module is used to collect triaxial acceleration data and triaxial angular velocity data during vehicle operation.
[0020] The vehicle attitude module is used to obtain the current vehicle body attitude information;
[0021] The vehicle speed acquisition module is used to acquire vehicle speed information;
[0022] The feature extraction module, connected to the inertial sensor data acquisition module, is used to perform sliding window segmentation and bandpass filtering on the data acquired by the inertial sensor, and extract the time-domain and frequency-domain features within each window to form a feature vector;
[0023] The road surface condition classification module is connected to the feature extraction module and the vehicle speed acquisition module. It has a built-in pre-trained road surface condition classification model, which is used to output the road surface condition classification result based on the feature vector.
[0024] The correction parameter generation module is connected to the road surface state classification module and the vehicle posture module. It has a built-in mapping relationship between road surface state and correction parameters and is used to generate geometric correction parameters, optical focus parameters and rendering optimization parameters based on the road surface state classification results and vehicle posture information.
[0025] The projection correction execution module is connected to the correction parameter generation module and the HD projection optical engine. It is used to send geometric correction parameters, optical focusing parameters and rendering optimization parameters to the geometric correction unit, optical focusing unit and rendering unit of the HD projection system, respectively, and to perform real-time correction.
[0026] The feature extraction module includes: a bandpass filter; a time-domain feature calculation submodule, used to extract the mean, variance, peak-to-peak value, kurtosis, skewness, and zero-crossing rate of vertical acceleration; and a frequency-domain feature calculation submodule, used to calculate the energy distribution, dominant frequency component, and spectral centroid of each frequency band through fast Fourier transform.
[0027] The road surface condition classification model is a lightweight convolutional neural network model deployed on an on-board edge computing platform, used to output classification results in real time, including at least asphalt pavement, cement pavement, gravel pavement, wet and slippery pavement, and bumpy pavement.
[0028] The calibration parameter generation module includes:
[0029] The mapping relationship storage unit is used to store the geometric correction intensity coefficient, optical focus adjustment step size and rendering mode parameters corresponding to each road surface condition;
[0030] The parameter synthesis unit is used to calculate the basic geometric transformation matrix based on the pitch angle and roll angle in the vehicle body attitude information, and to correct the basic geometric transformation matrix using the geometric correction strength coefficient, and synthesize the final geometric correction parameters, optical focus parameters and rendering optimization parameters.
[0031] The technical effects of this invention are as follows:
[0032] (1) This invention eliminates geometric distortion of HD projection images by jointly correcting the vehicle body posture and the road surface. A basic geometric transformation matrix is constructed by combining the vehicle body pitch angle and roll angle to compensate for the translation, rotation and trapezoidal distortion of the projected virtual image caused by the vehicle body posture; then, the transformation matrix is dynamically corrected according to the correction intensity coefficients corresponding to different road surfaces to achieve adaptive adjustment of geometric correction and ensure that the position and shape of the projected virtual image are always accurately matched with the road surface.
[0033] (2) The present invention has comprehensive data acquisition and strong perception capabilities. It adopts an on-board six-axis inertial sensor combined with the vehicle CAN bus and attitude calculation algorithm to synchronously collect multi-dimensional data such as acceleration, angular velocity, vehicle speed, pitch angle, and roll angle. The hardware deployment is simple and easy to install. It can perceive the vehicle driving status and body posture in real time, providing complete and accurate raw data support for subsequent road surface recognition and projection correction, and is suitable for all vehicle driving conditions.
[0034] (3) The signal processing of the present invention is accurate and effectively eliminates interference information. By using a sliding window segmentation strategy, the real-time performance of data processing and signal stability are taken into account; with the addition of a bandpass filter, the ultra-low frequency components caused by vehicle pitch and roll and the high frequency noise of the sensor are accurately filtered out, while the core vibration signal caused by road surface unevenness is completely preserved, thereby improving the accuracy of road surface feature extraction from the source.
[0035] (4) This invention has high accuracy in road classification. It integrates time-domain and frequency-domain feature systems. The time-domain features can accurately characterize vibration intensity, impact characteristics and signal distribution patterns. The frequency domain features quantify vibration frequency characteristics through frequency band division, energy ratio, main frequency and spectral centroid. Multiple feature combinations form a highly discriminative feature vector. It relies on a lightweight convolutional neural network to complete road classification, which can accurately identify a variety of typical road surfaces such as asphalt, cement, gravel, wet and slippery, and bumpy. The classification results are highly reliable.
[0036] (5) This invention improves the user experience of vehicle projection by simultaneously carrying out correction work from multiple dimensions such as signal filtering, road surface recognition, geometric distortion, optical focusing and image rendering. It compensates for various interferences caused by vehicle driving, road conditions and vehicle posture in multiple dimensions, and finally achieves stable, clear and regular virtual images of road projection during vehicle driving, which greatly improves the user effect and driving experience of vehicle high-definition projection. Attached Figure Description
[0037] This manual includes the following figures, which illustrate the following:
[0038] Figure 1 This is a flowchart of an HD projection sharpness correction method and system based on an inertial sensor according to the present invention. Detailed Implementation
[0039] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings, in order to help those skilled in the art to have a more complete, accurate and in-depth understanding of the inventive concept and technical solution of the present invention, and to facilitate its implementation.
[0040] This invention provides a method for HD projection sharpness correction based on inertial sensors, comprising the following steps:
[0041] Step 1: Collect real-time data from the inertial sensors during vehicle operation. The real-time data includes three-axis acceleration data and three-axis angular velocity data, and simultaneously collect vehicle speed information.
[0042] Step 2: Obtain the vehicle's current body posture information;
[0043] Step 3: Preprocess and extract features from the real-time data acquired by the inertial sensor;
[0044] Step 4: Input the extracted feature vector into the pre-trained road surface state classification model to obtain the classification result of the current road surface state;
[0045] Step 5: Based on the classification results of road surface conditions and vehicle posture information, generate multi-dimensional projection correction parameters; the multi-dimensional projection correction parameters include geometric correction parameters, optical focusing parameters, and rendering optimization parameters;
[0046] Step Six: Perform real-time correction on the HD projection system using multi-dimensional projection correction parameters, adjust the geometric transformation matrix of the projected image, drive the optical focusing mechanism, and update the display parameters of the rendering engine.
[0047] In step three, the real-time data collected by the inertial sensor is segmented by a sliding window and bandpass filtered to remove sensor noise and low-frequency motion components of the vehicle itself; for the data in each time window, time-domain features and frequency-domain features are extracted to form a feature vector.
[0048] The time-domain features include the mean, variance, peak-to-peak value, kurtosis, skewness, and zero-crossing rate of the vertical acceleration; the frequency-domain features include the energy distribution, dominant frequency component, and spectral centroid of each preset frequency band obtained through fast Fourier transform.
[0049] In step four, the output types of the road surface condition classification model include at least: asphalt pavement, cement pavement, gravel pavement, wet and slippery pavement, and bumpy pavement. The road surface condition classification model is a classification network based on deep learning, and its training data includes feature vectors of inertial sensors collected under different road surface conditions and corresponding road surface condition labels.
[0050] Step five involves generating multi-dimensional projection correction parameters, including: querying a preset mapping table between road surface conditions and correction parameters; determining the geometric correction intensity coefficient, optical focus adjustment step size, and rendering refresh strategy based on the road surface condition classification results; calculating the basic geometric transformation matrix of the projected virtual image based on the pitch and roll angles in the vehicle body attitude information; using the geometric correction intensity coefficient to perform weighted correction on the basic geometric transformation matrix to obtain the final geometric correction parameters, and adjusting the optical focus parameters and rendering optimization parameters accordingly.
[0051] This invention provides a system for applying an HD projection sharpness correction method based on an inertial sensor, comprising:
[0052] The inertial sensor data acquisition module is used to collect triaxial acceleration data and triaxial angular velocity data during vehicle operation.
[0053] The vehicle attitude module is used to obtain the current vehicle body attitude information;
[0054] The vehicle speed acquisition module is used to acquire vehicle speed information;
[0055] The feature extraction module, connected to the inertial sensor data acquisition module, is used to perform sliding window segmentation and bandpass filtering on the data acquired by the inertial sensor, and extract the time-domain and frequency-domain features within each window to form a feature vector;
[0056] The road surface condition classification module is connected to the feature extraction module and the vehicle speed acquisition module. It has a built-in pre-trained road surface condition classification model, which is used to output the road surface condition classification result based on the feature vector.
[0057] The correction parameter generation module is connected to the road surface state classification module and the vehicle posture module. It has a built-in mapping relationship between road surface state and correction parameters and is used to generate geometric correction parameters, optical focus parameters and rendering optimization parameters based on the road surface state classification results and vehicle posture information.
[0058] The projection correction execution module is connected to the correction parameter generation module and the HD projection optical engine. It is used to send geometric correction parameters, optical focusing parameters and rendering optimization parameters to the geometric correction unit, optical focusing unit and rendering unit of the HD projection system, respectively, and to perform real-time correction.
[0059] The feature extraction module includes: a bandpass filter; a time-domain feature calculation submodule, used to extract the mean, variance, peak-to-peak value, kurtosis, skewness, and zero-crossing rate of vertical acceleration; and a frequency-domain feature calculation submodule, used to calculate the energy distribution, dominant frequency component, and spectral centroid of each frequency band through fast Fourier transform.
[0060] The road surface condition classification model is a lightweight convolutional neural network model deployed on an on-board edge computing platform, used to output classification results in real time, including at least asphalt pavement, cement pavement, gravel pavement, wet and slippery pavement, and bumpy pavement.
[0061] The calibration parameter generation module includes:
[0062] The mapping relationship storage unit is used to store the geometric correction intensity coefficient, optical focus adjustment step size and rendering mode parameters corresponding to each road surface condition;
[0063] The parameter synthesis unit is used to calculate the basic geometric transformation matrix based on the pitch angle and roll angle in the vehicle body attitude information, and to correct the basic geometric transformation matrix using the geometric correction strength coefficient, and synthesize the final geometric correction parameters, optical focus parameters and rendering optimization parameters.
[0064] The present invention will now be described in detail.
[0065] A six-axis inertial sensor installed on the vehicle body collects three-axis acceleration and three-axis angular velocity data in real time during vehicle operation. The six-axis inertial sensor includes a three-axis accelerometer and a three-axis gyroscope; the accelerometer collects three-axis acceleration data, and the gyroscope collects three-axis angular velocity data. Simultaneously, real-time vehicle speed signals are obtained from the vehicle's CAN bus, and current vehicle attitude information is acquired through a vehicle attitude module, which can use attitude sensors. The vehicle attitude information includes pitch and roll angles.
[0066] This step is achieved through a feature extraction module, with the aim of extracting sensitive features from the original IMU vibration signal that can distinguish different road surface conditions.
[0067] The acceleration data collected continuously by inertial sensors is segmented into sliding windows, with a window length of 2 seconds and a step size of 0.5 seconds to ensure real-time performance and stability. The signal within each window is bandpass filtered, with the filter's passband frequency set between 0.5Hz and 50Hz. This effectively filters out ultra-low-frequency motion components such as vehicle pitch and roll, as well as high-frequency sensor noise, while retaining the core vibration information transmitted from road surface unevenness excitation to the vehicle body through the suspension.
[0068] Time-domain and frequency-domain features were extracted from the filtered window signal. Time-domain features included calculating the mean, variance, peak-to-peak value, kurtosis, skewness, and zero-crossing rate of the vertical acceleration. Variance and peak-to-peak value reflect vibration intensity; kurtosis and skewness reflect the impact characteristics and distribution pattern of the vibration signal, providing significant differentiation for various road surface types. Frequency-domain features included performing a Fast Fourier Transform on the window data to obtain the energy distribution of each frequency band. The 0.5Hz-50Hz band was divided into three sub-bands: low-frequency band (0.5-5Hz), mid-frequency band (5-20Hz), and high-frequency band (20-50Hz). The energy distribution percentage, dominant frequency component frequency, and spectral centroid of each sub-band were calculated.
[0069] Feature vectors are generated based on the extracted features and then fed into a pre-trained pavement condition classification model in real time. This model is pre-trained offline using a large amount of labeled inertial sensor data collected in standard test tracks and actual roads. Label types include at least asphalt pavement, cement pavement, gravel pavement, slippery pavement, and bumpy pavement. Each time a new window of feature vectors is received, the model outputs the probability distribution of the current pavement condition type, and the category with the highest probability is taken as the classification result.
[0070] After receiving the road condition classification results and the current vehicle attitude information, the correction parameter generation module queries the internally stored mapping table between road conditions and correction parameters, which is pre-calibrated. Based on the real-time pitch and roll angles in the vehicle attitude information, it calculates the fundamental geometric transformation matrix of the projected virtual image. This matrix is used to compensate for the overall translation, rotation, and trapezoidal distortion of the virtual image caused by changes in vehicle attitude.
[0071] The geometric correction strength coefficients obtained from the mapping table between road surface conditions and correction parameters are used to dynamically correct the basic geometric transformation matrix. Optical focusing parameters and rendering optimization parameters are generated simultaneously.
[0072] The projection correction execution module distributes the generated correction parameters to the corresponding control unit of the HD projection optical engine. The geometric correction unit receives these parameters and performs perspective transformation on the source projected image in real time to ensure the virtual image's position and shape accurately match the road surface. The optical focusing unit receives focusing position commands and drives high-speed actuators such as voice coil motors or liquid lenses to fine-tune the projection focal length, compensating for instantaneous changes in projection distance caused by suspension vibration and ensuring virtual image sharpness. The rendering unit receives rendering optimization parameters and adjusts the refresh mechanism and pixel processing algorithms in the image generation pipeline to optimize the visual presentation of the virtual image under complex road surface dynamics.
[0073] The above three dimensions of correction are executed simultaneously to form a closed loop, ultimately presenting a stable and clear HD projection virtual image on the road surface below or in front of the vehicle.
[0074] The following is a detailed description of an HD projection sharpness correction system based on an inertial sensor according to the present invention.
[0075] The inertial sensor data acquisition module uses a six-axis inertial sensor, which is fixed to the vehicle body, to collect three-axis acceleration data and three-axis angular velocity data during vehicle movement.
[0076] The vehicle attitude module uses data collected by inertial sensors to obtain real-time pitch and roll angles through attitude calculation algorithms, thereby acquiring the vehicle's current body attitude information.
[0077] The vehicle speed acquisition module reads the vehicle speed from the vehicle bus.
[0078] The feature extraction module integrates a bandpass filter, a time-domain feature calculation submodule, and a frequency-domain feature calculation submodule. The bandpass filter is used to perform sliding window segmentation and bandpass filtering on the data acquired by the inertial sensor. The time-domain feature calculation submodule is used to extract the mean, variance, peak-to-peak value, kurtosis, skewness, and zero-crossing rate of the vertical acceleration. The frequency-domain feature calculation submodule is used to calculate the energy distribution, dominant frequency component, and spectral centroid of each frequency band through fast Fourier transform, and finally generate feature vectors.
[0079] The road surface condition classification model is a lightweight convolutional neural network model deployed on an on-board edge computing platform, used to output classification results in real time, including at least asphalt pavement, cement pavement, gravel pavement, wet and slippery pavement, and bumpy pavement.
[0080] The calibration parameter generation module has a built-in mapping relationship between road surface conditions and calibration parameters, which is used to generate geometric calibration parameters, optical focusing parameters and rendering optimization parameters based on the road surface condition classification results and vehicle body posture information.
[0081] The projection correction execution module is connected to the correction parameter generation module and the HD projection optical engine. It is used to send geometric correction parameters, optical focusing parameters and rendering optimization parameters to the geometric correction unit, optical focusing unit and rendering unit of the HD projection system respectively, and perform real-time correction.
[0082] The beneficial effects of the present invention are described in detail below.
[0083] This invention eliminates geometric distortion of HD projection images by jointly correcting vehicle posture and road surface. A basic geometric transformation matrix is constructed by combining the vehicle's pitch and roll angles to compensate for translation, rotation, and trapezoidal distortion of the projected virtual image caused by vehicle posture. Then, the transformation matrix is dynamically adjusted according to the corresponding correction intensity coefficients for different road surfaces, achieving adaptive adjustment of geometric correction and ensuring that the position and shape of the projected virtual image always precisely match the road surface.
[0084] This invention features comprehensive data acquisition and strong sensing capabilities. It employs an onboard six-axis inertial sensor combined with the vehicle's CAN bus and attitude calculation algorithms to simultaneously collect multi-dimensional data such as acceleration, angular velocity, vehicle speed, pitch angle, and roll angle. The hardware deployment is simple and easy to install, enabling real-time perception of the vehicle's driving status and body posture. This provides complete and accurate raw data support for subsequent road surface recognition and projection correction, adapting to all vehicle driving conditions.
[0085] The present invention features precise signal processing and effectively eliminates interference. By employing a sliding window segmentation strategy, it balances real-time data processing with signal stability; coupled with a bandpass filter, it precisely filters out ultra-low frequency components caused by vehicle pitch and roll, as well as high-frequency noise from sensors, while fully preserving the core vibration signals caused by road surface unevenness, thus improving the accuracy of road feature extraction from the source.
[0086] This invention achieves high accuracy in road surface classification by integrating a dual feature system of time and frequency domains. The time domain features can accurately characterize vibration intensity, impact characteristics, and signal distribution patterns, while the frequency domain features quantify vibration frequency characteristics through frequency band division, energy proportion, dominant frequency, and spectral centroid. Multiple feature combinations form a highly discriminative feature vector. Relying on a lightweight convolutional neural network to complete road surface classification, it can accurately identify various typical road surfaces such as asphalt, cement, gravel, slippery, and bumpy surfaces, with highly reliable classification results.
[0087] This invention enhances the user experience of in-vehicle projection by simultaneously performing corrections from multiple dimensions, including signal filtering, road surface recognition, geometric distortion, optical focusing, and image rendering. It compensates for various interferences caused by vehicle movement, road conditions, and vehicle posture, ultimately achieving a stable, clear, and well-formed virtual image of the road surface projection during vehicle movement, significantly improving the user effect and driving experience of in-vehicle high-definition projection.
[0088] The present invention has been described above by way of example with reference to the accompanying drawings. Obviously, the specific implementation of the present invention is not limited to the above-described manner. Any non-substantial improvements made using the inventive concept and technical solution; or the direct application of the inventive concept and technical solution to other situations without modification, are all within the protection scope of the present invention.
Claims
1. A method for HD projection sharpness correction based on inertial sensors, characterized in that, Includes the following steps: Step 1: Collect real-time data from the inertial sensors during vehicle operation. The real-time data includes three-axis acceleration data and three-axis angular velocity data, and simultaneously collect vehicle speed information. Step 2: Obtain the vehicle's current body posture information; Step 3: Preprocess and extract features from the real-time data acquired by the inertial sensor; Step 4: Input the extracted feature vector into the pre-trained road surface state classification model to obtain the classification result of the current road surface state; Step 5: Based on the classification results of road surface conditions and vehicle posture information, generate multi-dimensional projection correction parameters; the multi-dimensional projection correction parameters include geometric correction parameters, optical focusing parameters, and rendering optimization parameters; Step Six: Perform real-time correction on the HD projection system using multi-dimensional projection correction parameters, adjust the geometric transformation matrix of the projected image, drive the optical focusing mechanism, and update the display parameters of the rendering engine.
2. The HD projection sharpness correction method based on an inertial sensor as described in claim 1, characterized in that: In step three, the real-time data collected by the inertial sensor is segmented by a sliding window and bandpass filtered to remove sensor noise and low-frequency motion components of the vehicle itself; for the data in each time window, time-domain features and frequency-domain features are extracted to form a feature vector.
3. The HD projection sharpness correction method based on an inertial sensor as described in claim 2, characterized in that: The time-domain features include the mean, variance, peak-to-peak value, kurtosis, skewness, and zero-crossing rate of the vertical acceleration; the frequency-domain features include the energy distribution, dominant frequency component, and spectral centroid of each preset frequency band obtained through fast Fourier transform.
4. The HD projection sharpness correction method based on an inertial sensor as described in claim 1, characterized in that: In step four, the output types of the road surface condition classification model include at least: asphalt pavement, cement pavement, gravel pavement, wet and slippery pavement, and bumpy pavement. The road surface condition classification model is a classification network based on deep learning, and its training data includes feature vectors of inertial sensors collected under different road surface conditions and corresponding road surface condition labels.
5. The HD projection sharpness correction method based on an inertial sensor as described in claim 1, characterized in that: Step five involves generating multi-dimensional projection correction parameters, including: querying a preset mapping table between road surface conditions and correction parameters; determining the geometric correction intensity coefficient, optical focus adjustment step size, and rendering refresh strategy based on the road surface condition classification results; calculating the basic geometric transformation matrix of the projected virtual image based on the pitch and roll angles in the vehicle body attitude information; using the geometric correction intensity coefficient to perform weighted correction on the basic geometric transformation matrix to obtain the final geometric correction parameters, and adjusting the optical focus parameters and rendering optimization parameters accordingly.
6. A system for applying the HD projection sharpness correction method based on inertial sensors as described in any one of claims 1-5, characterized in that, include: The inertial sensor data acquisition module is used to collect triaxial acceleration data and triaxial angular velocity data during vehicle operation. The vehicle attitude module is used to obtain the current vehicle body attitude information; The vehicle speed acquisition module is used to acquire vehicle speed information; The feature extraction module, connected to the inertial sensor data acquisition module, is used to perform sliding window segmentation and bandpass filtering on the data acquired by the inertial sensor, and extract the time-domain and frequency-domain features within each window to form a feature vector; The road surface condition classification module is connected to the feature extraction module and the vehicle speed acquisition module. It has a built-in pre-trained road surface condition classification model, which is used to output the road surface condition classification result based on the feature vector. The correction parameter generation module is connected to the road surface state classification module and the vehicle posture module. It has a built-in mapping relationship between road surface state and correction parameters and is used to generate geometric correction parameters, optical focus parameters and rendering optimization parameters based on the road surface state classification results and vehicle posture information. The projection correction execution module is connected to the correction parameter generation module and the HD projection optical engine. It is used to send geometric correction parameters, optical focusing parameters and rendering optimization parameters to the geometric correction unit, optical focusing unit and rendering unit of the HD projection system, respectively, and to perform real-time correction.
7. The HD projection sharpness correction system based on an inertial sensor as described in claim 6, characterized in that: The feature extraction module includes: a bandpass filter; a time-domain feature calculation submodule, used to extract the mean, variance, peak-to-peak value, kurtosis, skewness, and zero-crossing rate of vertical acceleration; and a frequency-domain feature calculation submodule, used to calculate the energy distribution, dominant frequency component, and spectral centroid of each frequency band through fast Fourier transform.
8. The HD projection sharpness correction system based on an inertial sensor as described in claim 6, characterized in that: The road surface condition classification model is a lightweight convolutional neural network model deployed on an on-board edge computing platform, used to output classification results in real time, including at least asphalt pavement, cement pavement, gravel pavement, wet and slippery pavement, and bumpy pavement.
9. The HD projection sharpness correction system based on an inertial sensor as described in claim 6, characterized in that: The calibration parameter generation module includes: The mapping relationship storage unit is used to store the geometric correction intensity coefficient, optical focus adjustment step size and rendering mode parameters corresponding to each road surface condition; The parameter synthesis unit is used to calculate the basic geometric transformation matrix based on the pitch angle and roll angle in the vehicle body attitude information, and to correct the basic geometric transformation matrix using the geometric correction strength coefficient, and synthesize the final geometric correction parameters, optical focus parameters and rendering optimization parameters.
Citation Information
Patent Citations
Method and device for sequential ground scene image projection synthesis and complex scene reconstruction
CN111442776B