Head-up display device and anti-shake method, apparatus and computer storage medium thereof

Abstract

The embodiment of the present disclosure relates to the technical field of head-up display, and particularly relates to a head-up display device and a method and device for anti-shake thereof and a computer storage medium, which are used to solve the technical problem of poor anti-shake effect of the head-up display device. The anti-shake method of the head-up display device can comprise determining a target anti-shake algorithm according to a current driving environment of a vehicle; and correcting an image projection position of the head-up display device by using the target anti-shake algorithm based on inertial data capable of reflecting a change in a posture of the vehicle, so as to realize anti-shake. The present disclosure can improve the anti-shake effect of the head-up display device.

Description

Technical Field

[0001] This disclosure relates to the field of head-up display technology, and more particularly to a head-up display device and its anti-shake method, apparatus, and computer storage medium. Background Technology

[0002] Head-up display (HUD) technology projects key driving information such as vehicle speed and navigation directly onto the windshield using optical projection, allowing drivers to view driving data without looking down. This technology significantly reduces eye movement, helping drivers maintain continuous attention to the road and thus improving driving safety. However, in actual driving, dynamic scenarios such as road bumps and acceleration / deceleration can cause noticeable shaking in the projected image, which directly affects the readability and recognizability of the information.

[0003] Image stabilization technology is crucial for HUD systems, directly impacting core driving safety requirements. When the projected image shakes, the time it takes for the driver to switch their gaze increases, leading to distraction and significantly increasing the risk of traffic accidents. Especially at high speeds or in complex road conditions, a stable image display is key to ensuring the driver receives information quickly and accurately.

[0004] Current image stabilization methods fail to balance the relationship between stabilization accuracy and latency, and thus lack a dynamic balance model. This results in insufficient stabilization compensation accuracy in critical scenarios such as bumpy roads, while over-stabilization compensation occurs on smooth roads, leading to poor stabilization performance. Summary of the Invention

[0005] In view of this, the present disclosure aims to provide a head-up display device and its stabilization method, apparatus and computer storage medium; which can solve the technical problem that the existing stabilization methods cannot balance the relationship between stabilization accuracy and latency, resulting in poor stabilization effect.

[0006] The technical solution of this disclosure embodiment is implemented as follows: In a first aspect, embodiments of this disclosure provide a method for stabilizing a head-up display device, including: The target anti-shake algorithm is determined based on the vehicle's current driving environment; Based on inertial data that reflects changes in the vehicle's attitude, the target anti-shake algorithm is used to correct the image projection position of the head-up display device in order to achieve anti-shake.

[0007] Secondly, embodiments of this disclosure provide a shake stabilization device for a head-up display device, comprising: The algorithm determination module is used to determine the target anti-shake algorithm based on the vehicle's current driving environment; An inertial measurement module is used to acquire inertial data that reflects changes in the vehicle's attitude. The FPGA position mapping module is used to correct the image projection position of the head-up display device based on the inertial data and the target anti-shake algorithm, so as to achieve anti-shake.

[0008] Thirdly, embodiments of this disclosure provide a head-up display device, the head-up display device including: a processor and a memory; the processor is configured to execute instructions stored in the memory to implement the head-up display device anti-shake method described in the first aspect.

[0009] Fourthly, embodiments of this disclosure provide a computer storage medium storing at least one instruction, which is executed by a processor to implement the head-up display device stabilization method as described in the first aspect.

[0010] This disclosure provides a head-up display (HUD) device and its anti-shake method, apparatus, and computer storage medium. By dynamically determining the target anti-shake algorithm based on the vehicle's current driving environment, the environmental adaptability of the anti-shake method is improved. The mechanism of dynamically selecting the target anti-shake algorithm addresses the performance fluctuation problem of traditional solutions under complex road conditions, ensuring that algorithm resources are allocated as needed. This avoids resource waste caused by using highly complex algorithms in simple environments, or insufficient accuracy due to using simplified algorithms in challenging scenarios. Based on inertial data that reflects changes in vehicle attitude, the target anti-shake algorithm is used to correct the image projection position of the HUD device, achieving real-time compensation. Inertial data provides direct feedback on vehicle motion, ensuring that the correction process is synchronized with attitude changes, eliminating software processing layer delays, making the projection position adjustment more closely match the actual shaking mode, and maintaining image stability. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of the architecture of a head-up display device provided in this disclosure.

[0012] Figure 2 An exemplary top view of the vehicle provided in this disclosure.

[0013] Figure 3 An exemplary perspective view from a vehicle driver's seat is provided for this disclosure.

[0014] Figure 4 A flowchart illustrating a head-up display device anti-shake method provided in an embodiment of this disclosure.

[0015] Figure 5 This is a data flow diagram of a head-up display device anti-shake method provided in an embodiment of the present disclosure.

[0016] Figure 6Data flow diagram of another head-up display device anti-shake method provided in an embodiment of this disclosure.

[0017] Figure 7 This is a schematic diagram of the image projection position before adjustment, provided as an embodiment of the present disclosure.

[0018] Figure 8 This is a schematic diagram of an adjusted image projection position provided in an embodiment of the present disclosure.

[0019] Figure 9 This is a schematic diagram of hardware-induced latency data provided in an embodiment of this disclosure.

[0020] Figure 10 This is a schematic diagram of the structure of a head-up display device anti-shake device provided in an embodiment of the present disclosure.

[0021] Figure 11 This is a schematic diagram of the structure of a head-up display device provided in an embodiment of this disclosure.

[0022] The accompanying drawings have illustrated specific embodiments of this disclosure, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this disclosure to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0023] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0024] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0025] Reference Figure 1 The head-up display device 1 projects display information onto a portion of the windshield 14 through one or more holes in the dashboard. Although Figure 1The example size of the displayed information is shown, but the information can be presented in a larger or smaller area. Examples of displayed information include various vehicle information such as current vehicle speed, current gear of the vehicle's transmission, engine speed, vehicle direction, current infotainment system settings, and / or other vehicle information. The head-up display device 1 provides information to the vehicle driver without requiring the driver to take their eyes off objects in front of the vehicle.

[0026] See Figure 1 The exemplary implementation architecture of the head-up display device 1 shown includes a display control unit 12 generating a signal based on data processed by the data processing unit 11. The aforementioned communication control circuit is provided in the data processing unit 11 and transmits display information to the display control unit 12.

[0027] The display unit 13 may include a light source 131 and an optical path assembly 132. The light source 131 outputs light (e.g., a virtual image) based on a signal from the display control unit 12 to display on the windshield 14. For example, the light source 131 may include one or more lasers and output red, green, and blue light; that is, the light source 131 may include an image source display screen, i.e., a projection component.

[0028] The optical path assembly 132 can reflect the output of the light source 131 onto the windshield 14 through the aperture. A viewer (e.g., a driver) can view the displayed information in the display area on the windshield 14 where the information is projected.

[0029] Specifically, refer to Figure 2 and Figure 3 The head-up display can be installed on the vehicle, which includes a windshield 14 located at the front of the vehicle. The driver and passengers in the vehicle's passenger cabin 21 can see ahead of the vehicle through the windshield 14.

[0030] exist Figure 3 In this configuration, the windshield 14 is visually positioned above the vehicle's dashboard 24. The driver can turn the steering wheel 22 within the passenger cabin 21 to steer the vehicle, for example, to change lanes, merge, and park. In some embodiments, the steering wheel 22 may be retracted or omitted.

[0031] Reference Figure 3 The head-up display device 1 projects display information 17 (e.g., virtual images) onto a portion of the windshield 14 through one or more holes (e.g., hole 23) in the dashboard 24.

[0032] The head-up display (HUD) projects image information onto a specific area of ​​the windshield via an optical channel on the dashboard. Image jitter during this process is primarily caused by the coupling of three types of physical interference. First, mechanical vibrations from vehicle movement are directly transmitted to the HUD itself through the dashboard structure, causing micro-displacements in the light source components and optical path system. When the vehicle encounters road bumps, high-frequency vibrations in the suspension system induce resonance in the light source mounting bracket, resulting in a shift. This shift, amplified by reflection from the optical path components, manifests as visible image jitter in the windshield projection area. Therefore, image stabilization for the HUD is crucial.

[0033] In terms of image stabilization, multi-directional vibrations during vehicle movement cause continuous shaking in the projected image. While mechanical stabilization solutions can partially compensate, they are too bulky and expensive, while software stabilization algorithms based on general-purpose processors are difficult to implement in real time due to excessive processing latency. At the hardware architecture level, there is a dilemma: the computing power of general-purpose processors is insufficient to meet real-time processing requirements, while dedicated image chips, although improving computing power, sacrifice system flexibility, resulting in an inability to dynamically respond to complex road condition changes.

[0034] The evolution of image stabilization algorithms faces three major technical bottlenecks. The Kalman filter model exhibits phase lag in high-frequency vibration scenarios; optical flow algorithms are limited by the complexity of feature point calculations, and traditional hardware platforms cannot meet real-time processing requirements; while deep learning methods have theoretical advantages, they rely on massive amounts of training data and place stringent demands on hardware computing power during deployment. Existing hardware acceleration attempts have not overcome the limitations of parallel efficiency and power consumption, failing to achieve an effective balance between algorithm performance and hardware resources. These systemic shortcomings prevent traditional solutions from simultaneously achieving both response speed and compensation accuracy in dynamic driving environments.

[0035] Based on this, refer to Figure 4 This disclosure provides a method for stabilizing a head-up display device, which can be applied to the head-up display device. The method for stabilizing a head-up display device may include steps S410 to S420.

[0036] In step S410, the target anti-shake algorithm is determined based on the vehicle's current driving environment.

[0037] In some example implementations of this disclosure, the current driving environment may refer to the real-time external conditions when the vehicle is driving, including road surface type, driving scenario, and environmental parameters.

[0038] Among them, the road surface type can be such as bumpy road surface, smooth highway, etc., the driving scenario can include urban low speed, suburban curves, high-speed cruising, etc., and the environmental parameters can include vehicle speed, steering angle, etc. The specific type of the current driving environment can also be customized based on user needs, which will not be elaborated in this example implementation.

[0039] In some examples, the current driving environment can be obtained using camera modules installed in the vehicle, as well as various sensors, such as GPS and cameras, to collect environmental data.

[0040] After determining the vehicle's current driving environment, a target image stabilization algorithm corresponding to the current driving environment can be matched in the algorithm library. Optionally, the algorithm library may include multiple candidate image stabilization algorithms corresponding to multiple driving environments. The target image stabilization algorithm corresponding to the current driving environment can be matched in the algorithm library.

[0041] In step S420, based on inertial data that can reflect changes in vehicle attitude, the image projection position of the head-up display device is corrected using a target anti-shake algorithm to achieve anti-shake.

[0042] In some exemplary embodiments of this disclosure, inertial data may include physical motion parameters collected in real time by onboard sensors, including information that directly reflects the vehicle's dynamic balance and attitude, such as changes in vehicle tilt angle and acceleration fluctuations. The target stabilization algorithm, acting as the computational hub, receives and parses this data, converting it into the displacement correction amount required for image compensation. Specifically, the correction action involves dynamically adjusting the output coordinates of the projection device. For example, by remapping the pixel positions of the video buffer, the optical components can be controlled to generate a deflection angle opposite to the direction of vehicle vibration.

[0043] In some examples, the stabilization requirements can be determined based on the current driving environment, and a target stabilization algorithm can be matched based on these requirements. Optionally, the stabilization requirements can include accuracy requirements and latency requirements. Accuracy requirements refer to the intensity of the user's demand for image stability, while latency requirements represent the intensity of the user's demand for response speed. Accuracy and latency requirements are normalized to complementary percentage values, meaning their sum is always 1. The system's hardware resources are limited; high-precision algorithms inevitably increase computational complexity, leading to increased latency; while low-latency algorithms must simplify computational steps, resulting in decreased accuracy. These two factors constitute a trade-off, quantifying accuracy and latency requirements as complementary percentages.

[0044] In some examples, the vehicle's driving environment can be interpreted as a stabilization requirement, which includes both accuracy and latency requirements. When the difference between the latency requirement and the accuracy requirement exceeds a first threshold, the target stabilization algorithm is determined as the first compensation algorithm. This first compensation algorithm may include a compensation algorithm based on inter-frame difference. The compensation algorithm based on inter-frame difference aims to achieve a latency less than the first latency threshold, ensuring timely response by simplifying the computational hierarchy.

[0045] For example, when the latency requirement is 70% and the accuracy requirement is 30%, the target image stabilization algorithm can be determined as the first compensation algorithm mentioned above. The first threshold can be a threshold corresponding to the allowable error range, specifically 2%, 0.5%, etc., and can be customized according to requirements.

[0046] When the difference between the accuracy requirement and the latency requirement is equal within a threshold range (i.e., when the difference between the accuracy requirement and the latency requirement is within the threshold range), the target image stabilization algorithm switches to the second compensation algorithm. The second compensation algorithm may include a block matching motion estimation algorithm. The block matching motion estimation algorithm ensures that the stabilization accuracy is greater than the first accuracy threshold and the latency is less than the second latency threshold, achieving a balance between accuracy and latency control. When the difference between the accuracy requirement and the latency requirement is greater than the second threshold, the target image stabilization algorithm employs a third compensation algorithm. The third compensation algorithm may include feature point matching and optical flow algorithms to ensure that the stabilization accuracy is greater than the second accuracy threshold. The second threshold can be a threshold corresponding to an allowable error range, which can be equal to the first threshold, and can be 2%, 0.5%, etc., and can be customized according to requirements.

[0047] The threshold range mentioned above can be an error range, such as -2% to 2%, -0.5% to 0.5%, etc. For example, when the threshold range is -0.5% to 0.5%, and the accuracy requirement is 49.8% and the delay requirement is 50.2%, it can be determined that the accuracy requirement and the delay requirement are equal within the threshold range.

[0048] In some examples, the second delay threshold is greater than the first delay threshold, and the second precision threshold is greater than the first precision threshold. The specific values ​​of the first delay threshold, the second delay threshold, the first precision threshold, and the second precision threshold can be customized based on user needs, and will not be elaborated in this example implementation.

[0049] In some examples, the first threshold can be the absolute value of the lower limit of the threshold range, and the second threshold can be the upper limit of the threshold range. That is to say, no matter what kind of image stabilization requirement, only one algorithm will be involved at the same time.

[0050] In some examples, two algorithms can be used simultaneously. For instance, when the difference between the latency requirement and the accuracy requirement is greater than a first threshold, when both the inter-frame difference-based compensation algorithm and the block matching motion estimation algorithm are included, the historical calculation deviation between the block matching motion estimation algorithm and the inter-frame difference-based compensation algorithm can be used to supplement the calculation results of the inter-frame difference-based compensation algorithm. This improves the accuracy of the inter-frame difference-based compensation algorithm without increasing its latency.

[0051] For another example, when the difference between the accuracy requirement and the latency requirement is within a threshold range, the target stabilization algorithm can include only the block matching motion estimation algorithm, or it can include both the block matching motion estimation algorithm and the feature point matching and optical flow algorithm. The historical calculation deviations of the block matching motion estimation algorithm and the feature point matching and optical flow algorithm can be used to supplement the calculation results of the block matching motion estimation algorithm, thereby improving the accuracy of the block matching motion estimation algorithm without increasing the latency of the block matching motion estimation algorithm.

[0052] In some examples, refer to Figure 5 The anti-shake method of the head-up display device achieves anti-shake through the following hardware working together. Specifically, it may include a microcontroller unit (MCU) 52, an inertial measurement unit (IMU) 51, and a field-programmable gate array (FPGA) 53 chip, and finally display on the LCD screen 54.

[0053] Reference Figure 6 The MCU52 receives vehicle attitude change data collected by the IMU51 via the Controller Area Network (CAN) bus protocol. Optionally, the sampling frequency of the inertial data can be 500Hz, with one acquisition cycle completed every 2 seconds. When the Vertical Synchronization (VSYNC) signal generated by the FPGA53 triggers an interrupt, the MCU52 performs analog-to-digital conversion and integration processing on the continuously acquired inertial data, such as completing 8 sampling operations within a 16.6ms cycle per frame, to generate the anti-shake offset. The anti-shake offset is transmitted to the FPGA53 via the Serial Peripheral Interface (SPI) 531 for dynamic correction of the projection position of the next frame image.

[0054] It should be noted that the sampling frequency of inertial data can also be customized based on user needs, and the sampling period can also be set based on needs, which will not be elaborated in this example implementation.

[0055] Reference Figure 6The FPGA53 architecture constructs a complete video processing pipeline to perform real-time compensation. The video input signal (taking 1280×640 resolution as an example) enters the receiver module (RX module) 532 through a low-voltage differential signal interface. The deserializer parses the serial signal into an RGB parallel image data stream. After being segmented by the segmentation module 533, the parsed data undergoes row-level pre-buffering through a First-In-First-Out (FIFO) memory 534, with configurable buffer depth to adapt to different processing requirements. The dynamic storage layer employs DDR3 high-speed memory 55 to implement a frame buffering strategy.

[0056] Specifically, taking a cache partition consisting of 440 lines of valid display data and 200 lines of fill area as an example, during write operations, only the valid image area is stored, and the remaining area is filled with zero values. When reading data from the DDR3 high-speed memory 55, the system dynamically adjusts the row address offset based on the displacement coordinates transmitted in real time by the MCU 52, and achieves precise matching of the image projection position by filling the area before and after the valid data lines with zero-value areas.

[0057] In some examples, timing accuracy can be ensured through a delay-resistant synchronization mechanism. The falling edge of the VSYNC signal generated by FPGA53 triggers an interrupt response from MCU52, ensuring that the anti-shake offset is transmitted before the next display cycle. During the current frame display process, FPGA53 preloads the compensation parameters for the next frame, combining coordinate remapping and pipelined parallel processing to achieve synchronization between the projected position and the vehicle's attitude. The processed video stream is output to the LCD display 54 via the LVDS transmitter module (TX module) 535, and simultaneously transmitted to the backlight driver chip via the Localdimming module 536 to execute a local dimming algorithm, ensuring the brightness consistency of the compensated image.

[0058] When the inter-frame difference-based compensation algorithm is active, the first offset is calculated directly from the inertial data. This first offset is generated from the original attitude data through a kinematic model, accurately representing the spatial displacement relationship between the frame to be displayed and the current frame. Subsequently, a coordinate mapping operation can be performed to vector-superimpose the first offset with the display position of the current frame, directly outputting the image projection position of the frame to be displayed. This process completely avoids complex image analysis calculations, utilizes the hardware acceleration capabilities of the FPGA53 to achieve low response latency, and is suitable for low-speed urban traffic or simple road conditions with low accuracy requirements.

[0059] When the block matching motion estimation algorithm is activated, the frame to be displayed is first divided into multiple display image blocks, and the corresponding matching image block is determined by searching a reference frame. After calculating the motion vector based on the inter-block displacement relationship, the system introduces inertial data for data fusion. The motion vector and inertial data are input together into a filtering model (such as a Kalman filter) to calculate the second offset. The second offset drives the adjustment of the projection position of the display image block, forming a dual compensation in the spatial domain (image block displacement) and the temporal domain (inertial data synchronization), ensuring that the anti-shake accuracy is greater than the first accuracy threshold and the latency is less than the second latency threshold. For example, in the FPGA53 implementation, image block processing improves efficiency through distributed RAM and pipeline optimization, making it suitable for medium-complexity scenarios with bumpy suburban roads.

[0060] It should be noted that the reference frame can be the frame before the frame to be displayed or the three frames before it. The specific content of the reference frame can be customized according to the requirements, which will not be elaborated here.

[0061] During the feature point matching and optical flow algorithm operation, feature extraction can be performed to obtain multiple feature points. By tracking the motion trajectory of feature points in a reference frame, the system establishes a motion model and derives the motion offset. Simultaneously, inertial data is processed in parallel to generate inertial offsets. Then, the motion offset and inertial offset can be synthesized into a third offset through a weighted model. The third offset serves as the final correction benchmark, and the image projection position of the frame to be displayed is adjusted through a geometric transformation matrix to achieve high-precision spatial compensation.

[0062] Three algorithm modes constitute a complete anti-shake control system. The inter-frame difference-based compensation algorithm focuses on delay optimization, the block matching motion estimation algorithm achieves balanced control, and the feature point matching and optical flow algorithms ensure the accuracy limit. All algorithms establish a mapping relationship between vehicle attitude and projection position through inertial data. The correction execution process uniformly follows the offset-driven principle: the first offset is used for direct coordinate correction, the second offset achieves primary fusion of motion and inertia, and the third offset completes deep collaboration of multi-source data. The entire system is implemented in parallel processing through FPGA53 hardware architecture to ensure computational efficiency in different modes. Image projection position adjustment is achieved by remapping the coordinates of the display buffer, ultimately outputting a stable optical projection effect.

[0063] It should be noted that the first offset, the second offset, and the third offset mentioned above are three different manifestations of the anti-shake offset under three different algorithms.

[0064] In image stabilization methods for head-up display (HUD) devices, real-time image stabilization can be achieved through coordinate mapping and latency optimization. For example, initially, the user interface (UI) display area can be 1100×150 pixels, located at the center of a backlight illumination area of ​​1160×450 pixels. The displacement rows calculated by the IMU are limited to the range [-150, 150], where negative values ​​represent upward displacement, positive values ​​represent downward displacement, and zero values ​​remain in place.

[0065] To ensure strict synchronization between image movement and display frames, a coordinate linear transformation strategy can be implemented. For example, based on the anti-shake offset calculated above, when it is determined that the UI area needs to be shifted upwards by 245 lines, refer to... Figure 7 and Figure 8 The original displacement range [-150, 150] is mapped to the new coordinate range [95, 395]. This transformation is accomplished through a hardware-implemented linear scaling algorithm, ensuring precise alignment between the projected position adjustment and the display cycle. In this mode, upward displacement operations do not incur additional processing latency, with a compensation latency of approximately 3.85ms relative to the original UI, meeting real-time requirements.

[0066] Reference Figure 9 To address the timing synchronization issue between IMU data and video frames, the system employs a two-level delay control mechanism. The IMU51 data update cycle is 2ms, inherently asynchronous with the vertical synchronization (VSYNC) signal. Although the MCU52 output is synchronized with the falling edge of VSYNC, the asynchronous delay still fluctuates within a 0-2ms range, for example, 0.7ms. This delay is mitigated by the pre-load buffer mechanism of the FPGA53, which pre-loads the displacement parameters of the next frame when VSYNC is triggered. The IMU51 attitude calculation time is far less than 2ms (typically <0.1ms), achieving nanosecond-level processing through hardware acceleration. The anti-shake offset is directly connected to the FPGA53 data processing pipeline via SPI531, avoiding software-level delays.

[0067] In some examples, predictive inertial data is obtained based on road surface information in the current driving environment; based on the predictive inertial data, the target anti-shake algorithm is used to determine the predicted offset, and the image projection position is updated based on the predicted offset.

[0068] Specifically, onboard multi-source sensors can collect road information in the current driving environment in real time, covering key parameters such as road unevenness level, gradient of slope angle change, and obstacle distribution density. Based on the vehicle dynamics model and historical driving data, a road-attitude mapping relationship library is constructed, and a Kalman predictor generates predicted inertial data for a future preset time window.

[0069] Vehicle-mounted multi-source sensors may include visual cameras, millimeter-wave radar, and vehicle attitude sensors, etc., which will not be described in detail in this example embodiment.

[0070] After obtaining the aforementioned predicted inertial data, the target stabilization algorithm simultaneously processes the real-time inertial data and the predicted inertial data. It directly generates real-time offsets based on the current attitude data collected by the IMU51, and inputs the predicted inertial data into the target stabilization algorithm to calculate the predicted offsets for the next 3-5 frames. At the FPGA53 hardware layer, this process is pipelined and accelerated through a digital signal processing (DSP) module. Taking the feature point matching algorithm as an example, the system simultaneously loads the real-time feature point motion trajectory and the predicted trajectory vector, generating high-precision compensation parameters through weighted fusion of the motion model, thus eliminating the phase lag phenomenon in traditional solutions.

[0071] After obtaining the predicted offset, the image projection position is updated according to the predicted offset. Specifically, after the prediction time is reached, the real-time offset and the predicted offset can be weighted to determine the anti-shake offset.

[0072] In the image stabilization method for head-up display devices, a target stabilization algorithm is dynamically determined based on the vehicle's current driving environment. Combined with the FPGA53 hardware architecture, a three-level algorithm collaboration is achieved. The inter-frame difference-based compensation algorithm is used for high real-time demand scenarios. It directly generates the first offset based on inertial data through a kinematic model, achieving microsecond-level coordinate mapping. The block matching motion estimation algorithm is activated when the accuracy and latency requirements are balanced. It performs motion estimation on image blocks and fuses inertial data. The second offset is calculated through a filtering model to complete spatiotemporal dual compensation. The feature point matching and optical flow algorithm is used to deal with high-precision scenarios. The third offset is generated through feature point trajectory tracking and motion model, achieving sub-pixel-level spatial calibration.

[0073] The system enhances dynamic response capabilities through a predictive compensation mechanism. Based on road surface information, it generates predictive inertial data, and the target anti-shake algorithm simultaneously processes real-time and predicted data to calculate the predicted offset. The real-time and predicted offsets are then vector-superimposed, driving the DDR3 high-speed memory's 55-fold remapping engine to perform address offset calculations. The backlight co-control unit synchronously updates the local dimming area to ensure visual consistency. This mechanism significantly reduces image residual jitter on bumpy roads and minimizes pitch angle compensation errors in curved scenes.

[0074] The FPGA hardware acceleration architecture supports efficient execution. It can parse video input signals through a Low-Voltage Differential Signaling (LVDS) interface, and manage frame data using a FIFO534 pre-buffer and DDR3 high-speed memory 55 for dynamic storage. Vertical synchronization signals trigger MCU52 interrupt responses, and SPI531 transmits stabilization displacement parameters. An anti-delay synchronization mechanism stably controls end-to-end latency within a threshold. Dynamic partial reconfiguration technology enables on-demand activation of algorithm modules, significantly reducing system power consumption and supporting OTA remote updates to the road-attitude mapping model library, thus building an adaptive stabilization ecosystem.

[0075] Further reference Figure 10 As shown, this example embodiment also provides a head-up display device anti-shake device 1000, including an algorithm determination module 1010, an inertial measurement module 1020, and an FPGA position mapping module 1030. Wherein: The algorithm determination module 1010 can be used to determine the target anti-shake algorithm based on the vehicle's current driving environment.

[0076] In one example implementation, the algorithm determination module 1010 can be configured to determine the stabilization requirements based on the current driving environment and match a target stabilization algorithm based on the stabilization requirements.

[0077] In one example implementation, the image stabilization requirement includes an accuracy requirement and a latency requirement. The algorithm determination module 1010 can be configured such that if the difference between the latency requirement and the accuracy requirement is greater than a first threshold, the target image stabilization algorithm includes a first compensation algorithm so that the latency corresponding to the latency requirement is less than the first latency threshold.

[0078] In one example implementation, the stabilization requirement includes an accuracy requirement and a latency requirement. The algorithm determination module 1010 can be configured such that, when the difference between the accuracy requirement and the latency requirement is equal within a threshold range, the target stabilization algorithm includes a second compensation algorithm, such that the stabilization accuracy corresponding to the accuracy requirement is greater than a first accuracy threshold, and the latency corresponding to the latency requirement is less than a second latency threshold.

[0079] In one example implementation, the stabilization requirement includes an accuracy requirement and a latency requirement. The algorithm determination module 1010 can be configured to include a third compensation algorithm when the difference between the accuracy requirement and the latency requirement is greater than a second threshold, so that the stabilization accuracy corresponding to the accuracy requirement is greater than the second accuracy threshold.

[0080] The inertial measurement module 1020 is used to acquire inertial data that can reflect changes in the vehicle's attitude.

[0081] The FPGA position mapping module 1030 can be used to correct the image projection position of the head-up display device based on inertial data that reflects changes in vehicle attitude, using a target anti-shake algorithm to achieve anti-shake.

[0082] In one example implementation, when the target stabilization algorithm includes a second compensation algorithm, the FPGA position mapping module 1030 can be configured to divide the frame to be displayed into display image blocks, and determine a matching image block corresponding to the display image block in a reference frame; determine a motion vector based on the matching image block and the display image block; determine a second offset based on inertial data and the motion vector; and adjust the projection position of the display image block based on the second offset.

[0083] In one example implementation, when the target stabilization algorithm includes a third compensation algorithm, the FPGA position mapping module 1030 can be configured to extract features from the frame to be displayed to obtain multiple feature points; determine the motion trajectory of the multiple feature points in a reference frame, and determine a motion model based on the motion trajectory; determine a motion offset based on the feature points and the motion model; determine an inertial offset based on inertial data, and determine a third offset based on the motion offset and the inertial offset; and adjust the image projection position of the frame to be displayed according to the third offset.

[0084] In one example implementation, when the target stabilization algorithm includes a first compensation algorithm, the FPGA position mapping module 1030 can be configured to determine a first offset between the frame to be displayed and the current frame based on inertial data; and to determine the image projection position of the frame to be displayed based on the first offset and the display position of the current frame.

[0085] It should be understood that the above-described device embodiments are merely illustrative, and the device disclosed herein can be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, integrated into another system, or some features may be ignored or not executed.

[0086] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this disclosure can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0087] When integrated units / modules are implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the storage unit can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc.

[0088] If the integrated unit / module is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMSDD). Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this disclosure. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0089] Please refer to Figure 11 This illustration shows a structural block diagram of a head-up display (HUD) device provided in an exemplary embodiment of this disclosure. In some examples, the HUD device can be at least one of devices such as a smartphone, smartwatch, desktop computer, laptop, virtual reality terminal, augmented reality terminal, wireless terminal, and laptop computer. The HUD device has communication capabilities and can access wired or wireless networks. The term "HUD device" can refer to one of multiple terminals; those skilled in the art will understand that the number of terminals can be more or less. It is understood that the HUD device undertakes the computation and processing work of the technical solutions of this disclosure, and this disclosure does not limit this aspect.

[0090] like Figure 11 As shown, the head-up display device 1100 may include at least one processor 1110, a memory 1120, and a communication interface 1130.

[0091] The memory 1120 is used to store programs. Specifically, the program may include program code, which includes computer operation instructions.

[0092] The memory 1120 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage.

[0093] The processor 1110 is used to execute computer execution instructions stored in the memory 1120 to implement the head-up display device anti-shake method described in the foregoing method embodiments. The processor 1110 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this disclosure.

[0094] The head-up display device 1100 may also include a communication interface 1130, through which it can communicate and interact with external devices. In specific implementations, if the communication interface 1130, memory 1120, and processor 1110 are implemented independently, they can be interconnected via a bus to complete communication. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc., but this does not imply that there is only one bus or one type of bus.

[0095] Optionally, in a specific implementation, if the communication interface 1130, memory 1120 and processor 1110 are integrated on a single chip, then the communication interface 1130, memory 1120 and processor 1110 can communicate through an internal interface.

[0096] This disclosure also provides a computer storage medium, which may include various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory, a random access memory, a disk, or an optical disk. Specifically, the computer storage medium stores program instructions, which are used for the head-up display device anti-shake method in the above embodiments.

[0097] This disclosure also provides a computer program product including computer instructions stored in a computer storage medium; a processor of a head-up display device reads the computer instructions from the computer storage medium and executes the computer instructions, causing the head-up display device to perform the head-up display device anti-shake method of the above embodiments.

[0098] Those skilled in the art will recognize that the functions described in the embodiments of this disclosure in one or more of the foregoing examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0099] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0100] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention applied herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not claimed herein.

[0101] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for stabilizing a head-up display device, characterized in that, include: The anti-shake requirements are determined based on the current driving environment, and a target anti-shake algorithm is matched based on the anti-shake requirements. The current driving environment includes road surface type, driving scenario and environmental parameters, including vehicle speed and steering angle. The anti-shake requirements include accuracy requirements and latency requirements. If the difference between the latency requirement and the accuracy requirement is greater than a first threshold, the target anti-shake algorithm includes a first compensation algorithm to make the latency corresponding to the latency requirement less than the first latency threshold. When the difference between the accuracy requirement and the delay requirement is equal within a threshold range, the target image stabilization algorithm includes a second compensation algorithm to make the image stabilization accuracy corresponding to the accuracy requirement greater than the first accuracy threshold, and the delay corresponding to the delay requirement less than the second delay threshold. If the difference between the accuracy requirement and the delay requirement is greater than the second threshold, the target image stabilization algorithm includes a third compensation algorithm to make the image stabilization accuracy corresponding to the accuracy requirement greater than the second accuracy threshold, wherein the first delay threshold is less than the second delay threshold, and the first accuracy threshold is less than the second accuracy threshold. Based on inertial data that reflects changes in vehicle attitude, the target anti-shake algorithm is used to correct the image projection position of the head-up display device to achieve anti-shake. The inertial data includes changes in vehicle tilt angle and acceleration fluctuations.

2. The anti-shake method for a head-up display device according to claim 1, characterized in that, When the target image stabilization algorithm includes a first compensation algorithm, the step of correcting the image projection position of the head-up display device using the target image stabilization algorithm based on inertial data that reflects changes in the vehicle's attitude includes: The first offset between the frame to be displayed and the current frame is determined based on the inertial data; The image projection position of the frame to be displayed is determined based on the first offset and the display position of the current frame.

3. The anti-shake method for a head-up display device according to claim 1, characterized in that, When the target image stabilization algorithm includes the second compensation algorithm, the step of correcting the image projection position of the head-up display device using the target image stabilization algorithm based on inertial data that can reflect changes in the vehicle's attitude includes: The frame to be displayed is divided into display image blocks, and a matching image block corresponding to the display image block is determined in the reference frame; Motion vectors are determined based on the matched image blocks and the displayed image blocks; The second offset is determined based on the inertial data and the motion vector; Adjust the projection position of the displayed image block according to the second offset.

4. The anti-shake method for a head-up display device according to claim 1, characterized in that, When the target image stabilization algorithm includes a third compensation algorithm, the step of correcting the image projection position of the head-up display device using the target image stabilization algorithm based on inertial data that reflects changes in the vehicle's attitude includes: Perform feature extraction on the frame to be displayed to obtain multiple feature points; Determine the motion trajectories of multiple feature points in a reference frame, and determine a motion model based on the motion trajectories; The motion offset is determined based on the feature points and the motion model; The inertial offset is determined based on the inertial data, and the third offset is determined based on the motion offset and the inertial offset. Adjust the image projection position of the frame to be displayed based on the third offset.

5. The anti-shake method for a head-up display device according to claim 1, characterized in that, The method further includes: Based on the road surface information in the current driving environment, predictive inertial data is obtained; Based on the predicted inertial data, the predicted offset is determined using the target anti-shake algorithm; The image projection position is updated based on the predicted offset.

6. A shake stabilization device for a head-up display device, characterized in that, include: The algorithm determination module is used to determine the anti-shake requirements based on the current driving environment and match the target anti-shake algorithm based on the anti-shake requirements. The current driving environment includes road surface type, driving scenario and environmental parameters. The environmental parameters include vehicle speed and steering angle. The anti-shake requirements include accuracy requirements and latency requirements. If the difference between the latency requirement and the accuracy requirement is greater than a first threshold, the target anti-shake algorithm includes a first compensation algorithm to make the latency corresponding to the latency requirement less than the first latency threshold. When the difference between the accuracy requirement and the delay requirement is equal within a threshold range, the target image stabilization algorithm includes a second compensation algorithm to make the image stabilization accuracy corresponding to the accuracy requirement greater than the first accuracy threshold, and the delay corresponding to the delay requirement less than the second delay threshold. If the difference between the accuracy requirement and the delay requirement is greater than the second threshold, the target image stabilization algorithm includes a third compensation algorithm to make the image stabilization accuracy corresponding to the accuracy requirement greater than the second accuracy threshold, wherein the first delay threshold is less than the second delay threshold, and the first accuracy threshold is less than the second accuracy threshold. The inertial measurement module is used to acquire inertial data that reflects changes in vehicle attitude. The FPGA position mapping module is used to correct the image projection position of the head-up display device based on the inertial data and the target anti-shake algorithm to achieve anti-shake. The inertial data includes changes in vehicle body tilt angle and acceleration fluctuations.

7. A head-up display device, characterized in that, The head-up display device includes a processor and a memory; the processor is configured to execute instructions stored in the memory to implement the head-up display device anti-shake method as described in any one of claims 1 to 5.

8. A computer storage medium, characterized in that, The computer storage medium stores at least one instruction, which is executed by a processor to implement the head-up display device anti-shake method as described in any one of claims 1 to 5.

Images