Vehicle projection display control method and device, projection image processing control method, automobile, equipment and storage medium

Abstract

This application discloses a vehicle projection display control method and apparatus, a projection image processing control method, an automobile, equipment, and a storage medium. It determines, at least based on real-time collected vehicle motion data, the expected relative pose change of the vehicle relative to the target road surface at the projection moment; calculates image geometric transformation parameters at the projection moment based on the expected relative pose change, these parameters being used to offset projection offset caused by vehicle motion; performs a reverse geometric transformation on the original projection image based on the image geometric transformation parameters to generate a compensated projection image at the projection moment; and controls the projection module to project the compensated projection image onto the target road surface. By using sensors to perceive the vehicle's own motion posture in real time and using this data to proactively and inversely adjust the projection image size, the projected content remains stable on the target projection plane.

Description

Technical Field

[0001] This application relates to the fields of automotive lighting and intelligent driving assistance technology, and in particular to a vehicle projection display control method and device, a projection image processing control method, an automobile, equipment and storage medium. Background Technology

[0002] With the development of automotive intelligence, DLP (Digital Light Processing) projection headlights can not only achieve high-precision ADB (Adaptive Driving Beam) masking and programmable intelligent lighting, but also project interactive information such as navigation arrows, pedestrian warnings, and width indication light carpets onto the road surface. However, when a vehicle is driving on bumpy roads or undergoes dynamic changes in attitude, the relative position of the vehicle body and the road surface changes rapidly, causing unexpected jitter, displacement, or deformation of the image projected onto the road surface, seriously affecting the readability, safety, and user experience of the information.

[0003] Currently, most mass-produced vehicles do not implement projection anti-shake solutions, or avoid displaying the projected graphics for extended periods. Instead, they use methods such as flashing or breathing to reduce the duration of the projected icons on the road surface, thus mitigating the decline in user experience caused by bumps and vibrations. Summary of the Invention

[0004] The purpose of this application is to provide a projection display control method, a projection image processing control method, and related equipment, which collects vehicle motion data in real time and uses this data to proactively and inversely adjust the projection image size so that the projected content remains stable on the target projection plane.

[0005] The technical solution provided in this application is: a vehicle projection display control method, comprising: Based at least on the real-time collected vehicle motion data, determine the expected relative pose change of the vehicle relative to the target road surface at the projection time. The image geometric transformation parameters at the projection time are calculated based on the expected relative pose change, and the image geometric transformation parameters are used to offset the projection offset caused by vehicle motion. The original projected image is subjected to an inverse geometric transformation based on the image geometric transformation parameters to generate a compensated projected image at the projection time. The control projection module projects the compensated projection image onto the target road surface.

[0006] In a preferred embodiment, the expected relative pose change includes at least the incremental rotation angle of the vehicle body; The incremental rotation angle includes the changes in the vehicle's roll angle, pitch angle, and yaw angle.

[0007] In a preferred embodiment, the expected relative pose change also includes the vertical displacement change of the vehicle; Determining the expected relative pose change of the vehicle relative to the target road surface at the projection time further includes: estimating the vertical displacement change of the vehicle based on the motion data.

[0008] In a preferred embodiment, the change in vertical displacement is obtained through one or more of the following: a dual inertial measurement unit algorithm, a double integral of vertical acceleration, or data from a vehicle height sensor.

[0009] In a preferred embodiment, determining the expected relative pose change of the vehicle relative to the target road surface at the projection time further includes: The system response delay is obtained, and the system response delay includes at least one of the following: inertial measurement unit output delay, data processing delay, image compensation calculation delay, and projection module display refresh delay; Based on the motion data and the system response delay, the relative pose change of the vehicle relative to the target road surface at the moment of projection display is predicted.

[0010] In a preferred embodiment, the prediction of the relative pose change of the vehicle relative to the target road surface at the projection display time further includes: Based on motion data from the current and historical moments, a linear extrapolation or uniform angular velocity motion model is used to predict the changes in the vehicle body posture parameters at the time of projection display compared to the current time, which are then used as the expected relative pose change. The projection time is predicted based on the system response delay.

[0011] In a preferred embodiment, the image geometric transformation parameters at the projection time are further determined based on preset system calibration parameters. The system calibration parameters include the intrinsic parameter matrix of the projection optical engine, the relative pose relationship between the projection optical engine and the vehicle body coordinate system, and the plane equation parameters of the target road surface.

[0012] In a preferred embodiment, the image geometric transformation parameters include a compensation homography matrix, and the calculation of the image geometric transformation parameters at the projection time based on the expected relative pose change includes: The compensation homography matrix at the projection time is calculated based on the expected relative pose change and the system calibration parameters.

[0013] In a preferred embodiment, the compensated homography matrix is ​​calculated according to the following model: , Where K is the projection optical engine intrinsic parameter matrix, R is the inverse rotation matrix of the projection optical engine coordinate system relative to the virtual stable coordinate system during the compensation period, T is the inverse translation vector, n is the normal vector of the target road surface in the vehicle coordinate system, and d is the distance constant. Specifically, based on the relative pose relationship between the projection optical engine and the vehicle body coordinate system, the inverse rotation matrix R of the projection optical engine coordinate system relative to the virtual stable coordinate system during the compensation period is further calculated. Based on the plane equation parameters of the target road surface, the normal vector n of the target road surface in the vehicle body coordinate system is further obtained. Based on the relative pose change, the inverse translation vector T is further calculated.

[0014] In a preferred embodiment, the step of performing an inverse geometric transformation on the original projected image based on image geometric transformation parameters further includes: Based on the compensated homography matrix, the original projected image is subjected to real-time perspective transformation in the image processor through texture mapping or pixel remapping.

[0015] In a preferred embodiment, the motion data includes angular velocity data and / or linear acceleration data acquired by an inertial measurement unit.

[0016] In a preferred embodiment, determining the expected relative pose change of the vehicle relative to the target road surface at the projection time further includes: The motion data is fused using at least one of Kalman filtering, complementary filtering, and quaternion filtering algorithms to obtain the relationship between the real-time relative pose change and the expected relative pose change of the vehicle body.

[0017] In a preferred embodiment, the control projection module projecting the compensated projection image onto the target road surface further includes: The compensated projected image data is sent to a digital micromirror array and / or a transmissive liquid crystal panel and / or a reflective silicon-based liquid crystal panel, and their driving circuits for projection display.

[0018] In a preferred embodiment, before determining the expected relative pose change of the vehicle relative to the target road surface at the projection time based at least on the real-time acquired vehicle motion data, a system calibration step is further included: The intrinsic parameter matrix of the calibrated projection optical engine, the relative pose relationship between the calibrated projection optical engine and the vehicle body coordinate system, and the plane equation parameters of the target road surface are determined.

[0019] This application also provides a vehicle projection image processing control method, including: Based on real-time collected vehicle motion data and / or road condition information of the current driving surface, the vehicle projection display control method described above is used.

[0020] In a preferred embodiment, the vehicle projection display control method is controlled based on real-time collected vehicle motion data and / or road condition information of the current driving surface, including: The real-time vibration frequency of the vehicle is determined based on the real-time collected vehicle motion data. Based on the real-time vibration frequency of the vehicle, execute any of the following control methods: The image geometric transformation parameter update frequency and / or amplitude of the vehicle projection display control method are controlled. The system switches between the vehicle projection display control method and the control method that uses low-pass filtering compensation to generate a compensated projection image for projection.

[0021] In a preferred embodiment, in response to the real-time vibration frequency of the vehicle meeting a first preset condition, a first anti-shake projection control mode is executed; In response to the vehicle's real-time vibration frequency not meeting the first preset condition, the second anti-shake projection control mode is executed. The first image stabilization projection control method has a lower jitter suppression rate than the second image stabilization projection control method. The first image stabilization projection control method includes any one of the following control methods: Reduce the update frequency and / or amplitude of the image geometric transformation parameters in the vehicle projection display control method; A control method for projection that uses low-pass filtering compensation to generate a compensated projection image; The second image stabilization projection control method includes any one of the following control methods: Increase the update frequency and / or amplitude of the image geometric transformation parameters in the vehicle projection display control method; The vehicle projection display control method described above is adopted.

[0022] In a preferred embodiment, the first preset condition is: The real-time vibration frequency of the vehicle is greater than a first preset frequency threshold; or The proportion of frequencies exceeding a preset frequency threshold in the vibration frequency spectrum formed by the real-time vibration frequency of the vehicle is greater than the first proportional threshold.

[0023] In a preferred embodiment, in response to road condition information indicating that the road condition is a bumpy section, a first anti-shake projection control mode is executed; In response to the road condition information indicating that the road is flat, the second anti-shake projection control mode is executed. The first image stabilization projection control method has a lower jitter suppression rate than the second image stabilization projection control method. The first image stabilization projection control method includes any one of the following control methods: Reduce the update frequency and / or amplitude of the image geometric transformation parameters in the vehicle projection display control method; A control method for projection that uses low-pass filtering compensation to generate a compensated projection image; The second image stabilization projection control method includes any one of the following control methods: Increase the update frequency and / or amplitude of the image geometric transformation parameters in the vehicle projection display control method; The vehicle projection display control method described above is adopted.

[0024] In a preferred embodiment, the road condition information of the current driving surface is determined based on one or more of the vehicle's real-time vibration frequency, visual information, and radar information.

[0025] In a preferred embodiment, reducing the update frequency and / or amplitude of the image geometric transformation parameters of the vehicle projection display control method includes: making the update frequency and / or amplitude of the image geometric transformation parameters of the vehicle projection display control method inversely correlated with the real-time vehicle speed and / or the real-time vibration frequency of the vehicle.

[0026] In a preferred embodiment, increasing the update frequency and / or amplitude of the image geometric transformation parameters of the vehicle projection display control method includes making the update frequency and / or amplitude of the image geometric transformation parameters of the vehicle projection display control method positively correlated with the real-time vehicle speed and / or the real-time vibration frequency of the vehicle.

[0027] Based on the same concept, this application also provides a vehicle projection display control device, which employs the control method described in any one of the above-mentioned methods, including: The data processing module is used to determine, at least based on real-time collected vehicle motion data, the expected relative pose change of the vehicle relative to the target road surface at the projection time; and to calculate the image geometric transformation parameters at the projection time based on the expected relative pose change, the image geometric transformation parameters being used to offset the projection offset caused by vehicle motion. The image processing module is used to perform an inverse geometric transformation on the original projected image according to the image geometric transformation parameters, and generate a compensated projected image at the projection time. A projection module is used to project the compensated projection image onto the target road surface.

[0028] In a preferred embodiment, it further includes: The data acquisition module is used to collect the vehicle's motion data. The data acquisition module is an inertial measurement unit, which measures the angular velocity data and / or linear acceleration data of the vehicle body.

[0029] In a preferred embodiment, the projection module is a DLP projection module, which includes a digital micromirror array and / or a transmissive liquid crystal panel and / or a reflective silicon-based liquid crystal panel, and a driving circuit.

[0030] Based on the same concept, this application also provides an automobile, including the vehicle projection display control device described in any one of the above.

[0031] Based on the same concept, this application also provides a projection device, including: a processor, a memory, and a bus, wherein the memory stores program instructions executable by the processor, and when the projection device is running, the processor communicates with the memory via the bus, and the processor executes the program instructions to perform the steps of any of the control methods described above.

[0032] Based on the same concept, this application also provides a readable storage medium storing a processing program, which, when executed by a processor, implements the steps of the control method described in any of the above claims.

[0033] Compared with the prior art, this application has the following advantages: This application predicts the expected relative pose change between the vehicle body and the target road surface at the projection moment based on real-time collected vehicle motion data, and then calculates the corresponding image geometric transformation parameters (these parameters are opposite in direction and equal in magnitude to the expected offset), thereby enabling the inverse geometric transformation of the original image and generating a compensated projection image. After being projected by the projection module, the actual landing point of the light on the road surface coincides with the preset position of the original image, thereby offsetting the projection jitter, displacement and deformation caused by vehicle bumps or attitude changes. Attached Figure Description

[0034] Figure 1 This is a flowchart of the vehicle projection display control method of this application. Detailed Implementation

[0035] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be understood that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on this application. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0036] It should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0037] Example See Figure 1 This embodiment provides a vehicle projection display control method, including: Based at least on the real-time collected vehicle motion data, determine the expected relative pose change of the vehicle relative to the target road surface at the projection time. The image geometric transformation parameters at the projection time are calculated based on the expected relative pose change. These parameters are used to offset the projection offset caused by vehicle motion. The original projected image is subjected to an inverse geometric transformation based on the image geometric transformation parameters to generate a compensated projected image at the projection time. The control projection module projects the compensated projection image onto the target road surface.

[0038] In this embodiment, the expected relative pose change between the vehicle body and the target road surface at the projection moment is predicted based on the real-time collected vehicle motion data. Then, the corresponding image geometric transformation parameters are calculated (these parameters are opposite in direction and equal in magnitude to the expected offset), thereby enabling the inverse geometric transformation of the original image and generating a compensated projection image. After being projected by the projection module, the actual landing point of the light on the road surface coincides with the preset position of the original image, thereby offsetting the projection jitter, displacement and deformation caused by vehicle bumps or posture changes.

[0039] This application constructs a transformation from the "projection optical-mechanical coordinate system" and / or the "vehicle vehicle coordinate system" to the "virtual stable coordinate system". Assuming that content / images need to be projected onto a "virtual stable plane" in inertial space, which is relatively stationary with respect to the ground, the relationship between the "projection optical-mechanical coordinate system" and / or the "vehicle vehicle coordinate system" and the "virtual stable coordinate system" can be calculated through real-time changes in vehicle attitude. This allows for the calculation of the inverse geometric transformation parameters required by the projection module to output a stable image.

[0040] In a preferred embodiment, the expected relative pose change includes at least the incremental rotation angle of the vehicle body; The incremental rotation angle includes the changes in the vehicle's roll angle, pitch angle, and yaw angle.

[0041] In this embodiment, by real-time acquisition of incremental rotation angles (roll, pitch, and yaw angle changes) of the vehicle body using sensors, the three-dimensional rotational attitude changes of the vehicle body at the projection moment can be accurately predicted. This allows for the calculation of corresponding image reverse rotation / distortion compensation parameters, enabling the performance of a reverse geometric transformation on the original image. This compensates for the tilt, distortion, and other deformations of the projected image caused by the vehicle body's rotation on the road surface, ensuring that the projected information remains horizontal, upright, and accurately positioned. For example, a high-precision six-axis IMU (Inertial Measurement Unit) can be used to acquire incremental rotation angle data.

[0042] Since the projection module is rigidly connected to the vehicle body, its motion is consistent with that of the vehicle body. Based on this premise, the incremental rotation angle Δθ measured by the IMU is approximated as the incremental rotation angle of the projection center of the projection module. Assuming the target road surface is a plane, its equation relative to the vehicle body coordinate system at the initial moment is (obtained through calibration): (Where, n is the normal vector of the target road surface in the vehicle coordinate system, and d is a distance constant.) (This refers to the position coordinates of a point in the vehicle coordinate system).

[0043] When calculating the movement of the intersection point between the DLP headlight projection light path and the target plane (assuming the target road surface is a plane), in addition to the incremental rotation angle Δθ, the translation of the projection center point also needs to be considered.

[0044] When the car body rotates This causes the projection center of the projection module to shift. (After translation, which mainly comes from suspension extension and retraction) the projection light path of the projection module is at a point on the horizontal ground. The coordinate changes in the vehicle body coordinate system are as follows: , In a preferred embodiment, the expected relative pose change also includes the vertical displacement change of the vehicle; Determining the expected relative pose change of the vehicle relative to the target road surface at the projection time further includes estimating the vertical displacement change of the vehicle based on motion data.

[0045] This embodiment also comprehensively predicts the six-degree-of-freedom pose changes of the vehicle body at the projection moment by collecting vehicle motion data in real time and estimating the vertical displacement change, combined with incremental rotation angles (roll, pitch, yaw), thereby calculating the corresponding reverse compensation parameters such as image translation and distortion adjustment, and performing reverse geometric transformation on the original image to effectively offset the floating, stretching and distortion of the projected image caused by the vehicle body's vertical jump, tilt and steering, so that the road projection content still maintains a good display effect when bumpy and dynamic attitude changes.

[0046] In a preferred embodiment, the change in vertical displacement is obtained through one or more of the following: a dual inertial measurement unit algorithm, a double integral of vertical acceleration, or data from a vehicle height sensor.

[0047] This embodiment employs a dual inertial measurement unit algorithm, a double integral of vertical acceleration, or one or more of the vehicle height sensors to more accurately acquire the change in the vehicle's vertical displacement. Combined with rotation angle data, it improves the prediction accuracy of the vehicle's six-degree-of-freedom pose at the projection moment, thereby calculating more accurate image geometric compensation parameters.

[0048] In a preferred embodiment, determining the expected relative pose change of the vehicle relative to the target road surface at the projection time further includes: The system response delay is obtained, which includes at least one of the following: inertial measurement unit output delay, data processing delay, image compensation calculation delay, and projection module display refresh delay. Based on motion data and system response delay, the relative pose change of the vehicle relative to the target road surface at the moment of projection display is predicted.

[0049] This embodiment acquires the system response delay and predicts the pose change at the projection display time rather than the current time based on motion data and response delay time. This makes the image compensation parameters match the vehicle body posture at the actual time of projection illumination, thereby effectively eliminating the problem of compensation leading or lagging due to delays in various parts of the system, reducing residual jitter and position deviation of road projection, and improving the accuracy and real-time performance of projection compensation during dynamic driving.

[0050] In a preferred embodiment, predicting the relative pose change of the vehicle relative to the target road surface at the moment of projection display further includes: Based on motion data from the current and historical moments, a linear extrapolation or uniform angular velocity motion model is used to predict the changes in the vehicle body attitude parameters at the time of projection and the changes in the vehicle body attitude parameters at the current moment as the expected relative pose change. The projection moment is predicted based on the system response delay.

[0051] Based on current and historical motion data, linear extrapolation or uniform angular velocity motion models are used to predict the change in vehicle posture between the projection display time (considering system response delay) and the current time. Compared with using only current data, this can reduce prediction deviations caused by motion non-stationarity, thereby improving the matching accuracy between the compensated image and the actual projection pose.

[0052] In a preferred embodiment, the image geometric transformation parameters at the projection time are further determined based on preset system calibration parameters. The system calibration parameters include the intrinsic parameter matrix of the projection optical engine, the relative pose relationship between the projection optical engine and the vehicle body coordinate system, and the plane equation parameters of the target road surface.

[0053] This embodiment introduces system calibration parameters such as the projection optical engine intrinsic parameter matrix, the optical engine-vehicle relative pose, and the road surface plane equation. It can accurately map the vehicle body attitude compensation amount to the projection image coordinate system and take into account the road surface geometric constraints. This eliminates the projection position error caused by optical engine distortion, installation deviation, and road surface tilt, making the compensated image fall more accurately on the target road surface area and improving the geometric consistency of projection stabilization.

[0054] In a preferred embodiment, the image geometric transformation parameters include a compensation homography matrix, and the calculation of the image geometric transformation parameters at the projection time based on the expected relative pose change includes: Based on the expected relative pose change and system calibration parameters, calculate the compensation homography matrix at the projection time.

[0055] This embodiment incorporates image geometric transformation parameters into the compensation homography matrix and calculates the expected relative pose change and system calibration parameters. This allows the six-degree-of-freedom pose change of the vehicle body, along with optomechanical intrinsic and extrinsic parameters and road geometric constraints, to be unified into the same mathematical model. When performing inverse geometric transformation on the original image, translation, rotation, and distortion compensation can be completed simultaneously with a single matrix operation, improving the efficiency and accuracy of compensation calculations and enabling precise mapping of the projected image onto the target road surface.

[0056] In a preferred embodiment, the compensation homography matrix is ​​calculated according to the following model: , Where K is the projection optical engine intrinsic parameter matrix, R is the inverse rotation matrix of the projection optical engine coordinate system relative to the virtual stable coordinate system during the compensation period, T is the inverse translation vector, n is the normal vector of the target road surface in the vehicle coordinate system, and d is the distance constant. Specifically, based on the relative pose relationship between the projection optical engine and the vehicle body coordinate system, the inverse rotation matrix R of the projection optical engine coordinate system relative to the virtual stable coordinate system during the compensation period is further calculated. Based on the plane equation parameters of the target road surface, the normal vector n of the target road surface in the vehicle body coordinate system is further obtained. Based on the relative pose change, the inverse translation vector T is further calculated.

[0057] This embodiment unifies vehicle dynamic pose changes, lens distortion, and three-dimensional road surface geometric features into a single analytical expression, achieving precise mapping from physical space motion to pixel-level reverse compensation of images, ensuring the geometric accuracy of projection compensation under complex road conditions. The compensation period refers to the time span between the current moment of acquiring vehicle motion data and the "projection moment" when the projection module actually projects light onto the road surface. The virtual stable coordinate system is a pre-constructed reference benchmark that maintains an ideal stable state in space, for example, coinciding with the horizontal posture of the vehicle when stationary on a straight road. The reverse translation vector refers to the vector in the opposite direction of the actual physical displacement of the projection optical-mechanical coordinate system relative to the virtual stable coordinate system within the compensation period, i.e., the vector of reverse translation for each image pixel.

[0058] For each pixel projected by the projection module, the direction of the emitted light rays is fixed in the projection coordinate system P-frame. Taking DLP as an example, suppose that in the initial state, the intersection point of the light ray corresponding to a certain pixel on the DMD (Digital Micromirror Device) and the road plane is u. When the vehicle body posture changes, if no compensation is made, the light ray will hit a new point u' on the road surface, causing the projection position to shift.

[0059] To counteract this displacement, the pixel needs to be "reverse-moved" (relative to the vehicle body attitude change increment angle) beforehand. That is, the image input to the DMD (or LC, or LCOS) is modified so that the light emitted by the pixel u' in the new image falls exactly on the original target point u after passing through the vehicle body movement.

[0060] In a preferred embodiment, performing an inverse geometric transformation on the original projected image based on image geometric transformation parameters further includes: Based on the compensated homography matrix, real-time perspective transformation of the original projected image is performed in the image processor through texture mapping or pixel remapping.

[0061] The homography matrix H defines the pixel coordinate mapping relationship between the original image and the target compensated image. The image processor performs pixel transformation based on this matrix to achieve reverse compensation for deformations such as rotation of the original image, thus meeting the real-time processing requirements of projection display.

[0062] Based on the above embodiments, after obtaining the homography matrix H, for each frame of the original image to be projected... The compensated image is generated through perspective transformation. : , This process is performed in real time in the image processor via texture mapping or pixel remapping. The compensated image is then... Send to DMD, LC, or LCOS for display.

[0063] In a preferred embodiment, the motion data includes angular velocity data and / or linear acceleration data acquired by an inertial measurement unit.

[0064] This embodiment uses an inertial measurement unit to collect angular velocity and linear acceleration data, transforming high-frequency basic physical quantities into core inputs for calculating the expected pose change. This method is not affected by environmental factors such as external lighting or occlusion, ensuring reliable acquisition of motion data.

[0065] In a preferred embodiment, determining the expected relative pose change of the vehicle relative to the target road surface at the projection time further includes: At least one of the following algorithms—Kalman filtering, complementary filtering, and quaternion filtering—is used to fuse motion data to obtain the relationship between the real-time relative pose change and the expected relative pose change of the vehicle body.

[0066] The raw inertial data is accompanied by sensor noise. The filtering algorithm can remove high-frequency noise and suppress low-frequency drift. Combined with quaternion operations, the singularity problem in the attitude calculation process is avoided. The noisy raw data is transformed into a smooth, continuous and mathematically stable real-time pose change, providing an accurate input with a high signal-to-noise ratio for subsequent image compensation calculations.

[0067] In a preferred embodiment, controlling the projection module to project the compensated projection image onto the target road surface further includes: The compensated projected image data is sent to a digital micromirror array and / or a transmissive liquid crystal panel and / or a reflective silicon-based liquid crystal panel, and their driving circuits for projection display.

[0068] In this embodiment, digital compensation projection image data is sent to display chips and driving circuits such as DMD, LCD, or LCOS. The driving circuit converts the digital image signal into an electrical control signal, precisely driving the state changes of each micro-physical pixel unit on the display chip, thereby performing spatial light modulation on the projection light source. This achieves a complete closed loop from software algorithm compensation to hardware optomechanical execution, ensuring the anti-shake compensation effect.

[0069] In a preferred embodiment, before determining the expected relative pose change of the vehicle relative to the target road surface at the projection time, based at least on the real-time acquired vehicle motion data, a system calibration step is further included: The intrinsic parameter matrix of the calibrated projection optical engine, the relative pose relationship between the calibrated projection optical engine and the vehicle body coordinate system, and the plane equation parameters of the target road surface are determined.

[0070] Before dynamic compensation, the intrinsic parameters of the optomechanical system, the relative pose of the optomechanical system and the vehicle body, and the parameters of the road surface plane equation are obtained in advance. The intrinsic parameter matrix establishes the mapping benchmark between image pixels and light rays, the relative pose establishes the coordinate relationship between the optomechanical system and the vehicle body motion, and the plane equation of the target road surface determines the three-dimensional spatial constraints of the projection landing point. In this way, static variables such as hardware installation deviation and road surface geometric features are parameterized, providing coordinate system benchmarks and constraints for subsequent geometric transformation calculations of dynamic motion data, avoiding the intervention of inherent errors in the dynamic compensation process, and ensuring the geometric accuracy of the projection space mapping.

[0071] In some examples, the IMU is mounted at the center point of the vehicle's front axle, and this is used as the origin of the coordinate system. Definitions: the X-axis points towards the front of the vehicle, the Y-axis points to the left, and the Z-axis points upwards. The IMU can directly measure the angular velocity ω and acceleration a in this coordinate system. The origin of the projection optical engine coordinate system is at the optical center of the DLP module, and its relative installation relationship with the vehicle body coordinate system is determined through calibration (rotation matrix). Translation vector In the context of image stabilization, the world coordinate system can be simplified to an inertial reference system that is considered stationary during the compensation period.

[0072] According to another aspect of this application, a vehicle projection image processing control method is also provided. The vehicle projection image processing control method includes: controlling the vehicle projection display control method as described above based on real-time collected vehicle motion data and / or road condition information of the current driving surface.

[0073] Specifically, when a vehicle traverses a speed bump or unpaved gravel road, the front of the vehicle experiences a sudden, non-linear pitch change. If the aforementioned vehicle projection display control method continues to perform predictive compensation at this time, due to the delay in the sensor prediction model, the compensated image will experience a reverse overshoot at the moment the vehicle hits the speed bump—that is, the image suddenly jumps sharply in the opposite direction before returning to normal. This visual error can mislead the driver's judgment of the distance to obstacles ahead. Furthermore, the projected image is difficult to match with the ground, causing jerking and potentially leading to dizziness. Therefore, this application uses real-time collected vehicle motion data and / or road condition information of the current driving surface to control the parameters in the aforementioned vehicle projection display control method or switch to other anti-shake projection controls to improve the dizziness caused by the rapid, sudden rise and fall of the vehicle body due to the synchronous anti-shake image projection.

[0074] In some specific embodiments, the vehicle projection display control method is controlled based on real-time collected vehicle motion data and / or road condition information of the current driving surface. This includes: determining the real-time vibration frequency of the vehicle based on the real-time collected vehicle motion data; executing any of the following control methods based on the real-time vibration frequency of the vehicle: controlling the update frequency and / or amplitude of the image geometric transformation parameters of the vehicle projection display control method; and switching between the vehicle projection display control method and a control method that uses low-pass filtering compensation to generate a compensated projection image for projection.

[0075] Specifically, in this embodiment, incremental rotation angle data is collected using a high-precision six-axis IMU (Inertial Measurement Unit) as the vehicle's motion data. Based on the incremental rotation angle data collected at a predetermined sampling frequency, vibration frequency domain features can be extracted in real time using Fast Fourier Transform or a digital bandpass filter bank, thereby determining the vehicle's real-time vibration frequency. Based on the vehicle's real-time vibration frequency, it can be determined whether the vehicle is traveling on a bumpy road, a speed bump road, or a paved gravel road. Therefore, based on the identification results, the update frequency and / or amplitude of the image geometric transformation parameters of the vehicle projection display control method can be controlled to ensure normal operation of anti-shake on relatively smooth road sections, and to control the changes in the projected image on bumpy road sections, thereby alleviating dizziness caused by rapid and large changes in the projected image on bumpy road sections.

[0076] Preferably, this can be achieved by controlling the update frequency and / or amplitude of the image geometric transformation parameters in the vehicle projection display control method. In some variations, switching can be made between the aforementioned vehicle projection display control method and a control method that uses low-pass filtering compensation to generate a compensated projection image for projection. The aforementioned vehicle projection display control method is used to achieve projection stabilization, while the low-pass filtering compensation generates a compensated projection image that only performs simple geometric transformations, or only compensates for longitudinal displacement caused by vehicle speed and lateral curvature changes caused by steering wheel angle, without compensating for instantaneous pitch angles and vertical displacements caused by vertical bumps. This application can implement many more variations, which will not be elaborated here.

[0077] In some specific embodiments, the above steps further include: in response to the vehicle's real-time vibration frequency meeting a first preset condition, executing a first anti-shake projection control method; and in response to the vehicle's real-time vibration frequency not meeting the first preset condition, executing a second anti-shake projection control method. The first anti-shake projection control method has a lower shake suppression rate than the second anti-shake projection control method. Therefore, by matching the vehicle's real-time vibration frequency with the first preset condition, the road segment the vehicle is traveling on is distinguished, thereby differentiating different anti-shake projection control methods. On bumpier road segments, a control method with a relatively lower shake suppression rate (the first anti-shake projection control method) is executed; on relatively smooth road segments, a control method with a relatively higher shake suppression rate (the second anti-shake projection control method) is executed.

[0078] In some specific embodiments, the first anti-shake projection control method includes any of the following control methods: reducing the update frequency and / or amplitude of the image geometric transformation parameters of the vehicle projection display control method; and a control method that uses low-pass filtering compensation to generate a compensated projection image for projection. Specifically, reducing the update frequency of the image geometric transformation parameters of the vehicle projection display control method can help avoid excessively fast image updates; reducing the update amplitude of the image geometric transformation parameters of the vehicle projection display control method can help avoid excessive changes in image size. The update frequency and amplitude of the image geometric transformation parameters can be controlled simultaneously, or only one of them can be controlled. The low-pass filtering compensation to generate the compensated projection image only performs simple geometric transformations, or only compensates for longitudinal displacement caused by vehicle speed and lateral curvature changes caused by steering wheel angle, without compensating for instantaneous pitch angles and vertical displacements caused by vertical bumps.

[0079] Specifically, reducing the update frequency and / or amplitude of the image geometric transformation parameters in the vehicle projection display control method can be achieved by making the update frequency and / or amplitude of the image geometric transformation parameters in the vehicle projection display control method inversely correlated with the vehicle's real-time speed and / or the vehicle's real-time vibration frequency. That is, under very bumpy conditions, if the vehicle's real-time vibration frequency is higher and / or the vehicle speed is faster, the update frequency of the projected image will be lower and / or the amplitude of the change in the updated projected image will be smaller, thus alleviating dizziness.

[0080] In some specific embodiments, the second anti-shake projection control method includes any of the following control methods: increasing the update frequency and / or amplitude of the image geometric transformation parameters of the vehicle projection display control method; or employing the vehicle projection display control method. Specifically, increasing the update frequency of the image geometric transformation parameters of the vehicle projection display control method allows the projected image update to keep up with vehicle vibration; increasing the update amplitude of the image geometric transformation parameters of the vehicle projection display control method helps ensure road surface adhesion after changes in the projected image. The update frequency and amplitude of the image geometric transformation parameters can be controlled simultaneously, or only one of them can be controlled. Employing the vehicle projection display control method ensures the anti-shake effect of the projected image.

[0081] Specifically, increasing the update frequency and / or amplitude of the image geometric transformation parameters in the vehicle projection display control method can be achieved by making the update frequency and / or amplitude of the image geometric transformation parameters positively correlated with the real-time vehicle speed and / or the real-time vibration frequency of the vehicle. That is, under relatively smooth conditions, the lower the real-time vibration frequency of the vehicle and / or the slower the vehicle speed, the higher the update frequency of the projected image and / or the greater the amplitude of the changes in the projected image, so that the updates and / or changes of the projected image can keep up with the vehicle's vibration and movement.

[0082] In some specific embodiments, the first preset condition is that the real-time vibration frequency of the vehicle is greater than a first preset frequency threshold. The first preset frequency threshold can be, for example, 15Hz, 20Hz, etc., and this application is not limited to this. Judging by the real-time vibration frequency of the vehicle can speed up the judgment time. In some variations, the first preset condition can also be that the proportion of frequencies exceeding the preset frequency threshold in the vibration frequency spectrum formed by the real-time vibration frequency of the vehicle is greater than a first proportional threshold. Considering that the vibration spectrum of a vehicle is very complex, including both high-frequency, low-amplitude vibrations caused by road bumps and low-frequency, large-amplitude movements caused by acceleration, deceleration, and cornering, the accuracy of identifying bumpy road sections can be improved by ensuring that the proportion of frequencies exceeding the preset frequency threshold in the vibration frequency spectrum is greater than the first proportional threshold. The preset frequency threshold can be, for example, 15-20Hz, and the first proportional threshold can be, for example, 30-40%, and this application is not limited to this.

[0083] In some specific embodiments, different anti-shake projection control methods can also be executed by directly identifying the road condition information of the road surface ahead. For example, in response to the road condition information of the current driving road indicating a bumpy road section, a first anti-shake projection control method is executed; in response to the road condition information of the current driving road indicating a smooth road section, a second anti-shake projection control method is executed, wherein the shake suppression rate of the first anti-shake projection control method is lower than that of the second anti-shake projection control method. The smooth road section in this embodiment may include a relatively smooth road section. Further, the road condition information of the current driving road can be determined based on one or more of the vehicle's real-time vibration frequency, visual information, and radar information. In this embodiment, the first and second anti-shake projection control methods can be similar to the aforementioned methods, and will not be described in detail here.

[0084] Based on the same concept, this application also provides a vehicle projection display control device, which is controlled by any of the control methods described above, including: The data processing module is used to determine the expected relative pose change of the vehicle relative to the target road surface at the projection time, based at least on the real-time collected vehicle motion data; and to calculate the image geometric transformation parameters at the projection time based on the expected relative pose change, the image geometric transformation parameters being used to offset the projection offset caused by the vehicle motion. The image processing module is used to perform an inverse geometric transformation on the original projected image based on the image geometric transformation parameters, and generate a compensated projected image at the projection time. The projection module is used to project the compensated projection image onto the target road surface.

[0085] In a preferred embodiment, it further includes: The data acquisition module is used to collect vehicle motion data. The data acquisition module is an inertial measurement unit (IMU), which measures the angular velocity and / or linear acceleration data of the vehicle body.

[0086] In a preferred embodiment, the projection module is a DLP projection module, which includes a digital micromirror array and / or a transmissive liquid crystal panel and / or a reflective silicon-based liquid crystal panel, and a driving circuit.

[0087] Based on the same concept, this application also provides an automobile, including any of the above-mentioned vehicle projection display control devices.

[0088] Based on the same concept, this application also provides a projection device, including: a processor, a memory, and a bus. The memory stores program instructions that can be executed by the processor. When the projection device is running, the processor communicates with the memory through the bus, and the processor executes the program instructions to perform the steps of any of the control methods described above.

[0089] Based on the same concept, this application also provides a readable storage medium storing a processing program, which, when executed by a processor, implements the steps of the control method described above.

[0090] The embodiments of this application have been described in detail above with reference to the accompanying drawings, but this application is not limited to the above embodiments. Even if various changes are made to this application, if these changes fall within the scope of the claims of this application and their equivalents, they shall still fall within the protection scope of this application.

Claims

1. A vehicle projection display control method characterized by comprising: include: Based at least on the real-time collected vehicle motion data, determine the expected relative pose change of the vehicle relative to the target road surface at the projection time. The image geometric transformation parameters at the projection moment are calculated based on the expected relative pose change. These parameters are used to offset the projection offset caused by vehicle motion. The image geometric transformation parameters at the projection moment are also determined based on preset system calibration parameters, which include the intrinsic parameter matrix of the projection optical engine, the relative pose relationship between the projection optical engine and the vehicle body coordinate system, and the plane equation parameters of the target road surface. The original projected image is subjected to an inverse geometric transformation based on the image geometric transformation parameters to generate a compensated projected image at the projection time. The control projection module projects the compensated projection image onto the target road surface.

2. The vehicle projection display control method according to claim 1, characterized by, The expected relative pose change includes at least the incremental rotation angle of the vehicle body; The incremental rotation angle includes the changes in the vehicle's roll angle, pitch angle, and yaw angle.

3. The vehicle projection display control method according to claim 2, characterized by, The expected relative pose change also includes the vertical displacement change of the vehicle. Determining the expected relative pose change of the vehicle relative to the target road surface at the projection time further includes: estimating the vertical displacement change of the vehicle based on the motion data.

4. The vehicle projection display control method according to claim 3, characterized by, The change in vertical displacement is obtained through one or more of the following: a dual inertial measurement unit algorithm, a double integral of vertical acceleration, or data from a vehicle height sensor.

5. The vehicle projection display control method according to claim 1, characterized by, The determination of the expected relative pose change of the vehicle relative to the target road surface at the projection time further includes: The system response delay is obtained, and the system response delay includes at least one of the following: inertial measurement unit output delay, data processing delay, image compensation calculation delay, and projection module display refresh delay; Based on the motion data and the system response delay, the relative pose change of the vehicle relative to the target road surface at the moment of projection display is predicted.

6. The vehicle projection display control method according to claim 5, characterized by The predicted change in the relative pose of the vehicle with respect to the target road surface at the moment of projection display further includes: Based on motion data from the current and historical moments, a linear extrapolation or uniform angular velocity motion model is used to predict the changes in the vehicle body posture parameters at the time of projection display compared to the current time, which are then used as the expected relative pose change. The projection time is predicted based on the system response delay.

7. The vehicle projection display control method according to claim 1, characterized by, The image geometric transformation parameters include a compensation homography matrix, and the calculation of the image geometric transformation parameters at the projection time based on the expected relative pose change includes: The compensation homography matrix at the projection time is calculated based on the expected relative pose change and the system calibration parameters.

8. The vehicle projection display control method according to claim 7, characterized by, The compensation homography matrix is ​​calculated according to the following model: ， Where K is the projection optical engine intrinsic parameter matrix, R is the inverse rotation matrix of the projection optical engine coordinate system relative to the virtual stable coordinate system during the compensation period, T is the inverse translation vector, n is the normal vector of the target road surface in the vehicle coordinate system, and d is the distance constant. Specifically, based on the relative pose relationship between the projection optical engine and the vehicle body coordinate system, the inverse rotation matrix R of the projection optical engine coordinate system relative to the virtual stable coordinate system during the compensation period is further calculated. Based on the plane equation parameters of the target road surface, the normal vector n of the target road surface in the vehicle body coordinate system is further obtained. Based on the relative pose change, the inverse translation vector T is further calculated.

9. The vehicle projection display control method according to claim 7, characterized by, The step of performing an inverse geometric transformation on the original projected image based on image geometric transformation parameters further includes: Based on the compensated homography matrix, the original projected image is subjected to real-time perspective transformation in the image processor through texture mapping or pixel remapping.

10. The vehicle projection display control method according to claim 1, characterized by, Before determining the expected relative pose change of the vehicle relative to the target road surface at the projection moment, based at least on the real-time acquired vehicle motion data, a system calibration step is also included: The intrinsic parameter matrix of the calibrated projection optical engine, the relative pose relationship between the calibrated projection optical engine and the vehicle body coordinate system, and the plane equation parameters of the target road surface are determined.

11. A vehicle projection image processing control method characterized by comprising: include: The vehicle projection display control method as described in any one of claims 1 to 10 is controlled based on real-time collected vehicle motion data and / or road condition information of the current driving surface.

12. The vehicle projection image processing control method according to claim 11, characterized by, The vehicle projection display control method, based on real-time collected vehicle motion data and / or road condition information of the current driving surface, includes: The real-time vibration frequency of the vehicle is determined based on the real-time collected vehicle motion data. Based on the real-time vibration frequency of the vehicle, execute any of the following control methods: The image geometric transformation parameter update frequency and / or amplitude of the vehicle projection display control method are controlled. The system switches between the vehicle projection display control method and the control method that uses low-pass filtering compensation to generate a compensated projection image for projection.

13. The vehicle projection image processing control method as described in claim 12, characterized in that, In response to the real-time vibration frequency of the vehicle meeting the first preset condition, the first anti-shake projection control mode is executed; In response to the vehicle's real-time vibration frequency not meeting the first preset condition, the second anti-shake projection control mode is executed. The first image stabilization projection control method has a lower jitter suppression rate than the second image stabilization projection control method. The first image stabilization projection control method includes any one of the following control methods: Reduce the update frequency and / or amplitude of the image geometric transformation parameters in the vehicle projection display control method; A control method for projection that uses low-pass filtering compensation to generate a compensated projection image; The second image stabilization projection control method includes any one of the following control methods: Increase the update frequency and / or amplitude of the image geometric transformation parameters in the vehicle projection display control method; The vehicle projection display control method described above is adopted.

14. The vehicle projection image processing control method as described in claim 13, characterized in that, The first preset condition is: The real-time vibration frequency of the vehicle is greater than a first preset frequency threshold; or The proportion of frequencies exceeding a preset frequency threshold in the vibration frequency spectrum formed by the real-time vibration frequency of the vehicle is greater than the first proportional threshold.

15. The vehicle projection image processing control method as described in claim 12, characterized in that, In response to the road condition information indicating that the road is bumpy, the first anti-shake projection control mode is executed; In response to the road condition information indicating that the road is flat, the second anti-shake projection control mode is executed. The first image stabilization projection control method has a lower jitter suppression rate than the second image stabilization projection control method. The first image stabilization projection control method includes any one of the following control methods: Reduce the update frequency and / or amplitude of the image geometric transformation parameters in the vehicle projection display control method; A control method for projection that uses low-pass filtering compensation to generate a compensated projection image; The second image stabilization projection control method includes any one of the following control methods: Increase the update frequency and / or amplitude of the image geometric transformation parameters in the vehicle projection display control method; The vehicle projection display control method described above is adopted.

16. The vehicle projection image processing control method as described in claim 15, characterized in that, The road condition information of the current driving surface is determined based on one or more of the vehicle's real-time vibration frequency, visual information, and radar information.

17. The vehicle projection image processing control method according to any one of claims 13 to 16, characterized in that, The reduction of the update frequency and / or amplitude of the image geometric transformation parameters of the vehicle projection display control method includes: making the update frequency and / or amplitude of the image geometric transformation parameters of the vehicle projection display control method inversely correlated with the real-time vehicle speed and / or the real-time vibration frequency of the vehicle.

18. The vehicle projection image processing control method according to any one of claims 13 to 16, characterized in that, Increasing the update frequency and / or amplitude of the image geometric transformation parameters of the vehicle projection display control method includes making the update frequency and / or amplitude of the image geometric transformation parameters of the vehicle projection display control method positively correlated with the real-time vehicle speed and / or the real-time vibration frequency of the vehicle.

19. A vehicle projection display control device, characterized in that, Control is performed using the control method according to any one of claims 1-10, comprising: The data processing module is used to determine, at least based on real-time collected vehicle motion data, the expected relative pose change of the vehicle relative to the target road surface at the projection time; and to calculate the image geometric transformation parameters at the projection time based on the expected relative pose change, the image geometric transformation parameters being used to offset the projection offset caused by vehicle motion. The image processing module is used to perform an inverse geometric transformation on the original projected image according to the image geometric transformation parameters, and generate a compensated projected image at the projection time. A projection module is used to project the compensated projection image onto the target road surface.

20. A car, characterized in that, Includes the vehicle projection display control device as described in claim 19.

21. A projection device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores program instructions executable by the processor. When the projection device is running, the processor communicates with the memory via the bus, and the processor executes the program instructions to perform the steps of the control method as described in any one of claims 1 to 10; and / or the steps of the control method as described in any one of claims 11 to 18.

22. A readable storage medium, characterized in that, The readable storage medium stores a processing program that, when executed by a processor, implements the steps of the control method according to any one of claims 1-10; and / or the steps of the control method according to any one of claims 11 to 18.

Images