Method and apparatus for adjusting image of vehicle-mounted head-up display, and device and medium

Abstract

A method and apparatus for adjusting an image of a vehicle-mounted head-up display (HUD), and a device and a medium. The method comprises: determining an attitude transformation matrix of a current vehicle coordinate system relative to a world coordinate system at adjacent moments (S110); on the basis of the attitude transformation matrix, correcting a first position transformation matrix between a vehicle-mounted first camera coordinate system and the vehicle coordinate system relative to the world coordinate system (S120); determining a visual transformation matrix of the eyes of a driver in the vehicle coordinate system (S130); on the basis of the visual transformation matrix, correcting a second position transformation matrix between a rendering space coordinate system of a virtual image and a virtual camera coordinate system in a rendering space (S140); and on the basis of the corrected first position transformation matrix and the corrected second position transformation matrix, adjusting the position of the virtual image displayed on an HUD and the position of a target object in the virtual image (S150). The present invention solves the problem that objects in virtual images displayed on HUDs fail to align with real-world objects when vehicle attitudes change.

Description

Methods, devices, equipment and media for adjusting images in vehicle head-up displays

[0001] This application claims priority to Chinese Patent Application No. 202510121896.7, filed on January 24, 2025, with the State Intellectual Property Office of the People's Republic of China, entitled "Method, Apparatus, Device and Medium for Adjusting Images of Vehicle Head-Up Display", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of automotive electronics technology, and more specifically, to a method, apparatus, device, and medium for adjusting the image of an in-vehicle head-up display. Background Technology

[0003] AR-HUD (Augmented Reality Head-Up Display) is an in-vehicle display system that overlays virtual information onto the real world and projects this information directly into the driver's field of vision. However, as the vehicle moves, such as when it passes speed bumps, potholes, rocks, or goes uphill or downhill, changes in posture are inevitable, causing the virtual image displayed on the HUD to shake.

[0004] In related technologies, the object position information obtained by ADAS (Advanced Driver Assistance System) has too high a delay. In addition, the communication and image rendering delays of the HUD software itself will cause the objects in the virtual image to fail to match the actual position if the vehicle's posture changes frequently on the road. This will affect the driver's judgment and the driving experience. Summary of the Invention

[0005] This application provides a method, apparatus, device, and medium for adjusting images in a vehicle head-up display (HUD) to solve the problem that objects in the virtual image displayed by the HUD cannot match the actual objects when the vehicle's posture changes.

[0006] In a first aspect, this application provides a method for adjusting the image of a vehicle head-up display (HUD), wherein the HUD is used to display vehicle operation information and driving assistance information through virtual images. The method provided in this application includes:

[0007] Determine the attitude transformation matrix of the current vehicle coordinate system based on the world coordinate system at adjacent time points;

[0008] Based on the attitude transformation matrix, the first position transformation matrix between the vehicle-mounted first camera coordinate system and the vehicle coordinate system in the world coordinate system is corrected to obtain the corrected first position transformation matrix. The first position transformation matrix is ​​used to transform the position information of the target object in the vehicle-mounted first camera coordinate system to the vehicle coordinate system.

[0009] Based on the positional changes of the driver's gaze at adjacent moments, the visual transformation matrix of the driver's eyes in the vehicle coordinate system is determined, wherein the driver's eyes are identified by the vehicle's second camera;

[0010] Based on the visual transformation matrix, the second position transformation matrix between the rendering space coordinate system of the virtual image and the virtual camera coordinate system in the rendering space is corrected to obtain the corrected second position transformation matrix. The second position transformation matrix is ​​used to transform the position of the target object in the rendering space coordinate system to the virtual camera coordinate system. The rendering space coordinate system is consistent with the vehicle coordinate system.

[0011] Based on the corrected first position transformation matrix and the corrected second position transformation matrix, the position of the virtual image displayed by the HUD and the position of the target object in the virtual image are adjusted.

[0012] Optionally, determine the attitude transformation matrix of the current vehicle coordinate system based on the world coordinate system at adjacent time points, including:

[0013] Determine the current attitude information of the vehicle in the world coordinate system at the current moment, and obtain the historical attitude information of the vehicle in the world coordinate system at the previous moment.

[0014] Based on the current attitude information and historical attitude information, determine the attitude transformation matrix of the current vehicle coordinate system based on the world coordinate system at adjacent time points.

[0015] Optionally, determine the current attitude information of the vehicle in the world coordinate system at the current moment, including:

[0016] Based on the attitude change of the inertial measurement unit (IMU) at the current moment, the current attitude information of the vehicle in the world coordinate system at the current moment is determined.

[0017] Optionally, based on the attitude change of the inertial measurement unit (IMU) at the current moment, the current attitude information of the vehicle in the world coordinate system at the current moment is determined, including:

[0018] Based on the attitude change of the IMU at the current moment, determine the first rotation matrix of the current vehicle around its X-axis, the second rotation matrix around its Y-axis, and the third rotation matrix around its Z-axis in the world coordinate system at the current moment, wherein the X-axis, Y-axis and Z-axis are perpendicular to each other;

[0019] Multiply the third rotation matrix, the second rotation matrix, and the first rotation matrix in sequence to obtain the current attitude information of the vehicle in the world coordinate system at the current moment.

[0020] Optionally, based on the positional changes of the driver's line of sight at adjacent time points, a visual transformation matrix of the driver's eyes in the vehicle coordinate system is determined, including:

[0021] Based on the rotation matrix of the driver's gaze direction at adjacent moments and the position transformation matrix of the driver's eyes, the visual transformation matrix of the driver's eyes in the vehicle coordinate system is determined.

[0022] Optionally, based on the rotation matrix of the driver's gaze direction at adjacent moments and the position transformation matrix of the driver's eyes, a visual transformation matrix in the vehicle coordinate system is determined, including:

[0023] Based on the relative positional relationship between the vehicle-mounted second camera and the current vehicle, the positional information of the driver's eyes in the coordinate system of the vehicle-mounted second camera is transformed into the vehicle coordinate system to obtain the positional information of the driver's eyes based on the vehicle coordinate system.

[0024] The difference in the driver's eye position information at adjacent time points is used as the driver's eye position transformation matrix at adjacent time points;

[0025] Based on the rotation axis and rotation angle between the current gaze vector corresponding to the driver's gaze direction at the current moment and the historical gaze vector at the previous moment, determine the rotation matrix of the driver's gaze direction at adjacent moments;

[0026] The rotation matrix of the driver's gaze direction at adjacent moments and the position transformation matrix of the driver's eyes at adjacent moments are combined to form the visual transformation matrix of the driver's eyes in the vehicle coordinate system.

[0027] Optionally, based on the rotation axis and rotation angle between the current gaze vector corresponding to the driver's gaze direction at the current moment and the historical gaze vector at the previous moment, a rotation matrix for the driver's gaze direction at adjacent moments is determined, including:

[0028] Based on the Rodriguez formula, the rotation axis and rotation angle between the current gaze vector corresponding to the driver's gaze direction at the current moment and the historical gaze vector at the previous moment are used to form the rotation matrix of the driver's gaze direction at adjacent moments.

[0029] The rotation axis and rotation angle between the current gaze vector corresponding to the driver's gaze direction at the current moment and the historical gaze vector at the previous moment are obtained by the following formulas: i = z(t-1) × z(t);

[0030] Where z(t) represents the current gaze vector corresponding to the driver's gaze direction at the current moment, z(t-1) represents the historical gaze vector corresponding to the driver's gaze direction at the previous moment, i represents the rotation axis between the current gaze vector and the historical gaze vector at the previous moment, and δ represents the rotation angle between the current gaze vector and the historical gaze vector at the previous moment.

[0031] Optionally, based on the attitude transformation matrix, the first position transformation matrix between the vehicle-mounted first camera coordinate system and the vehicle coordinate system in the world coordinate system is corrected, including:

[0032] The first position transformation matrix between the vehicle-mounted first camera coordinate system and the vehicle coordinate system in the world coordinate system is corrected according to the following formula:

[0033] Among them, M′ M M represents the corrected first position transformation matrix. M This represents the first position transformation matrix before correction. This represents the attitude transformation matrix of the current vehicle coordinate system based on the world coordinate system at adjacent moments;

[0034] Based on the visual transformation matrix, the second position transformation matrix between the rendering space coordinate system of the virtual image and the virtual camera coordinate system in the rendering space is corrected, including:

[0035] The second position transformation matrix between the rendering space coordinate system of the virtual image and the virtual camera coordinate system in the rendering space is corrected according to the following formula:

[0036] Among them, M′ V M represents the corrected second position transformation matrix. V This represents the second position transformation matrix before correction. This represents the visual transformation matrix of the driver's eyes in the vehicle coordinate system at adjacent moments.

[0037] Secondly, embodiments of this application also provide a device for adjusting the image of a vehicle head-up display (HUD), wherein the HUD is used to display vehicle operation information and driving assistance information through virtual images. The device provided in this application includes:

[0038] The attitude transformation matrix determination module is configured to determine the attitude transformation matrix of the current vehicle coordinate system based on the world coordinate system at adjacent time points;

[0039] The first correction module is configured to correct the first position transformation matrix between the vehicle-mounted first camera coordinate system and the vehicle coordinate system in the world coordinate system according to the attitude transformation matrix, so as to obtain the corrected first position transformation matrix. The first position transformation matrix is ​​used to transform the position information of the target object in the vehicle-mounted first camera coordinate system to the vehicle coordinate system.

[0040] The visual transformation matrix determination module is configured to determine the visual transformation matrix of the driver's eyes in the vehicle coordinate system based on the positional changes of the driver's gaze at adjacent time points, wherein the driver's eyes are identified by the vehicle-mounted second camera;

[0041] The second correction module is configured to correct the second position transformation matrix between the rendering space coordinate system of the virtual image and the virtual camera coordinate system in the rendering space according to the visual transformation matrix, so as to obtain the corrected second position transformation matrix. The second position transformation matrix is ​​used to transform the position of the target object in the rendering space coordinate system to the virtual camera coordinate system. The rendering space coordinate system is consistent with the vehicle coordinate system.

[0042] The image position adjustment module is configured to adjust the position of the virtual image displayed by the HUD and the position of the target object in the virtual image based on the modified first position transformation matrix and the modified second position transformation matrix.

[0043] Optional, the attitude transformation matrix determination module includes:

[0044] The attitude information determination unit is configured to determine the current attitude information of the current vehicle in the world coordinate system at the current moment, and to obtain the historical attitude information of the current vehicle in the world coordinate system at the previous moment.

[0045] The attitude transformation matrix determination unit is configured to determine the attitude transformation matrix of the current vehicle coordinate system based on the world coordinate system at adjacent time points based on the current attitude information and the historical attitude information.

[0046] Optionally, the attitude information determination unit includes:

[0047] The attitude information determination subunit is configured to determine the current attitude information of the current vehicle in the world coordinate system based on the attitude change of the inertial measurement unit (IMU) at the current moment.

[0048] Optionally, the attitude information determination subunit is configured as follows:

[0049] Based on the attitude change of the IMU at the current moment, determine the first rotation matrix of the current vehicle around its X-axis, the second rotation matrix around its Y-axis, and the third rotation matrix around its Z-axis in the world coordinate system at the current moment, wherein the X-axis, Y-axis and Z-axis are perpendicular to each other;

[0050] Multiply the third rotation matrix, the second rotation matrix, and the first rotation matrix in sequence to obtain the current attitude information of the vehicle in the world coordinate system at the current moment.

[0051] Optional, the visual transformation matrix determination module includes:

[0052] The visual transformation matrix determination unit is configured to determine the visual transformation matrix of the driver's eye in the vehicle coordinate system based on the position change of the driver's line of sight at adjacent times, the rotation matrix of the direction the eye is looking at, and the position transformation matrix of the driver's eye.

[0053] Optionally, the visual transformation matrix determination unit includes:

[0054] The eye position information determination subunit is configured to transform the driver's eye position information in the coordinate system of the vehicle-mounted second camera to the vehicle coordinate system based on the relative positional relationship between the vehicle-mounted second camera and the current vehicle, thereby obtaining the driver's eye position information based on the vehicle coordinate system.

[0055] The eye position transformation matrix determination subunit is configured to use the difference in the position information of the driver's eyes at adjacent time points as the position transformation matrix of the driver's eyes at adjacent time points.

[0056] The gaze direction rotation matrix determination sub-unit is configured to determine the rotation matrix of the driver's gaze direction at adjacent times based on the rotation axis and rotation angle between the current gaze vector corresponding to the driver's gaze direction at the current time and the historical gaze vector at the previous time.

[0057] The visual transformation matrix forming sub-unit is configured to combine the rotation matrix of the driver's eye gaze direction at the adjacent time and the position transformation matrix of the driver's eye at the adjacent time to form the visual transformation matrix of the driver's eye in the vehicle coordinate system.

[0058] Optionally, the gaze direction rotation matrix determines the sub-unit, specifically configured as follows:

[0059] Based on the Rodriguez formula, the rotation axis and rotation angle between the current gaze vector corresponding to the driver's gaze direction at the current moment and the historical gaze vector at the previous moment are used to form the rotation matrix of the driver's gaze direction at adjacent moments.

[0060] The rotation axis and rotation angle between the current gaze vector corresponding to the driver's gaze direction at the current moment and the historical gaze vector at the previous moment are obtained by the following formulas: i = z(t-1) × z(t);

[0061] Where z(t) represents the current gaze vector corresponding to the driver's gaze direction at the current moment, z(t-1) represents the historical gaze vector corresponding to the driver's gaze direction at the previous moment, i represents the rotation axis between the current gaze vector and the historical gaze vector at the previous moment, and δ represents the rotation angle between the current gaze vector and the historical gaze vector at the previous moment.

[0062] Optionally, the first correction module is specifically configured as follows:

[0063] The first position transformation matrix between the vehicle-mounted first camera coordinate system and the vehicle coordinate system in the world coordinate system is corrected according to the following formula:

[0064] Among them, M′ M M represents the corrected first position transformation matrix. M This represents the first position transformation matrix before correction. This represents the attitude transformation matrix of the current vehicle coordinate system based on the world coordinate system at adjacent moments;

[0065] The second correction module is specifically configured as follows:

[0066] The second position transformation matrix between the rendering space coordinate system of the virtual image and the virtual camera coordinate system in the rendering space is corrected according to the following formula:

[0067] Among them, M′ V M represents the corrected second position transformation matrix. V This represents the second position transformation matrix before correction. This represents the visual transformation matrix of the driver's eyes in the vehicle coordinate system at adjacent moments.

[0068] Thirdly, embodiments of this application also provide an electronic device, including:

[0069] Memory containing executable program code;

[0070] A processor coupled to the memory;

[0071] The processor calls the executable program code stored in the memory to execute the method for adjusting the HUD image provided in any embodiment of this application.

[0072] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method for adjusting the image of a vehicle head-up display (HUD) provided in any embodiment of this application.

[0073] The technical solution provided in this application, when the vehicle posture changes, corrects the first position transformation matrix between the vehicle-mounted first camera coordinate system and the vehicle coordinate system in the world coordinate system, and the second position transformation matrix between the rendering space coordinate system and the virtual camera coordinate system in the rendering space. When drawing virtual images using the corrected first position transformation matrix, the corrected second position transformation matrix, and the projection matrix, the position of the virtual image and the position of the target object in the virtual image can be adjusted accordingly with the change of vehicle posture. That is, when the vehicle posture changes, the position of the target object in the virtual image can also be aligned with the actual position of the target object, avoiding the problem of driver dizziness caused by the target object in the virtual image not being aligned with the actual target object, and improving the driver's riding experience. Attached Figure Description

[0074] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0075] Figure 1 is a schematic diagram of the changes in the HUD screen display area;

[0076] Figure 2 is a schematic diagram of the coordinate system of each device in the vehicle system;

[0077] Figure 3 is a flowchart of a method for adjusting the image of a vehicle head-up display provided in Embodiment 1 of this application;

[0078] Figure 4 is a flowchart of the driver vision transformation matrix determination provided in Embodiment 1 of this application;

[0079] Figure 5 is a structural block diagram of an image adjustment device for a vehicle head-up display (HUD) provided in Embodiment 2 of this application;

[0080] Figure 6 is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of this application. Detailed Implementation

[0081] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0082] It should be noted that the terms "comprising" and "having," and any variations thereof, in the embodiments and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0083] This application discloses a method, apparatus, device, and medium for adjusting the image of a vehicle head-up display. To more clearly and understandably explain the contents of the various embodiments of this application, the implementation principle of this application will be briefly introduced below.

[0084] Figure 1 illustrates the change in the HUD display area. As shown in Figure 1, when a vehicle passes a rock, using the rear axle as a fulcrum, the front of the vehicle lifts by α°, which is equivalent to rotating α° along the x-axis. At this point, the vehicle's posture changes, and the driver's viewing angle of the target object changes from the lower region A to the upper region B. Due to the change in the image coordinate system transformation, the HUD virtual image cannot be adjusted to the appropriate position in time, and the target object in the virtual image cannot match the actual position of the target object. Furthermore, Figure 2 is a schematic diagram of the coordinate systems of various devices within the vehicle system. Figure 2 involves the ADAS camera coordinate system O. C X C Y C Z C Driver's head coordinate system O H X H Y H Z H DMS coordinate system O D X D Y D Z D Pixel coordinate system O2UV, Image coordinate system O2X A Y A Z A and vehicle coordinate system O b X b Y b Z bAs shown in Figure 2, when the target object changes its angle or position on the vehicle body, its coordinates in the vehicle coordinate system will also change, and the position of the human eye will also change. However, the change in the position of the human eye will not completely follow the translation and rotation of the vehicle body, but will be a random translation. This is the key to solving the anti-shake problem.

[0085] The purpose of this application is to correct the first position transformation matrix between the onboard first camera coordinate system and the vehicle coordinate system by using the attitude transformation matrix of the vehicle before and after the attitude change when the vehicle's posture changes, such as when passing over speed bumps, low-lying areas, or rocks, or when going up or down slopes. Furthermore, by determining the visual transformation matrix of the driver's eyes in the vehicle coordinate system before and after the vehicle's attitude change, the second position transformation matrix between the rendering space coordinate system and the virtual camera coordinate system in the rendering space can be corrected. When the HUD uses the corrected first and second position transformation matrices to draw virtual images, the position of the virtual image can be adjusted accordingly with the change in vehicle posture, thereby ensuring that the positions of icons and actual objects in the virtual image viewed by the driver are aligned in real time, avoiding driver dizziness and affecting the driving experience. The implementation process of this application is described in detail below.

[0086] Example 1

[0087] Figure 3 is a flowchart of a method for adjusting an image on a vehicle-mounted head-up display (HUD) according to Embodiment 1 of this application. This method can be applied to vehicle-mounted terminals such as HUDs, and can also be applied to servers; this embodiment does not limit its application. The method provided in this embodiment can be applied to application scenarios where the vehicle's posture changes, such as when the vehicle passes over speed bumps, low-lying areas, or rocks, or when the vehicle travels uphill or downhill. As shown in Figure 3, the method provided in this embodiment specifically includes:

[0088] S110. Determine the attitude transformation matrix of the current vehicle coordinate system based on the world coordinate system at adjacent time points.

[0089] In this context, adjacent moments refer to the current moment and the moment before the current moment. The world coordinate system, also known as the geodetic coordinate system, is an absolute coordinate system that does not rotate with the object. The vehicle coordinate system is a coordinate system located on the vehicle body. This coordinate system typically has the rear axle center as the origin, the vehicle's direction of travel as the X-axis, the direction perpendicular to the X-axis upwards as the Z-axis, and the direction perpendicular to both the X and Z axes as the Y-axis. This embodiment does not specifically limit the method for setting the position of the vehicle coordinate system.

[0090] In this embodiment, the attitude transformation matrix of the current vehicle coordinate system based on the world coordinate system at adjacent time points can be obtained based on the current attitude information of the current vehicle in the world coordinate system at the current time point and the historical attitude information of the current vehicle in the world coordinate system at the previous time point. The attitude information of the vehicle in the world coordinate system at each time point can be measured by an onboard IMU (Inertial Measurement Unit). The onboard IMU is installed at the origin of the vehicle coordinate system. The initial attitude of the IMU is an identity matrix. Based on the attitude change of the IMU at the current time point, the current attitude information of the current vehicle in the world coordinate system at the current time point can be determined.

[0091] Specifically, the attitude change of the IMU at the current moment can be expressed as θ = (θ x θ y θ z Of the parameters mentioned above, θ x θ represents the roll angle (Roll) of rotation about the X-axis. y θ represents the pitch angle (or pitch) around the Y-axis. z This represents the yaw angle of rotation about the Z-axis. The angle is obtained by integrating the angular velocity information measured by the IMU's angular velocity meter.

[0092] Based on the attitude change of the IMU at the current moment, the first rotation matrix M of the current vehicle around its X-axis in the world coordinate system at the current moment can be determined. x (t), the second rotation matrix M around the Y-axis y (t), and the third rotation matrix M about the Z-axis. z (t), the specific formulas for each rotation matrix are as follows:

[0093] In this embodiment, the vehicle attitude transformation matrix can be obtained by three rotations in the order of Z-axis, Y-axis, and X-axis, that is, by multiplying the third rotation matrix, the second rotation matrix, and the first rotation matrix in sequence to obtain the current attitude information of the vehicle in the world coordinate system at the current moment. The specific formula is as follows:

[0094] Where t represents time, the subscript b represents the vehicle coordinate system, and the superscript i represents the IMU coordinate system, which can be used as the world coordinate system in this embodiment.

[0095] In this way, the rotation matrix of each axis is determined independently, the attitude change of each axis is determined, the accuracy of single-axis attitude description is improved, and the third rotation matrix, the second rotation matrix and the first rotation matrix are multiplied in sequence around the Z axis, the Y axis and the X axis, calculated step by step and fused in an orderly manner, so as to obtain the current attitude information of the vehicle in real time and accurately.

[0096] Based on the current attitude information of the vehicle in the world coordinate system at the current moment And the vehicle's historical attitude information in the world coordinate system at the previous moment. The attitude transformation matrix of the current vehicle coordinate system based on the world coordinate system at adjacent time steps can be obtained. Specifically, this can be expressed by the following formula:

[0097] in, express The transpose of .

[0098] This accurately reflects the dynamic changes in vehicle attitude, improving the real-time performance and accuracy of attitude transformation between the vehicle coordinate system and the world coordinate system. By precisely calculating the attitude transformation matrix of the vehicle coordinate system based on the world coordinate system at adjacent moments, and correcting the subsequent first and second position transformation matrices accordingly, it ensures that the position of the virtual image displayed on the HUD can be adjusted in real time according to changes in vehicle attitude. This allows the icons in the virtual image viewed by the driver to accurately align with the positions of actual objects, effectively avoiding driver dizziness caused by visual misalignment, reducing driving distraction, and thus improving the overall driving experience and safety.

[0099] S120. Based on the attitude transformation matrix, the first position transformation matrix between the vehicle-mounted first camera coordinate system and the vehicle coordinate system is corrected to obtain the corrected first position transformation matrix.

[0100] The vehicle-mounted first camera is the camera of the vehicle's ADAS (Advanced Driving Assistance System). This camera is used to capture target objects outside the vehicle. These target objects can be other vehicles, obstacles, signs, or traffic markings, etc., and this embodiment does not specifically limit them.

[0101] Those skilled in the art will understand that when rendering HUD virtual images, the MVP matrix (a collective term for the Model, View, and Projection matrices) of OpenGL (Open Graphics Library) can be used to draw the image. In the MVP matrix, the M matrix transforms the object from model space to world space, that is, transforms the object from its own coordinate system to the rendering space coordinate system. In this embodiment, the rendering space coordinate system completely coincides with the vehicle coordinate system, meaning the reference for model rendering is the vehicle coordinate system. Therefore, in this embodiment, the M matrix is ​​the first position transformation matrix between the vehicle-mounted first camera coordinate system and the vehicle coordinate system, based on the world coordinate system. In practical applications, due to changes in the vehicle's posture, the position of the target object captured by the vehicle-mounted first camera also changes relatively. Therefore, to ensure that the position of the rendered object remains unchanged while keeping the virtual camera position unchanged in the rendering space, the posture transformation matrix obtained in step S110 can be used to correct the OpenGL M matrix, that is, the first position transformation matrix between the vehicle-mounted first camera coordinate system and the vehicle coordinate system in the world coordinate system in this embodiment.

[0102] Specifically, the correction process for the first position transformation matrix can be as follows:

[0103] First, the three-dimensional attitude transformation matrix Extended to a four-dimensional rotation and translation matrix This setting is to unify the dimensions with subsequent matrices. Specifically, the extension can be achieved by adding a 3D pose rotation matrix. The next column adds a one-dimensional matrix with elements of 0, due to the three-dimensional attitude rotation matrix. Since the matrix is ​​3x3, and the added column matrix is ​​3x1, we need to add a 4x4 matrix to the last row of the original matrix. The elements of this 4x4 matrix are (0001), resulting in a 4×4 rotation and translation matrix. In the three-dimensional attitude transformation matrix Extended to a four-dimensional rotation and translation matrix Then, the rotation and translation matrix is... The addition is made to the current M matrix in OpenGL rendering by adding the rotation and translation matrices. Multiplying this by the current M matrix yields the corrected first positional transformation matrix, which can be represented by the following formula:

[0104] in, It is a three-dimensional matrix, representing the rotation matrix between the vehicle-mounted first camera coordinate system and the vehicle coordinate system in the world coordinate system, L. a=[s,0,h] T Let represent the translation matrix from the origin of the vehicle's first camera coordinate system to the origin of the vehicle's coordinate system, where s represents the distance difference between the vehicle's rear axle center and its lateral center in the X direction, and h represents the distance difference between the vehicle's rear axle center and its lateral center in the Z direction. M′ represents the uncorrected M matrix, which is the first position transformation matrix between the vehicle-mounted first camera coordinate system and the vehicle coordinate system based on the world coordinate system before correction. M This represents the corrected first position transformation matrix.

[0105] Furthermore, the modified M-matrix can be used to transform the target object detected by the vehicle-mounted first camera into the rotated vehicle coordinate system, thereby adjusting the position of the target object and preventing the virtual image from shaking due to changes in vehicle posture. This can be achieved through the following formula:

[0106] Where P is a 3×1 matrix, representing the position information of the target object in the coordinate system of the vehicle's first camera, and P′ is a 3×1 matrix, representing the position information of the target object in the vehicle coordinate system after the posture rotation, and also representing the first position information of the target object in the rendering space coordinate system after the rotation.

[0107] Thus, based on the attitude transformation matrix of the current vehicle coordinate system relative to the world coordinate system at the next time step, the rotation matrix between the vehicle-mounted first camera coordinate system and the vehicle coordinate system in the world coordinate system, and the translation matrix from the origin of the vehicle-mounted first camera coordinate system to the origin of the vehicle coordinate system are processed to obtain the corrected first position transformation matrix. Therefore, the corrected M matrix can be used to transform the target object recognized by the vehicle-mounted first camera to the rotated vehicle coordinate system (rendering space coordinate system), ensuring the accuracy of the target object's position information. This ensures that the position of the virtual object in the rendering space remains unchanged when the vehicle's attitude changes, effectively preventing HUD virtual image jitter caused by vehicle attitude changes and improving the stability of virtual image display.

[0108] S130. Determine the visual transformation matrix of the driver's eyes in the vehicle coordinate system based on the positional changes of the driver's line of sight at adjacent time points.

[0109] The driver's eyes can be identified and tracked by the vehicle's second camera in the DMS (Driver Monitoring System). This DMS can also detect changes in the driver's gaze direction. In this embodiment, the positional change of the driver's gaze can be represented by a rotation matrix of the driver's gaze direction and a positional change matrix of the driver's eyes. Thus, based on the rotation matrix of the driver's gaze direction and the positional change matrix of the driver's eyes at adjacent moments, the dynamic transformation information of the driver's eyes in the vehicle coordinate system can be accurately obtained, improving the accuracy of the analysis of the driver's visual state.

[0110] Those skilled in the art will understand that, in rendering HUD virtual images, the V matrix in the MVP matrix transforms coordinates from world space to visual space, that is, transforms the object's coordinates from the rendering space coordinate system to the camera coordinate system within the rendering space. In this embodiment, when the vehicle's posture changes, the driver's eye position relative to the vehicle coordinate system changes. To ensure that the position of the virtual image seen by the driver does not change, the camera position in the rendering space also needs to be adjusted accordingly, i.e., the V matrix in the OpenGL rendering pipeline needs to be adjusted. In this embodiment, the V matrix can be adjusted based on the driver's visual transformation matrix.

[0111] In this embodiment, as shown in Figure 4, the visual transformation matrix of the driver's eyes can be determined through the following steps S131-S134:

[0112] S131. Based on the relative positional relationship between the vehicle-mounted second camera and the current vehicle, the positional information of the driver's eyes in the coordinate system of the vehicle-mounted second camera is transformed into the vehicle coordinate system to obtain the positional information of the driver's eyes based on the vehicle coordinate system.

[0113] Among them, the positional relationship between the vehicle-mounted second camera and the vehicle is fixed, that is, the positional transformation relationship between the coordinate system of the vehicle-mounted second camera and the coordinate system of the vehicle is fixed.

[0114] In this embodiment, step S131 can be implemented using the following formula:

[0115] in, This indicates the position information of the driver's eyes in the coordinate system of the vehicle's second camera. This indicates the position information of the driver's eyes in the vehicle coordinate system. This represents the rotation matrix from the coordinate system of the second onboard camera to the vehicle coordinate system. This represents the translation matrix from the coordinate system of the second onboard camera to the vehicle coordinate system.

[0116] S132. Use the difference in the driver's eye position information at adjacent time points as the driver's eye position transformation matrix at adjacent time points.

[0117] The position transformation matrix of the driver's eyes can be represented by the following formula:

[0118] [Corrected according to Rule 91, 23.01.2026] Where t represents time. This indicates the current position of the driver's eyes in the vehicle coordinate system. This indicates the position information of the driver's eyes in the vehicle coordinate system at the previous moment. This represents the matrix showing the change in the driver's eye position at adjacent moments.

[0119] Thus, by transforming the eye position from the camera coordinate system to the vehicle coordinate system, coordinate system unification is achieved, improving the consistency and accuracy of eye position information. Using the difference in eye position between adjacent moments as the position transformation matrix, the dynamic changes in eye position can be captured in real time, adapting to driver head movement scenarios and improving the real-time performance of position change description. The rotation matrix is ​​determined by the rotation axis and angle of the current and historical gaze vectors, accurately describing the rotational changes in the gaze direction. Combining the rotation matrix and the position transformation matrix to form the visual transformation matrix comprehensively reflects the dynamic changes in eye position and gaze direction, improving the comprehensiveness and accuracy of the description of the driver's eye state changes between adjacent moments.

[0120] S133. Based on the rotation axis and rotation angle between the current gaze vector corresponding to the driver's gaze direction at the current moment and the historical gaze vector at the previous moment, determine the rotation matrix of the driver's gaze direction at adjacent moments.

[0121] In this embodiment, the DMS can detect the driver's gaze direction and determine the unit vector of the driver's gaze direction at the current moment as z(t) and the unit vector of the driver's gaze direction at the previous moment as z(t-1). The rotation axis of the gaze vectors corresponding to the driver's gaze direction at these two adjacent moments can be obtained by cross product of the two unit vectors, i.e.: i = z(t-1) × z(t).

[0122] Furthermore, by multiplying the two unit vectors mentioned above, the rotation angle δ between them can be obtained, as shown in the following formula:

[0123] According to Rodriguez's formula, the above rotation axis and rotation angle can be transformed into a rotation matrix.

[0124] Thus, by combining the Rodriguez formula with vector data from adjacent time points, a rotation matrix can be quickly generated, which meets the requirements of real-time monitoring of the driver's gaze direction and is conducive to accurately quantifying the rotation process of the gaze direction at adjacent time points.

[0125] S134. Combine the rotation matrix of the direction the driver's eyes are looking at with the position transformation matrix of the driver's eyes to form the visual transformation matrix of the driver's eyes in the vehicle coordinate system.

[0126] The rotation matrix obtained in step S134 Combining the position transformation matrix obtained in step S132, we can obtain the visual transformation matrix of the driver's eyes in the vehicle coordinate system, i.e., the homogeneous translation and rotation matrix. The rotation matrix obtained in step S134 The purpose of combining the position transformation matrix obtained in step S132 is to transform the three-dimensional rotation matrix. When combined with a one-dimensional position transformation matrix, it forms a four-dimensional matrix that includes both rotation and translation. The principle of this combination is the same as that used in the three-dimensional attitude transformation matrix described above. Extended to a four-dimensional rotation and translation matrix The method is similar, specifically involving rotating a 3x3 dimensional matrix. Add a three-row, one-column position transformation matrix to the next column, and then add a four-row, four-column matrix (0001) to the last row of the matrix to obtain a 4×4 dimensional visual transformation matrix. Specifically, this can be expressed by the following formula:

[0127] in, This represents the matrix showing the change in the driver's eye position at adjacent moments.

[0128] S140. Based on the visual transformation matrix, the second position transformation matrix between the rendering space coordinate system of the virtual image and the virtual camera coordinate system in the rendering space is corrected to obtain the corrected second position transformation matrix.

[0129] In this embodiment, the position transformation matrix between the rendering space coordinate system of the virtual image and the virtual camera coordinate system in the rendering space is corrected based on the visual transformation matrix. Specifically, the V matrix in OpenGL is modified. With the current second position transformation matrix M V Multiply M V ,Right now The corrected second position transformation matrix is ​​obtained. Thus, by correcting the second position transformation matrix, the position of the virtual camera coordinate system in the rendering space changes with the vehicle's posture. Using the corrected second position transformation matrix, the pose information of the object in the rendering space coordinate system can be transformed into the corrected virtual camera coordinate system, facilitating subsequent adjustments to the virtual image position. This ensures that the position of the virtual image seen by the driver is adjusted accordingly when the vehicle's posture changes.

[0130] S150. Based on the corrected first position transformation matrix and the corrected second position transformation matrix, adjust the position of the virtual image displayed by the HUD and the position of the target object in the virtual image.

[0131] Those skilled in the art will understand that when drawing a virtual image in rendering space using OpenGL's MVP matrix, the target object's position is transformed to the vehicle coordinate system (i.e., the rendering space coordinate system) using a modified first position matrix (modified M matrix). Then, a modified second position matrix (modified V matrix) is used to transform the target object from the rendering space coordinate system to the virtual camera coordinate system within the rendering space. Finally, the target object in the virtual camera coordinate system is projected onto the HUD's 2D screen using a projection matrix. In this embodiment, when the vehicle's posture changes, the position of the target object in the virtual image seen by the driver can be adjusted by modifying the first position matrix, and the position of the virtual image seen by the driver can be adjusted by modifying the second position matrix. This ensures that the position of the virtual image seen by the driver and the position of the target object in the virtual image are adjusted accordingly when the vehicle's posture changes, solving the problem of the target object's position not matching the actual position when the vehicle's posture changes.

[0132] In this embodiment, when the vehicle posture changes, the first position transformation matrix between the vehicle-mounted first camera coordinate system and the vehicle coordinate system in the world coordinate system, and the second position transformation matrix between the rendering space coordinate system and the virtual camera coordinate system in the rendering space are corrected. When drawing the virtual image using the corrected first position transformation matrix, the corrected second position transformation matrix, and the projection matrix, the position of the virtual image and the position of the target object in the virtual image can be adjusted accordingly with the change of the vehicle posture. That is, when the vehicle posture changes, the position of the target object in the virtual image can also be aligned with the actual position of the target object, avoiding the problem of driver dizziness caused by the target object in the virtual image not being aligned with the actual target object, and improving the driver's riding experience.

[0133] Example 2

[0134] Figure 5 is a structural block diagram of an image adjustment device for a vehicle-mounted head-up display (HUD) provided in Embodiment 2 of this application. As shown in Figure 5, the device includes: a posture transformation matrix determination module 210, a first correction module 220, a visual transformation matrix determination module 230, a second correction module 240, and an image position adjustment module 250.

[0135] The attitude transformation matrix determination module 210 is configured to determine the attitude transformation matrix of the current vehicle coordinate system based on the world coordinate system at adjacent time points;

[0136] The first correction module 220 is configured to correct the first position transformation matrix between the vehicle-mounted first camera coordinate system and the vehicle coordinate system in the world coordinate system according to the attitude transformation matrix, so as to obtain the corrected first position transformation matrix. The first position transformation matrix is ​​used to transform the position information of the target object in the vehicle-mounted first camera coordinate system to the vehicle coordinate system.

[0137] The visual transformation matrix determination module 230 is configured to determine the visual transformation matrix of the driver's eyes in the vehicle coordinate system based on the rotation matrix of the direction in which the driver's eyes are looking at adjacent moments and the position transformation matrix of the driver's eyes, wherein the driver's eyes are identified by the vehicle-mounted second camera.

[0138] The second correction module 240 is configured to correct the second position transformation matrix between the rendering space coordinate system of the virtual image and the virtual camera coordinate system in the rendering space according to the visual transformation matrix, so as to obtain the corrected second position transformation matrix. The second position transformation matrix is ​​used to transform the position of the target object in the rendering space coordinate system to the virtual camera coordinate system. The rendering space coordinate system is consistent with the vehicle coordinate system.

[0139] The image position adjustment module 250 is configured to adjust the position of the virtual image displayed by the HUD and the position of the target object in the virtual image based on the modified first position transformation matrix and the modified second position transformation matrix.

[0140] Optionally, the attitude transformation matrix determination module 210 includes:

[0141] The attitude information determination unit is configured to determine the current attitude information of the current vehicle in the world coordinate system at the current moment, and to obtain the historical attitude information of the current vehicle in the world coordinate system at the previous moment.

[0142] The attitude transformation matrix determination unit is configured to determine the attitude transformation matrix of the current vehicle coordinate system based on the world coordinate system at adjacent time points based on the current attitude information and the historical attitude information.

[0143] Optionally, the attitude information determination unit includes:

[0144] The attitude information determination subunit is configured to determine the current attitude information of the current vehicle in the world coordinate system based on the attitude change of the inertial measurement unit (IMU) at the current moment.

[0145] Optionally, the attitude information determination subunit is configured as follows:

[0146] Based on the attitude change of the IMU at the current moment, determine the first rotation matrix of the current vehicle around its X-axis, the second rotation matrix around its Y-axis, and the third rotation matrix around its Z-axis in the world coordinate system at the current moment, wherein the X-axis, Y-axis and Z-axis are perpendicular to each other;

[0147] Multiply the third rotation matrix, the second rotation matrix, and the first rotation matrix in sequence to obtain the current attitude information of the vehicle in the world coordinate system at the current moment.

[0148] Optionally, the visual transformation matrix determination module 230 includes:

[0149] The visual transformation matrix determination unit is configured to determine the visual transformation matrix of the driver's eye in the vehicle coordinate system based on the position change of the driver's line of sight at adjacent times, the rotation matrix of the direction the eye is looking at, and the position transformation matrix of the driver's eye.

[0150] Optionally, the visual transformation matrix determination unit includes:

[0151] The eye position information determination subunit is configured to transform the driver's eye position information in the coordinate system of the vehicle-mounted second camera to the vehicle coordinate system based on the relative positional relationship between the vehicle-mounted second camera and the current vehicle, thereby obtaining the driver's eye position information based on the vehicle coordinate system.

[0152] The eye position transformation matrix determination subunit is configured to use the difference in the position information of the driver's eyes at adjacent time points as the position transformation matrix of the driver's eyes at adjacent time points.

[0153] The gaze direction rotation matrix determination sub-unit is configured to determine the rotation matrix of the driver's gaze direction at adjacent times based on the rotation axis and rotation angle between the current gaze vector corresponding to the driver's gaze direction at the current time and the historical gaze vector at the previous time.

[0154] The visual transformation matrix forming sub-unit is configured to combine the rotation matrix of the driver's eye gaze direction at the adjacent time and the position transformation matrix of the driver's eye at the adjacent time to form the visual transformation matrix of the driver's eye in the vehicle coordinate system.

[0155] Optionally, the gaze direction rotation matrix determines the sub-unit, specifically configured as follows:

[0156] Based on the Rodriguez formula, the rotation axis and rotation angle between the current gaze vector corresponding to the driver's gaze direction at the current moment and the historical gaze vector at the previous moment are used to form the rotation matrix of the driver's gaze direction at adjacent moments.

[0157] The rotation axis and rotation angle between the current gaze vector corresponding to the driver's gaze direction at the current moment and the historical gaze vector at the previous moment are obtained by the following formulas: i = z(t-1) × z(t);

[0158] Where z(t) represents the current gaze vector corresponding to the driver's gaze direction at the current moment, z(t-1) represents the historical gaze vector corresponding to the driver's gaze direction at the previous moment, i represents the rotation axis between the current gaze vector and the historical gaze vector at the previous moment, and δ represents the rotation angle between the current gaze vector and the historical gaze vector at the previous moment.

[0159] Optionally, the first correction module 220 is specifically configured as follows:

[0160] The first position transformation matrix between the vehicle-mounted first camera coordinate system and the vehicle coordinate system in the world coordinate system is corrected according to the following formula:

[0161] Among them, M′ M M represents the corrected first position transformation matrix. M This represents the first position transformation matrix before correction. This represents the attitude transformation matrix of the current vehicle coordinate system based on the world coordinate system at adjacent moments;

[0162] The second correction module 240 is specifically configured as follows:

[0163] The second position transformation matrix between the rendering space coordinate system of the virtual image and the virtual camera coordinate system in the rendering space is corrected according to the following formula:

[0164] Among them, M′ V M represents the corrected second position transformation matrix. V This represents the second position transformation matrix before correction. This represents the visual transformation matrix of the driver's eyes in the vehicle coordinate system at adjacent moments.

[0165] Example 3

[0166] Please refer to Figure 6, which is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of this application. As shown in Figure 6, the electronic device may include:

[0167] Memory 701 storing executable program code;

[0168] Processor 702 coupled to memory 701;

[0169] The processor 702 calls the executable program code stored in the memory 701 to execute the method for adjusting the HUD image provided in any embodiment of this application.

[0170] This application discloses a computer-readable storage medium storing a computer program that causes a computer to execute the method for adjusting the image of a vehicle head-up display (HUD) provided in any embodiment of this application.

[0171] In the various embodiments of this application, it should be understood that the sequence number of each process does not necessarily imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0172] In the embodiments provided in this application, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined based on A. However, it should also be understood that determining B based on A does not mean determining B solely based on A; B can also be determined based on A and / or other information.

[0173] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0174] If the aforementioned integrated units are implemented as software functional units and sold or used as independent products, they can be stored in a computer-accessible memory. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several requests to cause a computer device (which can be a personal computer, server, or network device, specifically a processor in the computer device) to execute some or all of the steps of the methods described in the various embodiments of this application.

[0175] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0176] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of one embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing this application.

[0177] Those skilled in the art will understand that the modules in the apparatus of the embodiments can be distributed in the apparatus of the embodiments as described in the embodiments, or they can be located in one or more devices different from this embodiment with corresponding changes. The modules of the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.

[0178] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for adjusting the image of a vehicle-mounted head-up display (HUD), wherein, The HUD is used to display vehicle operation information and driving assistance information through virtual images, characterized in that the method includes: Determine the attitude transformation matrix of the current vehicle coordinate system based on the world coordinate system at adjacent time points; Based on the attitude transformation matrix, the first position transformation matrix between the vehicle-mounted first camera coordinate system and the vehicle coordinate system in the world coordinate system is corrected to obtain the corrected first position transformation matrix. The first position transformation matrix is ​​used to transform the position information of the target object in the vehicle-mounted first camera coordinate system to the vehicle coordinate system. Based on the positional changes of the driver's line of sight at adjacent moments, a visual transformation matrix of the driver's eyes in the vehicle coordinate system is determined, wherein the driver's eyes are identified by the vehicle-mounted second camera; Based on the visual transformation matrix, the second position transformation matrix between the rendering space coordinate system of the virtual image and the virtual camera coordinate system in the rendering space is modified to obtain the modified second position transformation matrix. The second position transformation matrix is ​​used to transform the position of the target object in the rendering space coordinate system to the virtual camera coordinate system. The rendering space coordinate system is consistent with the vehicle coordinate system. Based on the modified first position transformation matrix and the modified second position transformation matrix, the position of the virtual image rendered by the HUD and the position of the target object in the virtual image are adjusted.

2. The method according to claim 1, characterized in that, The determination of the attitude transformation matrix of the current vehicle coordinate system based on the world coordinate system at adjacent time moments includes: Determine the current attitude information of the vehicle in the world coordinate system at the current moment, and obtain the historical attitude information of the vehicle in the world coordinate system at the previous moment. Based on the current attitude information and the historical attitude information, determine the attitude transformation matrix of the current vehicle coordinate system based on the world coordinate system at adjacent time points.

3. The method according to claim 2, characterized in that, Determining the current attitude information of the vehicle in the world coordinate system at the current moment includes: Based on the attitude change of the inertial measurement unit (IMU) at the current moment, determine the first rotation matrix of the current vehicle around its X-axis, the second rotation matrix around its Y-axis, and the third rotation matrix around its Z-axis in the world coordinate system at the current moment, wherein the X-axis, Y-axis, and Z-axis are perpendicular to each other; Multiply the third rotation matrix, the second rotation matrix, and the first rotation matrix in sequence to obtain the current attitude information of the vehicle in the world coordinate system at the current moment.

4. The method according to any one of claims 1-3, characterized in that, The step of determining the visual transformation matrix of the driver's eyes in the vehicle coordinate system based on the positional changes of the driver's line of sight at adjacent moments includes: Based on the rotation matrix of the driver's gaze direction at adjacent moments and the position transformation matrix of the driver's eyes, the visual transformation matrix of the driver's eyes in the vehicle coordinate system is determined.

5. The method according to claim 4, characterized in that, The step of determining the visual transformation matrix of the driver's eyes in the vehicle coordinate system based on the rotation matrix of the direction the driver's eyes are looking at at adjacent moments and the position transformation matrix of the driver's eyes includes: Based on the relative positional relationship between the vehicle-mounted second camera and the current vehicle, the positional information of the driver's eyes in the coordinate system of the vehicle-mounted second camera is transformed into the vehicle coordinate system to obtain the positional information of the driver's eyes based on the vehicle coordinate system. The difference in the driver's eye position information at adjacent time points is used as the driver's eye position transformation matrix at adjacent time points; Based on the rotation axis and rotation angle between the current gaze vector corresponding to the driver's gaze direction at the current moment and the historical gaze vector at the previous moment, determine the rotation matrix of the driver's gaze direction at adjacent moments; The rotation matrix of the driver's gaze direction at adjacent moments and the position transformation matrix of the driver's eyes at adjacent moments are combined to form the visual transformation matrix of the driver's eyes in the vehicle coordinate system.

6. The method according to claim 5, characterized in that, The step of determining the rotation matrix of the driver's gaze direction at adjacent moments based on the rotation axis and rotation angle between the current gaze vector corresponding to the driver's gaze direction at the current moment and the historical gaze vector at the previous moment includes: Based on the Rodriguez formula, the rotation axis and rotation angle between the current gaze vector corresponding to the driver's gaze direction at the current moment and the historical gaze vector at the previous moment are used to form the rotation matrix of the driver's gaze direction at adjacent moments. The rotation axis and rotation angle between the current gaze vector corresponding to the driver's gaze direction at the current moment and the historical gaze vector at the previous moment are obtained by the following formulas: i=z(t-1)×z(t); Where z(t) represents the current gaze vector corresponding to the driver's gaze direction at the current moment, z(t-1) represents the historical gaze vector corresponding to the driver's gaze direction at the previous moment, i represents the rotation axis between the current gaze vector and the historical gaze vector at the previous moment, and δ represents the rotation angle between the current gaze vector and the historical gaze vector at the previous moment.

7. The method according to any one of claims 1-6, characterized in that, The step of correcting the first position transformation matrix between the vehicle-mounted first camera coordinate system and the vehicle coordinate system in the world coordinate system according to the attitude transformation matrix includes: The first position transformation matrix between the vehicle-mounted first camera coordinate system and the vehicle coordinate system in the world coordinate system is corrected according to the following formula: Among them, M′ M M represents the corrected first position transformation matrix. M This represents the first position transformation matrix before correction. This represents the attitude transformation matrix of the current vehicle coordinate system based on the world coordinate system at adjacent moments; The step of correcting the second position transformation matrix between the rendering space coordinate system of the virtual image and the virtual camera coordinate system in the rendering space according to the visual transformation matrix includes: The second position transformation matrix between the rendering space coordinate system of the virtual image and the virtual camera coordinate system in the rendering space is corrected according to the following formula: Among them, M′ V M represents the corrected second position transformation matrix. V This represents the second position transformation matrix before correction. This represents the visual transformation matrix of the driver's eyes in the vehicle coordinate system at adjacent moments.

8. A device for adjusting the image of a vehicle-mounted head-up display (HUD), wherein, The HUD is used to display vehicle operation information and driving assistance information through virtual images, characterized in that the device includes: The attitude transformation matrix determination module is configured to determine the attitude transformation matrix of the current vehicle coordinate system based on the world coordinate system at adjacent time points; The first correction module is configured to correct the first position transformation matrix between the vehicle-mounted first camera coordinate system and the vehicle coordinate system in the world coordinate system according to the attitude transformation matrix, so as to obtain the corrected first position transformation matrix. The first position transformation matrix is ​​used to transform the position information of the target object in the vehicle-mounted first camera coordinate system to the vehicle coordinate system. The visual transformation matrix determination module is configured to determine the visual transformation matrix of the driver's eyes in the vehicle coordinate system based on the rotation matrix of the direction in which the driver's eyes are looking at adjacent moments and the position transformation matrix of the driver's eyes, wherein the driver's eyes are identified by the vehicle-mounted second camera. The second correction module is configured to correct the second position transformation matrix between the rendering space coordinate system of the virtual image and the virtual camera coordinate system in the rendering space according to the visual transformation matrix, so as to obtain the corrected second position transformation matrix. The second position transformation matrix is ​​used to transform the position of the target object in the rendering space coordinate system to the virtual camera coordinate system. The rendering space coordinate system is consistent with the vehicle coordinate system. The image position adjustment module is configured to adjust the position of the virtual image displayed by the HUD and the position of the target object in the virtual image based on the modified first position transformation matrix and the modified second position transformation matrix.

9. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method for adjusting the image of the vehicle head-up display (HUD) as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the method for adjusting the image of the vehicle head-up display (HUD) as described in any one of claims 1-7.

Images