Method, device and equipment for adjusting head-up display image of vehicle and medium

Abstract

The embodiment of the application discloses a kind of adjustment method, device, equipment and medium of vehicle head-up display image, the method comprises: determining adjacent time current vehicle coordinate system based on the pose conversion matrix of world coordinate system;According to the first position transformation matrix between the first camera coordinate system of world coordinate system and vehicle coordinate system of vehicle head-up display, it is corrected;Determine the visual transformation matrix of driver's eyes in vehicle coordinate system;According to the second position transformation matrix between the rendering space coordinate system of virtual image and the virtual camera coordinate system in rendering space, it is corrected;Based on the first position transformation matrix after correction and the second position transformation matrix after correction, the position of virtual image displayed by HUD and the position of target object in virtual image are adjusted.By using the above technical solution, the problem that the object in the virtual image displayed by HUD cannot be attached to the actual object when the attitude of the vehicle changes is solved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of automotive electronics, in particular, to a method and device for adjusting images of a head-up display (HUD) in a vehicle, and a medium. BACKGROUND

[0002] AR-HUD (Augmented Reality Head Up Display) is a kind of vehicle display system, which can superimpose virtual information in the real world and display these information directly in the driver's field of view through projection equipment. However, due to the change of the vehicle's posture during driving, such as passing through a speed bump, a low-lying area, or a stone, or going uphill or downhill, the virtual image displayed by the HUD will inevitably shake.

[0003] In related technologies, the object position information obtained by the ADAS (Advanced Driver Assistance System) has a high delay, and the communication and image drawing of the HUD software itself also have a delay. If the vehicle's posture changes frequently on the road, the objects in the virtual image will not be able to match the actual objects, which will affect the driver's judgment and driving experience. SUMMARY

[0004] Embodiments of the present application provide a method and device for adjusting images of a head-up display (HUD) in a vehicle, to solve the problem that the objects in the virtual image displayed by the HUD cannot match the actual objects when the vehicle's posture changes.

[0005] In a first aspect, the present application provides a method for adjusting images of a head-up display (HUD) in a vehicle, wherein the HUD is used to display vehicle operation information and driving assistance information through a virtual image. The method provided by embodiments of the present application comprises the following steps:

[0006] determining a posture conversion matrix of a current vehicle coordinate system based on a world coordinate system at a previous time point;

[0007] correcting a first position conversion matrix between a first camera coordinate system in the vehicle and the vehicle coordinate system in the world coordinate system according to the posture conversion matrix, to obtain a corrected first position conversion matrix, wherein the first position conversion matrix is used to convert position information of a target object in the first camera coordinate system in the vehicle to the vehicle coordinate system;

[0008] determining a visual conversion matrix of the driver's eyes in the vehicle coordinate system according to a position conversion relationship of the driver's line of sight at the previous time point, wherein the driver's eyes are recognized by a second camera in the vehicle;

[0009] According to the visual transformation matrix, a second position transformation matrix between a rendering space coordinate system of the virtual image and a virtual camera coordinate system in the rendering space is corrected to obtain a corrected second position transformation matrix, wherein the second position transformation matrix is used to convert a position of a target object in the rendering space coordinate system to the virtual camera coordinate system, and the rendering space coordinate system is consistent with the vehicle coordinate system;

[0010] Based on the corrected first position transformation matrix and the corrected second position transformation matrix, the position of the virtual image displayed by the HUD and the position of the target object in the virtual image are adjusted.

[0011] Optionally, the determination of the posture transformation matrix of the current vehicle coordinate system based on the world coordinate system at the adjacent time point comprises:

[0012] The current posture information of the current vehicle in the world coordinate system at the current time point is determined, and historical posture information of the current vehicle in the world coordinate system at a previous time point of the current time point is obtained;

[0013] According to the current posture information and the historical posture information, the posture transformation matrix of the current vehicle coordinate system based on the world coordinate system at the adjacent time point is determined.

[0014] Optionally, the determination of the current posture information of the current vehicle in the world coordinate system at the current time point comprises:

[0015] Based on the attitude change amount of the inertial measurement unit (IMU) at the current time point, the current posture information of the current vehicle in the world coordinate system at the current time point is determined.

[0016] Optionally, the determination of the current posture information of the current vehicle in the world coordinate system at the current time point based on the attitude change amount of the inertial measurement unit (IMU) at the current time point comprises:

[0017] Based on the attitude change amount of the IMU at the current time point, a first rotation matrix of the current vehicle in the world coordinate system around an X-axis, a second rotation matrix around a Y-axis, and a third rotation matrix around a Z-axis are determined, wherein the X-axis, the Y-axis and the Z-axis are perpendicular to each other;

[0018] The third rotation matrix, the second rotation matrix and the first rotation matrix are multiplied in sequence to obtain the current posture information of the current vehicle in the world coordinate system at the current time point.

[0019] Optionally, the determination of the visual transformation matrix of the driver's eyes in the vehicle coordinate system according to the position transformation relationship of the driver's line of sight at the adjacent time point comprises:

[0020] According to the rotation matrix of the direction of the driver's eyes at the adjacent time point and the position transformation matrix of the driver's eyes, the visual transformation matrix of the driver's eyes in the vehicle coordinate system is determined.

[0021] Optionally, the visual transformation matrix of the driver's eyes in the vehicle coordinate system is determined according to the rotation matrix of the direction of the driver's eyes at the adjacent time and the position transformation matrix of the driver's eyes, and includes:

[0022] The position information of the driver's eyes in the vehicle coordinate system is obtained by converting the position information of the driver's eyes in the second vehicle-mounted camera coordinate system to the vehicle coordinate system based on the relative position relationship between the second vehicle-mounted camera and the current vehicle.

[0023] The difference value of the position information of the driver's eyes at the adjacent time is used as the position transformation matrix of the driver's eyes at the adjacent time.

[0024] The rotation matrix of the direction of the driver's eyes at the adjacent time is determined according to the rotation axis and the rotation angle between the current gaze vector corresponding to the direction of the driver's eyes at the current time and the historical gaze vector at the previous time.

[0025] The rotation matrix of the direction of the driver's eyes at the adjacent time and the position transformation matrix of the driver's eyes at the adjacent time are combined to form the visual transformation matrix of the driver's eyes in the vehicle coordinate system.

[0026] Optionally, the rotation matrix of the direction of the driver's eyes at the adjacent time is determined according to the rotation axis and the rotation angle between the current gaze vector corresponding to the direction of the driver's eyes at the current time and the historical gaze vector at the previous time, and includes:

[0027] The rotation matrix of the direction of the driver's eyes at the adjacent time is formed based on the rotation axis and the rotation angle between the current gaze vector corresponding to the direction of the driver's eyes at the current time and the historical gaze vector at the previous time according to the Rodrigues formula.

[0028] Wherein, the rotation axis and the rotation angle between the current gaze vector corresponding to the direction of the driver's eyes at the current time and the historical gaze vector at the previous time are obtained by the following formula respectively:

[0029] i=z(t-1)×z(t)

[0030]

[0031] Wherein, z(t) represents the current gaze vector corresponding to the direction of the driver's eyes at the current time, z(t-1) represents the historical gaze vector corresponding to the direction of the driver's eyes at the previous time, i represents the rotation axis between the current gaze vector and the historical gaze vector at the previous time; δ represents the rotation angle between the current gaze vector and the historical gaze vector at the previous time.

[0032] Optionally, according to the attitude conversion matrix, a first position transformation matrix between the vehicle-mounted first camera coordinate system in the world coordinate system and the vehicle coordinate system is corrected, including:

[0033] According to the following formula, the first position transformation matrix between the vehicle-mounted first camera coordinate system in the world coordinate system and the vehicle coordinate system is corrected:

[0034]

[0035] M' = M * R M M' represents the corrected first position transformation matrix, M represents the first position transformation matrix before correction, M R represents the attitude conversion matrix of the current vehicle coordinate system based on the world coordinate system at the adjacent time;

[0036] According to the visual transformation matrix, a second position transformation matrix between the rendering space coordinate system of the virtual image and the virtual camera coordinate system in the rendering space is corrected, including:

[0037] According to the following formula, the second position transformation matrix between the rendering space coordinate system of the virtual image and the virtual camera coordinate system in the rendering space is corrected:

[0038]

[0039] M' = M * R V M' represents the corrected second position transformation matrix, M represents the second position transformation matrix before correction, V R represents the visual transformation matrix of the driver's eyes in the vehicle coordinate system at the adjacent time.

[0040] In a second aspect, the embodiment of the present application further provides an image adjustment device of a vehicle-mounted head-up display (HUD), wherein the HUD is used to display vehicle operation information and driving assistance information through a virtual image, and the device provided by the embodiment of the present application includes:

[0041] The attitude conversion matrix determination module is configured to determine the attitude conversion matrix of the current vehicle coordinate system based on the world coordinate system at the adjacent time;

[0042] The first correction module is configured to correct the first position transformation matrix between the vehicle-mounted first camera coordinate system in the world coordinate system and the vehicle coordinate system according to the attitude conversion matrix, to obtain a corrected first position transformation matrix, wherein the first position transformation matrix is used to convert position information of a target object in the vehicle-mounted first camera coordinate system to the vehicle coordinate system;

[0043] ​​The visual transformation matrix determination module is configured to determine a visual transformation matrix of the driver's eyes in the vehicle coordinate system according to a position transformation relationship of the driver's line of sight at adjacent time points, wherein the driver's eyes are identified by the vehicle-mounted second camera.

[0044] The second correction module is configured to correct a second position transformation matrix between a rendering space coordinate system of the virtual image and a virtual camera coordinate system in a rendering space according to the visual transformation matrix, to obtain a corrected second position transformation matrix, wherein the second position transformation matrix is used to convert a position of a target object in the rendering space coordinate system to the virtual camera coordinate system, and the rendering space coordinate system is consistent with the vehicle coordinate system.

[0045] The image position adjustment module is configured to adjust a position of the virtual image rendered by the HUD and a position of a target object in the virtual image based on the corrected first position transformation matrix and the corrected second position transformation matrix.

[0046] Optionally, the attitude transformation matrix determination module comprises:

[0047] The attitude information determination unit is configured to determine current attitude information of the current vehicle in the world coordinate system at the current time point, and obtain historical attitude information of the current vehicle in the world coordinate system at the previous time point at the current time point.

[0048] The attitude transformation matrix determination unit is configured to determine an attitude transformation matrix of the current vehicle coordinate system based on the world coordinate system at an adjacent time point according to the current attitude information and the historical attitude information.

[0049] Optionally, the attitude information determination unit comprises:

[0050] The attitude information determination sub-unit is configured to determine the current attitude information of the current vehicle in the world coordinate system at the current time point based on an attitude change amount of an inertial measurement unit (IMU) at the current time point.

[0051] Optionally, the attitude information determination sub-unit is specifically configured to:

[0052] determine, based on the attitude change amount of the IMU at the current time point, a first rotation matrix of the current vehicle in the world coordinate system around an X-axis, a second rotation matrix around a Y-axis, and a third rotation matrix around a Z-axis at the current time point, wherein the X-axis, the Y-axis, and the Z-axis are perpendicular to each other.

[0053] multiply the third rotation matrix, the second rotation matrix, and the first rotation matrix in sequence to obtain the current attitude information of the current vehicle in the world coordinate system at the current time point.

[0054] Optionally, the visual transformation matrix determination module comprises:

[0055] a visual transformation matrix determination unit configured to determine a visual transformation matrix of the driver's eyes in the vehicle coordinate system according to a position transformation of the driver's line of sight at the adjacent time, a rotation matrix of the direction of the driver's gaze and a position transformation matrix of the driver's eyes.

[0056] Optionally, the visual transformation matrix determination unit comprises:

[0057] an eye position information determination sub-unit configured to convert the position information of the driver's eyes in the vehicle coordinate system of the vehicle-mounted second camera to the vehicle coordinate system based on the relative position relationship between the vehicle-mounted second camera and the current vehicle, to obtain the position information of the driver's eyes in the vehicle coordinate system;

[0058] an eye position transformation matrix determination sub-unit configured to take the difference of the position information of the driver's eyes at the adjacent time as the position transformation matrix of the driver's eyes at the adjacent time;

[0059] a gaze direction rotation matrix determination sub-unit configured to determine the rotation matrix of the direction of the driver's gaze at the adjacent time according to the rotation axis and rotation angle between the current gaze vector corresponding to the direction of the driver's gaze at the current time and the historical gaze vector at the previous time;

[0060] a visual transformation matrix formation sub-unit configured to combine the rotation matrix of the direction of the driver's gaze at the adjacent time and the position transformation matrix of the driver's eyes at the adjacent time to form the visual transformation matrix of the driver's eyes in the vehicle coordinate system.

[0061] Optionally, the gaze direction rotation matrix determination sub-unit is specifically configured to:

[0062] form the rotation matrix of the direction of the driver's gaze at the adjacent time according to the rotation axis and rotation angle between the current gaze vector corresponding to the direction of the driver's gaze at the current time and the historical gaze vector at the previous time based on the Rodrigues formula;

[0063] wherein the rotation axis and rotation angle between the current gaze vector corresponding to the direction of the driver's gaze at the current time and the historical gaze vector at the previous time are obtained by the following formulas respectively:

[0064] i = z(t-1) x z(t)

[0065]

[0066] Wherein, z(t) represents a current gaze vector corresponding to the gaze direction of the driver's eyes at the current moment, z(t-1) represents a historical gaze vector corresponding to the gaze direction of the driver's eyes at the previous moment, and i represents a rotation axis between the current gaze vector and the historical gaze vector at the previous moment; and δ represents a rotation angle between the current gaze vector and the historical gaze vector at the previous moment.

[0067] Optionally, the first correction module is specifically configured to:

[0068] The first position transformation matrix between the vehicle coordinate system and the first camera coordinate system in the world coordinate system is corrected according to the following formula:

[0069]

[0070] Wherein, M' M represents the first position transformation matrix before correction, and M M represents the first position transformation matrix before correction, represents a pose transformation matrix of the current vehicle coordinate system based on the world coordinate system at the adjacent moment;

[0071] The second correction module is specifically configured to:

[0072] The second position transformation matrix between the rendering space coordinate system of the virtual image and the virtual camera coordinate system in the rendering space is corrected according to the following formula:

[0073]

[0074] Wherein, M' V represents the second position transformation matrix before correction, and M V represents the second position transformation matrix before correction, represents a visual transformation matrix of the driver's eyes in the vehicle coordinate system at the adjacent moment.

[0075] In a third aspect, an electronic device is also provided, comprising:

[0076] A memory storing executable program codes;

[0077] A processor coupled with the memory;

[0078] The processor invokes the executable program codes stored in the memory to execute the image adjustment method of the head-up display (HUD) provided by any embodiment of the present application.

[0079] In a fourth aspect, the embodiments of the present application further provide a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the image adjustment method of the vehicle head-up display (HUD) provided by any of the embodiments of the present application.

[0080] The technical scheme provided by the embodiments of the present application can make the position of the virtual image and the position of the target object in the virtual image be adjusted correspondingly following the change of the vehicle posture when the first position transformation matrix between the first camera coordinate system of the vehicle in the world coordinate system and the vehicle coordinate system and the second position transformation matrix between the rendering space coordinate system and the virtual camera coordinate system in the rendering space are corrected, and the drawing of the virtual image is performed by using the corrected first position transformation matrix, the corrected second position transformation matrix and the projection matrix, that is, when the vehicle posture changes, the position of the target object in the virtual image can also be fitted together with the actual position of the target object, thereby avoiding the problem of driver dizziness caused by the target object in the virtual image not being fitted with the actual target object, and improving the driving experience of the driver. BRIEF DESCRIPTION OF DRAWINGS

[0081] In order to more clearly illustrate the technical scheme in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0082] Figure 1 A schematic diagram for the change of the HUD picture display area;

[0083] Figure 2 A flowchart of the image adjustment method of the vehicle head-up display provided by the first embodiment of the present application;

[0084] Figure 3 A flowchart of the determination of the driver visual transformation matrix provided by the first embodiment of the present application;

[0085] Figure 4 A structure block diagram of the image adjustment device of the vehicle head-up display (HUD) provided by the second embodiment of the present application;

[0086] Figure 5 A structure schematic diagram of an electronic device provided by the third embodiment of the present application. DETAILED DESCRIPTION

[0087] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0088] It should be noted that the terms "comprising" and "having," and any variations thereof, in the embodiments and drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0089] This invention discloses a method, apparatus, device, and medium for adjusting images on a vehicle-mounted head-up display. To more clearly explain the contents of the various embodiments of this invention, the implementation principle of this invention will be briefly introduced below.

[0090] Figure 1 This is a diagram illustrating the changes in the HUD display area. (Example:) Figure 1 As shown, when a vehicle passes a rock, using the rear axle as a fulcrum, the front of the vehicle lifts by α°, which is equivalent to rotating α° along the x-axis. At this moment, the overall posture of the vehicle changes, and the driver's viewing angle of the target object changes from the lower region A to the upper region B. Due to the change in the image coordinate system transformation, the HUD virtual image cannot be adjusted to the appropriate position in time, and the target object in the virtual image cannot match the actual position of the target object.

[0091] The purpose of this invention is to correct the first position transformation matrix between the onboard first camera coordinate system and the vehicle coordinate system by using the attitude transformation matrix of the vehicle before and after the attitude change when the vehicle undergoes a change in posture, such as when passing over speed bumps, low-lying areas, or rocks, or when going uphill or downhill. Furthermore, by determining the visual transformation matrix of the driver's eyes in the vehicle coordinate system before and after the vehicle attitude change, the second position transformation matrix between the rendering space coordinate system and the virtual camera coordinate system in the rendering space can be corrected. When the HUD uses the corrected first and second position transformation matrices to draw virtual images, the position of the virtual image can be adjusted accordingly with the change in vehicle posture, thereby ensuring that the positions of icons and actual objects in the virtual image viewed by the driver are aligned in real time, avoiding dizziness and affecting the driving experience. The implementation process of this invention will be described in detail below.

[0092] Example 1

[0093] Figure 2 A flowchart of an adjustment method of a vehicle-mounted head-up display image is provided for an embodiment of the present application. The method can be applied to a vehicle-mounted terminal such as a HUD (Head Up Display) and can also be applied to a server. The present application does not limit the embodiment to this. The method provided by the present embodiment can be applied to application scenarios in which the vehicle posture changes, such as when the vehicle passes through a speed reduction belt, a low-lying area, or a stone, or when the vehicle passes through an uphill or downhill section. As shown in FIG. 1, the method provided by the present embodiment specifically includes the following steps. Figure 2

[0094] S110, determining a posture conversion matrix of a current vehicle coordinate system based on a world coordinate system at a neighboring time.

[0095] The neighboring time is the current time and the previous time of the current time. The world coordinate system is also referred to as the geodetic coordinate system, which is an absolute coordinate system that does not rotate with the object. The vehicle coordinate system is a coordinate system on the vehicle body, which is usually taken with the center of the rear axle as the origin, the vehicle driving direction as the X-axis direction, the direction perpendicular to the X-axis upward as the Z-axis direction, and the direction perpendicular to the X-axis and the Z-axis as the Y-axis direction. The present embodiment does not specifically limit the setting method of the vehicle coordinate system position.

[0096] In the present embodiment, the posture conversion matrix of the current vehicle coordinate system based on the world coordinate system at the neighboring time can be obtained according to the current posture information of the current vehicle in the world coordinate system at the current time and the historical posture information of the current vehicle in the world coordinate system at the previous time. The posture information of the vehicle in the world coordinate system at each time can be measured by a vehicle-mounted IMU (Inertial Measurement Unit). The vehicle-mounted IMU is installed at the origin position of the vehicle coordinate system. The initial posture of the IMU is a unit matrix. Based on the posture change amount of the IMU at the current time, the current posture information of the current vehicle in the world coordinate system at the current time can be determined.

[0097] Specifically, the posture change amount of the IMU at the current time can be represented as θ = (θ x , θ y , θ z ). In the above parameters, θ x represents the roll angle (Roll) of rotation around the X-axis, θ y represents the pitch angle (Pitch) of rotation around the Y-axis, and θ z represents the yaw angle (Yaw) of rotation around the Z-axis. The above angles are obtained by integrating the angular velocity information measured by the angular velocity meter of the IMU.

[0098] ​Based on the attitude change amount of the IMU at the current time, the first rotation matrix M of the current vehicle around the X axis in the world coordinate system at the current time can be determined x (t), the second rotation matrix M y (t) around the Y axis, and the third rotation matrix M z (t) around the Z axis, and the specific formula of each rotation matrix is as follows:

[0099]

[0100] In the embodiment, the transformation matrix of the vehicle attitude can be obtained through three rotations in the order of rotation around the Z axis, rotation around the Y axis, and rotation around the X axis, that is, the third rotation matrix, the second rotation matrix, and the first rotation matrix are sequentially multiplied to obtain the current attitude information of the current vehicle in the world coordinate system at the current time The specific formula is:

[0101]

[0102] Wherein, t represents the time, subscript b represents the vehicle coordinate system, superscript i represents the IMU coordinate system, and the embodiment can take it as the world coordinate system.

[0103] According to the current attitude information of the current vehicle in the world coordinate system at the current time And the historical attitude information of the vehicle in the world coordinate system at the previous time The attitude conversion matrix of the current vehicle coordinate system based on the world coordinate system at the adjacent time can be obtained The specific formula is as follows:

[0104]

[0105] Wherein, Indicates the transpose of the matrix .

[0106] S120, according to the attitude conversion matrix, the first position transformation matrix between the first vehicle-mounted camera coordinate system and the vehicle coordinate system is corrected to obtain the corrected first position transformation matrix.

[0107] Wherein, the first vehicle-mounted camera is the camera of the vehicle ADAS (Advanced Driving Assistance System, Advanced Driving Assistance System). The camera is used to collect target objects outside the vehicle. Wherein, the target object can be other vehicles, obstacles, signboards or traffic markings, etc., which are not limited in the embodiment.

[0108] Those skilled in the art can understand that in the process of realizing the drawing of the HUD virtual image, the MVP matrix (a combination of Model, View and Projection matrices) of OpenGL (Open Graphics Library) can be used to draw the image. In the MVP matrix, the role of the M matrix is to convert the object from the model space to the world space, that is, to convert the object from its own coordinate system to the rendering space coordinate system. In the embodiment, the rendering space coordinate system is completely coincident with the vehicle coordinate system, that is, the reference for model drawing is the vehicle coordinate system, that is, the M matrix in the embodiment is a first position transformation matrix between the vehicle-mounted first camera coordinate system based on the world coordinate system and the vehicle coordinate system. In actual application, due to the change of the vehicle body posture, the position of the target object photographed by the vehicle-mounted first camera also changes relatively, so in the condition that the virtual camera position in the rendering space is unchanged, in order to ensure that the position of the rendered object is unchanged, the posture conversion matrix obtained by the above step S110 can be used to modify the M matrix of OpenGL, that is, the first position transformation matrix between the vehicle-mounted first camera coordinate system under the world coordinate system and the vehicle coordinate system in the embodiment.

[0109] Specifically, the modification process of the first position transformation matrix can be:

[0110] The three-dimensional posture conversion matrix is first expanded into a four-dimensional rotation translation matrix This setting is to unify the dimensions of subsequent matrices. The specific expansion method can be: a one-dimensional matrix with elements of 0 is added to the next column of the three-dimensional posture rotation matrix Since the three-dimensional posture rotation matrix is a three-row and three-column matrix, the added column matrix is a three-row and one-column matrix, so a four-column matrix with elements of (0 0 0 1) needs to be supplemented in the last row of the matrix, obtaining a 4x4-dimensional rotation translation matrix After expanding the three-dimensional posture conversion matrix into a four-dimensional rotation translation matrix , the rotation translation matrix is added to the current M matrix in the rendering of OpenGL, and the addition process is to multiply the rotation translation matrix by the current M matrix to obtain the modified first position transformation matrix. The specific process can be represented by the following formula:

[0111]

[0112] wherein, is a three-dimensional matrix, which represents a rotation matrix between the vehicle-mounted first camera coordinate system under the world coordinate system and the vehicle coordinate system, and La = [s, 0, h] T represents a translation matrix from the origin of the first on-board camera coordinate system to the origin of the vehicle coordinate system, wherein s represents a distance difference between the rear axle center of the vehicle and the lateral center of the vehicle in the X direction, and h represents a distance difference between the rear axle center of the vehicle and the lateral center of the vehicle in the Z direction, represents the M matrix before correction, that is, the first position transformation matrix between the first on-board camera coordinate system based on the world coordinate system and the vehicle coordinate system before correction, M' M represents the first position transformation matrix after correction.

[0113] Further, the target object recognized by the first on-board camera can be converted to the rotated vehicle coordinate system by using the M matrix after correction, so as to adjust the position of the target object, to prevent the virtual image from shaking due to the change of the vehicle posture. The adjustment can be realized by the following formula:

[0114]

[0115] wherein P is a 3x1 matrix, representing the position information of the target object in the first on-board camera coordinate system, and P' is a 3x1 matrix, representing the position information of the target object in the vehicle coordinate system after the posture rotation, and also representing the first position information of the target object in the rendering space coordinate system after the rotation.

[0116] S130, determining a visual transformation matrix of the driver's eyes in the vehicle coordinate system according to the position change relationship of the driver's line of sight at adjacent time points.

[0117] wherein the driver's eyes can be recognized and tracked by the second on-board camera in the DMS (Driver Monitor System, driver monitoring system), and the DMS can also detect the change of the driver's line of sight, that is, the change of the driver's gaze direction. In the embodiment, the position transformation of the driver's line of sight can be represented by the rotation matrix of the direction in which the driver's eyes gaze and the position transformation matrix of the driver's eyes.

[0118] It can be understood by those skilled in the art that, in the realization of the drawing of the HUD virtual image, the V matrix in the MVP matrix is used to convert the coordinates in the world space to the visual space, that is, to convert the object in the rendering space coordinate system to the camera coordinate system in the rendering space. In the embodiment, when the vehicle posture changes, the position of the driver's eyes relative to the vehicle coordinate system changes. In order to ensure that the position of the virtual image seen by the driver does not change, the position of the camera in the rendering space also needs to be adjusted accordingly, that is, the V matrix in the OpenGL rendering pipeline needs to be adjusted. In the embodiment, the V matrix can be adjusted according to the visual transformation matrix of the driver's eyes.

[0119] In this embodiment, as shown in Figure 3 The determination of the visual transformation matrix of the driver's eyes can be achieved through steps S131-S134 as follows:

[0120] S131, based on the relative position relationship between the vehicle-mounted second camera and the current vehicle, the position information of the driver's eyes in the vehicle-mounted second camera coordinate system is converted to the vehicle coordinate system to obtain the position information of the driver's eyes based on the vehicle coordinate system.

[0121] Wherein, the position relationship between the vehicle-mounted second camera and the vehicle is fixed, that is, the position transformation relationship between the vehicle-mounted second camera coordinate system and the vehicle coordinate system is fixed.

[0122] In this embodiment, step S131 can be realized through the following formula:

[0123]

[0124] Wherein, represents the position information of the driver's eyes in the vehicle-mounted second camera coordinate system, represents the position information of the driver's eyes in the vehicle coordinate system, represents the rotation matrix from the vehicle-mounted second camera coordinate system to the vehicle coordinate system, represents the translation matrix from the vehicle-mounted second camera coordinate system to the vehicle coordinate system.

[0125] S132, the difference value of the position information of the driver's eyes at adjacent time is taken as the position transformation matrix of the driver's eyes at adjacent time.

[0126] Wherein, the position transformation matrix of the driver's eyes can be represented by the following formula:

[0127]

[0128] Wherein, t represents the time, represents the position information of the driver's eyes in the vehicle coordinate system at the current time, represents the position information of the driver's eyes in the vehicle coordinate system at the previous time at the current time, represents the position transformation matrix of the driver's eyes at adjacent time.

[0129] S133, according to the rotation axis and rotation angle between the current gaze vector corresponding to the gaze direction of the driver's eyes at the current time and the historical gaze vector at the previous time, the rotation matrix of the gaze direction of the driver's eyes at adjacent time is determined.

[0130] In this embodiment, the DMS can detect the gaze direction of the driver's eyes, determine the unit vector of the driver's eye gaze direction at the current time as z(t), the unit vector of the driver's eye gaze direction at the previous time as z(t-1), and the rotation axis of the gaze vectors corresponding to the driver's eye gaze directions at the two adjacent times can be obtained by cross-multiplying the two unit vectors, that is:

[0131] i = z(t-1) x z(t)

[0132] And the rotation angle δ between the two unit vectors can be obtained by dot-multiplying the two unit vectors, and the specific formula is:

[0133]

[0134] According to the Rodrigues formula, the rotation axis and the rotation angle can be converted into a rotation matrix

[0135] S134, combine the rotation matrix of the direction in which the driver's eyes are gazing and the position transformation matrix of the driver's eyes to form a visual transformation matrix of the driver's eyes in the vehicle coordinate system.

[0136] Combine the rotation matrix obtained in step S134 with the position transformation matrix obtained in step S132 to obtain a visual transformation matrix of the driver's eyes in the vehicle coordinate system, that is, a homogeneous translation rotation matrix wherein the combination of the rotation matrix obtained in step S134 and the position transformation matrix obtained in step S132 is to combine the three-dimensional rotation matrix with the one-dimensional position transformation matrix into a four-dimensional matrix that contains both rotation and translation, and the combination principle is similar to the method of extending the three-dimensional pose conversion matrix to a four-dimensional rotation translation matrix , which is to add a three-row one-column position transformation matrix to the next column of the three-row three-column rotation matrix , and then supplement a row of four-column matrix (0 0 0 1) to the last row of the matrix to obtain a 4x4 visual transformation matrix which can be represented by the following formula:

[0137]

[0138] wherein, represents the position transformation matrix of the driver's eyes at the adjacent time.

[0139] S140, correcting the second position transformation matrix between the rendering space coordinate system of the virtual image and the virtual camera coordinate system in the rendering space according to the visual transformation matrix, to obtain a corrected second position transformation matrix.

[0140] In this embodiment, the position transformation matrix between the rendering space coordinate system of the virtual image and the virtual camera coordinate system in the rendering space is corrected according to the visual transformation matrix, that is, the V matrix in OpenGL is corrected. Specifically, the visual transformation matrix is multiplied by the current second position transformation matrix M V V , that is to obtain the corrected second position transformation matrix. By correcting the second position transformation matrix, the position of the virtual camera coordinate system in the rendering space coordinate system can change with the change of the vehicle attitude. Using the corrected second position transformation matrix, the pose information of the object in the rendering space coordinate system can be converted to the corrected virtual camera coordinate system.

[0141] S150, adjusting the position of the virtual image rendered by the HUD and the position of the target object in the virtual image based on the corrected first position transformation matrix and the corrected second position transformation matrix.

[0142] Those skilled in the art can understand that when the virtual image is rendered in the rendering space through the MVP matrix of OpenGL, the target object position is converted to the vehicle coordinate system, that is, the rendering space coordinate system, through the corrected first position matrix (corrected M matrix), and then the target object is converted from the rendering space coordinate system to the virtual camera coordinate system in the rendering space through the corrected second position matrix (corrected V matrix), and then the target object in the virtual camera coordinate system is projected to the 2D screen of the HUD through the projection matrix. In this embodiment, when the vehicle attitude changes, the position of the target object in the virtual image seen by the driver can be adjusted by correcting the first position matrix, and the position of the virtual image seen by the driver can be adjusted by correcting the second position matrix, so that the position of the virtual image seen by the driver and the position of the target object in the virtual image can be adjusted accordingly when the vehicle attitude changes, solving the problem that the position of the target object does not match the actual position when the vehicle attitude changes.

[0143] ​​In the embodiment, when the vehicle posture changes, the first position transformation matrix between the vehicle-mounted first camera coordinate system in the world coordinate system and the vehicle coordinate system and the second position transformation matrix between the rendering space coordinate system and the virtual camera coordinate system in the rendering space are corrected. When the virtual image is drawn by using the corrected first position transformation matrix, the corrected second position transformation and the projection matrix, the position of the virtual image and the position of the target object in the virtual image can be adjusted according to the change of the vehicle posture, that is, when the vehicle posture changes, the position of the target object in the virtual image can be consistent with the actual position of the target object, the problem of driver dizziness caused by the inconsistency between the target object in the virtual image and the actual target object is avoided, and the driving experience of the driver is improved.

[0144] Embodiment two

[0145] Figure 4 A structural block diagram of an image adjustment device of a vehicle-mounted head-up display (HUD) provided in the second embodiment of the present application is shown in FIG. 2, which comprises a posture conversion matrix determination module 210, a first correction module 220, a visual transformation matrix determination module 230, a second correction module 240 and an image position adjustment module 250. Figure 4 The posture conversion matrix determination module 210 is configured to determine a posture conversion matrix of a current vehicle coordinate system based on a world coordinate system at an adjacent time point.

[0146] The posture conversion matrix determination module 210 is configured to determine a posture conversion matrix of a current vehicle coordinate system based on a world coordinate system at an adjacent time point.

[0147] The first correction module 220 is configured to correct a first position transformation matrix between a vehicle-mounted first camera coordinate system in the world coordinate system and the vehicle coordinate system according to the posture conversion matrix, to obtain a corrected first position transformation matrix, wherein the first position transformation matrix is used to convert the position information of a target object in the vehicle-mounted first camera coordinate system to the vehicle coordinate system.

[0148] The visual transformation matrix determination module 230 is configured to determine a visual transformation matrix of a driver's eyes in the vehicle coordinate system according to the position transformation relationship of the driver's line of sight at the adjacent time point, wherein the driver's eyes are identified by a vehicle-mounted second camera.

[0149] The second correction module 240 is configured to correct a second position transformation matrix between a rendering space coordinate system of a virtual image and a virtual camera coordinate system in the rendering space according to the visual transformation matrix, to obtain a corrected second position transformation matrix, wherein the second position transformation matrix is used to convert the position of a target object in the rendering space coordinate system to the virtual camera coordinate system, and the rendering space coordinate system is consistent with the vehicle coordinate system.

[0150] The image position adjustment module 250 is configured to adjust the position of the virtual image rendered by the HUD and the position of the target object in the virtual image based on the corrected first position transformation matrix and the corrected second position transformation matrix.

[0151] Optionally, the attitude conversion matrix determination module 210 comprises:

[0152] The attitude information determination unit is configured to determine current attitude information of the current vehicle in the world coordinate system at the current time, and obtain historical attitude information of the current vehicle in the world coordinate system at a previous time at the current time.

[0153] The attitude conversion matrix determination unit is configured to determine, according to the current attitude information and the historical attitude information, an attitude conversion matrix of the vehicle coordinate system based on the world coordinate system at the adjacent time.

[0154] Optionally, the attitude information determination unit comprises:

[0155] The attitude information determination sub-unit is configured to determine, based on an attitude change amount of the inertial measurement unit (IMU) at the current time, the current attitude information of the current vehicle in the world coordinate system at the current time.

[0156] Optionally, the attitude information determination sub-unit is specifically configured to:

[0157] Determine, based on the attitude change amount of the IMU at the current time, a first rotation matrix of the current vehicle in the world coordinate system around an X-axis, a second rotation matrix around a Y-axis, and a third rotation matrix around a Z-axis at the current time, wherein the X-axis, the Y-axis, and the Z-axis are perpendicular to each other.

[0158] Multiply the third rotation matrix, the second rotation matrix, and the first rotation matrix in sequence to obtain the current attitude information of the current vehicle in the world coordinate system at the current time.

[0159] Optionally, the visual transformation matrix determination module 230 comprises:

[0160] The visual transformation matrix determination unit is configured to determine, according to a position transformation of the driver's line of sight at the adjacent time, a rotation matrix of the direction in which the driver's eyes are gazing, and a position transformation matrix of the driver's eyes, a visual transformation matrix of the driver's eyes in the vehicle coordinate system.

[0161] Optionally, the visual transformation matrix determination unit comprises:

[0162] The eye position information determination sub-unit is configured to convert, based on a relative position relationship between the vehicle-mounted second camera and the current vehicle, position information of the driver's eyes in the vehicle-mounted second camera coordinate system to the vehicle coordinate system to obtain position information of the driver's eyes based on the vehicle coordinate system.

[0163] The eye position transformation matrix determination subunit is configured to take the difference value of the position information of the driver's eyes at adjacent time points as the position transformation matrix of the driver's eyes at adjacent time points.

[0164] The gaze direction rotation matrix determination subunit is configured to determine the rotation matrix of the gaze direction of the driver's eyes at adjacent time points according to the rotation axis and rotation angle between the current gaze vector corresponding to the gaze direction of the driver's eyes at the current time point and the historical gaze vector at the previous time point.

[0165] The visual transformation matrix formation subunit is configured to combine the rotation matrix of the gaze direction of the driver's eyes at adjacent time points and the position transformation matrix of the driver's eyes at adjacent time points to form the visual transformation matrix of the driver's eyes in the vehicle coordinate system.

[0166] Optionally, the gaze direction rotation matrix determination subunit is specifically configured to:

[0167] form the rotation matrix of the gaze direction of the driver's eyes at adjacent time points based on the Rodrigues formula according to the rotation axis and rotation angle between the current gaze vector corresponding to the gaze direction of the driver's eyes at the current time point and the historical gaze vector at the previous time point.

[0168] wherein the rotation axis and rotation angle between the current gaze vector corresponding to the gaze direction of the driver's eyes at the current time point and the historical gaze vector at the previous time point are obtained through the following formulas respectively:

[0169] i = z(t-1) x z(t)

[0170]

[0171] wherein z(t) represents the current gaze vector corresponding to the gaze direction of the driver's eyes at the current time point, z(t-1) represents the historical gaze vector corresponding to the gaze direction of the driver's eyes at the previous time point, i represents the rotation axis between the current gaze vector and the historical gaze vector at the previous time point, and δ represents the rotation angle between the current gaze vector and the historical gaze vector at the previous time point.

[0172] Optionally, the first correction module 220 is specifically configured to:

[0173] correct the first position transformation matrix between the first vehicle-mounted camera coordinate system in the world coordinate system and the vehicle coordinate system according to the following formula:

[0174]

[0175] wherein M' M represents the corrected first position transformation matrix, MM denotes the first position transformation matrix before correction, denotes a posture conversion matrix of the current vehicle coordinate system based on the world coordinate system at the adjacent time;

[0176] The second correction module 240 is specifically configured to:

[0177] The second position transformation matrix between the rendering space coordinate system of the virtual image and the virtual camera coordinate system in the rendering space is corrected according to the following formula:

[0178]

[0179] wherein M' V denotes the second position transformation matrix after correction, M V denotes the second position transformation matrix before correction, denotes a visual transformation matrix of the driver's eyes in the vehicle coordinate system at the adjacent time.

[0180] Embodiment three

[0181] Please refer to Figure 5 , Figure 5 is a structural schematic diagram of an electronic device provided in the embodiment three of the present application. As shown in the figure, Figure 5 The electronic device can include:

[0182] a memory 701 storing executable program codes;

[0183] a processor 702 coupled with the memory 701;

[0184] The processor 702 calls the executable program codes stored in the memory 701 to execute the image adjustment method of the head-up display (HUD) provided in any embodiment of the present application.

[0185] The embodiment of the present application discloses a computer readable storage medium storing a computer program, wherein the computer program makes the computer execute the image adjustment method of the head-up display (HUD) provided in any embodiment of the present application.

[0186] In various embodiments of the present application, it should be understood that the size of the serial number of each process does not mean the inevitable sequence of execution, and the execution sequence of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0187] In the embodiments provided in the present application, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that the determination of B according to A does not mean that B is determined only according to A, but B can also be determined according to A and / or other information.

[0188] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0189] When the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-accessible memory. Based on this understanding, the technical solutions of the present application, essentially or the part that contributes to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. The computer software product is stored in a memory and includes a number of steps for causing a computer device (which can be a personal computer, a server, or a network device, and specifically can be a processor in the computer device) to execute the above-mentioned methods of each embodiment of the present application.

[0190] Those skilled in the art can understand that all or part of the steps in the above-mentioned embodiments can be completed by programs instructing relevant hardware, and the programs can be stored in a computer-readable storage medium, including a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disk storage, a magnetic disk storage, a magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0191] Those skilled in the art can understand that the accompanying drawings are only schematic diagrams of one embodiment, and the modules or flows in the drawings are not necessarily required to implement the present application.

[0192] Those skilled in the art can understand that the modules in the device in the embodiments can be distributed in the device in the embodiments as described in the embodiments, or can be changed to be located in one or more devices different from the embodiments. The modules in the above embodiments can be combined into one module, or can be further split into multiple sub-modules.

[0193] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method of adjusting a head-up display (HUD) image for a vehicle, wherein, The HUD is used for displaying vehicle operation information and driving assistance information through a virtual image, and the method comprises the following steps: determining a posture conversion matrix of a current vehicle coordinate system based on a world coordinate system at a next time point; correcting a first position conversion matrix between a first camera coordinate system of a vehicle and the vehicle coordinate system in the world coordinate system according to the posture conversion matrix, to obtain a corrected first position conversion matrix, wherein the first position conversion matrix is used for converting position information of a target object in the first camera coordinate system of the vehicle to the vehicle coordinate system; determining a visual conversion matrix of driver's eyes in the vehicle coordinate system according to a position conversion relationship of a line of sight of the driver at the next time point, wherein the driver's eyes are recognized by a second camera of the vehicle; correcting a second position conversion matrix between a rendering space coordinate system of the virtual image and a virtual camera coordinate system in a rendering space according to the visual conversion matrix, to obtain a corrected second position conversion matrix, wherein the second position conversion matrix is used for converting a position of the target object in the rendering space coordinate system to the virtual camera coordinate system, and the rendering space coordinate system is consistent with the vehicle coordinate system; adjusting a position of the virtual image rendered by the HUD and a position of the target object in the virtual image based on the corrected first position conversion matrix and the corrected second position conversion matrix.

2. The method of claim 1, wherein, The determination of the posture conversion matrix of the current vehicle coordinate system based on the world coordinate system at the next time point comprises the following steps: determining current posture information of the current vehicle in the world coordinate system at a current time point, and obtaining historical posture information of the current vehicle in the world coordinate system at a previous time point of the current time point; determining the posture conversion matrix of the current vehicle coordinate system based on the world coordinate system at the next time point according to the current posture information and the historical posture information.

3. The method of claim 2, wherein, The determination of the current posture information of the current vehicle in the world coordinate system at the current time point comprises the following steps: determining a first rotation matrix of the current vehicle in the world coordinate system around an X-axis, a second rotation matrix around a Y-axis, and a third rotation matrix around a Z-axis at the current time point based on a posture change amount of an inertial measurement unit (IMU) at the current time point, wherein the X-axis, the Y-axis, and the Z-axis are perpendicular to each other; multiplying the third rotation matrix, the second rotation matrix, and the first rotation matrix in sequence to obtain the current posture information of the current vehicle in the world coordinate system at the current time point.

4. The method according to any of claims 1 to 3, characterized in that, The determination of the visual conversion matrix of the driver's eyes in the vehicle coordinate system according to the position conversion relationship of the line of sight of the driver's eyes at the next time point comprises the following steps: determining the visual conversion matrix of the driver's eyes in the vehicle coordinate system according to a rotation matrix of a gazing direction of the driver's eyes at the next time point and a position conversion matrix of the driver's eyes.

5. The method of claim 4, wherein, The determination of the visual conversion matrix of the driver's eyes in the vehicle coordinate system according to the rotation matrix of the gazing direction of the driver's eyes at the next time point and the position conversion matrix of the driver's eyes comprises the following steps: Convert the position information of the driver's eyes in the second camera coordinate system to the vehicle coordinate system based on the relative position relationship between the second camera and the current vehicle, to obtain the position information of the driver's eyes in the vehicle coordinate system; The difference value of the position information of the driver's eyes at adjacent time points is taken as the position transformation matrix of the driver's eyes at adjacent time points; Determine the rotation matrix of the direction in which the driver's eyes are looking at adjacent time points according to the rotation axis and rotation angle between the current gaze vector corresponding to the direction in which the driver's eyes are looking at the current time and the historical gaze vector at the previous time; Combine the rotation matrix of the direction in which the driver's eyes are looking at adjacent time points and the position transformation matrix of the driver's eyes at adjacent time points to form the visual transformation matrix of the driver's eyes in the vehicle coordinate system.

6. The method of claim 5, wherein, The rotation matrix of the direction in which the driver's eyes are looking at adjacent time points is determined according to the rotation axis and rotation angle between the current gaze vector corresponding to the direction in which the driver's eyes are looking at the current time and the historical gaze vector at the previous time, including: Based on the Rodrigues formula, the rotation axis and rotation angle between the current gaze vector corresponding to the direction in which the driver's eyes are looking at the current time and the historical gaze vector at the previous time are formed into the rotation matrix of the direction in which the driver's eyes are looking at adjacent time points; Wherein, the rotation axis and rotation angle between the current gaze vector corresponding to the direction in which the driver's eyes are looking at the current time and the historical gaze vector at the previous time are obtained by the following formula respectively: i=z(t-1)×z(t) Wherein, z(t) represents the current gaze vector corresponding to the direction in which the driver's eyes are looking at the current time, z(t-1) represents the historical gaze vector corresponding to the direction in which the driver's eyes are looking at the previous time, i represents the rotation axis between the current gaze vector and the historical gaze vector at the previous time; δ represents the rotation angle between the current gaze vector and the historical gaze vector at the previous time.

7. The method according to any of claims 1 to 6, characterized in that, The first position transformation matrix between the first camera coordinate system and the vehicle coordinate system in the world coordinate system is corrected according to the posture transformation matrix, including: The first position transformation matrix between the first camera coordinate system and the vehicle coordinate system in the world coordinate system is corrected according to the following formula: wherein M' M represents the first position conversion matrix after correction, M M represents the first position conversion matrix before correction, represents the posture conversion matrix of the current vehicle coordinate system based on the world coordinate system at the adjacent time The second position transformation matrix between the rendering space coordinate system of the virtual image and the virtual camera coordinate system in the rendering space is corrected according to the visual transformation matrix, including: The second position transformation matrix between the rendering space coordinate system of the virtual image and the virtual camera coordinate system in the rendering space is corrected according to the following formula: where M' = M - M V represents the second position conversion matrix after correction, M V represents the second position conversion matrix before correction, represents the visual transformation matrix of the driver's eyes in the vehicle coordinate system at the adjacent time.

8. An image adjustment device of a head-up display (HUD) for a vehicle, wherein The HUD is used to display vehicle operation information and driving assistance information through virtual images, characterized in that the device comprises: A posture transformation matrix determination module is configured to determine the posture transformation matrix of the current vehicle coordinate system based on the world coordinate system at adjacent time points; The first correction module is configured to correct a first position transformation matrix between the vehicle coordinate system and a first camera coordinate system of the vehicle in the world coordinate system according to the attitude transformation matrix, to obtain a corrected first position transformation matrix, wherein the first position transformation matrix is used to convert position information of a target object in the first camera coordinate system of the vehicle to the vehicle coordinate system. The visual transformation matrix determination module is configured to determine a visual transformation matrix of the driver's eyes in the vehicle coordinate system according to a position transformation relationship of the driver's line of sight at an adjacent time, wherein the driver's eyes are identified by a second camera of the vehicle. The second correction module is configured to correct a second position transformation matrix between a rendering space coordinate system of the virtual image and a virtual camera coordinate system in a rendering space according to the visual transformation matrix, to obtain a corrected second position transformation matrix, wherein the second position transformation matrix is used to convert a position of the target object in the rendering space coordinate system to the virtual camera coordinate system, and the rendering space coordinate system is consistent with the vehicle coordinate system. The image position adjustment module is configured to adjust a position of the virtual image rendered by the HUD and a position of the target object in the virtual image based on the corrected first position transformation matrix and the corrected second position transformation matrix.

9. An electronic device, comprising: The electronic device includes: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the image adjustment method of the head-up display (HUD) of the vehicle as claimed in any one of claims 1-7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the image adjustment method of the head-up display (HUD) of the vehicle as claimed in any one of claims 1-7.

Images