Image processing apparatus and method, head-up display apparatus, and computer storage medium

Abstract

The invention provides an image processing device and method, a head-up display device and a computer storage medium, and belongs to the technical field of image processing, and the image processing device comprises a memory which is configured to store a corresponding relation between eye box partitions and distortion parameters; the processor is in communication connection with the memory and is configured to determine a target distortion parameter corresponding to a target subarea through a corresponding relation based on the target subarea where the eye spot position of the driver is located in the eye box area; and correcting the to-be-projected image through the target distortion parameter. The distortion of an HUD image projected to a windshield can be in a range which can not be observed by human eyes.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of image processing, and in particular to an image processing device and method, a head-up display device and a computer storage medium. BACKGROUND

[0002] A head-up display (HUD) device is an optical device that projects key driving information (such as vehicle speed, navigation, warnings, etc.) to the driver's field of view in front, which can enable the driver to obtain driving assistance information without looking down at the instrument panel, thereby effectively reducing distraction and improving driving safety and interaction experience.

[0003] However, due to the complex optical structure of the HUD system itself, and the fact that its imaging medium is usually a non-flat free-form windshield, the virtual image produced by the projection generally has distortion problems. This distortion usually manifests as a distorted, stretched, scaled or curved image, which seriously affects the clarity, accuracy and visual perception of the image. SUMMARY

[0004] The present disclosure provides an image processing device and method, a head-up display device and a computer storage medium, which can make the distortion of the HUD image projected onto the windshield within the range of human eyes.

[0005] The technical solution of the present disclosure is implemented as follows: In a first aspect, the present disclosure provides an image processing device, which comprises: a memory configured to store a correspondence between eyebox partitions and distortion parameters; a processor in communication connection with the memory, configured to determine a target distortion parameter corresponding to a target partition in which the eye point position of the driver is located in the eyebox region based on the target partition, and correct a to-be-projected image by using the target distortion parameter.

[0006] In a second aspect, the present disclosure provides an image processing method, which comprises: determining a target distortion parameter corresponding to a target partition in which the eye point position of the driver is located in the eyebox region based on the target partition, and correcting a to-be-projected image by using the target distortion parameter.

[0007] In a third aspect, the present disclosure provides a head-up display device, which comprises the image processing device of the first aspect and a display part in communication connection with the processor in the image processing device; the display part is used to project the corrected to-be-projected image to the windshield of the vehicle for display.

[0008] In a fourth aspect, the disclosure provides a computer-readable storage medium storing at least one instruction for execution by a processor to implement the image processing method according to the second aspect.

[0009] In a fifth aspect, the disclosure provides a vehicle comprising the head-up display device according to the third aspect.

[0010] The disclosure provides an image processing device, which, by means of a pre-stored correspondence between eyebox partitions and distortion parameters, maps a current eye point position of a driver to a preset target partition after real-time acquisition, and determines a target distortion parameter corresponding to the target partition from the correspondence, so as to correct a to-be-projected image by means of the target distortion parameter. This mechanism enables generation of distortion-free images meeting visual requirements at any position within the full eyebox range, significantly improves the fidelity and visual comfort of virtual display, avoids problems such as image distortion, ghosting and reduced information readability, and thus improves user experience and driving safety. BRIEF DESCRIPTION OF DRAWINGS

[0011] Figure 1 A schematic diagram of a vehicle-mounted system according to the disclosure is provided.

[0012] Figure 2 A schematic diagram of the architecture of a head-up display device according to the disclosure is provided.

[0013] Figure 3 A schematic diagram of the structure of an image processing device according to the disclosure is provided.

[0014] Figure 4 A schematic diagram of the execution flow of a processor according to the disclosure is provided.

[0015] Figure 5 A schematic diagram in a vehicle coordinate system according to the disclosure is provided.

[0016] Figure 6 A schematic diagram of eyebox partitioning according to the disclosure is provided.

[0017] Figure 7 A schematic diagram of the imaging principle of a 3D HUD according to the disclosure is provided.

[0018] Figure 8 A schematic diagram of the generation of left-eye and right-eye crosstalk of a 3D HUD according to the disclosure is provided.

[0019] Figure 9 A schematic diagram of left-eye and right-eye images according to the disclosure is provided.

[0020] Figure 10 A schematic diagram of the difference between a standard image and a measured image according to the disclosure is provided.

[0021] Figure 11 A flowchart of an image processing method provided by the present disclosure.

[0022] Figure 12 A flowchart of another image processing method provided by the present disclosure.

[0023] Figure 13 A flowchart of still another image processing method provided by the present disclosure.

[0024] Figure 14 A flowchart of yet another image processing method provided by the present disclosure. DETAILED DESCRIPTION

[0025] The technical solutions in the present disclosure will be described clearly and completely below in conjunction with the drawings in the present disclosure.

[0026] Reference Figure 1 which shows a system 10 for implementing the image processing method of the present disclosure. The system 10 is mounted on an exemplary vehicle 11. Although the vehicle 11 shown in the figure is a passenger car, it should be understood that the vehicle 11 can be any type of vehicle without departing from the scope of the present disclosure. The system 10 generally includes a controller 12, vehicle sensors 13, a heads-up display device 14, and a Human Machine Interface (HMI) 15.

[0027] The controller 12 includes at least one processing unit 121 and a non-transitory computer-readable storage device or medium 122. The processing unit 121 can be a central processing unit (CPU), a graphics processing unit (GPU), an auxiliary processor associated with the controller 12, a microprocessor, a macroprocessor, or a combination thereof, or a device generally used to execute instructions.

[0028] The computer-readable storage device or medium 122 can include volatile and nonvolatile storage in, for example, read-only memory (ROM), random-access memory (RAM), and keep-alive memory (KAM). KAM is a persistent or nonvolatile memory that can be used to store various operating variables when the processing unit 121 is powered down. The computer-readable storage device or medium 122 can be implemented using a number of storage devices, such as programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or other electric, magnetic, optical, or combined storage devices capable of storing data.

[0029] The heads-up display device 14 is used to display display content related to the current vehicle and its environment on the windshield 16 of the vehicle 11. The controller 12 is in electrical communication with the vehicle sensors 13, the heads-up display device 14, and the HMI 15. The electrical communication can be established using, for example, a CAN bus, a Wi-Fi network, a cellular data network, or the like. It should be understood that various additional wired and wireless technologies and communication protocols for communicating with the controller 12 are within the scope of the present disclosure.

[0030] The vehicle sensors 13 are used to acquire information about the environment surrounding the vehicle 11. In the exemplary embodiment, the vehicle sensors 13 include a camera 131, a vehicle communication system 132, and an electronic ranging sensor 133. It should be understood that additional sensors can be included in the vehicle sensors 13 for determining characteristics of the vehicle 11, such as vehicle speed, road curvature, and / or vehicle steering, without departing from the scope of the present disclosure. As discussed above, the vehicle sensors 13 are in electrical communication with the controller 12.

[0031] Camera 131 is used to capture images and / or videos of the environment surrounding vehicle 11. In an exemplary embodiment, camera 131 is a photographic and / or video camera positioned to observe environmental information in front of vehicle 11. In one example, camera 131 is fixed inside vehicle 11 (e.g., in the roof lining of vehicle 11) and has a field of view through windshield 16. In another example, camera 131 is fixed outside vehicle 11, for example, on the roof of vehicle 11, and has a view of the environment in front of vehicle 11. It should be understood that cameras with various sensor types are within the scope of this disclosure, such as charge-coupled device (CCD) sensors, complementary metal-oxide-semiconductor (CMOS) sensors, and / or high dynamic range (HDR) sensors, etc. Furthermore, cameras with various lens types, including, for example, wide-angle lenses and / or narrow-angle lenses, are also within the scope of this disclosure.

[0032] The controller 12 uses the vehicle communication system 132 to communicate with other systems outside the vehicle 11. For example, the vehicle communication system 132 includes the ability to communicate with remote systems and / or personal devices, such as vehicles, infrastructure, remote call centers, etc. The vehicle communication system 132 may include one or more antennas and / or communication transceivers for receiving and / or transmitting signals. The vehicle communication system 132 is configured to wirelessly transmit information between the vehicle 11 and another vehicle. Furthermore, the vehicle communication system 132 is configured to wirelessly transmit information between the vehicle 11 and infrastructure or other vehicles.

[0033] Electronic distance sensor 133 is used to determine the range (i.e., distance) between vehicle 11 and objects in the environment surrounding the vehicle. Electronic distance sensor 133 may utilize electromagnetic waves (e.g., radar), sound waves (e.g., ultrasound), and / or light (e.g., lidar) to determine the distance. Figure 1 In the exemplary embodiments shown, the electronic ranging sensor 133 is a lidar sensor. It should be understood that embodiments in which the electronic ranging sensor 133 includes radar sensors, ultrasonic sensors, lidar sensors, and / or other sensors configured to determine range (i.e., distance) fall within the scope of this disclosure.

[0034] refer to Figure 2An exemplary implementation architecture of a head-up display device 14 is shown. The head-up display device 14 is used to display a HUD image 22 (i.e., a notification symbol providing visual information to the occupants) on the windshield 16 of a vehicle 11. The head-up display device 14 includes an image processing unit 20 and a display unit 21. As discussed above, the head-up display device 14 is in electrical communication with a controller 12.

[0035] The image processing device 20 generates a signal 202 based on the data 201 received by the controller 12 via electrical communication. The display unit 21 may include a light source 211 and an optical path assembly 212. The light source 211 outputs a light signal 203 based on the signal 202 from the image processing device 20 to display a HUD image 22 on the windshield 16. For example, the light source 211 may include one or more lasers and output red, green, and blue light.

[0036] The optical path assembly 212 reflects the output of the light source 211 onto the windshield 16. A viewer (e.g., a driver) can view the HUD image 22 in the display area on the windshield 16 where the light signal 203 is projected. In some examples, the optical path assembly 212 may include one or more reflectors (plane mirrors) and concave mirrors (magnifying glasses). The output of the light source 211 is reflected back via the reflector and magnified by the concave mirror before being reflected back onto the windshield 16 to form a HUD image 22 that can be visually observed by the driver. The visual effect of this HUD image 22 is that it is projected onto a projection surface 23 at a predetermined distance in front of the vehicle, but the real environment remains visible through the projection surface 23. In some examples, the optical path assembly 212 may be omitted, and the light source 211 may directly project the light signal 203 onto the windshield 16 to form the HUD image 22 on the projection surface 23.

[0037] Because the projection medium of the HUD image is the windshield 16, which is not an ideal optical plane but a large-curvature freeform surface designed to meet aerodynamic and aesthetic requirements, this complex asymmetrical curvature acts like a non-ideal lens, causing non-uniform refraction and reflection of the projected light, resulting in distortion, stretching, and pincushion or barrel distortion of the virtual image. Furthermore, the aberrations (such as coma and astigmatism) inherent in the precision optical components (such as aspherical mirrors and freeform mirrors) within the HUD optical module during design and manufacturing, as well as the minute assembly tolerances and angular deviations in the installation process, further introduce deviations in the light propagation path. In addition, changes in the driver's viewing position cause distortion of the virtual image. These systematic errors, the characteristics of the windshield, and changes in the driver's viewing position combine to destroy the geometric fidelity of the HUD image, resulting in visible distortion that affects the visual effect and driving experience.

[0038] Based on the above description, this disclosure aims to provide an image processing apparatus 20, such as... Figure 3 As shown, the image processing device 20 includes a memory 301 and a processor 302. The memory 301 is communicatively connected to the processor 302, and the memory 301 stores the correspondence between eyebox partitions and distortion parameters. Figure 4 As shown, processor 302 is configured to perform the following steps S401 and S402.

[0039] In step S401, based on the target partition where the driver's eye point is located in the eye box region, the target distortion parameters corresponding to the target partition are determined through the correspondence between the eye box partition and the distortion parameters.

[0040] Eye position refers to the three-dimensional spatial coordinates (X, Y, Z) of the driver's eye position (e.g., the center of the pupil) in the vehicle coordinate system. This position can be obtained in real time through an eye-tracking module. Figure 5 As shown, a vehicle coordinate system is established with the vehicle as the reference. This coordinate system is a right-handed coordinate system. In this coordinate system, the positive X-axis is the direction in which the front of the vehicle is facing, the positive Y-axis is the direction along the width of the vehicle pointing to the left, and the positive Z-axis is the height of the vehicle. Based on the above three coordinate axes, three planes are formed with the projection of the center of the vehicle's front onto the ground as the origin. Among them, plane XOY is the ground, plane YOZ is a plane parallel to the front of the vehicle, and plane XOZ is a plane along the center of the front and rear of the vehicle and perpendicular to the ground.

[0041] The eye-box area refers to a pre-defined, fixed three-dimensional spatial range in front of the driver. The driver can see the complete HUD image if their eye position is within this area. Exceeding this range may result in incomplete display or even complete blindness. For example, the eye-box area can be defined as the volume enclosed by the space bounded by [450, 550] mm in the X direction, [300, 400] mm in the Y direction, and [600, 700] mm in the Z direction.

[0042] Eyebox partitioning refers to further dividing the eyebox area into several smaller subspaces in the horizontal (X) and vertical (Y) directions, with each partition associated with an independent set of distortion parameters. For example, dividing the eyebox into 2 columns in the X direction and 2 rows in the Y direction results in a total of 2×2=4 eyebox partitions. Figure 6 As shown, the eye box region is a cross-sectional area 601 perpendicular to the center point. The eye box region is divided into four areas. Within the eye box region, the human eye can observe the virtual image projected by the head-up display device 14 onto the windshield 16, namely the HUD image 22.

[0043] Distortion parameters refer to a set of mathematical parameters used to pre-distort the projected image, with the aim of compensating for distortions caused by the optical system, driver position changes, etc. For example, a distortion parameter can be a transformation matrix, with each eye box section corresponding to a unique matrix.

[0044] The correspondence between eyebox partitions and distortion parameters is a pre-established and stored mapping table. Specifically, the eyebox area is divided into multiple eyebox partitions (e.g., 6). A high-precision camera module is used to precisely move to the center point of each eyebox partition and acquire a standard test image (e.g., a grid image) projected by the HUD at that point. The acquired test image is analyzed using image processing algorithms and compared with the standard test image to determine the distortion parameters that can correct the distortion at the center point of that eyebox partition. The eyebox partition number and its corresponding distortion parameter are stored as a key-value pair in memory 301. The above operation is repeated until the distortion parameters corresponding to each eyebox partition are determined and stored.

[0045] In step S402, the projected image is corrected using the target distortion parameters.

[0046] The image to be projected refers to the raw image data that needs to be displayed to the driver by the HUD, such as an RGB pixel matrix containing information such as vehicle speed, navigation arrows, and warning signs. The image to be projected is pre-distorted using target distortion parameters, so that the corrected image, when projected onto the windshield, appears undistorted or within the permissible distortion range to the driver.

[0047] This embodiment of the disclosure utilizes a pre-stored correspondence between eye-box partitions and distortion parameters to obtain the driver's current eye position in real time. This position is then mapped to a preset target partition, and the corresponding target distortion parameter is determined from the correspondence. This target distortion parameter is then used to correct the projected image. This mechanism enables the generation of a distortion-free image that meets visual requirements at any position within the entire eye-box, significantly improving the fidelity and visual comfort of the virtual display. It avoids problems such as image distortion, ghosting, and reduced information readability, thereby enhancing user experience and driving safety.

[0048] Conventional 2D HUDs project important driving information (such as vehicle speed, navigation arrows, warning signs, etc.) into the driver's forward field of vision. However, with the development of 3D display technology, 3D HUDs are also widely used, enabling virtual images to present different levels of depth and seamlessly integrate with the real world, such as "fitting" navigation arrows onto the real road and highlighting lane lines.

[0049] 3D HUDs generate images with binocular parallax (i.e., subtle differences between the images for the left and right eyes), utilizing the human brain's fusion mechanism to create virtual images with a realistic sense of depth. For example...Figure 7 As shown, there is a certain difference between the left eye image 701 and the right eye image 702. The distance between the left eye image 701 and the right eye image 702 is the disparity (the distance between the two arrows in the figure). The left eye is at position 703 and the right eye is at position 704. The left eye observes the left eye image 701 at position 703 and the right eye observes the right eye image 702 at position 704. Based on the disparity, the brain fuses the observed left eye image 701 and right eye image 702. Finally, the human eye perceives a stereoscopic image 705.

[0050] The display effect of a 3D HUD relies on projecting at least two slightly different images to the left and right eyes respectively. Ideally, the image projected to the left eye should only enter the left eye, and the image projected to the right eye should only enter the right eye. However, due to optical system defects and changes in viewing position, some light from the right eye image may "leak" into the left eye area, and vice versa, causing optical crosstalk. This crosstalk can lead to ghosting and blurring of the image, severely compromising the 3D HUD display effect and visual comfort. Figure 8 As shown, the dashed line represents the ideal beam. The beam corresponding to the left-eye image reaches position 703 (left eye location), and the beam corresponding to the right-eye image reaches position 704 (right eye location). At this point, there is no optical crosstalk, and the fused stereoscopic image is distortion-free. The solid line represents the actual beam. The beams corresponding to the left-eye and right-eye images crosstalk at region 801, and the beam corresponding to the right-eye image reaches position 703 (left eye location), causing distortion in the fused stereoscopic image.

[0051] Therefore, in some embodiments of this disclosure, the distortion parameters include left-eye crosstalk distortion parameters and right-eye crosstalk distortion parameters; the correspondence between the eyebox partition and the distortion parameters includes: left-eye crosstalk distortion parameters corresponding to the eyebox partition where the left-eye region is located, determined based on the first left-eye image information and the first right-eye image information obtained in the left-eye region, and right-eye crosstalk distortion parameters corresponding to the eyebox partition where the right-eye region is located, determined based on the second left-eye image information and the second right-eye image information obtained in the right-eye region.

[0052] The first left-eye image information refers to the relevant data corresponding to the left-eye image acquired by an image acquisition device (such as a camera) deployed in the left-eye region (a preset area corresponding to the left eye). The first right-eye image information refers to the relevant data corresponding to the right-eye image acquired by an image acquisition device deployed in the left-eye region. The first right-eye image information acquired in the left-eye region can be understood as a crosstalk signal. The more information contained in the first right-eye image information, the more severe the crosstalk, and the more severe the distortion of the final stereoscopic image. The ratio of the first right-eye image information to the first left-eye image information can be used to characterize the left-eye crosstalk rate. The smaller the ratio, the smaller the distortion, and vice versa.

[0053] Similarly, the second left-eye image information refers to the relevant data corresponding to the left-eye image acquired by an image acquisition device (such as a camera) deployed in the right-eye region (the preset region corresponding to the left eye). The second right-eye image information refers to the relevant data corresponding to the right-eye image acquired by an image acquisition device deployed in the right-eye region. The second left-eye image information acquired in the right-eye region can be understood as a crosstalk signal. The more information contained in the second left-eye image information, the more severe the crosstalk, and the more severe the distortion of the final stereoscopic image. The ratio of the second left-eye image information to the second right-eye image information can be used to characterize the right-eye crosstalk rate. The smaller the ratio, the smaller the distortion, and vice versa.

[0054] It should be noted that the first left-eye image information can also be determined by directly parsing the left-eye image, and the second right-eye image information can also be determined by directly parsing the right-eye image. That is, there is no need to collect the image through an image acquisition device. The information corresponding to the standard left-eye image can be directly used as the first left-eye image information, and the information corresponding to the standard right-eye image can be used as the second right-eye image information.

[0055] In some feasible methods, the acquisition of first and second left- and right-eye image information can be achieved by simultaneously displaying the left- and right-eye images. A single frame is captured in the left-eye region and designated as the first left- and right-eye image, while a single frame is captured in the right-eye region and designated as the second left- and right-eye image. The first and second left- and right-eye image information is then extracted from the first left- and right-eye image, and vice versa. This method requires extracting different image information from a single frame, placing high demands on the algorithm, but the acquisition process is simple.

[0056] In other feasible methods, only the left-eye image can be displayed. A first left-eye image is acquired in the left-eye region, then the left-eye image is turned off and only the right-eye image is displayed, and a first right-eye image is acquired in the left-eye region. Similarly, only the left-eye image can be displayed, a second left-eye image is acquired in the right-eye region, then the left-eye image is turned off and only the right-eye image is displayed, and a second right-eye image is acquired in the right-eye region. The first left-eye image information is then parsed from the first left-eye image, the first right-eye image information is parsed from the first right-eye image, the second left-eye image information is parsed from the second left-eye image, and the second right-eye image information is parsed from the second right-eye image. In this method, one frame contains the information of one image. The algorithm parsing process is simple, but the acquisition process is complex. The specific process for obtaining the first and second left-eye image information is determined according to actual needs, and this disclosure does not limit it.

[0057] The left-eye crosstalk distortion parameter is a parameter used to indicate the adjustment of the beam projection path of the left-eye and right-eye images, aiming to reduce the amount of projected light from the right-eye image entering the left-eye region to below the crosstalk threshold; the right-eye crosstalk distortion parameter is a parameter used to indicate the adjustment of the beam projection path of the left-eye and right-eye images, aiming to reduce the amount of projected light from the left-eye image entering the right-eye region to below the crosstalk threshold, thereby avoiding distortion of the projected stereoscopic image.

[0058] The correspondence between the eyebox partition and the distortion parameter is pre-stored in the memory 301. In some possible implementations, based on the first left-eye image information and the first right-eye image information, when it is determined that the crosstalk rate is greater than or equal to the crosstalk threshold, the beam projection direction of the left-eye image and the right-eye image is adjusted according to a preset step size. After adjustment, the first left-eye image information and the first right-eye image information are acquired again, and this process is repeated until the ratio of the adjusted first right-eye image information to the first left-eye image information is less than the crosstalk threshold. At this point, the adjustment amount during the adjustment process is determined as the left-eye crosstalk distortion parameter corresponding to the eyebox partition where the left-eye region is located.

[0059] Similarly, based on the second left-eye image information and the second right-eye image information, when the crosstalk rate is determined to be greater than or equal to the crosstalk threshold, the beam projection direction of the left-eye and right-eye images is adjusted according to a preset step size. After adjustment, the second left-eye image information and the second right-eye image information are acquired again, and this process is repeated until the ratio of the adjusted second left-eye image information to the second right-eye image information is less than the crosstalk threshold. The adjustment amount during the adjustment process is then determined as the right-eye crosstalk distortion parameter corresponding to the eyebox partition where the right-eye region is located. In this method, the left-eye crosstalk distortion parameter and the right-eye crosstalk distortion parameter are a translation amount, used to indicate whether to perform translation processing on the left-eye image or the right-eye image to change the path of light entering the human eye, thereby solving the crosstalk problem.

[0060] In other feasible approaches, based on the first left-eye image information and the first right-eye image information, when the crosstalk rate is determined to be greater than or equal to the crosstalk threshold, a left-eye crosstalk distortion parameter is determined based on the first right-eye image information to cancel out the crosstalk distortion of the first right-eye image information. That is, this left-eye crosstalk distortion parameter indicates the processing strategy for the image acquired in the left-eye region. For example, if the processing strategy is convolution processing, the left-eye crosstalk distortion parameter is a convolution kernel, and may also include coefficients to avoid over- or under-compensation. Since the area where crosstalk occurs is basically fixed after the HUD is fixed, this left-eye distortion crosstalk parameter is used by software processing to compensate for the crosstalk of the first right-eye image information to the left-eye region, so that the compensated first right-eye image information is weakened or eliminated. Ultimately, the image seen in the left-eye region contains only left-eye image information, and the image seen in the right-eye region contains only right-eye image information, with all other crosstalk components being compensated out.

[0061] Similarly, based on the second left-eye image information and the second right-eye image information, when the crosstalk rate is determined to be greater than or equal to the crosstalk threshold, a right-eye crosstalk distortion parameter is determined based on the second left-eye image information to cancel out the crosstalk distortion of the second left-eye image information. That is, this right-eye crosstalk distortion parameter is used to indicate the processing strategy for the image acquired in the right-eye region, performing image compensation on the second left-eye image information that crosstalks to the right-eye region, so that the compensated second left-eye image information is weakened or disappears.

[0062] The process of determining the above two correspondences can be understood as either a hardware compensation process, which involves shifting the image source, or a software compensation process, which involves processing the acquired image that has crosstalk.

[0063] It should be noted that for each eyebox section, either of the two methods mentioned above is used to determine the corresponding distortion parameters, and the final correspondence between each eyebox section and the distortion parameters in all eyebox sections is as follows.

[0064] For example, such as Figure 9 As shown, the left-eye image 901 is a rectangular array, and the right-eye image 902 is a hexagonal array. The extracted features of the image acquired in the left-eye region are brightness information, based on which the first left-eye image information and the first right-eye image information are obtained. Specifically: when only the left-eye image 901 is projected, the image is acquired in the left-eye region, and the average brightness value of all feature points (i.e., rectangular points) in the image is calculated as the first left-eye image information (denoted as CnL); when the system only projects the right-eye image 902, the image is acquired in the left-eye region, and the average brightness value of all feature points (i.e., hexagonal points) in the image is calculated as the first right-eye image information (denoted as CnR). The crosstalk rate CnR to CnL is calculated and compared with a preset crosstalk threshold (e.g., 3%). If the crosstalk rate exceeds the crosstalk threshold, a control command is generated (e.g., shifting the right eye image beam to the right by Δx units). The above acquisition, calculation and comparison process is iteratively executed until the crosstalk rate is lower than the crosstalk threshold. At this point, the set of all control commands is the left eye crosstalk distortion parameter.

[0065] This embodiment of the invention introduces independent left-eye crosstalk distortion parameters and right-eye crosstalk distortion parameters for each eye box partition. This not only effectively suppresses image ghosting and blurring caused by light leakage in traditional 3D displays, but also ensures that the geometric deformation and color brightness deviation of the virtual stereoscopic image are strictly controlled below the distortion threshold perceived by human vision at different eye box positions, thereby significantly improving visual clarity and comfort.

[0066] In some embodiments, distortion parameters include image distortion parameters.

[0067] One way to determine the correspondence between eyebox partitions and image distortion parameters is to obtain measured images from each eyebox partition and determine the image distortion parameters corresponding to each eyebox partition based on the differences between the measured images and the standard test images.

[0068] The measured image refers to the standard image displayed. The image is acquired by the image acquisition device in each eye box partition. Based on the difference between each pixel of the measured image and the standard image, the correction parameters for each pixel are obtained, and the images are combined to obtain the image distortion parameters corresponding to each eye box partition.

[0069] Another method involves displaying a standard test image, then acquiring a reference image for the corresponding eyebox partition at the center of the eyebox area using an image acquisition device. Based on the differences between each pixel in the reference image and the standard test image, correction parameters are obtained for each pixel, and these are combined to obtain the reference distortion parameters for the center region of the eyebox. Then, based on the differences between the measured images acquired in each eyebox partition and the reference image, and the reference distortion parameters, the image distortion parameters for each eyebox partition are determined. Since the differences between the measured images acquired in different eyebox partitions and the reference image are usually regular variations, such as rotation or translation of the measured images relative to the reference image, etc., this method is not recommended. Figure 10 As shown, the measured image 1001 is rotated as a whole compared to the reference image 1002. Therefore, it is not necessary to store the change parameters corresponding to each pixel of each eye box partition. Only the reference distortion parameters and the correction parameters of each eye box partition relative to the reference image (such as rotating 1 degree clockwise, shifting 2 pixels to the left, etc.) need to be stored. This can save a lot of storage space and reduce the complexity of the image correction process.

[0070] Specifically, feature points can be extracted from a reference image to obtain a reference feature point matrix. This reference feature point matrix is ​​then compared with the standard feature point matrix of the corresponding standard image to determine standard distortion parameters. These standard distortion parameters can be an image transformation matrix, used to indicate the transformation process applied to each point in the image, ensuring that the distortion of the processed image is less than a distortion threshold. Alternatively, they can be an image position mapping table, mapping pixels one-to-one between the standard and reference images to determine scaling, stretching, translation, etc., for each pixel. Then, the measured feature point matrix of each measured image is determined and compared with the reference feature point matrix to determine the parameters for processing the entire image. Specific feature point extraction algorithms can refer to existing image processing algorithms; this disclosure is not limited to any particular algorithm.

[0071] Based on the same concept as the image processing device described above, this disclosure also provides an image processing method, which includes: determining the target distortion parameter corresponding to the target partition based on the target partition where the driver's eye point is located in the eye box region, through the correspondence between the eye box partition and the distortion parameter; and correcting the projected image through the target distortion parameter.

[0072] In some embodiments, the distortion parameters include left-eye crosstalk distortion parameters and right-eye crosstalk distortion parameters; such as Figure 11 As shown, the above image processing method includes the following steps S1101 to S1104.

[0073] S1101. Based on the first left-eye image information and the first right-eye image information obtained in the left-eye region, determine the left-eye crosstalk distortion parameters corresponding to the eyebox partition where the left-eye region is located.

[0074] S1102. Based on the second left-eye image information and the second right-eye image information obtained in the right-eye region, determine the right-eye crosstalk distortion parameters corresponding to the eyebox partition where the right-eye region is located.

[0075] S1103. Based on the target partition where the eye point is located in the eyebox region, determine the target distortion parameters corresponding to the target partition through the correspondence between the eyebox partition and the distortion parameters.

[0076] S1104. Correct the projected image using the target distortion parameters.

[0077] In some embodiments, such as Figure 12 As shown, the above image processing method includes the following steps S1201 to S1204.

[0078] S1201. Based on the first left-eye image information and the first right-eye image information, the beam projection direction of the left-eye image and the right-eye image is adjusted until the ratio of the adjusted first right-eye image information to the first left-eye image information is less than the crosstalk threshold, and the left-eye crosstalk distortion parameter corresponding to the eye box partition where the left-eye region is located is obtained.

[0079] S1202. Based on the second left-eye image information and the second right-eye image information, the beam projection direction of the left-eye image and the right-eye image is adjusted until the ratio of the adjusted second left-eye image information and the second right-eye image information is less than the crosstalk threshold, and the right-eye crosstalk distortion parameter corresponding to the eye box partition where the right-eye region is located is obtained.

[0080] S1203. Based on the target partition where the eye point is located in the eyebox region, determine the target distortion parameters corresponding to the target partition through the correspondence between the eyebox partition and the distortion parameters.

[0081] S1204. Correct the projected image using the target distortion parameters.

[0082] In some embodiments, the first left-eye image information and the first right-eye image information are obtained by parsing the acquired first left and right eye images; the second left-eye image information and the second right-eye image information are obtained by parsing the acquired second left and right eye images.

[0083] In some embodiments, the first left-eye image information is obtained by parsing the acquired first left-eye image, the first right-eye image information is obtained by parsing the acquired first right-eye image, the second left-eye image information is obtained by parsing the acquired second left-eye image, and the second right-eye image information is obtained by parsing the acquired second right-eye image.

[0084] In some embodiments, distortion parameters include image distortion parameters; such as Figure 13 As shown, the above image processing method includes the following steps S1301 to S1303.

[0085] S1301. Based on the measured images and standard test images obtained in each eyebox partition, determine the correspondence between each eyebox partition and the distortion parameters.

[0086] S1302. Based on the target partition where the driver's eye point is located in the eye box area, determine the target distortion parameters corresponding to the target partition through the correspondence between the eye box partition and the distortion parameters.

[0087] S1303, Correct the projected image using the target distortion parameters.

[0088] In some embodiments, distortion parameters include image distortion parameters; such as Figure 14 As shown, the above image processing method includes the following steps S1401 to S1404.

[0089] S1401. Determine the baseline distortion parameters based on the difference between the baseline image acquired in the central region of the eye box area and the standard test image.

[0090] S1402. Based on the differences between the measured images obtained in each eyebox partition and the reference image, as well as the reference distortion parameters, determine the correspondence between each eyebox partition and the distortion parameters.

[0091] S1403. Based on the target partition where the eye point is located in the eyebox region, determine the target distortion parameters corresponding to the target partition through the correspondence between the eyebox partition and the distortion parameters.

[0092] S1404. Correct the projected image using the target distortion parameters.

[0093] It should be noted that the above image processing method can achieve the same technical effect as the above image processing device 20, and will not be described in detail here.

[0094] Based on the same inventive concept as the aforementioned technical solutions, this disclosure provides an image processing apparatus, which includes: a determining part configured to determine target distortion parameters corresponding to the target partition based on the target partition where the driver's eye point is located in the eye box region, through the correspondence between the eye box partition and distortion parameters; and a correction part configured to correct the projected image through the target distortion parameters.

[0095] In some embodiments, the distortion parameters include left-eye crosstalk distortion parameters and right-eye crosstalk distortion parameters; the determining part is configured to determine the left-eye crosstalk distortion parameters corresponding to the eyebox partition where the left-eye region is located, based on the first left-eye image information and the first right-eye image information obtained in the left-eye region; and to determine the right-eye crosstalk distortion parameters corresponding to the eyebox partition where the right-eye region is located, based on the second left-eye image information and the second right-eye image information obtained in the right-eye region.

[0096] In some embodiments, the image processing apparatus further includes: an adjustment section configured to, based on first left-eye image information and first right-eye image information, adjust the beam projection direction of the left-eye image and the right-eye image until the ratio of the adjusted first right-eye image information to the first left-eye image information is less than a crosstalk threshold, to obtain left-eye crosstalk distortion parameters corresponding to the eyebox partition where the left-eye region is located; and, based on second left-eye image information and second right-eye image information, adjust the beam projection direction of the left-eye image and the right-eye image until the ratio of the adjusted second left-eye image information to the second right-eye image information is less than a crosstalk threshold, to obtain right-eye crosstalk distortion parameters corresponding to the eyebox partition where the right-eye region is located.

[0097] In some embodiments, the first left-eye image information and the first right-eye image information are obtained by parsing the acquired first left and right eye images; the second left-eye image information and the second right-eye image information are obtained by parsing the acquired second left and right eye images.

[0098] In some embodiments, the first left-eye image information is obtained by parsing the acquired first left-eye image, the first right-eye image information is obtained by parsing the acquired first right-eye image, the second left-eye image information is obtained by parsing the acquired second left-eye image, and the second right-eye image information is obtained by parsing the acquired second right-eye image.

[0099] In some embodiments, the distortion parameters include image distortion parameters; the determining part is configured to determine the correspondence between each eyebox partition and the distortion parameters based on the measured images and standard test images obtained in each eyebox partition; and, based on the target partition where the driver's eye point is located in the eyebox region, determine the target distortion parameters corresponding to the target partition through the correspondence between the eyebox partition and the distortion parameters.

[0100] In some embodiments, the distortion parameters include image distortion parameters; the determining portion is configured to determine a reference distortion parameter based on the difference between a reference image acquired in the central region of the eye box region and a standard test image; and to determine the correspondence between each eye box region and the distortion parameter based on the difference between the test image acquired in each eye box region and the reference image, and the reference distortion parameter.

[0101] Based on the same inventive concept as the aforementioned technical solutions, this disclosure also provides a head-up display device, combined with Figure 2 and Figure 3 The head-up display device includes: the image processing device 20 and the display unit 21; the display unit 21 is communicatively connected to the processor 302 in the image processing device 20; the display unit 21 is used to project the corrected image to be projected onto the windshield of the vehicle for display.

[0102] This disclosure also provides a computer-readable storage medium storing at least one instruction that is executed by a processor to implement the image processing methods described in the above embodiments.

[0103] This disclosure also provides a computer program product including computer instructions stored in a computer-readable storage medium; a processor of a computing device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computing device to perform the image processing methods described in the above embodiments.

[0104] Those skilled in the art will recognize that the functions described in this disclosure in one or more of the examples above can be implemented using hardware, software, firmware, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium accessible to a general-purpose or special-purpose computer.

[0105] It should be noted that the technical solutions described in this disclosure can be combined arbitrarily as long as they do not conflict.

[0106] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An image processing apparatus characterized by comprising: The method comprises: a memory configured to store a correspondence between eyebox partitions and distortion parameters; a processor in communication connection with the memory, configured to determine a target distortion parameter corresponding to a target partition in which a driver's eye point position is located in an eyebox region based on the target partition, through the correspondence; and correcting a to-be-projected image through the target distortion parameter.

2. The image processing apparatus according to claim 1, characterized by, The distortion parameters comprise left-eye crosstalk distortion parameters and right-eye crosstalk distortion parameters; The correspondence between the eyebox partitions and the distortion parameters comprises left-eye crosstalk distortion parameters corresponding to an eyebox partition in which a left-eye region is located, which are determined based on first left-eye image information and first right-eye image information acquired in the left-eye region, and right-eye crosstalk distortion parameters corresponding to an eyebox partition in which a right-eye region is located, which are determined based on second left-eye image information and second right-eye image information acquired in the right-eye region.

3. The image processing device according to claim 2, wherein The left-eye crosstalk distortion parameters are obtained by adjusting the beam projection directions of left-eye images and right-eye images until the ratio of the adjusted first right-eye image information to the first left-eye image information is less than a crosstalk threshold value, based on the first left-eye image information and the first right-eye image information; The right-eye crosstalk distortion parameters are obtained by adjusting the beam projection directions of left-eye images and right-eye images until the ratio of the adjusted second left-eye image information to the second right-eye image information is less than a crosstalk threshold value, based on the second left-eye image information and the second right-eye image information.

4. The image processing apparatus according to claim 2, characterized by The first left-eye image information and the first right-eye image information are obtained by analyzing first left-eye images and first right-eye images collected; and the second left-eye image information and the second right-eye image information are obtained by analyzing second left-eye images and second right-eye images collected.

5. The image processing apparatus according to claim 2, characterized by The first left-eye image information is obtained by analyzing first left-eye images collected, the first right-eye image information is obtained by analyzing first right-eye images collected, the second left-eye image information is obtained by analyzing second left-eye images collected, and the second right-eye image information is obtained by analyzing second right-eye images collected.

6. The image processing apparatus according to any one of claims 1 to 5, characterized by The distortion parameters comprise image distortion parameters; The correspondence between the eyebox partitions and the distortion parameters is determined according to a difference between a measured image acquired in each eyebox partition and a standard test image.

7. The image processing apparatus according to any one of claims 1 to 5, characterized by The distortion parameters comprise image distortion parameters; The correspondence between the eyebox partitions and the distortion parameters is determined according to a difference between a measured image acquired in each eyebox partition and a reference image, and a reference distortion parameter corresponding to the reference image, the reference image being collected in a central region of the eyebox region, and the reference distortion parameter being determined according to a difference between the reference image and a standard test image.

8. An image processing method characterized by, The method comprises: determining a target distortion parameter corresponding to a target partition in which a driver's eye point position is located in an eyebox region based on the target partition, through a correspondence between eyebox partitions and distortion parameters; and correcting a to-be-projected image through the target distortion parameter.

9. The image processing method of claim 8, wherein, The distortion parameters comprise left-eye crosstalk distortion parameters and right-eye crosstalk distortion parameters, and the image processing method further comprises: determine, based on the first left-eye image information and the first right-eye image information acquired in the left-eye region, a left-eye crosstalk distortion parameter corresponding to the eyebox partition in which the left-eye region is located; determine, based on the second left-eye image information and the second right-eye image information acquired in the right-eye region, a right-eye crosstalk distortion parameter corresponding to the eyebox partition in which the right-eye region is located.

10. The image processing method of claim 9, wherein, The determining, based on the first left-eye image information and the first right-eye image information acquired in the left-eye region, a left-eye crosstalk distortion parameter corresponding to the eyebox partition in which the left-eye region is located includes: adjusting the beam projection direction of the left-eye image and the right-eye image based on the first left-eye image information and the first right-eye image information until the ratio of the adjusted first left-eye image information and the first right-eye image information is less than a crosstalk threshold, to obtain the left-eye crosstalk distortion parameter corresponding to the eyebox partition in which the left-eye region is located; The determining, based on the second left-eye image information and the second right-eye image information acquired in the right-eye region, a right-eye crosstalk distortion parameter corresponding to the eyebox partition in which the right-eye region is located includes: adjusting the beam projection direction of the left-eye image and the right-eye image based on the second left-eye image information and the second right-eye image information until the ratio of the adjusted second left-eye image information and the second right-eye image information is less than a crosstalk threshold, to obtain the right-eye crosstalk distortion parameter corresponding to the eyebox partition in which the right-eye region is located.

11. The image processing method of any one of claims 8 to 10, characterized in that, The distortion parameter includes an image distortion parameter, and the image processing method further includes: determining a reference distortion parameter according to the difference between a reference image collected in a central region of the eyebox region and a standard test image; determining a corresponding relationship between each eyebox partition and a distortion parameter according to the difference between a measured image acquired in each eyebox partition and the reference image, and the reference distortion parameter.

12. A head-up display device, characterized by comprising: The head-up display device includes: The image processing device of any one of claims 1 to 7; and The display unit is in communication connection with the processor in the image processing device, and is configured to project the corrected to-be-projected image to the windshield of the vehicle for display.

13. A computer storage medium, comprising, The computer storage medium stores at least one instruction, and the at least one instruction is used for being executed by the processor to implement the image processing method of claims 8 to 11.

Images