Left and right eye fusion image head-up display method and device based on distortion correction

Abstract

A left and right eye fusion image head-up display method and device based on distortion correction. The method comprises: capturing real-time position data of left and right eyes of a user, and on the basis of the real-time position data of left and right eyes, determining a target partition from among a plurality of partitions of a virtual image coordinate space (S110); acquiring left and right eye display images to be fused for display, and using a distortion correction parameter corresponding to the target partition to perform distortion correction on said left and right eye display images, so as to obtain left and right eye corrected images (S120); respectively acquiring pixel points at different positions from the left and right eye corrected images, and performing a plurality of fusions, so as to obtain a plurality of fused images (S130); and on the basis of the real-time position data of left and right eyes and the plurality of fused images, acquiring a target fused image, and projecting the target fused image onto a display screen for head-up display (S140).

Description

A method and device for head-up display based on distortion correction and left-right eye fusion images.

[0001] This application claims priority to Chinese Patent Application No. 202411023401.9, filed with the Chinese Patent Office on July 29, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of image processing technology, for example to a method and apparatus for head-up display based on distortion correction of left and right eye fused images. Background Technology

[0003] In traditional HUD displays, images are typically flat, lacking depth and a sense of three-dimensionality. Naked-eye 3D HUD technology, however, utilizes the parallax principle of the human eye, allowing the left and right eyes to see different images, thus creating a sense of depth. However, due to factors such as the optical principles of the HUD system and the shape of the vehicle's windshield, the displayed image may be distorted, resulting in inaccurate, blurry, or unclear images. This distortion can affect the driver's accurate understanding and judgment of information, reducing the effectiveness of the HUD.

[0004] Currently, related technologies correct image distortion by applying fixed distortion correction parameters to the entire image. However, since the distortion effect varies at different locations, using fixed distortion correction parameters alone can result in poor image accuracy, thus affecting the user experience. Summary of the Invention

[0005] This application provides a method and device for head-up display of left and right eye fused images based on distortion correction. By correcting the distortion of the left and right eye fused images, it can prevent important information from being difficult to recognize due to distortion, improve visual accuracy, and enhance user experience.

[0006] According to one aspect of this application, a method for left-right eye fusion image head-up display based on distortion correction is provided, the method comprising:

[0007] Capture the user's real-time left and right eye position data, and determine the target partition in multiple partitions of the virtual image coordinate space based on the real-time left and right eye position data;

[0008] Obtain the left and right eye display images to be fused and displayed, and use the distortion correction parameters corresponding to the target partition to perform distortion correction on the left and right eye display images to obtain the left and right eye corrected images;

[0009] Pixels at different locations are obtained from the corrected images of the left and right eyes and fused multiple times to obtain multiple fused images;

[0010] Based on real-time position data of the left and right eyes and multiple fused images, a target fused image is obtained and projected onto the display screen for head-up display.

[0011] Optionally, before using the distortion correction parameters corresponding to the target partition to perform distortion correction on the left and right eye display images, the method further includes: after displaying a standard dot matrix image on a display screen, obtaining a distorted display image corresponding to each observation position by simulating the display images observed by the human eye at different observation positions; determining the partition corresponding to each observation position in the virtual image coordinate space; and generating distortion correction parameters corresponding to each partition based on the image differences between the standard dot matrix image and each distorted display image.

[0012] The advantage of this setup is that by displaying a standard bitmap image and simulating human observation from different positions, a distorted display image corresponding to each observation position can be obtained, which helps to comprehensively understand the distortion of the displayed image under different viewing angles. By generating distortion correction parameters, the distortion characteristics of each partition can be accurately described, thereby achieving more accurate distortion correction.

[0013] Optionally, the system captures the real-time position data of the user's left and right eyes, and determines the target partition in multiple partitions of the virtual image coordinate space based on the real-time position data of the left and right eyes, including: capturing the real-time position of the user's left eye and the real-time position of the right eye through the driver monitoring system; and determining the target partition based on the partition in the virtual image coordinate space in which the real-time position of the left eye and the real-time position of the right eye fall.

[0014] The advantage of this setup is that by monitoring the real-time position data of the user's left and right eyes, the displayed content can be adjusted promptly to adapt to changes in the user's viewing angle. After determining the target area, targeted image correction can be performed to improve the accuracy and clarity of the display, avoid image distortion or blurring, and ensure that the user always receives a good visual experience.

[0015] Optionally, before obtaining multiple fused images by fusing pixels from different locations in the left and right eye corrected images, the method further includes: determining the number of fusions of the fused images, and dividing the display screen into multiple regions based on the number of fusions; wherein the number of regions matches the number of fusions.

[0016] The advantage of this setup is that by determining the number of blending iterations, the degree and effect of the blending can be controlled. Different blending iterations can produce different visual effects, meeting the needs of different users or adapting to different application scenarios.

[0017] Optionally, pixels at different locations are obtained from the left and right eye corrected images and fused multiple times to obtain multiple fused images, including: expanding each pixel of the left and right eye corrected images into RGB channel values ​​to obtain the left eye RGB channel layout map and the right eye RGB channel layout map; and determining the fused image under each fusion round corresponding to the number of fusions based on the left eye RGB channel layout map and the right eye RGB channel layout map.

[0018] The advantage of this setup is that by expanding each pixel of the left and right eye corrected images into RGB channel values, the color information of the image can be analyzed and processed in more detail, and color details can be better preserved and integrated during the fusion process. Fusing the left and right eye corrected images enhances the stereoscopic and depth perception of the image, allowing users to see a three-dimensional image.

[0019] Optionally, based on the left-eye RGB channel layout map and the right-eye RGB channel layout map, determine the fused image for each fusion round corresponding to the number of fusions, including: generating a full-eye RGB channel layout map to be filled corresponding to the fused image; determining the fusion strategy corresponding to each fusion round, and according to the fusion strategy, selecting RGB channel values ​​from the left-eye RGB channel layout map and the right-eye RGB channel layout map to fill the matching image positions in the full-eye RGB channel layout map to be filled, thereby obtaining the fused image for each fusion round.

[0020] The advantage of this setup is that different fusion rounds and fusion strategies can adapt to different eye positions of the user, allowing the user's left and right eyes to achieve the best observation effect in any line of sight.

[0021] Optional fusion strategies include: the number of RGB channel value groups, the number of indents after the first line, and the number of vertical indents.

[0022] The advantage of this setup is that different fusion strategies can achieve different fusion effects, providing users with flexibility and customizability to meet the requirements of image fusion in different situations.

[0023] Optionally, according to the fusion strategy, RGB channel values ​​are selected from the left-eye RGB channel layout image and the right-eye RGB channel layout image to fill the matching image positions in the full-eye RGB channel layout image to be filled, respectively, to obtain the fused image under each fusion round. This includes: obtaining the current fusion strategy corresponding to the current fusion round, and determining the initial filling method of the first row corresponding to the full-eye RGB channel layout image to be filled based on the number of RGB channel value groups in the current fusion strategy; adjusting the initial filling method of the first row according to the number of indentations after the first row corresponding to the current fusion round to obtain the target filling method of the first row; determining each non-first row filling method sequentially starting from the second row based on the target filling method of the first row and the number of vertical indentations; and selecting RGB channel values ​​from the left-eye RGB channel layout image and the right-eye RGB channel layout image to fill the matching image positions in the full-eye RGB channel layout image to be filled, based on the target filling method of the first row and each non-first row filling method, to obtain the fused image under the current fusion round.

[0024] The advantage of this setup is that by adjusting the initial fill method of the first row through post-indentation, an indentation effect can be achieved between the first row and other rows, thereby creating a sense of depth and three-dimensionality in the merged image. Following the fusion strategy ensures consistency in the fill method of each row, guaranteeing the quality of the image fusion.

[0025] Optionally, in each fusion strategy under different fusion rounds, the number of RGB channel value groups and the number of vertical indentation bits are the same; in adjacent fusion rounds, the number of first-line indentation bits in the later fusion round is 1 more than the number of first-line indentation bits in the previous fusion round.

[0026] The advantage of this setup is that by maintaining the same number of RGB channel value groups and vertical indentation bits across different fusion rounds, the stability of the fusion effect can be improved. Gradually increasing the first-line indentation bits can add a sense of depth to the fused image, enhancing its visual appeal.

[0027] Optionally, the number of RGB channel value groups is 4, and the vertical indentation is 1 bit.

[0028] The advantage of this setting is that by setting the specific number of RGB channel value groups and the vertical indentation, the fusion process can be controlled more precisely, ensuring the accuracy of image fusion.

[0029] Optionally, based on real-time position data of the left and right eyes and multiple fused images, a target fused image is obtained, including: based on the mapping relationship between the real-time position data of the left and right eyes and the pre-established region and fusion round, obtaining the fusion round corresponding to each region on the display screen; and using the fused images under the fusion round corresponding to each region, stitching together to obtain the target fused image.

[0030] The advantage of this setup is that it can dynamically determine the fusion round corresponding to each region based on the real-time position data of the user's left and right eyes, ensuring that the stitched target fused image matches the user's perspective and provides the user with a better visual experience.

[0031] Optionally, pixels at different locations are obtained from the left and right eye corrected images and fused multiple times to obtain multiple fused images. This includes calling the shader plugin in the 3D image processing engine to obtain pixels at different locations from the left and right eye corrected images and fused multiple times to obtain multiple fused images.

[0032] The advantage of this setup is that by calling shader plugins in the 3D image processing engine multiple times for fusion, it can simulate the human eye's perception of different depths and positions, thereby enhancing the realism and three-dimensionality of the image. Shader plugins can execute in parallel on the graphics processor, improving image processing efficiency and performance while reducing processing time and latency.

[0033] According to another aspect of this application, a head-up display device for left-right eye fusion images based on distortion correction is provided, the device comprising:

[0034] The target partition determination module is used to capture the real-time position data of the user's left and right eyes, and determine the target partition among multiple partitions in the virtual image coordinate space based on the real-time position data of the left and right eyes.

[0035] The image distortion correction module is used to acquire the left and right eye display images to be fused and displayed, and to use the distortion correction parameters corresponding to the target partition to correct the distortion of the left and right eye display images to obtain the corrected left and right eye images.

[0036] The image fusion module is used to obtain pixels from different locations in the left and right eye corrected images and fuse them multiple times to obtain multiple fused images;

[0037] The target fusion image determination and display module is used to obtain a target fusion image based on real-time position data of the left and right eyes and multiple fusion images, and project the target fusion image onto the display screen for head-up display.

[0038] According to another aspect of this application, an electronic device is provided, the electronic device comprising:

[0039] At least one processor; and

[0040] A memory communicatively connected to the at least one processor; wherein,

[0041] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform a left-right eye fusion image head-up display method based on distortion correction as described in any embodiment of this application.

[0042] According to another aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement a distortion correction-based left-right eye fusion image head-up display method as described in any embodiment of this application.

[0043] The technical solution of this application, by capturing real-time position data of the user's left and right eyes, can dynamically adjust the displayed content according to the user's viewing angle. By correcting the left and right eye display images using distortion correction parameters corresponding to the target partition, image distortion can be eliminated, making the image more realistic and accurate, and improving visual quality. By obtaining pixels from different positions in the corrected left and right eye images and fusing them multiple times, the image details and three-dimensionality can be enhanced, providing a more realistic and vivid visual effect. By projecting the target fused image onto the display screen for head-up display, users can obtain important information without looking down at the instrument panel, improving driving safety. Attached Figure Description

[0044] Figure 1 is a flowchart of a left-right eye fusion image head-up display method based on distortion correction according to Embodiment 1 of this application;

[0045] Figure 2 is a schematic diagram of a distorted image generation process according to Embodiment 1 of this application;

[0046] Figure 3 is a schematic diagram of a distorted image generation effect according to Embodiment 1 of this application;

[0047] Figure 4 is a schematic diagram of a hardware correction process according to Embodiment 1 of this application;

[0048] Figure 5 is a schematic diagram of a software correction process according to Embodiment 1 of this application;

[0049] Figure 6 is a schematic diagram of a distortion correction effect provided according to Embodiment 1 of this application;

[0050] Figure 7 is a schematic diagram of a distortion mapping relationship mesh according to Embodiment 1 of this application;

[0051] Figure 8 is a schematic diagram of a head-up display imaging process according to Embodiment 1 of this application;

[0052] Figure 9 is a schematic diagram of a naked-eye 3D imaging principle according to Embodiment 1 of this application;

[0053] Figure 10 is a schematic diagram of a head-up display process for correcting image distortion according to Embodiment 1 of this application;

[0054] Figure 11 is a flowchart of another left-right eye fusion image head-up display method based on distortion correction according to Embodiment 2 of this application;

[0055] Figure 12 is a schematic diagram of a head-up display device for left and right eye fusion images based on distortion correction according to Embodiment 3 of this application;

[0056] Figure 13 is a schematic diagram of the structure of a vehicle that implements a distortion correction-based left and right eye fusion image head-up display method according to an embodiment of this application. Detailed Implementation

[0057] The embodiments of this application will now be clearly and completely described with reference to the accompanying drawings. These described embodiments are only a part of, and not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort should fall within the scope of protection of this application.

[0058] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0059] Example 1

[0060] Figure 1 is a flowchart of a left-right eye fusion image head-up display method based on distortion correction provided in Embodiment 1 of this application. This embodiment is applicable to situations where distortion processing is performed according to the user's eye position and a real-time head-up display is displayed. This method can be executed by a left-right eye fusion image head-up display device based on distortion correction. This left-right eye fusion image head-up display device based on distortion correction can be implemented in hardware and / or software and can be configured in a vehicle. As shown in Figure 1, the method includes:

[0061] S110. Capture the real-time position data of the user's left and right eyes, and determine the target partition among multiple partitions in the virtual image coordinate space based on the real-time position data of the left and right eyes.

[0062] The real-time position data for the left and right eyes refers to the specific position information of the user's left and right eyes in the virtual image coordinate space, which is monitored and acquired in real time. This includes the real-time position of the left eye and the real-time position of the right eye, represented in coordinate form. The virtual image coordinate space is a mathematical tool used to describe and process the position of virtual images. It can represent eye positions by setting a two-dimensional or three-dimensional coordinate system. In practical applications, the coordinate system of the virtual image coordinate space is usually mapped to other coordinate systems to facilitate interaction and processing in different systems and environments. For example, converting coordinates in the virtual image coordinate space to pixel coordinates on the actual screen.

[0063] Specifically, the processor can capture the real-time position data of the user's left and right eyes using eye detection devices. Then, based on the real-time positions of the user's left and right eyes, it determines the partition corresponding to the real-time position of the left eye and the partition corresponding to the real-time position of the right eye from among many pre-defined partitions in the virtual image coordinate space. This allows for targeted processing and optimization of the display effect in subsequent steps. The multiple partitions in the virtual image coordinate space are designed to more precisely correspond to different eye position data, facilitating targeted processing and optimization of the display effect.

[0064] Optionally, the system captures the real-time position data of the user's left and right eyes, and determines the target partition in multiple partitions of the virtual image coordinate space based on the real-time position data of the left and right eyes, including: capturing the real-time position of the user's left eye and the real-time position of the right eye through the driver monitoring system; and determining the target partition based on the partition in the virtual image coordinate space in which the real-time position of the left eye and the real-time position of the right eye fall.

[0065] The driver monitoring system refers to a system that monitors and tracks the real-time positions of the driver's left and right eyes. The real-time positions of the left and right eyes include the real-time pupil positions of the left and right eyes.

[0066] Specifically, the driver monitoring system can include sensors and cameras capable of accurately capturing the user's eye positions. The sensors and cameras can monitor subtle movements and changes in the user's eyes in real time, thereby determining the real-time positions of the user's left and right eyes. The processor then determines the target partitions into which the acquired real-time left and right eye positions fall based on their coordinates in the virtual image coordinate space.

[0067] For example, a positional model can be established to associate the positional data of the user's left and right eyes with the positions and ranges of virtual image coordinate space partitions, and a threshold range for the left and right eye positions can be set for each partition. When the position of the user's left and right pupils falls within the threshold range of a certain partition, that partition can be determined as the target partition. It should be noted that the determination of the target partition is not limited to which partition the pupil position is located in; it can also be determined by which partition the area of ​​the two-dimensional area of ​​the eye image falls into is larger. In addition, the target partition is dynamically adjusted, that is, as the user's eye position changes, the processor will update the positional data in real time and dynamically adjust the determination of the target partition.

[0068] Furthermore, driver monitoring systems can determine the real-time positions of a user's left and right eyes using either infrared tracking or image recognition technology. Infrared tracking technology involves emitting infrared light towards the user's eyes and then using sensors to receive the reflected light, calculating the real-time positions of the left and right eyes based on changes in light intensity and time differences. Image recognition technology involves using a camera to capture images of the user's eyes and using image processing algorithms to identify key features such as the center of the pupil and the corners of the eyes, thereby determining the real-time positions of the left and right eyes.

[0069] In one specific implementation, the driver monitoring system may include a DMS (Eye-tracking System), which is a device that uses optical and infrared cameras installed in the vehicle to acquire real-time information about the driver's eye state. It analyzes the acquired information using deep learning algorithms to determine the driver's state, enabling driver identification, driver fatigue monitoring, driver attention monitoring, and monitoring of dangerous driving behaviors, and providing different levels of warnings.

[0070] In summary, by determining the target partitions corresponding to the real-time positions of the user's left and right eyes, a display effect that better meets the user's needs can be provided, and it has strong real-time adaptability, allowing it to quickly adjust according to real-time changes in the user's eye position.

[0071] S120. Obtain the left and right eye display images to be fused and displayed, and use the distortion correction parameters corresponding to the target partition to perform distortion correction on the left and right eye display images to obtain the left and right eye corrected images.

[0072] Distortion correction parameters are matrices used to correct image distortions caused by optical systems or other factors. Different partitions correspond to different distortion correction parameters, so the distortion correction parameters corresponding to the target partition must be used for correction. Distortion correction parameters can effectively correct the image according to the characteristics of the partition, thereby obtaining a clearer and more accurate image.

[0073] It is understood that the imaging process is essentially a transformation of several coordinate systems. First, a point in space is transformed from the world coordinate system to the camera coordinate system, then it is projected onto the imaging plane (image physical coordinate system), and finally the data on the imaging plane is transformed to the image plane (image pixel coordinate system). Figure 2 is a schematic diagram of the distortion image generation process provided by Embodiment 1 of this application, and Figure 3 is a schematic diagram of the distortion image generation effect provided by Embodiment 1 of this application. In Figure 3, the left side of the arrow is the original image, and the right side of the arrow is the distorted image observed by the human eye. In Figure 2, after the original image is imaged by the curved mirror of the head-up display, a distorted image is generated due to the change in the coordinate correspondence. That is, the distortion-free coordinates (U,V) in the image pixel coordinate system (uOv coordinate system) fall on (Ud,Vd) in the uOv coordinate system after radial and tangential distortion. In other words, the relationship between the real image imgR and the distorted image imgD is: imgR(U,V)=imgD(Ud,Vd). The distortion correction process involves determining distortion correction parameters by mapping the real image to the distorted image. Through distortion correction, the distorted image can be restored to the original image.

[0074] Distortion correction in related technologies includes hardware correction and software correction. Hardware distortion correction reduces or eliminates image distortion by designing and adjusting the hardware of the imaging system. Software distortion correction corrects distortion after image acquisition using image processing algorithms. Software distortion correction can rearrange and adjust image pixels through mathematical transformations. For example, Figure 4 illustrates a hardware correction process according to Embodiment 1 of this application. In Figure 4, after the original image is input into the head-up display system, a pre-distorted image is obtained through chip correction processing, and then a normal image is displayed after projection through a curved mirror. Figure 5 illustrates a software correction process according to Embodiment 1 of this application. In Figure 5, a pre-distorted image is generated after distortion correction processing of the original image, and then a normal image is displayed after projection through a curved mirror into the head-up display system. Software correction methods correct the displayed image through image processing algorithms, but may be limited by computing power and real-time performance. Hardware correction methods require additional optical components or mechanical structures, increasing the complexity and cost of the system.

[0075] Furthermore, the technical solution of this application embodiment, when performing real-time distortion correction, can acquire images frame by frame, read the pixel value of each pixel coordinate point in the image, calculate the corresponding pixel value through mapping relationships, and output the final normal image frame by frame. This enables high-quality head-up display image correction without increasing system complexity or cost, improving the driver's visual experience and driving safety. Figure 6 is a schematic diagram of the distortion correction effect provided in Embodiment 1 of this application. In Figure 6, the image to the left of the arrow is the original image, and the image to the right of the arrow is the display image generated after distortion correction processing.

[0076] Optionally, before using the distortion correction parameters corresponding to the target partition to perform distortion correction on the left and right eye display images, the method further includes: after displaying a standard dot matrix image on a display screen, obtaining a distorted display image corresponding to each observation position by simulating the display images observed by the human eye at different observation positions; determining the partition corresponding to each observation position in the virtual image coordinate space; and generating distortion correction parameters corresponding to each partition based on the image differences between the standard dot matrix image and each distorted display image.

[0077] Specifically, after displaying a standard dot matrix image on a screen, a camera device can simulate the displayed image as seen by the human eye at different observation positions, thus obtaining a distorted display image corresponding to each observation position, and determining a corresponding partition in the virtual image coordinate space. The processor compares and analyzes the distorted display image with the standard dot matrix image, including comparing differences in pixel arrangement, shape, size, and position, to evaluate the image distortion produced by each screen partition. Based on the comparative analysis, the distortion correction parameters corresponding to each partition can be determined.

[0078] In one specific implementation, the distortion correction parameters for each partition can be determined using the following steps: First, a grid image with a regular geometric structure can be used as a standard bitmap image for head-up display. Then, a camera device simulates the distorted display image corresponding to each observation position of the human eye, and feature points are extracted from the standard bitmap image and the distorted display image. By matching the feature points, the mapping relationship between the standard bitmap image and the distorted display image can be determined, and based on the mapping relationship, a distortion parameter matrix for each partition can be further constructed as the distortion correction parameters.

[0079] Furthermore, the mapping relationship between normal images (standard raster images) and anti-distortion images (distorted display images) can be saved by constructing a mesh. Figure 7 is a schematic diagram of a distortion mapping mesh provided in Embodiment 1 of this application, showing the mapping relationship between standard raster images and distorted display images. The constructed mesh mapping relationship can be bound to a canvas. By outputting the image to the canvas, the canvas calls the corresponding distortion correction parameters for processing, thereby generating a distortion-corrected image.

[0080] In summary, once the distortion correction parameters are determined, the distortion correction parameters corresponding to the target partition can be used to correct the distortion of the image during the subsequent image display process. The corrected image can more realistically reflect the shape and proportion of the object, reduce visual distortion and deformation, and thus provide users with a more accurate display effect.

[0081] S130. Pixels at different positions are obtained from the corrected images of the left and right eyes and fused multiple times to obtain multiple fused images.

[0082] Here, a pixel refers to the smallest unit of an image. A fused image is a new image obtained by processing the corrected images for the left and right eyes according to fusion rules.

[0083] Specifically, the processor can acquire the left and right eye corrected images that need to be fused and displayed. These images can be pre-prepared or generated based on real-time application scenarios and user needs; images include, but are not limited to, pictures, videos, and video streams. The processor will select corresponding pixels from different locations in the left and right eye corrected images according to pre-set fusion rules and fuse them sequentially to obtain multiple fused images.

[0084] Optionally, pixels at different locations are obtained from the left and right eye corrected images and fused multiple times to obtain multiple fused images. This includes calling the shader plugin in the 3D image processing engine to obtain pixels at different locations from the left and right eye corrected images and fused multiple times to obtain multiple fused images.

[0085] Specifically, the processor invokes the shader plugin within the 3D image processing engine. The shader plugin is a component of the 3D image processing engine, possessing 3D image processing capabilities. The processor can invoke the shader plugin and, according to preset fusion rules, extract pixels at different locations from the left and right eye corrected images, performing multiple fusions. In each fusion process, the shader plugin combines the selected pixels according to the preset fusion rules, ultimately generating multiple fused images.

[0086] In summary, multiple fusions through shader plugins can generate multiple fused images, thereby better meeting different application needs and visual effect requirements.

[0087] S140. Based on the real-time position data of the left and right eyes and multiple fused images, obtain the target fused image and project the target fused image onto the display screen for head-up display.

[0088] Figure 8 is a schematic diagram of the head-up display imaging process provided in Embodiment 1 of this application. A head-up display (HUD) is a driver assistance instrument used in vehicles, and is a comprehensive electronic display device composed of electronic components, display components, controllers, etc. The HUD can project vehicle speed, navigation information, warning information, etc., in the form of images and characters onto the driver's front through optical components. The HUD allows images or information to be directly projected into the user's field of vision, enabling the user to see relevant content without looking down.

[0089] Figure 9 is a schematic diagram of the naked-eye 3D imaging principle provided in Embodiment 1 of this application. Naked-eye 3D display technology refers to a 3D display technology that allows users to directly view three-dimensional images with the naked eye without wearing special 3D glasses, presenting a 3D effect. The front-end display part of the naked-eye 3D display technology needs to calculate through pixel imaging units so that the left and right eyes see different images, which are then merged into a 3D image in the brain. In Figure 9, through optical design of the HUD, the user's left eye sees image P1 through the imaging structure, such as the windshield, while the right eye sees image P2. Due to the binocular parallax formed by images P1 and P2, the objects seen by the user have a sense of depth and space, and the image seen by the right eye is synthesized into a stereoscopic image with a sense of depth in the user's brain. Naked-eye 3D imaging can adjust the binocular parallax by changing the position between two images, thus changing the perceived distance of the virtual image (the actual distance of the virtual image remains unchanged). The closer the two images are, the closer the perceived distance of the virtual image; conversely, the farther the two images are, the farther the perceived distance of the virtual image.

[0090] Optionally, based on real-time position data of the left and right eyes and multiple fused images, a target fused image is obtained, including: based on the mapping relationship between the real-time position data of the left and right eyes and the pre-established region and fusion round, obtaining the fusion round corresponding to each region on the display screen; and using the fused images under the fusion round corresponding to each region, stitching together to obtain the target fused image.

[0091] Specifically, each location data point has a corresponding preset fusion rule. This rule includes specifying the nth time each region's image will be fused for display. The processor, based on real-time left and right eye position data and a pre-established mapping between regions and fusion rounds, determines the fusion round corresponding to each region on the display screen. Then, the fused images corresponding to each region's fusion round are stitched together to obtain the target fused image. It's important to note that the stitching process considers pixel alignment and matching to ensure the stitched target fused image is visually continuous and consistent.

[0092] Specific application scenario: Figure 10 is a schematic diagram of the image distortion correction head-up display process provided by this application. As shown in Figure 10, the position information of the left and right eyes is obtained in real time through the DMS device, and the distortion correction matrix parameters of the corresponding different regions are used to correct the distortion of the input material images for the left and right eyes. The image fusion rules and display content are switched in real time according to the corrected image and the monitored human eye position, thereby achieving a naked-eye 3D effect in AR HUD display. By combining the eye box position partitioning scheme of 3DHUD, the left and right fused images are calculated in real time, and the target fusion object is determined in the target screen partition for dynamic head-up display to the user.

[0093] The head-up display technology described in this embodiment can be applied to various driving scenarios involving vehicles. For example, in car driving, it can display important data such as vehicle speed, navigation information, vehicle fault warnings, and traffic sign recognition on the windshield, allowing drivers to keep their eyes on the road without looking down at the dashboard, thus improving driving safety. In aircraft piloting, the pilot's helmet or cockpit windshield can display key information such as flight altitude, speed, heading, and target distance, helping the pilot quickly obtain important data during flight and reducing distractions. In cycling, such as motorcycle riding, speed, RPM, and navigation guidance can be displayed on the helmet visor, allowing the rider to keep their eyes on the dashboard while riding.

[0094] The technical solution of this application, by capturing real-time position data of the user's left and right eyes, can dynamically adjust the displayed content according to the user's viewing angle. By correcting the left and right eye display images using distortion correction parameters corresponding to the target partition, image distortion can be eliminated, making the image more realistic and accurate, and improving visual quality. By obtaining pixels from different positions in the corrected left and right eye images and fusing them multiple times, the image details and three-dimensionality can be enhanced, providing a more realistic and vivid visual effect. By projecting the target fused image onto the display screen for head-up display, users can obtain important information without looking down at the instrument panel, improving driving safety.

[0095] Example 2

[0096] Figure 11 is a flowchart of a left-right eye fusion image head-up display method based on distortion correction provided in Embodiment 2 of this application. This embodiment adds a specific process to Embodiment 1 above, where pixels at different positions are obtained from the left and right eye corrected images and fused multiple times to obtain multiple fused images. The specific content of steps S210-S220 is largely the same as steps S110-S120 in Embodiment 1, therefore it will not be described again in this embodiment. As shown in Figure 2, the method includes:

[0097] S210. Capture the real-time position data of the user's left and right eyes, and determine the target partition among multiple partitions in the virtual image coordinate space based on the real-time position data of the left and right eyes.

[0098] Optionally, the system captures the real-time position data of the user's left and right eyes, and determines the target partition in multiple partitions of the virtual image coordinate space based on the real-time position data of the left and right eyes, including: capturing the real-time position of the user's left eye and the real-time position of the right eye through the driver monitoring system; and determining the target partition based on the partition in the virtual image coordinate space in which the real-time position of the left eye and the real-time position of the right eye fall.

[0099] S220. Obtain the left and right eye display images to be fused and displayed, and use the distortion correction parameters corresponding to the target partition to perform distortion correction on the left and right eye display images to obtain the left and right eye corrected images.

[0100] Optionally, before using the distortion correction parameters corresponding to the target partition to perform distortion correction on the left and right eye display images, the method further includes: after displaying a standard dot matrix image on a display screen, obtaining a distorted display image corresponding to each observation position by simulating the display images observed by the human eye at different observation positions; determining the partition corresponding to each observation position in the virtual image coordinate space; and generating distortion correction parameters corresponding to each partition based on the image differences between the standard dot matrix image and each distorted display image.

[0101] S230. Determine the number of times the image is fused, and divide the display screen into multiple regions based on the number of fusions, wherein the number of regions matches the number of fusions.

[0102] S240. Expand each pixel of the corrected images for the left and right eyes into RGB channel values ​​to obtain the RGB channel layout diagram for the left eye and the RGB channel layout diagram for the right eye.

[0103] Specifically, the processor will expand each pixel in the left-eye corrected image and the right-eye corrected image into RGB channel values ​​to describe the color composition of each pixel.

[0104] S250. Based on the RGB channel layout diagram of the left eye and the RGB channel layout diagram of the right eye, determine the fused image under each fusion round corresponding to the number of fusions.

[0105] Specifically, the processor performs each fusion round sequentially based on the RGB channel layout of the left and right eyes, according to a pre-set number of fusion cycles. In each fusion round, the RGB channel values ​​of the left and right eyes are extracted and combined according to the set fusion strategy.

[0106] Optionally, based on the left-eye RGB channel layout map and the right-eye RGB channel layout map, determine the fused image for each fusion round corresponding to the number of fusions, including: generating a full-eye RGB channel layout map to be filled corresponding to the fused image; determining the fusion strategy corresponding to each fusion round, and according to the fusion strategy, selecting RGB channel values ​​from the left-eye RGB channel layout map and the right-eye RGB channel layout map to fill the matching image positions in the full-eye RGB channel layout map to be filled, thereby obtaining the fused image for each fusion round.

[0107] The image to be filled with the full eye RGB channel layout is a blank image with the same size and pixel layout as the fused image, used to store the RGB channel values ​​selected during the fusion process.

[0108] Specifically, the processor can select the corresponding RGB channel values ​​from the left-eye and right-eye RGB channel layout images in each fusion round according to the fusion strategy, and fill them into the corresponding positions in the full-eye RGB channel layout image to be filled. It should be noted that the fused image obtained in each fusion is a complete full-eye RGB channel layout image, and the processor can perform different degrees of fusion in each fusion round according to the fusion strategy.

[0109] Optional fusion strategies include: the number of RGB channel value groups, the number of indents after the first line, and the number of vertical indents.

[0110] The values ​​involved in the fusion strategy need to be adapted to the arrangement of the beam splitter. The number of RGB channel value groups is related to the number of pixels covered by the grating unit. For example, when one grating unit can cover 8 pixels (i.e., the backlight LEDs), the number of RGB channel value groups needs to be set to 4. In this case, out of every 8 LEDs, the 4 LEDs on the left and the 4 LEDs on the right will shine in different directions, thus being seen by the left and right eyes respectively.

[0111] Specifically, in each round, the different indentation bits of different rows are also to adapt to various different beam splitter grating arrangements. By using the tilted grating arrangement, crosstalk can be effectively reduced.

[0112] It's important to note that the indentation in different rounds is primarily for traversing various states. Different states correspond to the display of pixels (LED beads) covered by the raster. It's crucial to understand that there's an upper limit to the number of states. For example, in a group of four, if each round indents by one position, the maximum number of states is reached after eight rounds. In this case, the ninth round would be identical to the first. Therefore, the upper limit of the number of rounds is essentially the upper limit of the number of states. When selecting the number of fusion rounds for partitioning, it's essentially choosing the corresponding states for the fusion operation. The number of fusion rounds needs to be selected based on specific needs and conditions to achieve the best fusion effect and the desired goal.

[0113] Optionally, according to the fusion strategy, RGB channel values ​​are selected from the left-eye RGB channel layout image and the right-eye RGB channel layout image to fill the matching image positions in the full-eye RGB channel layout image to be filled, respectively, to obtain the fused image under each fusion round. This includes: obtaining the current fusion strategy corresponding to the current fusion round, and determining the initial filling method of the first row corresponding to the full-eye RGB channel layout image to be filled based on the number of RGB channel value groups in the current fusion strategy; adjusting the initial filling method of the first row according to the number of indentations after the first row corresponding to the current fusion round to obtain the target filling method of the first row; determining each non-first row filling method sequentially starting from the second row based on the target filling method of the first row and the number of vertical indentations; and selecting RGB channel values ​​from the left-eye RGB channel layout image and the right-eye RGB channel layout image to fill the matching image positions in the full-eye RGB channel layout image to be filled, based on the target filling method of the first row and each non-first row filling method, to obtain the fused image under the current fusion round.

[0114] Specifically, based on the number of RGB channel value groups, it can be determined which channel values ​​to select from the left and right eye RGB channel layout maps to fill the first row of pixels in the full-eye RGB channel layout map to be filled. Then, based on the indentation number of the first row in the current fusion round, the fill position of the first row is shifted to the right by a certain number of positions to adjust the initial fill method of the first row. Starting from the second row, the processor can determine the fill method for each non-first row sequentially based on the target fill method of the first row and the vertical indentation number, where the vertical indentation number determines the vertical offset of the fill position of each row relative to the previous row.

[0115] Furthermore, based on the first row target filling method and each non-first row filling method, the corresponding RGB channel values ​​are sequentially selected from the left and right eye RGB channel layout images and filled into the corresponding pixels in the full eye RGB channel layout image to be filled. This completes the image filling and fusion for the current fusion round, resulting in the fused image for that round. By repeating the above steps for multiple fusion rounds, the processor can ultimately obtain the fused images corresponding to each fusion round.

[0116] Optionally, in each fusion strategy under different fusion rounds, the number of RGB channel value groups and the number of vertical indentation bits are the same; in adjacent fusion rounds, the number of first-line indentation bits in the later fusion round is 1 more than the number of first-line indentation bits in the previous fusion round.

[0117] In different fusion rounds, the same number of RGB channel value groups and vertical indentation means that the number of RGB channel values ​​fused in each fusion round and the vertical spacing between rows remain constant. In adjacent fusion rounds, the indentation of the first line of the later fusion round being one more than that of the previous fusion round means that as the fusion round increases, the rightward indentation of the first line increases by one.

[0118] Specifically, the same number of RGB channel value groups ensures consistency in the division and processing of color channel values ​​in each fusion round, thereby maintaining the stability and continuity of color information processing throughout the fusion process.

[0119] Optionally, the number of RGB channel value groups is 4, and the vertical indentation is 1 bit.

[0120] Specifically, a grouping of 4 RGB channel values ​​means that each horizontal row consists of four RGB values, and a vertical indentation of 1 means that each row of RGB values ​​is indented one position downwards. By sequentially grouping the horizontal RGB values ​​into groups of four and indenting each row of RGB values ​​one position downwards, the left and right eye images are fused N times. Each fusion indents one position from the first row, and the resulting N fused images are saved. N is the total number of fusion operations corresponding to the number of screen partitions.

[0121] S260. Based on the real-time position data of the left and right eyes, obtain the fusion round corresponding to each region on the display screen in the pre-established mapping relationship between regions and fusion rounds.

[0122] S270. Using the fusion images corresponding to the fusion rounds for each region, stitch together to obtain the target fusion image.

[0123] The technical solution of this application embodiment expands each pixel of the left and right eye corrected images to obtain the RGB channel layout diagrams of the left and right eyes, which allows for a more intuitive understanding of the differences and distribution of color channels in the left and right eye images, enabling the fused image to better integrate information from both eyes. Fusing the left and right eye corrected images enhances the stereoscopic and depth perception of the image, allowing users to see a three-dimensional image.

[0124] Example 3

[0125] Figure 12 is a schematic diagram of a head-up display device based on distortion correction for left and right eye fusion images provided in Embodiment 3 of this application. As shown in Figure 12, the device includes: a target partition determination module 310, used to capture the real-time position data of the user's left and right eyes, and determine the target partition in multiple partitions of the virtual image coordinate space based on the real-time position data of the left and right eyes;

[0126] The image distortion correction module 320 is used to acquire the left and right eye display images to be fused and displayed, and to use the distortion correction parameters corresponding to the target partition to correct the distortion of the left and right eye display images to obtain the corrected left and right eye images.

[0127] The image fusion module 330 is used to obtain pixels from different positions in the left and right eye corrected images and fuse them multiple times to obtain multiple fused images;

[0128] The target fusion image determination and display module 340 is used to obtain a target fusion image based on real-time position data of the left and right eyes and multiple fusion images, and project the target fusion image onto the display screen for head-up display.

[0129] Optionally, the device further includes: a distortion correction parameter generation module, used to, before using the distortion correction parameters corresponding to the target partition to perform distortion correction on the left and right eye display images, display a standard dot matrix image on the display screen, and obtain a distorted display image corresponding to each observation position by simulating the display images observed by the human eye at different observation positions; determine the partition corresponding to each observation position in the virtual image coordinate space; and generate distortion correction parameters corresponding to each partition based on the image differences between the standard dot matrix image and each distorted display image.

[0130] Optionally, the target partition determination module 310 is specifically used to: capture the user's real-time left eye position and real-time right eye position through the driver monitoring system; and determine the target partition based on the partitions that the real-time left eye position and real-time right eye position fall into in the virtual image coordinate space.

[0131] Optionally, the device further includes: a display screen division module, used to determine the number of times the fused images are fused before obtaining multiple fused images by fusing pixels at different positions from the left and right eye corrected images, and to divide the display screen into multiple regions according to the number of fusions; wherein the number of regions matches the number of fusions.

[0132] Optionally, the image fusion module 330 specifically includes: an RGB layout map generation unit, used to expand each pixel of the left and right eye corrected images into RGB channel values ​​respectively, to obtain the left eye RGB channel layout map and the right eye RGB channel layout map; and a fused image generation unit, used to determine the fused image under each fusion round corresponding to the number of fusions based on the left eye RGB channel layout map and the right eye RGB channel layout map.

[0133] Optionally, the fused image generation unit specifically includes: a full-eye RGB channel layout generation subunit, used to generate a full-eye RGB channel layout map to be filled corresponding to the fused image; and an image fusion filling subunit, used to determine the fusion strategy corresponding to each fusion round, and according to the fusion strategy, select RGB channel values ​​from the left-eye RGB channel layout map and the right-eye RGB channel layout map respectively to fill the matching image positions in the full-eye RGB channel layout map to be filled, thereby obtaining the fused image under each fusion round.

[0134] Optionally, the image fusion filling subunit is specifically used for: obtaining the current fusion strategy corresponding to the current fusion round, and determining the initial filling method of the first row corresponding to the full-eye RGB channel layout image to be filled based on the number of RGB channel value groups in the current fusion strategy; adjusting the indentation of the initial filling method of the first row based on the number of indentations of the first row corresponding to the current fusion round to obtain the target filling method of the first row; determining each non-first row filling method sequentially starting from the second row based on the target filling method of the first row and the number of vertical indentations; selecting RGB channel values ​​from the left-eye RGB channel layout image and the right-eye RGB channel layout image to fill the matching image position in the full-eye RGB channel layout image to be filled based on the target filling method of the first row and each non-first row filling method to obtain the fused image under the current fusion round.

[0135] Optionally, the target fusion image determination and display module 340 is specifically used to: obtain the fusion round corresponding to each region on the display screen based on the mapping relationship between the real-time position data of the left and right eyes and the pre-established region and fusion round; and stitch together the fusion images under the fusion rounds corresponding to each region to obtain the target fusion image.

[0136] Optionally, the image fusion module 330 is specifically used to: call the shader plugin in the 3D image processing engine to obtain pixels from different positions in the left and right eye corrected images and fuse them multiple times to obtain multiple fused images.

[0137] The technical solution of this application, by capturing real-time position data of the user's left and right eyes, can dynamically adjust the displayed content according to the user's viewing angle. By correcting the left and right eye display images using distortion correction parameters corresponding to the target partition, image distortion can be eliminated, making the image more realistic and accurate, and improving visual quality. By obtaining pixels from different positions in the corrected left and right eye images and fusing them multiple times, the image details and three-dimensionality can be enhanced, providing a more realistic and vivid visual effect. By projecting the target fused image onto the display screen for head-up display, users can obtain important information without looking down at the instrument panel, improving driving safety.

[0138] The left-right eye fusion image head-up display device based on distortion correction provided in this application embodiment can execute the left-right eye fusion image head-up display method based on distortion correction provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects of executing the method.

[0139] Example 4

[0140] Figure 13 shows a schematic diagram of the structure of an electronic device 10 that can be used to implement embodiments of this application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the application described and / or claimed herein.

[0141] As shown in Figure 13, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded into the RAM 13 from storage unit 18. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0142] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0143] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as a distortion correction-based left-right eye fusion image head-up display method.

[0144] In some embodiments, a distortion-corrected left-right eye fusion image head-up display method can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the distortion-corrected left-right eye fusion image head-up display method described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform a distortion-corrected left-right eye fusion image head-up display method by any other suitable means (e.g., by means of firmware).

[0145] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0146] Computer programs used to implement the methods of this application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0147] In the context of this application, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0148] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0149] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0150] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

Claims

1. A distortion correction-based left and right eye fusion image head-up display method, comprising: capturing real-time left and right eye position data of a user, and determining a target partition in a plurality of partitions in a virtual image coordinate space according to the real-time left and right eye position data; obtaining left and right eye display images to be fused and displayed, and performing distortion correction on the left and right eye display images using distortion correction parameters corresponding to the target partition to obtain left and right eye corrected images; obtaining pixel points at different positions from the left and right eye corrected images for multiple times of fusion to obtain a plurality of fusion images; obtaining a target fusion image according to the real-time left and right eye position data and the plurality of fusion images, and projecting the target fusion image into a display screen for head-up display.

2. The method of claim 1, before performing distortion correction on the left and right eye display images using distortion correction parameters corresponding to the target partition, the method further comprises: after displaying a standard dot matrix image using the display screen, obtaining distortion display images corresponding to each observation position by simulating the display images observed by a human eye at different observation positions; determining partitions corresponding to each observation position in the virtual image coordinate space; generating distortion correction parameters corresponding to each partition according to the image difference between the standard dot matrix image and each distortion display image.

3. The method of claim 1, wherein, capturing real-time left and right eye position data of a user, and determining a target partition in a plurality of partitions in a virtual image coordinate space according to the real-time left and right eye position data, comprising: capturing real-time left eye position and real-time right eye position of a user through a driver monitoring system; determining a target partition according to the partitions in which the real-time left eye position and the real-time right eye position fall in the virtual image coordinate space.

4. The method of claim 1, before obtaining pixel points at different positions from the left and right eye corrected images for multiple times of fusion to obtain a plurality of fusion images, the method further comprises: determining the number of fusion times of the fusion image, and dividing the display screen into a plurality of regions according to the number of fusion times; wherein the number of regions matches the number of fusion times.

5. The method of claim 4, wherein, obtaining pixel points at different positions from the left and right eye corrected images for multiple times of fusion to obtain a plurality of fusion images, comprising: expanding each pixel point of the left and right eye corrected images into RGB channel values respectively to obtain left eye RGB channel arrangement images and right eye RGB channel arrangement images; determining fusion images under each fusion round corresponding to the number of fusion times according to the left eye RGB channel arrangement images and the right eye RGB channel arrangement images.

6. The method of claim 5, wherein, determining fusion images under each fusion round corresponding to the number of fusion times according to the left eye RGB channel arrangement images and the right eye RGB channel arrangement images, comprising: generating a to-be-filled full-eye RGB channel arrangement image corresponding to the fusion image; determining a fusion strategy corresponding to each fusion round, and selecting RGB channel values from the left eye RGB channel arrangement images and the right eye RGB channel arrangement images according to the fusion strategy to fill into the matching image positions of the to-be-filled full-eye RGB channel arrangement image to obtain the fusion images under each fusion round.

7. The method of claim 6, wherein, The fusion strategy includes: RGB channel value grouping quantity, first row post-indentation bit number and vertical post-indentation bit number.

8. The method of claim 7, wherein, According to the fusion strategy, RGB channel values are selected from the left eye RGB channel arrangement diagram and the right eye RGB channel arrangement diagram and filled into matched image positions in the full-eye RGB channel arrangement diagram to be filled to obtain a fusion image under each fusion round, including: A current fusion strategy corresponding to a current fusion round is acquired, and a first row initial filling mode corresponding to the full-eye RGB channel arrangement diagram to be filled is determined according to the RGB channel value grouping quantity in the current fusion strategy; According to the first row post-indentation bit number corresponding to the current fusion round, the first row initial filling mode is adjusted by post-indentation to obtain a first row target filling mode; According to the first row target filling mode and the vertical post-indentation bit number, each non-first row filling mode is sequentially determined starting from the second row; According to the first row target filling mode and each non-first row filling mode, RGB channel values are selected from the left eye RGB channel arrangement diagram and the right eye RGB channel arrangement diagram and filled into matched image positions in the full-eye RGB channel arrangement diagram to be filled to obtain a fusion image under the current fusion round.

9. The method of claim 7, wherein, In each fusion strategy under different fusion rounds, the RGB channel value grouping quantity and the vertical post-indentation bit number are the same; In adjacent fusion rounds, the first row post-indentation bit number of the next fusion round is 1 more than the first row post-indentation bit number of the previous fusion round.

10. The method of claim 9, wherein, The RGB channel value grouping quantity is 4, and the vertical post-indentation bit number is 1.

11. The method of claim 4, wherein, According to the left and right eye real-time position data and the plurality of fusion images, a target fusion image is acquired, including: According to the mapping relationship between the left and right eye real-time position data and the fusion rounds in a pre-established region, a fusion round corresponding to each region in the display screen is acquired; Using the fusion image under the fusion round corresponding to each region, the target fusion image is spliced to obtain.

12. The method of any one of claims 1-11, wherein, From the left and right eye corrected images, pixel points at different positions are respectively acquired for multiple times of fusion to obtain a plurality of fusion images, including: A shader plug-in in a three-dimensional image processing engine is called to acquire pixel points at different positions from the left and right eye corrected images for multiple times of fusion to obtain a plurality of fusion images.

13. A left and right eye fusion image head-up display device based on distortion correction, comprising: A target partition determination module is configured to capture left and right eye real-time position data of a user, and determine a target partition in a plurality of partitions of a virtual image coordinate space according to the left and right eye real-time position data; An image distortion correction module is configured to acquire left and right eye display images to be fused, and correct the left and right eye display images by using distortion correction parameters corresponding to the target partition to obtain left and right eye corrected images; An image fusion module is configured to acquire pixel points at different positions from the left and right eye corrected images for multiple times of fusion to obtain a plurality of fusion images; A target fusion image determination and display module is configured to acquire a target fusion image according to the left and right eye real-time position data and the plurality of fusion images, and project the target fusion image into a display screen for head-up display.

14. A carrier comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-12.

15. A computer storage medium storing computer instructions for causing a processor to implement the method of any one of claims 1-12 when executed.

Images