Display system, display method and vehicle-mounted system for binocular distortion correction

Abstract

The application provides a display system, a display method and a vehicle-mounted system for binocular distortion correction, which can be applied to intelligent cars, networked cars and self-driving cars. The display system comprises an image generator, a binocular disparity generator and an optical lens group. The image generator is configured to generate a left-eye pre-distortion image and a right-eye pre-distortion image, and interleave pixels in the left-eye pre-distortion image with pixels in the right-eye pre-distortion image to generate a pixel-interleaved composite image. The binocular disparity generator is configured to make the pixel-interleaved composite image pass through the optical lens group and be pushed to the binocular vision of an observer. In the pixel-interleaved composite image, the pixels of the left-eye pre-distortion image are pushed to the left eye of the observer, and the pixels of the right-eye pre-distortion image are pushed to the right eye of the observer. The application can make the binocular vision simultaneously receive non-distorted images and coincide, thereby improving the overall visual experience.

Description

Technical Field

[0001] This application relates to the field of image processing, and more specifically, to a display system, display method, and vehicle-mounted system for binocular distortion correction. Background Technology

[0002] Known visual display systems generally include an image generation unit (PGU), a projector, and an optical lens assembly. The projector receives and projects an image supplied by the image generator, and then the light path passes through a precisely designed optical lens assembly, ultimately reflecting the image onto the observer's field of vision—the eyebox area.

[0003] However, due to design limitations of components in the display system, such as irregular curvature of the reflector or deviations in assembly process, the image is distorted to varying degrees. Furthermore, since the observer's two eyes are located at different positions in the eye box, the two eyes simultaneously receive images with different distortions without overlapping, ultimately greatly diminishing the overall visual experience.

[0004] Therefore, improving the overall visual experience is an urgent problem to be solved. Summary of the Invention

[0005] This application provides a display system, display method, and vehicle-mounted system for binocular distortion correction, which can improve the overall visual experience.

[0006] In a first aspect, a display system for binocular distortion correction is provided, comprising: an image generator, a binocular parallax generator, and an optical lens assembly; the image generator is used to generate a left-eye pre-distortion image and a right-eye pre-distortion image, and to interleave pixels in the left-eye pre-distortion image with pixels in the right-eye pre-distortion image to generate a pixel interleaving composite image; the binocular parallax generator causes the pixel interleaving composite image to be pushed to the observer's binoculars through the optical lens assembly, wherein pixels of the left-eye pre-distortion image in the pixel interleaving composite image are pushed to the observer's left eye, and pixels of the right-eye pre-distortion image in the pixel interleaving composite image are pushed to the observer's right eye.

[0007] The binocular distortion correction display system provided in this application includes an image generator, a binocular parallax generator, and an optical lens assembly. The image generator generates a left-eye pre-distortion image and a right-eye pre-distortion image, and interweaves pixels from the left-eye pre-distortion image with pixels from the right-eye pre-distortion image to generate a pixel-interlaced composite image. The binocular parallax generator pushes pixels from the left-eye and right-eye pre-distortion images in the pixel-interlaced composite image to the observer's left and right eyes respectively through the optical lens assembly. This solution allows both eyes to simultaneously receive distortion-free images that overlap, thereby improving the overall visual experience.

[0008] In conjunction with the first aspect, in some implementations of the first aspect, the image generator is further configured to: overlap the left pre-distorted image and the right pre-distorted image; determine an envelope range that covers the valid pixels in the left pre-distorted image and the right pre-distorted image; and interweave the pixels in the left pre-distorted image and the pixels in the right pre-distorted image within the envelope range to generate a pixel interleaved composite image.

[0009] In conjunction with the first aspect, in some implementations of the first aspect, the display system further includes: a distortion corrector; the distortion corrector is used to acquire a left-eye distortion model and a right-eye distortion model, the left-eye distortion model being generated based on a target source image sample and an uncorrected left-eye distortion image sample, and the right-eye distortion model being generated based on a target source image sample and an uncorrected right-eye distortion image sample; the image generator is further used to acquire a target source image; and to generate a left-eye pre-distorted image and a right-eye pre-distorted image based on the target source image, the left-eye distortion model, and the right-eye distortion model.

[0010] In conjunction with the first aspect, in some implementations of the first aspect, the target source image is a source image taken at the target viewpoint; the image generator is also used to generate the left eye pre-distortion image based on the target source image and the left eye distortion model, and to generate the right eye pre-distortion image based on the target source image and the right eye distortion model.

[0011] Optionally, the target viewpoint can be a viewpoint between the left and right eyes, or it can be other views; this application does not limit this.

[0012] In the embodiments of this application, the target source image is a source image taken at the target viewpoint, and a left-eye pre-distortion image and a right-eye pre-distortion image are obtained by pre-distortion processing based on the same source image (i.e., the target source image), so that the imaging observed by both eyes is a distortion-free two-dimensional 2D imaging.

[0013] In conjunction with the first aspect, in some implementations of the first aspect, the target source image includes a left-eye view target source image and a right-eye view target source image; the image generator is further configured to generate the left-eye pre-distorted image based on the left-eye view target source image and the left-eye distortion model, and to generate the right-eye pre-distorted image based on the right-eye view target source image and the right-eye distortion model.

[0014] In the embodiments of this application, the target source image includes a target source image from the left eye view and a target source image from the right eye view. The images from different viewpoints (i.e., the target source image from the left eye view and the target source image from the right eye view) are subjected to pre-distortion processing to obtain a pre-distorted image from the left eye and a pre-distorted image from the right eye, so that the imaging observed by both eyes is a distortion-free three-dimensional 3D imaging.

[0015] In conjunction with the first aspect, in some implementations of the first aspect, the display system further includes: an eye tracker; the eye tracker is used to detect the coordinates of the observer's binoculars; the distortion corrector is also used to obtain the left eye distortion model and the right eye distortion model based on the coordinates of the observer's binoculars.

[0016] In this embodiment of the application, the display system may further include: an eye tracker, which can be used to detect the coordinates of the observer's two eyes, so as to obtain the left eye distortion model and the right eye distortion model according to the coordinates of the two eyes, so that the left eye distortion model and the right eye distortion model can be updated in real time according to the coordinates of the two eyes, thereby enabling the observer to always see distortion-free images at different positions in the eye box, which can further improve the overall visual experience.

[0017] In conjunction with the first aspect, in some implementations of the first aspect, the uncorrected left-eye distortion image sample and the uncorrected right-eye distortion image sample are obtained based on the spatial transformation relationship between the coordinates of the observer's binoculars and the coordinates of the target position on the final imaging plane, the target position being the intersection of the extension lines of the left-eye line of sight and the right-eye line of sight.

[0018] In conjunction with the first aspect, in some implementations of the first aspect, the optical mirror assembly includes a plane mirror and a freeform mirror.

[0019] In this embodiment of the application, the optical lens assembly may also include a self-curved mirror, which has the function of imaging magnification and expanding the eye box range, so that the observer can enjoy a better viewing experience.

[0020] In conjunction with the first aspect, in some implementations of the first aspect, the binocular parallax generator is a parallax barrier or a directional light source component.

[0021] In conjunction with the first aspect, in some implementations of the first aspect, the parallax barrier is an aperture stop or a cylindrical lens.

[0022] In conjunction with the first aspect, in some implementations of the first aspect, the display system is an office entertainment display system or a vehicle head-up display system.

[0023] Secondly, a display method for binocular distortion correction is provided, comprising: generating a left-eye pre-distortion image and a right-eye pre-distortion image; interleaving pixels in the left-eye pre-distortion image with pixels in the right-eye pre-distortion image to generate a pixel interleaving composite image; and, under the action of a binocular parallax generator, pushing the pixel interleaving composite image through an optical lens group to the observer's binoculars, wherein pixels of the left-eye pre-distortion image in the pixel interleaving composite image are pushed to the observer's left eye, and pixels of the right-eye pre-distortion image in the pixel interleaving composite image are pushed to the observer's right eye.

[0024] In conjunction with the second aspect, in some implementations of the second aspect, the step of interleaving the pixels in the left pre-distorted image and the pixels in the right pre-distorted image to generate a pixel interleaved composite image includes: overlapping the left pre-distorted image and the right pre-distorted image; determining an envelope range that covers the effective pixels in the left pre-distorted image and the right pre-distorted image; and interleaving the pixels in the left pre-distorted image and the pixels in the right pre-distorted image within the envelope range to generate a pixel interleaved composite image.

[0025] In conjunction with the second aspect, in some implementations of the second aspect, the method further includes: obtaining a left-eye distortion model and a right-eye distortion model, wherein the left-eye distortion model is generated based on a target source image sample and an uncorrected left-eye distortion image sample, and the right-eye distortion model is generated based on a target source image sample and an uncorrected right-eye distortion image sample; generating the left-eye pre-distortion image and the right-eye pre-distortion image includes: obtaining a target source image; generating the left-eye pre-distortion image and the right-eye pre-distortion image based on the target source image, the left-eye distortion model, and the right-eye distortion model.

[0026] In conjunction with the second aspect, in some implementations of the second aspect, the target source image is a source image captured at the target viewpoint; generating the left-eye pre-distorted image and the right-eye pre-distorted image based on the target source image, the left-eye distortion model, and the right-eye distortion model includes: generating the left-eye pre-distorted image based on the target source image and the left-eye distortion model, and generating the right-eye pre-distorted image based on the target source image and the right-eye distortion model.

[0027] In conjunction with the second aspect, in some implementations of the second aspect, the target source image includes a left-eye view target source image and a right-eye view target source image; generating the left-eye pre-distorted image and the right-eye pre-distorted image based on the target source image, the left-eye distortion model, and the right-eye distortion model includes: generating the left-eye pre-distorted image based on the left-eye view target source image and the left-eye distortion model, and generating the right-eye pre-distorted image based on the right-eye view target source image and the right-eye distortion model.

[0028] In conjunction with the second aspect, in some implementations of the second aspect, the method further includes: detecting the binocular coordinates of the observer; obtaining the left eye distortion model and the right eye distortion model includes: obtaining the left eye distortion model and the right eye distortion model based on the binocular coordinates of the observer.

[0029] In conjunction with the second aspect, in some implementations of the second aspect, the uncorrected left-eye distortion image sample and the uncorrected right-eye distortion image sample are obtained based on the spatial transformation relationship between the coordinates of the observer's binoculars and the coordinates of the target position on the final imaging plane, where the target position is the intersection of the extension lines of the left-eye line of sight and the right-eye line of sight.

[0030] In conjunction with the second aspect, in some implementations of the second aspect, the optical mirror assembly includes a plane mirror and a freeform mirror.

[0031] In conjunction with the second aspect, in some implementations of the second aspect, the binocular parallax generator is a parallax barrier or a directional light source component.

[0032] In conjunction with the second aspect, in some implementations of the second aspect, the parallax barrier is an aperture stop or a cylindrical lens.

[0033] Thirdly, a controller is provided, including an input / output interface, a processor, and a memory. The processor controls the input / output interface to send and receive signals or information, the memory stores a computer program, and the processor calls and runs the computer program from the memory, causing the controller to perform a display method for binocular distortion correction as described in the second aspect or any possible implementation thereof.

[0034] Fourthly, an in-vehicle system is provided, including a display system for binocular distortion correction as described in the first aspect or any possible implementation thereof.

[0035] Fifthly, a desktop display system is provided, including a display system for binocular distortion correction as described in the first aspect or any possible implementation thereof.

[0036] A sixth aspect provides a vehicle including a display system for binocular distortion correction as described in the first aspect or any possible implementation thereof.

[0037] A seventh aspect provides a computing device comprising: at least one processor and a memory, the at least one processor being coupled to the memory for reading and executing instructions in the memory to perform a display method for binocular distortion correction as described in the second aspect or any possible implementation thereof.

[0038] Eighthly, a computer program product containing instructions is provided, which, when run on a computer, causes the computer to perform the display method for binocular distortion correction described in the second aspect or any possible implementation thereof.

[0039] A ninth aspect provides a computer-readable storage medium for storing a computer program, the computer program including instructions for performing a display method for binocular distortion correction as described in the second aspect or any possible implementation thereof.

[0040] In a tenth aspect, a computer program is provided, the computer program including instructions for performing a display method for binocular distortion correction in the second aspect or any possible implementation of the second aspect.

[0041] Eleventhly, a chip is provided, the chip including a processor and a data interface, the processor reading instructions stored in a memory through the data interface to execute the display method for binocular distortion correction in the second aspect or any possible implementation of the second aspect.

[0042] Optionally, as one implementation, the chip may further include a memory storing instructions, and the processor is configured to execute the instructions stored in the memory. When the instructions are executed, the processor is configured to execute the display method for binocular distortion correction in the second aspect or any possible implementation of the second aspect.

[0043] In a twelfth aspect, a chip system is provided, the chip system including at least one processor for supporting the implementation of the functions involved in the second aspect or some implementations thereof, such as receiving or processing data and / or information involved in the methods described above.

[0044] In one possible design, the chip system further includes a memory for storing program instructions and data, which may be located within or outside the processor. The chip system may consist of chips or may include chips and other discrete devices. Attached Figure Description

[0045] Figure 1 This is a schematic diagram illustrating the principle of distortion correction for a single image provided in an embodiment of this application;

[0046] Figure 2 This is an example diagram of a display system for binocular distortion correction provided in an embodiment of this application;

[0047] Figure 3 This is an example diagram of generating a pixel interlaced composite image provided in an embodiment of this application;

[0048] Figure 4 This is an example diagram illustrating the working principle of a binocular parallax generator provided in an embodiment of this application;

[0049] Figure 5 This is a system architecture example diagram provided in an embodiment of this application;

[0050] Figure 6 This is a system architecture example diagram provided in an embodiment of this application for use in office and entertainment display scenarios;

[0051] Figure 7 This is a system architecture example diagram for an in-vehicle head-up display scenario provided in an embodiment of this application;

[0052] Figure 8 This is an example diagram of a binocular distortion correction method provided in an embodiment of this application;

[0053] Figure 9 This is an example diagram of a display method for binocular distortion correction provided in an embodiment of this application;

[0054] Figure 10 This is an exemplary block diagram of the hardware structure of a device provided in an embodiment of this application. Detailed Implementation

[0055] To facilitate understanding, the background technology involved in the embodiments of this application will be described in detail first.

[0056] Known visual display systems generally include an image generation unit (PGU), a projector, and an optical lens assembly. The projector receives and projects an image supplied by the image generator, and then the light path passes through a precisely designed optical lens assembly, ultimately reflecting the image onto the observer's field of vision—the eyebox area.

[0057] However, due to design limitations of components in the display system, such as irregular curvature of the reflector or assembly process deviations, varying degrees of image distortion occur. Furthermore, because the observer's left and right eyes are located at different positions within the eye box, both eyes simultaneously receive images with different distortions that do not overlap. Ultimately, this significantly diminishes the overall visual experience.

[0058] To address the image distortion issues caused by the limitations of the aforementioned imaging principles, the current mainstream image processing method involves correcting the distortion of the entire image seen by one eye, while the image seen by the other eye is a superposition of its own distortion effect and the pre-distortion correction processed for the first eye. However, this correction scheme is less effective at correcting image distortion when there is a significant difference in distortion between binocular imaging.

[0059] Figure 1 This is a schematic diagram illustrating the principle of distortion correction for a single image provided in an embodiment of this application. For example... Figure 1 As shown, we first define I as the original image projected by the projector, and J as the image received by the observer. Then, we can establish a mapping model M from the pixels in image J to the pixels in the original image I by using the corresponding pixel pairs between the original image I and the image J. Thus, we have:

[0060] I = M(J) (1)

[0061] If the original image I is a distortion-free image and J is an image with distortion, then the mapping model M can be denoted as the distortion model M. The specific representation of the distortion model M can then be obtained from formula (1). Next, the mapping model M is inversely mapped to obtain the inverse mapping model M from the original image to the final image. -1 .

[0062] Finally, the pre-distorted image I' in the projector is calculated according to the mapping model M obtained by formula (1). This allows the pre-distorted image I' to be projected onto the inverse mapping model M if distortion-free imaging is desired in actual operation. -1 The imaging yields a distortion-free final image J'. That is:

[0063] J'=M -1 (I') (2)

[0064] Because the observer's left and right eyes are located at different positions within the eye box, the two eyes may simultaneously receive images with different distortions that do not overlap. To address this, a distortion model can be obtained for each of the left and right views, and two pre-distorted images (left and right) can be prepared based on these models. These two pre-distorted images are then pushed to the precise positions of the two eyes to ensure that the final images overlap, thereby improving the user's visual experience.

[0065] However, existing technologies typically obtain two pre-distorted images, one for the left and one for the right, and then use an image generator to periodically switch between the left and right pre-distorted images at high speed. Then, using parallax barrier technology or directional light source technology, the left pre-distorted image is transmitted to the observer's left eye through an optical lens group in a time-sharing manner, and the right pre-distorted image is transmitted to the observer's right eye through an optical lens group.

[0066] However, the human eye requires a frame rate of 60fps or higher to perceive the transition between left and right images seamlessly. This means that the existing solution must switch between the left and right views within one frame, requiring the image generator to have a switching speed of 2 * 60Hz = 120Hz or higher. This high switching rate requirement places a significant computational burden on the data reading system, especially when processing high-definition images.

[0067] Based on this, this application provides a display system for binocular distortion correction. The display system generates a pixel interleaving composite image by interleaving the pixels in the left pre-distorted image with the pixels in the right pre-distorted image. Then, under the action of the binocular parallax generator, the pixel interleaving composite image is pushed to the binoculars in a spatially divided multiplexing manner, thereby improving the user's visual experience while reducing the computational pressure of the data reading system.

[0068] The technical solution of this application will now be described with reference to the accompanying drawings.

[0069] Figure 2 This is an example diagram of a display system for binocular distortion correction provided in an embodiment of this application. Optionally, the display system can be an office entertainment display system or a vehicle head-up display system; this application does not limit this. Figure 2 As shown, the display system 200 includes an image generator 210, a binocular parallax generator 220, and an optical lens group 230.

[0070] The image generator 210 is used to generate a left-eye pre-distortion image and a right-eye pre-distortion image.

[0071] Image generator 210 is also used to interleave pixels in the left pre-distorted image with pixels in the right pre-distorted image to generate a pixel-interleaved composite image.

[0072] The binocular parallax generator 220 is used to push the pixel interlacing composite image through the optical lens group 230 to the observer's binoculars, wherein pixels of the left pre-distorted image in the pixel interlacing composite image are pushed to the observer's left eye, and pixels of the right pre-distorted image in the pixel interlacing composite image are pushed to the observer's right eye.

[0073] Optionally, the binocular parallax generator is a parallax barrier or a directional light source component. Optionally, the binocular parallax generator can be mounted on the image generator or independent of the image generator. Optionally, the parallax barrier is an aperture stop or a cylindrical lens.

[0074] Optionally, the image generator 220 can generate a pixel-interlaced composite image by: overlapping the left-eye pre-distortion image and the right-eye pre-distortion image; determining the envelope range, which covers the effective pixels in the left-eye pre-distortion image and the right-eye pre-distortion image; and interlacing the pixels in the left-eye pre-distortion image with the pixels in the right-eye pre-distortion image within the envelope range to generate a pixel-interlaced composite image.

[0075] Optionally, when the left and right pre-distorted images are overlapped, it can be based on the center overlap of the left and right pre-distorted images; it can also be based on the alignment of the displayed content of the left and right pre-distorted images; or it can be based on overlap at other positions. This application does not specifically limit the overlap method. However, for ease of description, it will be described below. Figure 3 The lieutenant general will use the example of center overlap to describe this.

[0076] Optionally, the pixels in the left-eye pre-distortion image and the pixels in the right-eye pre-distortion image can be interleaved in a manner of LRLR… (where L is the pixel in the left-eye pre-distortion image and R is the pixel in the right-eye pre-distortion image), or in a manner of LLRRLLRR…, or in other irregular manners such as LLRLR…. This application does not limit this, as long as it can push the pixels in the left-eye pre-distortion image in the pixel interleaving composite image to the observer's left eye and the pixels in the right-eye pre-distortion image in the pixel interleaving composite image to the observer's right eye under the action of the matching binocular disparity generator. For ease of description, in the following… Figure 3 The lieutenant general will use the first type of interlacing mentioned above as an example for description.

[0077] It should be understood that the purpose of defining the envelope is to ensure that no pixels are lost in the final image due to the misalignment of the shapes of the left and right pre-distorted images. Optionally, the envelope can be rectangular or other shapes, and this application does not limit this.

[0078] For example, Figure 3 This is an example diagram illustrating the generation of a pixel-interlaced composite image provided in an embodiment of this application. It should be understood that... Figure 3 This is merely an example and does not constitute a limitation of this application. Specifically, the centers of the left pre-distortion image 202 and the right pre-distortion image 203 are first overlapped. Next, a rectangular envelope 201 is determined, which can cover all valid pixels in the left pre-distortion image 202 and the right pre-distortion image 203, such as... Figure 3 As shown in (a) above. Finally, the image generator 210 takes the effective pixels of the left pre-distortion image 202 and the right pre-distortion image 203 within the envelope 201, and as shown in (a). Figure 3 As shown in (b), the pixels of the left pre-distortion image 202 and the right pre-distortion image 203 are interleaved according to the LRLR… interleaving method to generate a new pixel interleaving composite image 301. After obtaining the pixel interleaving composite image 301, under the action of the parallax barrier 302 matching the pixel interleaving composite image 301, the pixel interleaving composite image 301 is pushed to the observer's binoculars. Among them, the pixels of the left pre-distortion image in the pixel interleaving composite image 301 are pushed to the observer's left eye, and the pixels of the right pre-distortion image in the pixel interleaving composite image 301 are pushed to the observer's right eye, as shown in (b). Figure 4 As shown.

[0079] In this embodiment, a pixel-interlaced composite image is generated by interlacing pixels from the left-eye pre-distortion image and the right-eye pre-distortion image. Then, under the action of a binocular disparity generator, pixels from the left-eye and right-eye pre-distortion images in the pixel-interlaced composite image are pushed to the observer's binoculars in a spatially divided multiplexing manner, thereby improving the overall visual experience. Moreover, it effectively avoids the computational pressure on the data reading system caused by the high-frequency switching of left and right pre-distortion images in the time-division multiplexing method of the prior art.

[0080] Optionally, the image generator 210 can also simultaneously generate multiple (more than two) pre-distorted images and interleave the pixels in the multiple pre-distorted images to generate a pixel-interleaved composite image. This application does not limit the method of interleaving the pixels in the multiple pre-distorted images. The multiple pre-distorted images can correspond to multiple pre-distorted images from different viewpoints. Then, with the cooperation of the binocular parallax generator 230, the pixels corresponding to two of the multiple pre-distorted images are respectively pushed to the observer's binoculars. Furthermore, as the observer's binoculars move, the two pre-distorted images projected to the binoculars continuously change. For example, the image generator can simultaneously generate pre-distorted image 1, pre-distorted image 2, pre-distorted image 3, and pre-distorted image 4, and interleave the pixels in the above four pre-distorted images to generate a pixel-interleaved composite image. If the current binoculars are at position 1, then with the cooperation of the binocular disparity generator 230, the pixels corresponding to the pre-distorted image 1 and pre-distorted image 2 in the pixel interleaving composite image will be pushed to the observer's binoculars respectively; if the observer's binoculars move to position 2, then with the cooperation of the binocular disparity generator 230, the pixels corresponding to the pre-distorted image 3 and pre-distorted image 4 in the pixel interleaving composite image will be pushed to the observer's binoculars respectively. However, for ease of description, the following description will use the generation of the left and right pre-distorted images by the image generator as an example.

[0081] Optionally, the optical lens group 230 may include a plane mirror and a freeform mirror, as detailed in the description of the optical lens group in system architectures 500 to 700 below, which will not be repeated here.

[0082] Optionally, the display system 200 may further include a distortion corrector, which may exist within the image generator 210 or independently of it; this application does not limit this. The distortion corrector can be used to obtain a left-eye distortion model and a right-eye distortion model. The left-eye distortion model is generated based on the target source image sample and an uncorrected left-eye distortion image sample, and the right-eye distortion model is generated based on the target source image sample and an uncorrected right-eye distortion image sample.

[0083] It should be understood that uncorrected left-eye distortion imaging samples and uncorrected right-eye distortion imaging samples can be collectively referred to as uncorrected distortion imaging samples. Uncorrected left-eye distortion imaging samples are those observed at the left-eye coordinate position, and uncorrected right-eye distortion imaging samples are those observed at the right-eye coordinate position. The left-eye coordinate position and right-eye coordinate position can also be understood as different coordinate positions within the eye box.

[0084] Optionally, the distortion corrector can obtain the left and right eye distortion models directly from the display system. It should be understood that in this case, the display system needs to generate the corresponding left and right distortion models for both eyes before leaving the factory or before use, and store them in the system so that the models can be directly read during actual use. Specifically, the left eye distortion model needs to be generated in advance based on the target source image sample and the uncorrected left eye distortion image sample, and the right eye distortion model needs to be generated based on the target source image sample and the uncorrected right eye distortion image sample (using formula (1)).

[0085] Optionally, the distortion corrector may acquire the distortion model by acquiring its own generated distortion model. It should be understood that in this case, the distortion corrector may first acquire the target source image sample, the uncorrected left eye distortion image sample, and the uncorrected right eye distortion image sample. Then, the distortion corrector generates the left eye distortion model based on the target source image sample corresponding to the left eye and the uncorrected left eye distortion image sample, and generates the right eye distortion model based on the target source image sample corresponding to the right eye and the uncorrected right eye distortion image sample (using formula (1)).

[0086] Optionally, after obtaining the left eye distortion model and the right eye distortion model, the image generator 210 can obtain the target source image and generate the left eye pre-distortion image and the right eye pre-distortion image based on the target source image, the left eye distortion model and the right eye distortion model.

[0087] It should be understood that the target source image is the source image that the display system needs to display during actual use, that is, the image that needs to be displayed to the user.

[0088] Optionally, the target source image can be a source image taken at the target viewpoint. The image generator 210 can generate a left-eye pre-distortion image based on the target source image and the left-eye distortion model, and generate a right-eye pre-distortion image based on the target source image and the right-eye distortion model (using formula (2)).

[0089] Optionally, the target viewpoint can be a viewpoint between the left and right eyes, or it can be another viewpoint; this application does not limit this. In the embodiments of this application, the target source image is a source image captured at the target viewpoint, and in this case, pre-distortion processing is performed on the same source image (i.e., the target source image) to obtain a left-eye pre-distortion image and a right-eye pre-distortion image, thereby making the imaging observed at both eyes a distortion-free 2D imaging.

[0090] Optionally, the target source image may include a left-eye view target source image and a right-eye view target source image. The image generator 210 can generate a left-eye pre-distortion image based on the left-eye view target source image and the left-eye distortion model, and generate a right-eye pre-distortion image based on the right-eye view target source image and the right-eye distortion model (using formula (2)).

[0091] Optionally, the target source image from the left eye view and the target source image from the right eye view can be obtained by taking pictures of the target source image from the left eye view and the right eye view respectively; or the target source image can be obtained by taking pictures of the target source image from one of the left eye view or the right eye view, and then the other view can be generated by combining the obtained view with the image processing algorithm; or the left eye view and the right eye view can be generated by taking pictures of other views. This application does not limit the method of obtaining the target source image from the left eye view and the target source image from the right eye view.

[0092] In the embodiments of this application, images from different perspectives (i.e., target source images from the left eye perspective and target source images from the right eye perspective) are pre-distorted to obtain left eye pre-distorted images and right eye pre-distorted images, so that the imaging observed by both eyes is distortion-free 3D imaging.

[0093] In practical applications, the observer's binocular positions are dynamic, not fixed. Using fixed left and right eye distortion models would fail to meet the observer's requirement of consistently seeing a distortion-free image within the eye box. Therefore, optionally, the display system 200 may also include an eye tracker. The eye tracker can be used to detect the coordinates of the observer's binoculars.

[0094] At this point, the distortion corrector can also be used to obtain the left and right eye distortion models based on the coordinates of the observer's binoculars.

[0095] Optionally, the distortion corrector can directly read the left-eye distortion model and right-eye distortion model corresponding to the current coordinates of the left and right eyes from the parameter table in the system based on the current coordinate information of the binoculars. It should be understood that in this case, the display system needs to generate the distortion model corresponding to different coordinate positions of the eye box in advance before leaving the factory or using it, and store it in the system so that in actual use, the corresponding left-eye distortion model can be directly read according to the left-eye coordinate position and the corresponding right-eye distortion model can be read according to the right-eye coordinate position. This means that the display system needs to generate the distortion model corresponding to different coordinate positions based on the target source image sample and the uncorrected distortion image sample corresponding to different coordinate positions of the eye box in advance before leaving the factory or using it (using formula (1)).

[0096] Optionally, the distortion corrector can acquire the distortion model by acquiring its own generated distortion model. It should be understood that in this case, the distortion corrector can first acquire the target source image sample corresponding to the current coordinate position of the left eye and the uncorrected left eye distortion image sample, as well as the target source image sample corresponding to the current coordinate position of the right eye and the uncorrected right eye distortion image sample. Then, the distortion corrector generates the left eye distortion model based on the target source image sample corresponding to the current coordinate position of the left eye and the uncorrected left eye distortion image sample, and generates the right eye distortion model based on the target source image sample corresponding to the current coordinate position of the right eye and the uncorrected right eye distortion image sample (using formula (1)). It should be understood that in this case, the system needs to save the target source image samples and uncorrected distortion image samples corresponding to different coordinate positions of the eye box in advance; or save the images of each uncorrected distortion image sample located in the imaging area at different coordinate positions of the camera in the eye box in advance, and then in actual operation, obtain the camera images corresponding to the current coordinate positions of the two eyes respectively, and then combine the spatial transformation relationship between the current coordinate positions of the two eyes and the coordinates of the target position on the final imaging surface to convert the camera images into the sample images required above.

[0097] It should be understood that uncorrected distortion imaging samples can be obtained based on the spatial transformation relationship between different coordinate positions of the eye box and the coordinates of the target position on the final imaging plane. The target position is the intersection of the extended lines of the left and right eye lines. The method for determining this spatial transformation relationship is described below. It should be understood that the spatial transformation relationship in the embodiments of this application can be generated by a distortion corrector, an eye tracker, or it can be generated in advance by the system and stored in the system so that it can be directly obtained from the system when used. This application does not limit this.

[0098] The following is a brief introduction to the methods for obtaining target source image samples, uncorrected left eye distortion image samples, and uncorrected right eye distortion image samples.

[0099] First, it should be understood that, as can be seen from formula (1), the acquisition of the distortion model M depends on the comparison between the undistorted original image I and the distorted image J. The undistorted original image I is the undistorted original image projected by the projector, which is referred to as the target source image sample in this application; the distorted image J is the distorted image received at the target position of the eye box, which is referred to as the uncorrected distorted image sample in this application. Specifically, the image received at the left eye coordinate position of the eye box is the uncorrected left eye distorted image sample, and the image received at the right eye coordinate position of the eye box is the uncorrected right eye distorted image sample.

[0100] In practice, after determining the imaging position of the uncorrected distorted image sample, it is necessary to acquire the uncorrected distorted image sample. It should be understood that this application does not limit the method of acquiring the uncorrected distorted image sample. Optionally, it can be acquired by taking pictures with a camera. Specifically, the uncorrected distorted image sample can be acquired by placing a camera at the target position of the eye box. However, due to the geometric relationship between the projection and mapping of the three-dimensional information (including image information) captured by the camera onto the two-dimensional imaging plane, the uncorrected distorted image sample cannot be directly acquired by placing a camera at the target position of the eye box and taking pictures separately. Furthermore, this geometric relationship (i.e., spatial transformation relationship) can be obtained through camera calibration. Therefore, the aforementioned uncorrected distortion image sample can be obtained through this geometric relationship and the image captured by the camera. Specifically, the uncorrected left eye distortion image sample can be obtained through the geometric relationship between the left eye coordinate position and the target position on the final imaging surface and the image captured by the camera at the left eye position, and the uncorrected right eye distortion image sample can be obtained through the geometric relationship between the right eye coordinate position and the target position on the final imaging surface and the image captured by the camera at the right eye position.

[0101] Specifically, the geometric relationships are determined as follows: Define a three-dimensional coordinate vector [X,Y,Z] at any position within the eyebox of the display system. T And the coordinates of the target position on the final imaging plane (i.e., the two-dimensional coordinate system) are defined as [u,v]. T Then, using the pinhole camera model, we can obtain:

[0102] w[u,v,1] T =K[X,Y,Z,1] T (3)

[0103] Where K is the camera intrinsic parameter matrix, which depends on the camera's focal length and pixel displacement. The camera intrinsic parameter matrix K can be obtained by configuring the camera specifications. w is the scale factor, which is the scaling ratio between the image plane and the pixel plane, and can be obtained by calculating the actual image size and the size of the image captured by the camera.

[0104] It should be understood that since the size of the human eye is much smaller than the imaging distance, the pinhole camera model is suitable for the calculations in this embodiment.

[0105] It should be understood that in practice, the above geometric relationships can be directly determined using a calibration map for camera calibration. This calibration map can be a physical or virtual distortion-free standard pattern. Specifically, a physical calibration map or a virtual calibration map is placed at the imaging distance of the display system. Since the pattern size and arrangement of the calibration map are standardized, a camera can be placed at the observation position of the display system, and the calibration map can be photographed to determine the w and K parameters. If calibration is performed at multiple locations on the eyepiece, the w and K parameters corresponding to different locations on the eyepiece can be obtained. Subsequently, the transformation relationship between different locations on the eyepiece and the final imaging plane coordinates can be obtained.

[0106] It should be understood that after obtaining the uncorrected left eye distortion image sample and the uncorrected right eye distortion image sample, a distortion model can be constructed based on the target source image sample, the uncorrected left eye distortion image sample, and the uncorrected right eye distortion image sample.

[0107] It should be understood that this application does not limit the method of constructing the distortion model. Optionally, the correlation between the target source image sample projected by the image generator and the uncorrected left eye distortion image sample of the final image can be obtained through optical design software or physical modeling, and the left eye distortion model can be constructed; the correlation between the target source image sample projected by the image generator and the uncorrected right eye distortion image sample of the final image can be obtained, and the right eye distortion model can be constructed.

[0108] Optionally, to facilitate comparison between the target source image sample, calibration map, and uncorrected distorted image sample to determine the distortion model, a distortion model can also be constructed by comparing the distorted image with a reference image. Specifically, the distortion-free calibration map (i.e., the target source image sample) is imported into the image generator of the display system. After optical imaging, the final image projected is a distorted image (i.e., an uncorrected distorted image sample). Simultaneously, a physical or virtual distortion-free calibration map (i.e., the target source image sample) is placed at the imaging position of the display system. Its shape, size, and dimensions need to be proportionally matched with the calibration map displayed by the image generator. Then, cameras are placed at different positions in the eye box to simultaneously capture the distortion patterns of the calibration map and the image captured by the display system. Then, the captured calibration map and the distortion patterns of the image captured by the display system are transformed using the spatial transformation relationship mentioned above. The transformed patterns are then compared to obtain the distortion models M at multiple positions in the eye box. This distortion model can be in the form of a parameter table or a mathematical model, etc. The embodiments of this application do not impose any limitations on the form of the distortion model.

[0109] The following will combine Figures 5 to 7 The system architecture of the embodiments of this application will be briefly described in order to better understand the solution of this application.

[0110] Figure 5 This is a system architecture example diagram provided in an embodiment of this application. For example... Figure 5 As shown, the system architecture 500 includes: an eye tracker 510, an image generator 520, a binocular parallax generator 530, and an optical lens group 540. It should be understood that the system architecture 500 is merely an example and does not constitute a limitation of this application. Optionally, in an actual system architecture, the aforementioned eye tracker 510 may not be included, and this application does not limit this. The various components will be described below.

[0111] Among them, the eye tracker 510 is used to detect the coordinate information of the observer's two eyes.

[0112] Image generator 520 includes an interface for acquiring a target source image from an external source and a distortion corrector. The distortion corrector is used to acquire or generate a left-eye distortion model and a right-eye distortion model. Then, image generator 520 performs pre-distortion processing on the target source image based on the left-eye distortion model and the right-eye distortion model generated by the distortion corrector to generate a left-eye pre-distorted image and a right-eye pre-distorted image. Pixels from the left-eye pre-distorted image are then interleaved with pixels from the right-eye pre-distorted image to generate a pixel-interleaved composite image, as detailed in the description of display system 200 above. It should be understood that... Figure 5 As an example only, in a real system architecture, the above distortion corrector may also exist independently of the image generator 520, and this application does not impose any restrictions on this.

[0113] A binocular parallax generator 530, installed in the image generator 520, guides pixels from the left-eye pre-distorted image in the pixel interleaving composite image to be pushed to the observer's left eye in a spatially multiplexed manner, and simultaneously guides pixels from the right-eye pre-distorted image in the pixel interleaving composite image to be pushed to the observer's right eye, so that both eyes can simultaneously observe a distortion-free image. Optionally, the binocular parallax generator 530 can also be independent of the image generator 520; this application does not limit this. Optionally, the binocular parallax generator 530 can be a parallax barrier, such as an aperture or a cylindrical lens; it can also be a directional light source component; this application does not limit this.

[0114] The optical lens assembly 540, also known as an imaging combiner, is mainly used to image the image generated by the image generator 520 and transmit it to the observer's binoculars. Preferably, in this application, the optical lens assembly 540 includes a plane mirror and a freeform mirror, and the image generated by the image generator 520 is first transmitted to the binoculars through the plane mirror and then through the freeform mirror. However, it should be understood that the design of the optical lens assembly in this application is not limited to this.

[0115] It should be understood that the design of the 540 optical lens group has different requirements for different application scenarios.

[0116] As an example, Figure 6 This is a system architecture example diagram provided in an embodiment of this application for use in office and entertainment display scenarios. For example... Figure 6As shown, the optical lens group 540 in the system architecture 600 includes a plane mirror 541 and a freeform mirror 542. In this system architecture 600, the image generator 520, in conjunction with the binocular parallax generator 530, projects pixels from the left and right pre-distorted images in a spatially divided multiplexing manner, and transmits them to the binoculars through the plane mirror 541 and the freeform mirror 542 in the optical lens group 540, respectively. It should be understood that the freeform mirror 542 has the function of optical path extension and optical path folding, making the size design of the optical lens group 540 more flexible. Moreover, this example uses a freeform mirror 542 instead of a concave mirror because the freeform mirror can be manufactured according to process requirements, so that the pixels in the left and right pre-distorted images are accurately pushed to the observer's binoculars, and has the function of image magnification, allowing the observer to enjoy a better viewing experience. However, it should be understood that in actual operation, concave mirrors or other optical mirrors can also be used, and this application does not limit this, nor is the number of optical mirrors limited. Furthermore, the eye tracker 510 in the system architecture 600 can detect dynamic changes in binocular coordinates when the observer makes significant movements and feed this information back to the image generator 520. This allows the image generator 520 to update the pre-distorted images of the left and right eyes in real time based on the coordinate changes. This ensures that the observer always sees a distortion-free image within the eye box range, preserving the viewing experience without loss of quality.

[0117] As another example, Figure 7 This is an example diagram of a system architecture for an in-vehicle head-up display scenario provided in an embodiment of this application. Optionally, the system architecture 700 for the in-vehicle head-up display scenario can be in... Figure 6 The system architecture shown is obtained by slightly adjusting the system architecture 600. For example... Figure 7 As shown, the optical lens group 540 in system architecture 700 can be supplemented with a car reflective element 543 (such as a windshield) on the basis of the optical lens group 540 in system architecture 600, so that the image is accurately presented to the driver's (i.e., the observer's) eyes, avoiding the left and right views from not overlapping and interfering with driving. Moreover, the eye tracker 510 in system architecture 700 can ensure that the driver always sees a distortion-free image, especially in scenarios with frequent dynamic changes such as car shaking.

[0118] In addition, it is easy to see that the solution in this application only requires one image generator, which has the advantage of smaller system architecture, making it more suitable for vehicle-mounted installation applications or other scenarios with limited space.

[0119] The following section combines the above system architectures from 500 to 700, with... Figure 8 This paper will use an example to introduce the specific implementation method of binocular distortion correction in this application.

[0120] Figure 8 This is an example diagram of a binocular distortion correction method provided in an embodiment of this application. Figure 8 As shown, this method 800 includes steps S810 to S850. It should be understood that the embodiments of this application do not limit the order of the above steps; any solution that can implement this application through any order of the above steps falls within the protection scope of this application. It should also be understood that... Figure 8 This is merely an optional example and does not constitute a limitation of this application. The steps are described in detail below.

[0121] S810, an eye tracker, detects the coordinate information of the observer's two eyes.

[0122] S820, distortion corrector acquires distortion model.

[0123] Optionally, the distortion corrector can directly read the left-eye distortion model and right-eye distortion model corresponding to the current coordinates of the left and right eyes from the parameter table in the system based on the current coordinate information of the binoculars, as described above.

[0124] Alternatively, the distortion corrector can obtain the distortion model itself, as described above.

[0125] It should be understood that, in the embodiments, the distortion corrector may exist within the image generator, such as... Figures 5 to 7 As shown, it can also exist independently of the image generator, and this application does not limit this.

[0126] S830, the image generator generates a pre-distorted image for the left eye and a pre-distorted image for the right eye.

[0127] Specifically, the image generator needs to generate left-eye pre-distortion images and right-eye pre-distortion images based on the target source image, the left-eye distortion model, and the right-eye distortion model. It should be understood that the target source image is the image that the display system needs to display during actual use; that is, the image that needs to be displayed to the user.

[0128] In one implementation, the target source image is a source image taken at the target viewpoint, and this application does not limit the target viewpoint. A left-eye pre-distortion image can be generated based on the target source image and the left-eye distortion model, and a right-eye pre-distortion image can be generated based on the target source image and the right-eye distortion model (implemented using formula (2)).

[0129] In this case, pre-distortion processing is performed on the same source image (i.e., the target source image) to obtain the left eye pre-distortion image and the right eye pre-distortion image, so that the imaging observed by both eyes is a distortion-free 2D imaging.

[0130] In another implementation, the target source image includes a left-eye view target source image and a right-eye view target source image. The acquisition methods of the left-eye view target source image and the right-eye view target source image are not limited, as described above. A left-eye pre-distortion image can be generated based on the left-eye view target source image and the left-eye distortion model, and a right-eye pre-distortion image can be generated based on the right-eye view target source image and the right-eye distortion model (using formula (2)).

[0131] In this case, images from different perspectives (i.e., the target source image from the left eye perspective and the target source image from the right eye perspective) are pre-distorted to obtain the left eye pre-distorted image and the right eye pre-distorted image, so that the imaging observed by both eyes is a distortion-free 3D imaging.

[0132] S840, the image generator generates pixel-interwoven composite images.

[0133] For details, please refer to the above text. Figure 3 The details of the previous descriptions will not be repeated here.

[0134] The S850 projects a pixel-interlaced composite image using spatial multiplexing and then pushes it to the observer's eyes through an optical lens group.

[0135] It should be understood that this spatial multiplexing method is implemented in conjunction with a binocular parallax generator, as described above. Figure 4 Partial description.

[0136] Specifically, with the assistance of the binocular parallax generator, the pixels of the left-eye pre-distorted image in the pixel interlacing composite image generated by the image generator are pushed to the observer's left eye through the optical lens group, while the pixels of the right-eye pre-distorted image in the pixel interlacing composite image are pushed to the observer's right eye through the optical lens group, so that the observer's binoculars can see an undistorted image.

[0137] In this embodiment, a pixel-interlaced composite image is generated by interlacing pixels in the left pre-distorted image and pixels in the right pre-distorted image. Then, the pixel-interlaced composite image is pushed to the binoculars in a spatially divided multiplexing manner under the action of the binocular parallax generator, thereby improving the user's visual experience while reducing the computational pressure of the data reading system.

[0138] Figure 9 This is an example diagram of a display method for binocular distortion correction provided in an embodiment of this application. Figure 9 As shown, the method 900 includes steps S910 to S930, which are described in detail below.

[0139] S910 generates the left eye pre-distortion image and the right eye pre-distortion image.

[0140] Optionally, before generating the left-eye pre-distortion image and the right-eye pre-distortion image, the method 900 may further include: obtaining a left-eye distortion model and a right-eye distortion model. The left-eye distortion model is generated based on the target source image sample and an uncorrected left-eye distortion image sample, and the right-eye distortion model is generated based on the target source image sample and an uncorrected right-eye distortion image sample.

[0141] Optionally, generating the left-eye pre-distortion image and the right-eye pre-distortion image includes: acquiring the target source image; and generating the left-eye pre-distortion image and the right-eye pre-distortion image based on the target source image, the left-eye distortion model, and the right-eye distortion model.

[0142] Optionally, the target source image is a source image captured at the target viewpoint. Then, generating the left-eye pre-distortion image and the right-eye pre-distortion image based on the target source image, the left-eye distortion model, and the right-eye distortion model includes: generating the left-eye pre-distortion image based on the target source image and the left-eye distortion model, and generating the right-eye pre-distortion image based on the target source image and the right-eye distortion model.

[0143] Optionally, the target source image includes a left-eye view target source image and a right-eye view target source image. Then, generating the left-eye pre-distortion image and the right-eye pre-distortion image based on the target source image, the left-eye distortion model, and the right-eye distortion model includes: generating the left-eye pre-distortion image based on the left-eye view target source image and the left-eye distortion model, and generating the right-eye pre-distortion image based on the right-eye view target source image and the right-eye distortion model.

[0144] Optionally, before obtaining the left and right eye distortion models, method 900 may further include: detecting the coordinates of the observer's binoculars. Then, obtaining the left and right eye distortion models would include: obtaining the left and right eye distortion models based on the coordinates of the observer's binoculars.

[0145] Optionally, the uncorrected left-eye distortion image sample and the uncorrected right-eye distortion image sample are obtained based on the spatial transformation relationship between the observer's binocular coordinates and the coordinates of the target position on the final imaging plane, where the target position is the intersection of the extension lines of the left-eye line of sight and the right-eye line of sight.

[0146] S920, interweaves the pixels in the left pre-distortion image with the pixels in the right pre-distortion image to generate a pixel interweaving composite image.

[0147] Optionally, interleaving pixels in the left pre-distorted image with pixels in the right pre-distorted image to generate a pixel-interleaved composite image includes: overlapping the left pre-distorted image and the right pre-distorted image; determining an envelope range that covers the effective pixels in both the left and right pre-distorted images; and interleaving pixels in the left pre-distorted image with pixels in the right pre-distorted image within the envelope range to generate a pixel-interleaved composite image.

[0148] The S930, with the help of a binocular parallax generator, pushes the pixel-interlaced composite image through an optical lens group to the observer's binoculars.

[0149] In this process, pixels from the left-eye pre-distortion image in the pixel interleaving composite image are pushed to the observer's left eye, and pixels from the right-eye pre-distortion image in the pixel interleaving composite image are pushed to the observer's right eye.

[0150] Optionally, the optical lens assembly may include a plane mirror and a freeform mirror.

[0151] Alternatively, the binocular parallax generator can be a parallax barrier or a directional light source component.

[0152] Alternatively, the parallax barrier can be an aperture stop or a cylindrical lens.

[0153] Figure 10 This is an exemplary block diagram of the hardware structure of a device provided in an embodiment of this application. The device 1000 (specifically, it can be a computer device) includes a memory 1010, a processor 1020, a communication interface 1030, and a bus 1040. The memory 1010, the processor 1020, and the communication interface 1030 are interconnected via the bus 1040.

[0154] The memory 1010 may be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1010 may store a program, and when the program stored in the memory 1010 is executed by the processor 1020, the processor 1020 is used to execute the various steps of the display method of the embodiments of this application.

[0155] The processor 1020 may be a general-purpose central processing unit (CPU), microprocessor, application-specific integrated circuit (ASIC), graphics processing unit (GPU), or one or more integrated circuits, used to execute relevant programs to implement the display method of the embodiments of this application.

[0156] The processor 1020 can also be an integrated circuit chip with signal processing capabilities. In implementation, the display method of this application can be accomplished through integrated logic circuits in the processor 1020 or through software instructions.

[0157] The processor 1020 described above can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory 1010. The processor 1020 reads the information in memory 1010 and, in conjunction with its hardware, completes the functions required by the modules included in the apparatus of the embodiments of this application, or executes the display method of the method embodiments of this application.

[0158] The communication interface 1030 uses transceiver devices, such as, but not limited to, transceivers, to enable communication between the device 1100 and other devices or communication networks.

[0159] Bus 1040 may include a pathway for transmitting information between various components of device 1000 (e.g., memory 1010, processor 1020, communication interface 1030).

[0160] This application also provides a controller, including an input / output interface, a processor, and a memory. The processor is used to control the input / output interface to send and receive signals or information. The memory is used to store a computer program. The processor is used to call and run the computer program from the memory, so that the controller executes the display method for binocular distortion correction according to the method embodiment of this application.

[0161] This application also provides an in-vehicle system, including a display system for binocular distortion correction according to the device embodiments of this application.

[0162] This application also provides a desktop display system, including a display system for binocular distortion correction according to the device embodiments of this application.

[0163] This application also provides a vehicle, including the display system for binocular distortion correction according to the device embodiments of this application. Optionally, the vehicle involved in this application may be a conventional internal combustion engine vehicle, a hybrid vehicle, a pure electric vehicle, a centralized drive vehicle, or a distributed drive vehicle, etc., and this application does not limit it.

[0164] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to execute the display method for binocular distortion correction described in the above method embodiments.

[0165] This application also provides a computer-readable storage medium for storing a computer program, the computer program including instructions for executing the display method for binocular distortion correction according to the method embodiments of this application.

[0166] This application also provides a computer program that includes instructions for executing the display method for binocular distortion correction according to the method embodiments of this application.

[0167] This application also provides a chip, which includes a processor and a data interface. The processor reads instructions stored in a memory through the data interface and executes the display method for binocular distortion correction according to the method embodiment of this application.

[0168] Optionally, as one implementation, the chip may further include a memory storing instructions, and the processor is used to execute the instructions stored in the memory. When the instructions are executed, the processor is used to execute the display method for binocular distortion correction according to the method embodiment of this application.

[0169] This application also provides a chip system including at least one processor for supporting the functions involved in certain implementations of the display method for binocular distortion correction described in the above method embodiments, such as receiving or processing data and / or information involved in the above method.

[0170] In one possible design, the chip system further includes a memory for storing program instructions and data, which may be located within or outside the processor. The chip system may consist of chips or may include chips and other discrete devices.

[0171] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0172] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0173] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0174] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0175] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0176] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0177] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A display system for binocular distortion correction, characterized in that, include: Image generator, binocular parallax generator, and optical lens assembly; The image generator is used to generate a left-eye pre-distortion image and a right-eye pre-distortion image; Overlay the left eye pre-distortion image and the right eye pre-distortion image; Determine the envelope range, which covers the effective pixels in the left eye pre-distortion image and the right eye pre-distortion image; The pixels in the left pre-distorted image and the pixels in the right pre-distorted image are interleaved within the envelope to generate a pixel interleaved composite image; The binocular parallax generator causes the pixel interlacing composite image to be pushed to the observer's binoculars through the optical lens group, wherein pixels of the left-eye pre-distortion image in the pixel interlacing composite image are pushed to the observer's left eye, and pixels of the right-eye pre-distortion image in the pixel interlacing composite image are pushed to the observer's right eye.

2. The display system according to claim 1, characterized in that, The display system further includes: a distortion corrector; the distortion corrector is used for, Obtain a left eye distortion model and a right eye distortion model. The left eye distortion model is generated based on the target source image sample and an uncorrected left eye distortion image sample. The right eye distortion model is generated based on the target source image sample and an uncorrected right eye distortion image sample. The image generator is also used for, Obtain the target source image; generate the left eye pre-distortion image and the right eye pre-distortion image based on the target source image, the left eye distortion model, and the right eye distortion model.

3. The display system according to claim 2, characterized in that, The target source image is a source image captured from the target viewpoint; the image generator is also used for, The left pre-distorted image is generated based on the target source image and the left eye distortion model, and the right pre-distorted image is generated based on the target source image and the right eye distortion model.

4. The display system according to claim 2 or 3, characterized in that, The target source image includes a target source image from the left eye view and a target source image from the right eye view; The image generator is also used for, The left-eye pre-distortion image is generated based on the left-eye view target source image and the left-eye distortion model, and the right-eye pre-distortion image is generated based on the right-eye view target source image and the right-eye distortion model.

5. The display system according to any one of claims 2 to 4, characterized in that, The display system further includes: an eye tracker; the eye tracker is used for, Detect the coordinates of the observer's two eyes; The distortion corrector is also used to obtain the left eye distortion model and the right eye distortion model based on the binocular coordinates of the observer.

6. The display system according to claim 5, characterized in that, The uncorrected left eye distortion image sample and the uncorrected right eye distortion image sample are obtained based on the spatial transformation relationship between the observer's binocular coordinates and the coordinates of the target position on the final imaging plane, where the target position is the intersection of the extended lines of the left eye line of sight and the right eye line of sight.

7. The display system according to any one of claims 1 to 6, characterized in that, The optical mirror assembly includes a plane mirror and a freeform mirror.

8. The display system according to any one of claims 1 to 7, characterized in that, The binocular parallax generator is a parallax barrier or a directional light source component.

9. The display system according to claim 8, characterized in that, The parallax barrier is an aperture or a cylindrical lens.

10. The display system according to any one of claims 1 to 9, characterized in that, The display system is an office entertainment display system or a vehicle head-up display system.

11. A display method for binocular distortion correction, characterized in that, include: Generate pre-distortion images for the left and right eyes; Overlay the left eye pre-distortion image and the right eye pre-distortion image; Determine the envelope range, which covers the effective pixels in the left eye pre-distortion image and the right eye pre-distortion image; The pixels in the left pre-distorted image and the pixels in the right pre-distorted image are interleaved within the envelope to generate a pixel interleaved composite image; Under the action of the binocular parallax generator, the pixel interlacing composite image is pushed to the observer's binoculars through the optical lens group. Pixels of the left-eye pre-distortion image in the pixel interlacing composite image are pushed to the observer's left eye, and pixels of the right-eye pre-distortion image in the pixel interlacing composite image are pushed to the observer's right eye.

12. The display method according to claim 11, characterized in that, The method further includes: Obtain a left eye distortion model and a right eye distortion model. The left eye distortion model is generated based on the target source image sample and an uncorrected left eye distortion image sample. The right eye distortion model is generated based on the target source image sample and an uncorrected right eye distortion image sample. The generation of the left eye pre-distortion image and the right eye pre-distortion image includes: Acquire the target source image; The left-eye pre-distortion image and the right-eye pre-distortion image are generated based on the target source image, the left-eye distortion model, and the right-eye distortion model.

13. The display method according to claim 12, characterized in that, The target source image is a source image captured at the target viewpoint; the step of generating the left-eye pre-distortion image and the right-eye pre-distortion image based on the target source image, the left-eye distortion model, and the right-eye distortion model includes: The left pre-distorted image is generated based on the target source image and the left eye distortion model, and the right pre-distorted image is generated based on the target source image and the right eye distortion model.

14. The display method according to claim 12 or 13, characterized in that, The target source image includes a target source image from the left eye view and a target source image from the right eye view; The step of generating the left-eye pre-distortion image and the right-eye pre-distortion image based on the target source image, the left-eye distortion model, and the right-eye distortion model includes: The left-eye pre-distortion image is generated based on the left-eye view target source image and the left-eye distortion model, and the right-eye pre-distortion image is generated based on the right-eye view target source image and the right-eye distortion model.

15. The display method according to claim 13 or 14, characterized in that, The method further includes: Detect the coordinates of the observer's two eyes; The acquisition of the left eye distortion model and the right eye distortion model includes: The left eye distortion model and the right eye distortion model are obtained based on the binocular coordinates of the observer.

16. The display method according to claim 15, characterized in that, The uncorrected left eye distortion image sample and the uncorrected right eye distortion image sample are obtained based on the spatial transformation relationship between the observer's binocular coordinates and the coordinates of the target position on the final imaging plane, where the target position is the intersection of the extended lines of the left eye line of sight and the right eye line of sight.

17. The display method according to any one of claims 11 to 16, characterized in that, The optical mirror assembly includes a plane mirror and a freeform mirror.

18. The display method according to any one of claims 11 to 17, characterized in that, The binocular parallax generator is a parallax barrier or a directional light source component.

19. The display method according to claim 18, characterized in that, The parallax barrier is an aperture or a cylindrical lens.

20. A controller, characterized in that, The controller includes an input / output interface, a processor, and a memory. The processor controls the input / output interface to send and receive signals or information. The memory stores a computer program. The processor retrieves and runs the computer program from the memory, causing the controller to perform the display method for binocular distortion correction as described in any one of claims 11 to 19.

21. A vehicle-mounted system, characterized in that, The display system for binocular distortion correction includes any one of claims 1 to 10.

22. A desktop display system, characterized in that, The display system for binocular distortion correction includes any one of claims 1 to 10.

23. A vehicle, characterized in that, The display system for binocular distortion correction includes any one of claims 1 to 10.

24. A computer program product, characterized in that, Includes a computer program that, when run on a computer, causes the computer to perform a display method for binocular distortion correction as described in any one of claims 11 to 19.

25. A computer-readable medium, characterized in that, Used to store a computer program, the computer program including instructions for performing the display method for binocular distortion correction as described in any one of claims 11 to 19.

Images