Light field preconditioning for multi-view displays

By measuring and calculating correction factors in a multi-view display, adjusting pixel brightness and distribution, the problems of image blurring and stitching discontinuity in light field displays under high pixel density are solved, achieving clearer 3D image display.

CN123125849APending Publication Date: 2026-11-17AVALON HOLOGRAPHICS INC
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202580016621.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2024-02-23
Filing Date
2025-02-20
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing multi-view autostereoscopic displays struggle to achieve high-quality light field image output at high pixel densities. Manufacturing differences in optical components and diffraction effects lead to image blurring and stitching discontinuities, which existing methods cannot effectively correct.

Method used

The system receives holographic unit light through a positioning light sensor, displays pixel illumination patterns, measures light intensity, calculates correction factors to adjust pixel brightness and distribution, and applies correction factors to pre-adjust the light field to achieve sharpening and smoothing of the light field.

Benefits of technology

It improves the image quality of multi-view displays, reduces visual discontinuities between optical elements, and enhances the depth perception effect of 3D images.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123125849A_ABST
    Figure CN123125849A_ABST
Patent Text Reader

Abstract

A method for improving the light field image quality of a multi-view display through light field pre-adjustment. The holographic cell-based method guides image preprocessing by measuring the pixel luminance at a reference pixel within the holographic cell. By determining the pre-adjusted input light field for each pixel in the light field display, the resulting output light field closely approximates the desired ideal output, thereby providing a sharper and smoother image.
Need to check novelty before this filing date? Find Prior Art

Description

Cross-references to related applications

[0001] This application claims priority to U.S. Patent Application No. 18 / 585,864, filed February 23, 2024, which is incorporated herein by reference in its entirety. Technical Field

[0002] This disclosure relates to a method for enhancing the quality of a light field image in a multi-view display, such as a holographic display. More specifically, this disclosure relates to a method for measuring and pre-adjusting the light field of a multi-view display comprising multiple optical elements. Background Technology

[0003] Existing two-dimensional display technologies provide a single image output or output image to all viewers. In a two-dimensional display, all viewers see the same image with both eyes based on their position relative to the display, allowing them to see all the pixels on the display simultaneously. Because only a single image is displayed and both eyes receive the same image, it is essentially impossible to perceive any depth. In contrast, a three-dimensional display provides the viewer with two distinct images, one for each eye, thus enabling depth perception.

[0004] 3D image displays can generally be broadly classified into two categories: binocular stereoscopic displays and autostereoscopic displays. Binocular stereoscopic displays use special glasses to allow the left and right eyes to view two slightly different images; while autostereoscopic displays allow the viewer to see two different images with the naked eye—one image per eye. Each of the two different images received by the viewer depends on the distance between the eyes, meaning each eye is actually in a different viewing position relative to the display, thus receiving a different image. When each eye receives slightly different images (as people interact with the 3D world), these two images together create depth cues and a realistic 3D visual experience.

[0005] To simultaneously display multiple different images, autostereoscopic displays are generally divided into two types: eye-tracking displays and multi-view displays. Eye-tracking-based autostereoscopic displays collect the user's position relative to the display to calculate the image the display should output, thus enabling selective image output. Therefore, this type of autostereoscopic display requires some means to track the viewer's eye position and movement. This can be achieved through head-mounted equipment worn by the viewer (such as glasses, head-mounted detectors, or cameras), or through external detectors or front-facing cameras installed within and / or around the display. The viewer's position information can be provided to the display so that it projects images only at the user's visual perception location. Multi-view autostereoscopic displays, also known as multi-view displays, do not selectively output images specific to each view. Instead, they display all visible images corresponding to each viewing angle simultaneously, regardless of whether a viewer receives the image, without needing to know the viewer's position. Multi-view displays allow users to move easily and seamlessly relative to the display and simultaneously display holographic 3D images to many viewers without determining the position of each user.

[0006] Because multi-view displays require significant computing resources and energy, early multi-view autostereoscopic displays could only output a few different images (also known as views) at a time. However, modern multi-view light field displays can output a much larger number of images simultaneously, with sufficient image density to achieve a three-dimensional viewing experience. To reproduce a sufficient number of light rays from a three-dimensional scene to form a multi-view light field display, a light emission system is required to emit light precisely and with accurate directional light emission at multiple closely spaced viewing angles.

[0007] Light field display devices can use pixel arrays and groups of holographic units (hogels) composed of subpixels to provide a specific directionality for each ray of light. Generating a uniform and seamless image in light field displays with tens of millions to hundreds of billions of pixels is an extremely challenging task. In projector-based light field display systems and flat panel (i.e., LED, OLED, LCD, and MicroLED) displays, physical effects such as diffraction and actual alignment tolerances can lead to splicing, distortion, and blurring, thus degrading overall image quality. Furthermore, misalignment and differences in brightness and directionality of each subpixel in flat panel displays, or slight deviations in the configuration or characteristics of projector components in projector-based light field display systems, can all arise from manufacturing variations in the individual light-emitting optical elements. Image pre-adjustment—adjusting the brightness, color, sharpness, blur, and smoothness of pixels—can significantly improve image quality, thereby enhancing the viewing experience.

[0008] In one example of image conditioning for a two-dimensional display, Brown et al. (MS Brown, Peng Song, and Tat-Jen Cham, “Image Pre-Conditioning for Out-of-Focus Projector Blur,” 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06), New York, NY, USA, 2006, pp. 1956–1963, doi: 10.1109 / CVPR.2006.145.) describe a two-dimensional image sharpening pre-conditioning method based on output image measurements. Image blur in the output image is obtained by measuring the output of each pixel on the display. In another example, Zhou et al. (M. Zhou et al., “A Unified Method for Crosstalk Reduction in Multiview Displays,” Journal of Display Technology, vol. 10, no. 6, pp. 500-507, June 2014, doi:10.1109 / JDT.2014.2305995.) described a method to reduce crosstalk in multiview displays by reducing pixel overlap. Specifically, they addressed extrinsic crosstalk caused by manufacturing or assembly errors and intrinsic crosstalk caused by light leakage from adjacent views by adjusting the predefined view indices (i.e., corresponding viewpoint numbers) and grayscale intensities of each subpixel in the liquid crystal display (LCD). Although the display described by Zhou et al. is a multiview display, it has only nine views, and all different views are visible from a single viewing angle.

[0009] In another image enhancement method for 3D displays, U.S. Patent US10,523,928B2 to Hwang et al. describes a distortion correction method comprising: displaying a first image on a display panel based on a source image corresponding to a respective viewpoint of the display panel; generating a second image by photographing the display panel at a first position; and determining a first compensation value for compensating for distortion of the second image relative to the display panel.

[0010] In light field display technology, whether it's a flat panel display (i.e., LED, OLED, LCD, and MicroLED) or a projector-based display, pixel blurring can stem from defects in optical components or diffraction caused by finite aperture requirements, such as diffraction resulting from tightly packed projectors or holographic units. In an idealized light field display, light from each pixel is output as a narrow, symmetrical beam around the direction of light propagation. In projector-based systems, misalignment of the light propagation direction emitted by pixels relative to the ideal light direction can be corrected by adjusting the projector angle or using optical components such as lenses. Non-uniform distribution or blurring is more difficult to correct using hardware-based correction. The extent to which these optical deviations can be corrected by hardware after manufacturing is limited. Software solutions such as light field pre-adjustment can address pixel blurring to provide a sharper display image. A light field pre-adjustment method is still needed to obtain high-quality images in high-pixel-density multi-view autostereoscopic light field displays.

[0011] The background information provided above is intended to disclose information that the applicant believes may be relevant to this invention. This document is not intended to make any admission, nor should it be construed as any of the foregoing information constituting prior art. Summary of the Invention

[0012] One object of the present invention is to provide a pre-adjustment method for an input light field, the pre-adjustment method comprising: positioning a light sensor to receive light from a holographic unit in a light field display plane; displaying a pixel illumination pattern centered on a first pixel position in the holographic unit on the light field display plane; measuring a light intensity value at a reference pixel in the holographic unit using the light sensor; displaying the pixel illumination pattern at an offset position relative to the first pixel position on the light field display plane; measuring the light intensity value at the reference pixel when the pixel illumination pattern is at the offset position; generating a brightness distribution of the reference pixel based on the light received by the light sensor at the reference pixel; and calculating a correction factor for the reference pixel.

[0013] In one embodiment, the method further includes: displaying a pixel illumination pattern at a plurality of offset positions relative to the first pixel position, and measuring a light intensity value at a reference pixel at each of the plurality of offset positions.

[0014] In another embodiment, the method further includes applying a correction factor to the input light field of at least one pixel in the holographic unit.

[0015] In another embodiment, the method further includes recalculating the correction factor for each of the plurality of input optical fields.

[0016] In another embodiment, the method further includes applying a correction factor to the input light field of all pixels in the holographic unit.

[0017] In another embodiment, a correction factor adjusts the luminance of at least one pixel in the holographic unit.

[0018] In another embodiment, the pixel illumination pattern is a single pixel pattern, a linear pattern, a stepped linear pattern, a square pattern, a stepped square pattern, a rectangular pattern, a cross pattern, a stepped cross pattern, a criterion pattern, or a target pattern.

[0019] In another embodiment, the light intensity value at a reference pixel is measured in multiple color channels.

[0020] In another embodiment, a correction factor is calculated for each color channel.

[0021] In another embodiment, the correction factor for each color channel is combined and applied to the input light field.

[0022] In another embodiment, the offset position relative to the first pixel position is along one or both of the x-axis and y-axis of the holographic unit.

[0023] In another embodiment, the method further includes: calculating a pixel-by-pixel blur kernel based on the brightness distribution.

[0024] In another embodiment, the light field display plane is formed by a light field display including an array of oriented pixels, which is a projector-based light field display or a flat panel light field display. In another embodiment, the flat panel display is one of an LED, OLED, LCD, or MicroLED display.

[0025] In another embodiment, a correction factor is calculated such that the light field is subjected to one or more of sharpening and smoothing processes.

[0026] In another embodiment, the light sensor measures the light intensity from more than one holographic unit in the light field display plane.

[0027] In another embodiment, the light sensor simultaneously measures the light intensity values ​​in multiple color channels.

[0028] In another embodiment, the method further includes: positioning an additional light sensor to receive light from an additional reference pixel in a holographic unit in a light field display plane; displaying a pixel illumination pattern centered on the location of the additional pixel in the holographic unit on the light field display plane; measuring a light intensity value at the additional reference pixel in the holographic unit using the light sensor; displaying the pixel illumination pattern at an additional offset location relative to the location of the additional pixel on the light field display plane; measuring the light intensity value at the additional reference pixel when the pixel illumination pattern is at the additional offset location; and generating a brightness distribution of the additional reference pixel based on the light received by the additional light sensor at the additional reference pixel.

[0029] In another embodiment, calculating the correction factor includes incorporating an additional correction factor of the additional reference pixel into the correction factor.

[0030] On the other hand, a system for pre-adjusting a light field display is provided, the system comprising: a light sensor for measuring light intensity values ​​from a reference pixel direction in a holographic cell in a display plane of the light field display; a frame for supporting the light sensor in front of the display plane; and a processor configured to perform the following methods: displaying a pixel illumination pattern centered on a first pixel position in the holographic cell on the display plane; displaying the pixel illumination pattern on the display plane at an offset position relative to the first pixel position; and generating a brightness distribution based on the light intensity values ​​measured by the light sensor for the reference pixel at the first pixel position and the offset position, the brightness distribution describing light received from a light source on the display plane along the reference pixel direction.

[0031] In one embodiment, the processor calculates the correction factor for the reference pixel.

[0032] In another embodiment, the light field display includes a projector-based light field display or a directional pixel array in a flat panel light field display. In another embodiment, the flat panel display is one of an LED, OLED, LCD, or MicroLED display.

[0033] In another embodiment, the processor is further configured to perform the following steps: calculating a correction factor for a reference pixel; applying the correction factor to an input light field to generate a corrected light field; and displaying the corrected light field on a light field display.

[0034] In another embodiment, the light sensor is a single-frequency light sensor, a multi-frequency light sensor, a photodetector, a charge-coupled device (CCD), a liquid crystal on silicon (LCOS) sensor, or a camera.

[0035] In another embodiment, the light sensor includes one or more additional optical components for light deflection, lensing, collimation, or guiding.

[0036] In another embodiment, the offset position relative to the first pixel position is along one or both of the x-axis and y-axis of the holographic unit.

[0037] In another embodiment, the system includes more than one light sensor.

[0038] The embodiments of the present invention described herein can be combined or arranged in any combination or arrangement. Attached Figure Description

[0039] The accompanying drawings included in this disclosure illustrate some possible embodiments of the present application. These and other features of the invention will become clearer from the following detailed description with reference to the accompanying drawings.

[0040] Figure 1 An example of a projector-based multi-view (x,y) display plane is shown, where the light sensor is located at a reference position.

[0041] Figure 2 An example is shown of a single holographic unit in a display plane, which includes multiple oriented pixels.

[0042] Figure 3 An example is shown where three directional pixels in a holographic unit are illuminated according to a linear pixel illumination pattern.

[0043] Figure 4A An example 3×3 stepped cross-shaped pixel illumination pattern is shown around the origin pixel.

[0044] Figure 4B An example 5×5 stepped cross-shaped pixel illumination pattern is shown around the origin pixel.

[0045] Figure 5 Different example pixel lighting patterns are shown.

[0046] Figure 6 The displacement of the cross-shaped pixel illumination pattern around the reference pixel in the holographic unit is illustrated.

[0047] Figure 7 It is a graph of a dataset of luminance or brightness values ​​of a single reference pixel in a pixel array, which was obtained during a scan of the pixel illumination pattern along the y-axis.

[0048] Figure 8 It is a curve showing how the light intensity of multiple reference pixels in the display plane changes with the distance from the origin of the pixel illumination pattern.

[0049] Figure 9 This is a flowchart illustrating a pixel input optimization method for a light field display.

[0050] Figure 10This is a flowchart illustrating a method for generating pixel brightness.

[0051] Figure 11 This is a flowchart illustrating a method for characterizing the entire light field projector display through optical measurements and pixel input optimization.

[0052] Figure 12 The sub-sections of the light field before and after sharpening are illustrated and compared with the ideal light field output.

[0053] Figure 13 An example of a multi-view display plane arrangement based on a projector is shown, in which two optical sensors are located at different reference positions.

[0054] Figure 14 An example is a projector-based multi-view display with splicing optics configured to provide light to each pixel in each holographic unit of the multi-view display.

[0055] Figure 15 An example of a flat panel multi-view (x,y) display plane is shown, in which the light sensor is located at the reference position. Detailed Implementation

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0057] In this article, when used in conjunction with the term “including,” the words “a” or “an” can mean “one,” but also have the same meaning as “one or more,” “at least one,” and “more than one.”

[0058] As used herein, the terms “comprising,” “having,” “including,” and “containing,” and their grammatical variations, are inclusive or open-ended and do not exclude additional, unlisted elements and / or method steps. Compositions, devices, articles, systems, uses, or methods described herein that include certain elements and / or steps may also consist substantially of those elements and / or steps in some embodiments and in other embodiments, regardless of whether those embodiments are specifically mentioned.

[0059] As used herein, the term “about” refers to a change of approximately ±10% relative to a given value. It should be understood that such change is always included in any given value provided herein, whether or not specifically mentioned. The enumeration of ranges in this document is intended to indicate the range and the individual values ​​falling within it, with the same number of decimal places as the numbers used to represent the range, unless otherwise stated herein.

[0060] Any use of example or exemplary language, such as “such as,” “exemplary embodiment,” “illustrative embodiment,” and “for example,” is intended to illustrate or represent aspects, embodiments, variations, elements, or features relating to the invention and is not intended to limit the scope of the invention.

[0061] As used herein, the terms “connected” and “connected” refer to any direct or indirect physical association between elements or features of this disclosure. Thus, these terms can be understood to mean elements or features that are partially or wholly contained within, attached to, coupled to, disposed on, connected together, communicate with, or operatively associated with each other, even if other elements or features exist between the elements or features described as connected.

[0062] As used herein, the term "OLED" refers to an organic light-emitting diode, a photoelectric device that emits light when an external voltage is applied. An OLED has a light-emitting electroluminescent layer or an organic material or substance that emits light in response to an electric current. Unbound by theory, when an electric current is applied, the anode injects holes into the organic layer of the OLED, while the cathode injects electrons into the same organic layer. The injected holes and electrons migrate toward electrodes with opposite charges. When electrons and holes are localized on the same molecule, excitons are formed, which are localized electron-hole pairs with excited energy states. When the excitons relax through a light emission mechanism, light is emitted. Types of OLEDs include, but are not limited to, active-matrix OLEDs (AMOLEDs) and passive-matrix OLEDs (PMOLEDs). AMOLEDs have a complete cathode layer, an organic molecular layer, and an anode layer. The anode layer has a thin-film transistor (TFT) plane parallel to it to form a matrix. This allows individual pixels to be switched on or off as needed to form an image. Therefore, pixels can be turned off whenever they are not needed or when the display shows a black image, reducing the energy required for the display to emit light. This is the lowest power consumption type of OLED and has a fast refresh rate, making it suitable for video displays. PMOLED has a similar composition to AMOLED, but its cathode and anode lines are arranged at right angles to each other. Electrical control is achieved through the anode and cathode lines to activate pixels at their intersections, thus producing light. The background of a PMOLED display is always black, and the color displayed when a pixel is active is a predetermined color. PMOLED pixels are fixed to a single color and are not suitable for dynamic images or displays. OLEDs can be top-emitting or bottom-emitting. Top-emitting OLEDs have an opaque or reflective substrate. If the emitted light passes through a transparent or translucent bottom electrode and substrate, the OLED is bottom-emitting. Top-emitting OLEDs are generally more suitable for active-matrix applications because they are easier to integrate with opaque transistor backplanes.

[0063] As used herein, the term "pixel" refers to a spatially discrete light-emitting mechanism used to form a display. As used herein, a pixel array refers to an array of oriented pixels within a holographic unit.

[0064] As used herein, the term "light-emitting diode" or "LED" refers to a semiconductor device that emits light when an electric current passes through it. LEDs consist of multiple layers of semiconductor materials that release photons when an electric current is applied. These devices are highly efficient at converting electrical energy into light and are commonly used in a wide range of applications, such as lighting, displays, indicators, and signage. LEDs offer advantages including durability, energy efficiency, and flexibility in color and intensity, making them widely used in modern lighting solutions and electronic devices.

[0065] As used herein, the term "LCD" or "liquid crystal display" refers to a flat panel display technology that uses the properties of liquid crystals to generate images. In an LCD, liquid crystals are sandwiched between two glass or polymer substrates. When an electric current is applied to the liquid crystals, they twist and align to control the transmission of light. This manipulation of light allows LCDs to generate images and display content. LCDs are commonly used in electronic devices such as televisions, computer monitors, smartphones, and tablets due to their lightweight design, low power consumption, and ability to produce high-quality images with vibrant colors and sharp details.

[0066] As used herein, the term "MicroLED" refers to an emerging display technology featuring multiple tiny light-emitting diodes (LEDs) arranged in an array. Unlike traditional LED displays that use large LEDs, MicroLED displays use tiny individual LEDs, typically on the order of micrometers. Each MicroLED acts as a pixel and emits light independently. MicroLED displays offer several advantages, including high brightness, high contrast, wide color gamut, and fast response time. They also consume less power than traditional display technologies and have the potential for extended lifespan. MicroLED technology is considered a promising candidate for next-generation display applications, including large-screen televisions, augmented reality (AR) and virtual reality (VR) devices, wearable electronics, and automotive displays.

[0067] As used herein, the term "flat panel display" refers to an electronic visual display technology characterized by a thin, flat form factor and a uniform display surface. These displays use various technologies, such as liquid crystal displays (LCDs), plasma displays, organic light-emitting diodes (OLEDs), and MicroLEDs, to generate images. Unlike earlier cathode ray tube (CRT) displays, flat panel displays do not rely on bulky, curved tubes for image projection. Instead, they utilize thin layers of material or cells to generate images directly on a flat surface. Flat panel displays are commonly used in devices such as televisions, computer monitors, laptops, tablets, smartphones, and digital signage. Their advantages, such as space efficiency, lightweight design, high energy efficiency, and high image reproduction quality, make them popular in consumer electronics and professional applications.

[0068] As used in this article, the term "light field" at its basic level refers to a function describing the amount of light flowing through points in space in each direction under unobstructed conditions. Therefore, a light field represents radiance as a function of the position and direction of light in free space. Light fields can be generated synthetically through various rendering processes or captured by a light field camera or an array of light field cameras. More broadly, the term "light field" can be understood and described as an array or subset of holographic cells.

[0069] As used herein, the term "holographic unit" refers to a holographic element or holographic pixel, which is a cluster of pixels with directional control capabilities. Arrays of holographic units can generate light fields. Just as pixels describe the spatial resolution of a two-dimensional display, holographic pixels or holographic units describe the spatial resolution of a three-dimensional display.

[0070] As used herein, the term "light field display" is a device that reconstructs a light field from a finite number of light field radiance samples input to the device. The radiance samples represent the color components red, green, and blue (RGB). For the reconstruction in a light field display, the light field can also be understood as a mapping from four-dimensional space to a single RGB color. These four dimensions include the display's vertical and horizontal dimensions. and the description of the orientation components of the light field The light field has two dimensions. It is defined as a function of the following:

[0071]

[0072] For a fixed point in the light field , , This refers to a two-dimensional (2D) image, often called a "cell image." A cell image is derived from a fixed... , A positional light field orientation image. When multiple unit images are connected side-by-side, the resulting image is called an "integral image." The integral image can be understood as the entire light field required for a light field display. Parameters of a light field display can include one or more of the following: holographic unit spacing, pixel spacing, and focal length. Pixel spacing is defined as the distance from the center of one pixel to the center of the next pixel.

[0073] It is anticipated that any embodiment of the compositions, apparatus, articles, methods and uses disclosed herein can be implemented as is by those skilled in the art, or by making such modifications or adopting equivalents without departing from the scope of the invention.

[0074] This paper describes a method for enhancing the light field image quality in multi-view autostereoscopic displays, such as holographic displays, using light field preconditioning. This method guides image preprocessing by measuring the behavior of display pixels to improve the light field output; this improvement may also include sharpening and / or smoothing transitions between stitching elements in the viewed image. Light field preconditioning includes smoothing transitions between optical elements and sharpening the output light field based on a blur kernel measured across the light field display. Variations in the distribution or diffusion of light from a particular pixel (which can also be considered blur) are normal artifacts in light field displays. Differences in luminance and blur level can be caused by natural variations in the hardware and optical component structures. Luminance or light intensity values ​​can be detected by a light sensor and represent the measured light intensity received from the light source. In a pixel array, light is received from multiple pixels at a position in front of the display plane, such that light from more than one pixel contributes to the luminance or luminance received at the viewing position. Furthermore, for any display with discrete optical components, if the beam shape changes abruptly at transitions between components, a degree of visual discontinuity will appear at these transitions. By determining the actual brightness received at a specific location relative to the display plane, the image can be adjusted to normalize the pixel brightness across the light field display, thereby correcting variations in brightness and blur.

[0075] The method described in this paper detects discontinuities and variations in the resulting light field caused by light-emitting hardware and applies correction factors to the light field display to adjust the optical properties of the resulting light field, thereby sharpening and smoothing the viewed image and improving its image quality. By determining the actual brightness received at a specific location relative to the display plane, the image can be adjusted to normalize the pixel brightness across the light field display, thereby correcting for variations in brightness and blur. This method can be used to improve the viewing experience of any light field display with discrete components, including flat panel displays and projector-based displays, and can increase the perceived depth of projector-based displays and reduce and / or eliminate visual discontinuities at projector image boundaries.

[0076] The various features of the present invention will become clear from the following detailed description taken in conjunction with the accompanying drawings. The design factors, construction, and usage of the light field rendering techniques disclosed herein are described in conjunction with various examples representing embodiments, which are not intended to limit the scope of the invention as described and claimed herein. Those skilled in the art will understand that other variations, examples, and embodiments not disclosed herein may exist, which can be practiced based on the teachings of this disclosure without departing from the scope of the invention.

[0077] Figure 1 An example of a projector-based multi-view light field display is illustrated, which has a display plane 10 comprising a plurality of holographic units 14 arranged in an array in the xy plane, and a light sensor 12 positioned relative to the display plane 10. Each holographic unit 14 in the display plane 10 consists of a plurality of directional pixels 16, wherein each directional pixel 16 emits light toward the display plane 10 at an angle along a directional ray 20. In the projector-based light field display, a plurality of projectors 26 emit and guide light toward the light field display plane 10 to form a light field. The multi-view autostereoscopic display is capable of simultaneously outputting multiple images, each image output at a different viewing angle within a range of viewing angles. Because the display plane 10 comprises an array of directional pixels 16, the directional ray 20 can be guided relative to the display plane 10 at a wide range of angles, allowing the viewer to experience three-dimensional images relative to the display at different viewing angles. Furthermore, the autostereoscopic display generates multiple views simultaneously, allowing the viewer to receive two different images simultaneously, i.e., one image for each eye, thereby forming a holographic image or a three-dimensional image.

[0078] A projector-based light field display has an array of projectors 26, each projector having a series of optical components to generate a light field. To achieve the number of pixels required to realize a high-definition light field display, a large number of projectors are needed, and the multiple images generated by these projectors need to be optically aligned to form a smooth and high-quality light field display. Each projector typically includes multiple light-emitting diodes (LEDs), a projector body, and optical systems and components configured to form a multiplexed light field from the multiple light rays generated by the LEDs. In one example, the projector may sequentially include: a light source, such as a light-emitting diode (LED); a projection component for receiving light from the light source and directing the light into a single light path; a pixel forming component for receiving light and converting it into a pixel array; a magnifying optics component for magnifying the pixel array; and a collimating optics component for collimating the light from the pixel array and forming a collimated projected image. Optical components in a multi-view projector system may include, but are not limited to: one or more liquid crystal on silicon (LCOS) panels, digital micromirror devices (DMDs), pixel forming devices, plano-convex lenses, dichroic mirrors, lens arrays, microlens arrays, meniscus lenses, biconvex lenses, single prisms, folding prisms, light field projection (LFP) lenses, collimating lens arrays including multiple collimating sub-lenses, projection doublets, metasurfaces, and metalenses.

[0079] For any light field display with discrete optical components, visual discontinuities will occur at transitions where the beam shape changes abruptly between components. Differences in brightness between light sources (such as LEDs in a projector or subpixels in an OLED), and variations in light-guiding optics (whether within each light field display or between optics in a projector or OLED array), can lead to observable differences in optical measurements. Optical measurements include, but are not limited to, the color, luminance, size, luminance distribution, and position of each projected image. Hardware limitations within each light field display can make physical adjustments difficult; however, software adjustments based on light field display hardware measurements can apply correction factors to control the luminance of each light source, thereby achieving sharpening and smoothing of the projected image.

[0080] In a multi-holographic unit display plane, the degree of blurring of individual pixels in a light field display can vary significantly across the entire display and also within a single projector region or holographic unit region (referred to herein as a splicing optics). The same variable blurring can also occur in any display that includes an array of splicing optics, where abrupt changes occur in the light distribution between holographic units. Therefore, a single characterization result cannot be applied to the entire light field display because each pixel in a light field display may have different brightness and brightness distribution or diffusion across the display plane for each color. Modern multi-view displays have approximately 100 to 100 billion pixels compared to a two-dimensional (2D) 4K display with 8 million pixels, so even if each pixel could be uniquely characterized, the amount of data storage and computational demands required to correct each pixel is infeasible. Measuring only a set of values ​​for the entire display is insufficient to reflect the light emission range of each pixel, and measuring each pixel individually is also impractical. However, it has been found that by grouping pixels by holographic unit and measuring the brightness at a single reference pixel within the holographic unit, sufficient brightness and diffusion data can be obtained for pre-adjusting the light field, thus achieving significant and detectable sharpening and smoothing. The light field pre-adjustment and smoothing process presented in this paper can specifically target local abrupt changes in the measured distribution to obtain a smoother image. By using a complete per-holographic unit light distribution measurement, finer image sharpening can be achieved compared to averaging or unspecified image sharpening techniques. This process reduces the perception of transitions between different optical elements in the light field display, thus contributing to the formation of a visually continuous light field. Furthermore, this process addresses previously unrecognized changes in the physical properties of light emitted by individual components of the light field display. This method can perform light field correction on the light field display by measuring the display light distribution around the oriented reference pixel to achieve accurate input pre-adjustment, thereby correcting predicted light distribution defects and undesirable variations. Since this method does not rely on modeling and predicting the display light distribution, the computational intensity required for image pre-adjustment can be reduced, making it particularly efficient for splicing optics.

[0081] Traditional methods for measuring the light distribution of a display typically involve measuring each pixel on the display. However, applying this method to multi-view displays is impractical due to the much larger number of pixels. Furthermore, applying such methods to light field displays with even more pixels presents greater challenges. The disclosed method is suitable for measuring displays with non-uniform, asymmetric light distributions due to the inhomogeneous light distribution of individual splicing optics within the light field display. This method can measure the light distribution of a multi-view display without measuring all pixels in the display. Therefore, this method requires less data and enables customized downstream input pre-conditioning to correct for defects or undesirable variations in the display's light distribution.

[0082] Figure 2 An example is illustrated of a single holographic unit 14 in a projector-based display plane, which includes a plurality of directional pixels. In a three-dimensional multi-view display device, the display plane includes a plurality of holographic units 14 arranged in an array, wherein each holographic unit 14 includes a plurality of directional pixels 16, as shown, the pixel positions in each holographic unit 14 being defined by the x-axis and y-axis. The display plane generally refers to a set of points and orientations defined by the physical spacing of the flat panel display and its various light field holographic units 14 and pixel 16 elements. Each pixel 16 in the holographic unit 14 has a specific light emission direction and emits light along a directional ray 20 at an angle relative to the display plane. In each holographic unit 14, the directional pixel 16 guides light to the viewer along the directional ray 20 at an angle relative to the display plane; and each holographic unit 14 has a plurality of directional pixels 16 that guide light in multiple different directions to provide a multi-view image of an autostereoscopic display.

[0083] The directional light rays projected from different pixels in each holographic unit 14 have different directions of orientation, thus forming a multi-view display; however, all pixels 16 within the holographic unit 14 rely on similar electronic and optical hardware (such as the projector 26) and have been shown to have similar brightness distributions. Therefore, measuring the brightness distribution of a single reference pixel 18 in the holographic unit 14 can yield a representative correction factor used to correct the blurring and luminance of all pixels 16 in the holographic unit 14. In a display system where the pixels 16 in the holographic unit 14 have similar optical properties and outputs, measuring the angular distribution of light from the reference pixel 18 on a per-holographic-unit basis allows adjustment of the pixel input of each pixel in the holographic unit to make the resulting display light field closer to the ideal light field, thereby improving the display visual quality.

[0084] To measure the luminance distribution at each reference pixel 18, the light sensor 12 is aligned with the reference pixel 18 such that the reference pixel 18 is the brightest pixel in the holographic unit 14 received at the sensor origin 24. Pixels in each holographic unit 14 whose directional rays 20 directly point to the sensor origin 24 on the light sensor 12 are referred to herein as reference pixels 18. To collect luminance measurements from the display plane, the light sensor 12 is positioned at a reference location in the viewing area above the display plane. By measuring the light intensity received by the light sensor 12 from at least one reference pixel 18 in each holographic unit 14 at the sensor origin 24 and creating a pixel luminance matrix (L), a pre-adjustment correction factor for the holographic unit 14 containing the reference pixel 18 can be calculated. Applying the correction factor before displaying the light field helps to sharpen and smooth the image, thereby improving the overall quality of the light field display. The pixel luminance matrix (L) describes the contribution of multiple directional pixels to the light received at the sensor origin 24 and can be considered as the distribution shape of the light received along the direction of the reference pixel 18, as a contribution from multiple pixels in the pixel array (not just from the reference pixel 18).

[0085] Figure 3 An example is illustrated where three oriented pixels 16 in a holographic unit are illuminated according to a 1×3 linear pixel illumination pattern. The pixel illumination pattern includes one or more pixels illuminated according to a predetermined pattern. To calculate the contribution of each pixel in the holographic unit to the light received at the light sensor 12, the pixel illumination pattern can be projected at multiple locations on the holographic unit to generate a pixel brightness matrix (L). The pixel illumination pattern can include any number of adjacent pixels 28a, 28b surrounding the origin pixel 36 of the illumination pattern; Figure 3In the illustrated case, the origin pixel 36 of the illumination pattern is the same as the reference pixel 18, displayed in a coarse outline. The pixel pattern may include adjacent pixels located at different positions in the x and y directions relative to the origin pixel 36, and each of the origin pixel and adjacent pixels may have the same or different brightness or luminance. By changing the position of the origin pixel 36 in the pixel illumination pattern relative to the reference pixel in the xy holographic unit matrix, a pixel luminance matrix (L) is generated, which mathematically describes and approximates the light contribution of each pixel to the light received at the light sensor 12. The light sensor 12 is aligned with the directional light ray 20 emitted from the reference pixel 18. The reference pixel 18 is the pixel whose directional light ray 20 is most aligned with the sensor origin 24 and appears to the light sensor 12 as the brightest pixel in the holographic unit 14. When the reference pixel 18 and its adjacent pixels 28a and 28b are illuminated, the light sensor 12 simultaneously receives light from all three pixels at the sensor origin 24. Specifically, directional light 20 from reference pixel 18 is received at light sensor 12, but light sensor 12 also receives light from directional light 20a emitted from adjacent pixel 28a and light from directional light 20b emitted from adjacent pixel 28b. Light sensor 12 can be conceived as a single eye located above the display plane, receiving directional light from pixels in each holographic unit that are directly pointed at the light sensor. Light sensor 12 can be, for example, a single-frequency light sensor, a multi-frequency light sensor, a photosensor, or a camera, and optionally has one or more additional optical components for light deflection, lensing, collimation, or guidance, including but not limited to lenses or mirrors.

[0086] In an ideal light field display, the directional rays 20, 20a, 20b from each pixel would be highly collimated. However, due to blurring, each pixel has a non-zero light diffusion angle, causing the light emitted from multiple pixels to affect the image observed by the eye at the sensor origin 24. To measure the actual brightness distribution at each pixel in the display plane, the light contributions from the reference pixel 18 and surrounding pixels that contribute light should be considered. For this purpose, the light field device displays a pixel illumination pattern and detects the light contributed by the directional rays 20, 20a, 20b. When the display device illuminates a pixel with a specific illumination pattern, the total amount of light received by the light sensor 12 can accurately indicate the light received at the location of the light sensor 12 under that illumination pattern.

[0087] Precise navigation of a measurement device on a display plane is feasible but presents significant challenges due to its inherent complexity, requiring meticulous positioning and accurate position tracking. For light field displays with micron- and nanon-scale pixels and holographic units, accurately positioning and repositioning the light sensor at a distance from the display plane can be extremely challenging. To reduce this complexity, this method uses a fixed light sensor 12 and moves a digitally controlled pixel illumination pattern on the light field display to scan the light pattern across the light sensor 12 along one or both of the x- and y-axis. This allows for the use of a fixed measurement device within a rigid mechanical structure and fixes the position of the display plane relative to the light sensor 12, approximating the photometric deviation of each holographic unit throughout the light field display. In particular, by measuring the local blurring in the region surrounding a reference pixel 18 within a holographic unit, the blurring deviation within the holographic unit containing the reference pixel 18 can be approximated, enabling the use of sparse pixel measurements to enhance the sharpening process of light field displays comprising millions to billions of pixels. Once this is achieved, corrections can be applied to pre-adjust the light field to adjust the brightness of each pixel in the light field display, resulting in a more uniform image.

[0088] Figure 4A and Figure 4B Two different pixel illumination patterns are illustrated around the origin pixel, resulting in different numbers of adjacent pixels along both the x and y axes. A pixel illumination pattern is a pattern of pixels that can be illuminated in a display plane, used to measure how much light is received at the light sensor from pixels in the illumination pattern at a given position in the display plane relative to a reference pixel. Figure 4A It is an example of a lighting pattern, and is a 3×3 cross-shaped pixel lighting pattern 34a around the origin pixel 36, which has four adjacent pixels 38. Figure 4B An example of a 5×5 cross-shaped pixel illumination pattern 34b is shown, which has eight adjacent pixels 38 arranged around the origin pixel 36. Pixel illumination patterns can occupy 1×1 to n. x ×n y The pixel matrix is ​​defined by n, where n ranges from 1 to the number of pixels in the holographic unit array minus one. The pixel illumination pattern can have the same or different numbers of pixels in the x and y directions. By moving the pixel illumination pattern around a reference pixel in the light field display plane, the contribution of light received from the reference pixel and its surrounding pixels at the viewing position can be approximately determined.

[0089] Figure 5Different pixel illumination patterns are illustrated by varying arrangements of pixels along the x and y axes. Pixel illumination patterns can be formed based on the selection of the on or off states of certain pixels and the luminance of each pixel in the pattern. As shown, the state of each pixel ranges from completely off (white) to completely on (black), and each pixel in the pixel illumination pattern can also have an intermediate luminance (partially on) to obtain the desired measurement results. Ideally, a single pixel can be used in the pixel illumination pattern; however, it has been found that using more complex pixel illumination patterns can improve the accuracy of the resulting pixel luminance matrix L, thereby enabling better calibration and pre-adjustment of the light field device. Square, rectangular, and symmetrical and asymmetrical patterns surrounding a reference pixel can be used as pixel illumination patterns. The illuminated pixels in the pixel illumination pattern can be, for example, centered around an already illuminated or unilluminated origin pixel, and can include illuminated neighboring pixels in the x-direction, neighboring pixels in the y-direction, illuminated diagonal pixels, and / or illuminated neighboring pixels among neighboring pixels. In the simplest and most idealized case, a single pixel can be used as the pixel illumination pattern; however, light from a single pixel may be insufficient for accurate detection at the light sensor. It has been found that pixel illumination patterns comprising two or more pixels provide more light and are very useful in measuring light intensity and diffusion at a reference pixel. In one example, a 3×3 pixel square (with higher brightness than a single pixel), a stepped pixel square with a brighter origin pixel and darker adjacent pixels, and a 2×2 pixel square can be used. Cross shapes such as solid crosses and stepped crosses have been used, producing different pixel brightness matrices L compared to squares. Linear patterns, including solid pixel lines and pixel lines along which the luminance varies in stepwise proportions based on distance from the origin pixel, can also be used. Although the pixel illumination patterns illustrated herein are 1×1 to 5×5 patterns, it should be understood that larger pixel illumination patterns can also be used. Other patterns, such as stepped square patterns, stepped cross patterns, marker patterns, and target patterns, can have higher brightness while still having a single pixel with maximum pixel brightness at the origin. Conical patterns and other patterns with highly stepped luminance (meaning that each pixel in the pattern will have varying brightness) allow for finer measurements, with the valleys between the peaks providing near-pixel contour brightness data for the display.

[0090] Figure 6The displacement of a 3×3 cross-shaped pixel illumination pattern around a reference pixel in a holographic cell on the y-axis is illustrated. To measure the luminance matrix of each holographic cell in a light field display, a light sensor is placed at a reference position at a distance from the display plane, aligned with the reference pixel. The light intensity value at the reference pixel can be measured. The reference pixel (highlighted in bold) is the pixel within a particular holographic cell that has the closest angle of directional light rays to be received by the light sensor. The reference pixel can also be understood as: the brightest pixel in the holographic cell received by the light sensor at a defined position relative to the display plane, or the pixel with the highest luminance in any given holographic cell received at the light sensor. The reference pixel selected for measurement can serve as a representative of its surrounding directional pixels in the same holographic cell, and the same or similar corrections can be applied to all pixels in the holographic cell based on the pixel luminance measured at the reference pixel. The pixel luminance matrix (L) of the reference pixel (whose optical environment will be similar to that of its surrounding pixels) can then be pre-adjusted and corrected for illumination differences among pixels in the same holographic cell.

[0091] Light intensity values ​​can be measured across pixel offsets during scanning motion, such as... Figure 6 As shown along the y-axis. The figure illustrates when the pixel illumination pattern is located at y0, y... -1 y -2 y +1 and y +2 The luminance at the reference pixel is measured, where the cross-shaped center of the pixel illumination pattern (i.e., origin pixel 36) moves along the y-axis with an offset along with its adjacent pixels 38 in the pixel illumination pattern. The luminance measurement at the reference pixel is performed at the same reference position while the scan motion is being executed. The light emitted by each illuminated pixel in the pixel illumination pattern has a light distribution that overlaps with the output of its adjacent pixels, and the light from all illuminated pixels in the pixel illumination pattern contributes to the light received at the reference pixel, even if the reference pixel is not illuminated, as shown in the example y. -2 and y +2 This situation, where light distribution reduces the effective resolution of the resulting light field and thus the depth, is problematic. This pixel-by-pixel light distribution can also vary across the entire display. This method can improve the resolution and depth of the light field display by pre-computing the input pixel values, bringing it closer to an ideal level. This involves utilizing information about the physical light distribution across pixels, which contributes to the detected brightness at discrete viewing locations, allowing adjustments to those pixels to smooth and sharpen the overall received image. Furthermore, by employing light field pre-adjustment, the perceived transitions caused by boundaries between stitching optics (such as holographic units) can be reduced, leading to different light distributions, thereby improving the quality of the light field image and reducing stitching effects and fluctuating photometric effects.

[0092] Figure 7 This is a graph of a dataset of luminance or brightness values ​​of a single reference pixel located at y0 in a directional pixel array, based on a pixel illumination pattern across the pixel array from y0. -9 to y +9 The light intensity at y0 is shown when the origin pixel of the pixel illumination pattern is located at the reference pixel; the light intensity at y0 is also shown when the light intensity is offset from the reference pixel by -9 to +9 on the y-axis. It is worth noting that when the illumination pattern is at y... -2 When the light intensity or brightness detected at the reference pixel is centered at y0, it is unexpected but not uncommon. For discrete reference pixels, the measurement obtained by scanning the light distribution pattern along the axes in the pixel array can be represented as a pixel brightness matrix (L), which describes the light distribution or light blurring at a single reference pixel.

[0093] Figure 8 A graphical depiction of the light intensity of a reference pixel, based on the distance between the origin of the pixel illumination pattern and the reference pixel, is illustrated, showing multiple reference pixels in a projector-based display plane. The luminance versus offset graph shown is an example of a one-dimensional uniaxial (y-axis) pixel luminance matrix (L), which describes the luminance contribution of light from pixels surrounding the reference pixel at the origin 0 to the light detected at the light sensor. A similar two-dimensional biaxial (xy) pixel luminance matrix (L) can be used to represent the contribution of pixels adjacent to the reference pixel in the x and y directions of the pixel array to the light. Luminance or brightness values ​​(B) are measured and shown for multiple pixels in the green channel in the projector-based display plane. These graphs are labeled by the holographic cell positions (r<row number>, c<column number>) in the display plane, and a reference pixel is measured for each holographic cell. The asymmetry of the peaks and the abrupt transitions at the projector transitions (e.g., as shown in rows 64-66) can be readily observed.

[0094] The light received at the optical sensor from each oriented pixel in the light field display plane can have a wide and / or asymmetric distribution around the central brightness point. The light distribution of each pixel can depend on, for example: pixel position within the display plane, differences in pixel microcavities (in OLED displays), differences in optical components, differences in electronic control systems within and between pixels and holographic units, crosstalk or leakage within pixel-forming devices, asymmetric optical paths in imaging optics, and diffraction of light through optical apertures. Furthermore, abrupt changes in light output can occur during transitions between projectors or holographic units, potentially causing splicing effects at holographic unit boundaries. In addition to blurring of content far from the display plane, darkening at certain projector boundaries in projector-based displays can also be clearly observed in the projected image and can be quantified using this method. Pixel diffusion within the projector takes the form of a smoothing function, such that the light intensity “lost” from the expected pixel can be “regained” from adjacent pixels in any given view. At projector boundaries, the light distribution contribution from adjacent pixels decreases, which can create discontinuities, resulting in a net loss of luminous intensity in some views from pixels at the edges of the holographic unit. This photometric loss at the edges of holographic cells can also be corrected at the pixel level through light field pre-adjustment using this method. By measuring the light distribution at at least one reference pixel in each holographic cell in the display plane, the correction can be applied to all pixels in the holographic cell to reduce the effect of this non-ideal light distribution. This is based on the assumption that adjacent pixels have similar blurring and allows for approximation across the pixel array in the holographic cell. The result of the correction sharpens the resulting light field to correct for the wide light distribution, while smoothing the transitions in the light distribution to blend the light distribution across the projector boundaries. By correcting each pixel to form a customizable input signal, the adjusted output image can be sharpened and smoothed.

[0095] Figure 9This is a flowchart illustrating a pixel input optimization method for a light field display. In one example method, a light sensor is aligned 110 with a reference pixel in the directional pixel array of at least one holographic unit in the light field display. A pixel illumination pattern 112 is then displayed on the directional pixel array by providing instructions to the light field display to illuminate pixels in the display according to the pixel illumination pattern. Then, with the pixel illumination pattern illuminated, a light intensity value at the reference pixel is measured 114. The pixel illumination pattern is then offset 116 along an axis (e.g., the x-axis or y-axis). The process of displaying the pixel illumination pattern and measuring the light intensity at the reference pixel is repeated 118 for -n to +n pixels along one or both of the x-axis and y-axis, relative to the reference pixel in the pixel array. Then, based on the brightness measurements when the pixel illumination pattern is displayed at various positions on the directional pixel array, a pixel brightness matrix (L) 120 describing the light emitted in the direction of the reference pixel is generated. The pixel brightness matrix (L) can be a one-dimensional matrix displaying brightness variations across a single axis, or a two-dimensional matrix displaying brightness variations along both the x-axis and y-axis. Then, the pixel brightness matrix (L) measured at the reference pixel can be used to apply the correction factor to the pixel input of all pixels in the same holographic unit to sharpen and smooth the resulting image 122.

[0096] Figure 10 This is a flowchart illustrating a method for generating a pixel brightness distribution by displaying and collecting brightness data of a pixel illumination pattern at multiple offset distances from a reference pixel. First, a pixel illumination pattern is displayed on a display plane, with the origin pixel located at the reference pixel at position 130. Then, a light intensity value is measured at the reference pixel. In another embodiment, the pixel illumination pattern may begin at different positions on the display plane relative to the reference pixel. Then, the same pixel illumination pattern is displayed on the display plane, with the origin pixel at a certain pixel offset distance from the reference pixel, and a light intensity value 132 is measured at the reference pixel. This process is then repeated for multiple offset distances relative to the reference pixel, and the light intensity is measured at the reference pixel. Then, a measurement contribution M 134 is determined based on a pixel brightness matrix generated centered on the reference pixel and the brightness measured according to the following formula:

[0097]

[0098] in:

[0099] L px It is a pixel brightness matrix

[0100] M is the measurement contribution, L measure It is the measured brightness matrix.

[0101] Then, the pixel-by-pixel blur kernel K 136 can be calculated, and the pixel brightness distribution L 138 of the reference pixel can be stored.

[0102] View optimization using peak fitting

[0103] A light field projector display is characterized using optical measurements and pixel input optimization, where peak fitting of luminance versus offset measurements is used to identify the optimal correspondence between pixels and the viewer. This display characterization and pixel input optimization method reduces sensitivity to measurement noise and achieves correct allocation even when the intensity distribution differs significantly from a simple Gaussian function. In a projector-based display, all pixels within a holographic unit are essentially under the same conditions. Once a pixel luminance matrix is ​​generated, it can be applied to all pixels within the same holographic unit.

[0104] Measurements obtained at a reference pixel provide offset data in small increments in one of the x and y directions. The brightness of each holographic cell can be measured and stored as a function of the offset displacement. The brightness and displacement data can be processed to determine the optimal peak position, for example, using a brightness-weighted average position or fitting a brightness peak model. The optimal peak position can be used to update the pixel offset correction factor. This process can then be repeated for the other (x, y) direction. The process can be further repeated to improve the measurement signal and / or accuracy using different pixel illumination patterns or different pixel offsets. Two-dimensional scanning can also be used, i.e., changing x and y within a region and processing two-dimensional peaks simultaneously. The light sensor can be moved in a defined manner to collect additional peak measurement data. An example pixel illumination distribution of a single-pixel illumination pattern can be used to enhance the measurement signal s in the manner described by the brightness distribution:

[0105]

[0106] in:

[0107] V(x,y) is the sRGB value sent to the pixel.

[0108] r = max(xx i yy i ) is the distance from the center pixel in (x i , y i Chebyshev distance at ) and

[0109] R is the configured pattern size.

[0110] The initial measurements of the exemplary projector-based light field display included three different peak types. A peak model attempted to fit the data to each model type and determine the best fit, then assigned peak positions based on the fitting parameters and model type. The three peak types selected for use were:

[0111] Standard Gaussian:

[0112]

[0113] Equipeaker-shaped double Gaussian:

[0114]

[0115] Deflected Gaussian:

[0116]

[0117] The peak position for each peak type is as follows:

[0118] Standard Gaussian: µ

[0119] Equipeaker-shaped double Gaussian: (µ1+µ2) / 2

[0120] Skewed Gaussian: Peak value

[0121] To determine the holographic cell offset correction factor, an initial array of holographic cell offsets was determined, where (k x , k y ) is the holographic unit index, k c It is a color index (R / G / B), k d It is the offset direction (x, y). For each direction... and each color First, the offset displacement s relative to the reference pixel from -N to +N is performed through the following steps:

[0122] a) Generate an image as a pixel illumination pattern, where the center pixel is offset by s:

[0123]

[0124] Where S(k) x , k y , k c , k d ) is the initially estimated offset correction factor, and the transformation applies s only to the measured dimension, i.e.:

[0125]

[0126] b) Send the generated pixel illumination pattern image to the monitor under test.

[0127] c) Capture a camera image of the displayed pixel illumination pattern at the reference pixel.

[0128] d) Determine the per-holographic unit brightness from the image captured at the reference pixel, and

[0129] e) Store the brightness of the offset displacement s:

[0130]

[0131] For each holographic unit (k) x , k y ) and color / direction c and d, Then, it can be used as a function of s to fit the above brightness peak model and return the peak position. Then the offset array can be updated. Multiple iterations can be performed if necessary. It has been observed that with each iteration, the offset assignment improves, resulting in an increase in the measured brightness in step (d) above. After adjusting the holographic unit offset using this process, a similar process can be used to estimate the angular distribution of each pixel in the display. The pixel brightness distribution narrows and can be repeatedly determined for the x and y directions without updating the offset adjustment. The pixel spread at the reference pixel can then be estimated as the inverse result of the pixel offset brightness distribution and saved as a per-holographic unit pixel spread function. In this method of measuring the brightness and offset of a specific reference-oriented pixel, a correction can be applied to correct the angular distribution of light emitted at that pixel. By applying correction to the pixel input, the displayed image can be sharpened and smoothed, thereby improving perceived depth and reducing discontinuities at projector or pixel edges. By performing per-holographic unit measurements of the angular distribution of light from the reference pixel and applying the calculated correction factor to the input pixels in the light field display, the transitions between discrete optical components are fused by adjusting the input image to artificially match the angular light distributions of adjacent pixels, thereby reducing the perceived transitions caused by switching between optical elements.

[0132] Figure 11This is a flowchart illustrating a method for characterizing and optimizing pixel inputs of an entire light field projector display through optical measurements. The method for determining and applying corrections to individual holographic cells in a light field display can be applied simultaneously to multiple holographic cells in the display. This method enables simultaneous measurement of multiple pixels located at various positions because interference from light (if any) is minimal. To begin measurement, a light sensor is positioned at a distance from the display plane of the light field display and placed within the viewing area 150. Measurements are then performed by the light sensor at fixed positions and distances relative to the display plane and within the viewing area of ​​the display. Measurements begin by identifying which pixel(s) in each holographic cell in the display plane most directly points to the light sensor 152, referred to herein as the reference pixel in each holographic cell. Since each holographic cell in a light field display comprises multiple oriented pixels, the position of the light sensor will typically have a brightest pixel in each holographic cell, which is the oriented pixel from which directional light rays originate and are most aligned with and received by the optical sensor. The brightest oriented pixel in each holographic cell is defined as the reference pixel of each holographic cell at the fixed position of the light sensor relative to the display plane. Clearly, if the light sensor is placed at different positions (x, y) relative to the display plane, the different oriented pixels in each holographic unit will be identified as reference pixels for that light source position. Then, the set of reference pixels in the light field display (one for each holographic unit) is selected as the origin for illumination measurement.

[0133] To measure the light distribution of the entire light field display, a pixel illumination pattern is selected for the study, and a small group of oriented pixels in each holographic unit are illuminated within the pixel illumination pattern in an otherwise black display 154. Typically, the first position for displaying the pixel illumination pattern on each holographic unit will center the pixel illumination pattern or have an origin pixel located on the y-axis or x-axis of a reference pixel in the same holographic unit. While the pixel illumination pattern is illuminated, the light output at the reference pixel is measured at a light sensor or camera positioned above the display plane and pointing towards the display, and the luminance or luminance measurement value is stored 154. During scanning (from the observer's perspective), the illuminated pixels move around each holographic unit in the display, while the light sensor monitors the luminance or luminance at the reference pixel in each holographic unit 156, and stores the luminance measurement value at each location as a pixel luminance matrix centered on the reference pixel 158. The offset distance of the light received by the light sensor relative to the origin pixel to the reference pixel in either or both of the x- and y-directions, and for one or more pixel colors (R, G, B) in the display plane, is recorded. Collect brightness or light intensity curves centered on the reference pixel in each holographic unit.

[0134] In an RGB color light field display, the light field input has three color channels, and a color-specific correction factor can be calculated independently for each color channel. These three correction factors can then be combined into a single input light field for pre-adjusting the light field. In a dual-color light field display, a correction factor can be determined for each of the two color channels. To determine the correction factor for each color channel, the light sensor used can be color-sensitive or color-insensitive. For a light sensor that only detects photometric intensity and not color, each color channel can be detected independently. For a color-sensitive sensor, color filters can be used, allowing for the simultaneous measurement of multiple color channels.

[0135] The input light field can then be pre-adjusted using a pixel brightness matrix in a sharpening step compatible with spatially varying blur, as described herein. Sharpening and smoothing steps based on the pixel-wise light distribution within each holographic unit can also be used to identify achievable output target distributions to adjust the input light field. Deconvolution algorithms can also be used to solve for artificial blur to reduce it. In this way, light field smoothing can address abrupt regional changes in the measurement distribution, resulting in a smoother light field display image with fewer stitches, creating a visually more continuous light field.

[0136] Sharpening and Smoothing

[0137] In an ideal light field display, light from each pixel is output as a narrow, symmetrical beam around its principal ray direction. Current display characterization and calibration are used to adjust for pixel misalignment relative to the principal ray direction. However, the light distribution from any pixel may be wide and / or asymmetrical around the ideal central brightness point. Optical design constraints of light field displays can lead to contrast loss between adjacent pixels and blurring of the output image. Most optical blurring caused by hardware constraints remains unchanged after the light field display device is assembled; however, the data sent to individual pixels can be fully controlled, allowing for content preprocessing to generate an output light field that is closer to the ideal output (i.e., with less blur). The distribution of light in the light field can also depend on the pixel positions within the display and can change abruptly during transitions between projectors. The calibration described herein aims to reduce the effects of this non-ideal light distribution by adjusting the input to the light field display to sharpen the resulting light field image, correct for wide light distributions, and smooth the transitions in the light distribution to blend the distribution across projector boundaries. Preferably, smoothing is performed before sharpening to utilize a combination of physical and software kernels, allowing physical blur to be replaced by a combination of physical blur and software smoothing during the sharpening step. The result should be a smooth kernel, but with a sharper overall image. Note that in some cases, sharpening can be performed before smoothing. Image sharpening and smoothing processes can also be combined simultaneously to enhance image depth and smooth projector transitions. This requires solving for both the smoothing and sharpening kernels simultaneously to produce a combined kernel with acceptable smoothness and minimal spread.

[0138] Sharpen

[0139] The goal of sharpening is to make the output light field closer to the ideal light field LF. target In the sharpening method, the photometric data collected for each holographic unit using this method can be processed as a pixel brightness matrix using custom calculations, peak fitting, and data processing. Then, a blur kernel, mathematically describing pixel diffusion, can be applied to each holographic unit based on the pixel brightness matrix. Since pixels in different regions of the display plane and in different holographic units can have significantly different pixel brightness matrices, the blur kernel will also vary significantly across the display. The sharpening implementation described herein enables region correction based on the position of each pixel in the light field display and applies the varying blur kernel to the pixel input. Complete per-holographic-unit measurements enable more sensitive sharpening and customized solutions for each holographic unit. Compared to average image sharpening methods, this method has been shown to reduce the perception of transitions between different optical elements and produce a smoother, more continuous light field. Therefore, this method addresses a neglected change in the physical properties of light emitted by discrete elements in a light field display.

[0140] This sharpening method can be variably applied to light field displays on a per-holographic-unit basis. In one example, the sharpening method can use an iterative correction algorithm, where for each iteration k, the initial input is passed through an ideal light field and compared with the input LF from the current step. k The analog display output L•LF k The differences between them are updated. The input for the first step is an ideal light field LF consisting of narrow rays surrounding the principal ray. target Each iteration k can follow the following formula:

[0141]

[0142] in:

[0143] LF k It is the input light field of the current step.

[0144] LF k+1 The resulting light field is used as the input for the next iteration.

[0145] λ is a scaling parameter, and it is a variable parameter between different iterations.

[0146] L is the pixel brightness matrix, which is a measure of pixel diffusion, also known as pixel blur.

[0147] L•LF k It is L and LF k The convolution, and

[0148] LF target It is an ideal light field output, originally used as an input.

[0149] This sharpening process can be repeated a desired number of iterations to achieve the desired image sharpness. During the iterations, the first sharpening light field LF1 is set to approximate the target LF1 = LF. target Then, this can be achieved by first calculating the expected physical output light field (i.e., the convolution K•LF of the pixel blur and the input light field). k The sharpening light field is iteratively adjusted over k steps. Then, a convolution is computed for each group of pixel regions (in this case, each holographic unit in the display plane) such that for each holographic unit (a,b) in the display plane, K•LF is computed using the pixel blur K(a,b) of that holographic unit and the light field portion of that holographic unit. k The value of K•LF. Total output K•LF k It displays all K on the plane. (a,b) • LF k (a,b) The combination of .

[0150] Then, the difference between the output light field and the target light field in this step can be used to apply the sharpening equation described above. Update the input light field for the next step. The value of λ can then be set to accelerate solution convergence without causing output oscillation. Specifically, the value of λ can be varied between different iteration levels. For example, the value can initially be set higher to approximate the ideal light field, and then adjusted downwards for more accurate sharpening correction. In one example, the value of λ can be dynamically adjusted as an option to accelerate convergence. The final sharpened light field is the light field calculated in the last iteration step:

[0151]

[0152] in:

[0153] LF sharp It is the sharpened light field output by the sharpening process, and

[0154] LF C+1 The light field is obtained by iterating through the above process C.

[0155] The target light field can have any content, including a wide range of frequencies and contrast levels. Therefore, the sharpening algorithm should be general enough to adapt to various colors and intensities. As an alternative sharpening method, equation L•LF in =LF target The LF can be obtained by solving the deconvolution algorithm. in Many algorithms exist for similar deconvolutions, although these generally cannot handle variations in the light field L. The presence of a spatially varying kernel L is uncommon in other display systems because regional differences are typically absent in the display plane of 2D display systems that rely on fixed kernel measurements for sharpening. In autostereoscopic displays, the form of the kernel can vary drastically across the display, not only in luminance but also in luminance distribution. For example, some holographic cells have highly symmetrical peaks, while others have very asymmetric and bimodal peaks. This vast variation in luminance patterns makes it impossible to parameterize the kernel across the entire light field display. This solution measures the luminance and luminance pattern of each holographic cell in the display plane, and then applies a kernel specific to each holographic cell to each pixel in the display accordingly.

[0156] In a holographic cell-based light field display, by measuring the sharpening kernel for each discrete region and applying light field input correction based on pixel illumination for each holographic cell region, a customized sharpening solution can be provided for each region in the display plane. This iterative sharpening technique excels at identifying locally optimized kernels for regions with significant contrast variations and unrestricted input light fields, resulting in a highly sharpened field with pixel input.

[0157] Smoothing

[0158] In light field displays, splicing optics (such as holographic units or projectors) introduce abrupt changes in physical blur (K) at boundary regions, where one splicing optic ends as another begins. This sudden increase in blur level leads to visual artifacts, perceived as discontinuities in the 3D display content, such as abrupt changes in overall brightness along boundary edges. These anomalies can also manifest as gaps or overlaps between splicing optics. Due to the discrete nature of these transitions and the considerable spacing between holographic units in the light field, determining what constitutes a “sufficiently smooth” output image is not straightforward. That is, “smoothness” criteria applicable to 2D display characteristics (e.g., including the concave region) may not be directly applicable to holographic displays or stereoscopic light field displays. The oriented pixels in each splicing optic are affected by local conditions that alter the luminance and luminance distribution of each pixel within the splicing optic. Because each splicing optic exhibits different local behaviors, the smoothing process must be considered locally for each splicing optic, taking into account a subset of pixels within that element.

[0159] The basic concept behind smoothing is to ensure the physical output light field LF out There are no abrupt changes in output intensity at any location. This problem arises when the blur kernel K in the physical system changes, such as between holographic units. The input light field can be adjusted to hide abrupt transitions in K, thus making the light field image smoother. To reduce discontinuities caused by varying pixel diffusion at projector boundaries, the input can be adjusted to artificially add distribution to the pixels, thereby creating a continuous blur at projector transitions. The target image should result in a smooth and effective blur at the projector edges of the display output.

[0160] A software blur kernel G can be found and applied to the pixel input to smooth the image in boundary regions of any projector or holographic unit boundaries where abrupt transitions L are expected. The characterization process determines the software blur kernel G to be applied such that the combined convolution L·G of the physical blur and the software-applied blur is smooth. Therefore, a smoothing method can be applied to the correction factor, involving the application of an artificial distribution to multiple pixels in the holographic unit. The artificial distribution can be applied to the input image using the pixel brightness matrix L via the software blur kernel G, and then sent to the light field display such that the image sent to the display is (G • LF). in ), and the image emitted by the display is:

[0161]

[0162] in:

[0163] L is the measured pixel brightness matrix of the display (L), representing pixel diffusion.

[0164] G is the derived smoothing kernel, and

[0165] LF in It is the raw, unprocessed input light field.

[0166] An illustrative example of this implementation involves estimating an achievable smooth target kernel (denoted as H) based on the measured display kernel K. This estimation is performed independently for each holographic unit or pixel, depending on the processing level. Here, H is calculated as a weighted average of the surrounding measured kernels. When averaging only along the kernel direction, the one-dimensional average takes the following form:

[0167]

[0168] An example of possible weights is:

[0169]

[0170] Note that the weight of the central holographic unit is negative, making it more sensitive to... It has a negative correlation effect.

[0171] To determine the required software blur kernel G, the convolution equation K•G = H can be solved, for example, using the Richardson–Lucy algorithm. For each holographic unit index k, the pixel blur K at which that holographic unit is applied can be determined. k The most effective way to reproduce H afterwards k The software core of the kernel. This smoothing implementation reduces visual artifacts caused by splicing optical components at boundary regions, resulting in a more visually continuous light field output.

[0172] When light in a single holographic unit comes from more than one stitching optics, a smoothing preprocessing can be applied to the pixels within the holographic unit to make the light field image smoother. In one example, for holographic unit Q, a camera or light sensor at a specific position relative to the display plane primarily sees a specific pixel N. Pixel N is referred to as the reference pixel in the determination method described above. If light from pixel N passes through optics A, then all pixels within holographic unit Q that also pass through optics A can use the measured kernel K of that holographic unit. A(Q) However, for pixels in Q originating from optical element B, the measured kernel K is not used. A(Q) Instead, it extrapolates the kernels of these pixels from the kernels of the holographic units near the actual directly measured element B.

[0173] Figure 12The example illustrates sub-sections of the light field before and after sharpening, parallel to the ideal light field output. This light field is processed using an exemplary blur kernel K=2000, where λ=0.006 and gradually decreases to 5×10⁻⁶. -4 The sharpened image clearly shows that, compared to the unprocessed image, the sharpening method can reduce image degradation caused by blur, restore some overall brightness, and obtain clearer features.

[0174] Figure 13 An arrangement for measuring a projector-based multi-view display is illustrated, comprising a (x,y) display plane 10 and two light sensors 12a, 12b positioned at different reference points relative to the display plane 10. Light from individual pixels within the display plane can be captured by the light sensors 12a, 12b. These sensors detect light at different sets of reference pixels within each holographic unit on the display plane, light projected from the projector 26. The light intensity of different pixels is measured accordingly. This measurement can be used to improve the accuracy of per-holographic unit measurements, or to perform measurements and pre-adjustments at a finer resolution than per-holographic unit measurements. For example, four light sensors placed in different quadrants of the display viewing plane can provide per-quarter-holographic unit correction when needed.

[0175] Figure 14 An example is a projector-based multi-view display with splicing optics configured to provide light to each pixel in each holographic unit of the multi-view display. Projectors 26a-26e project light to form a display plane 10, which can be detected at a light sensor 12.

[0176] Figure 15 An example of a flat panel multi-view (x,y) display plane 10 is illustrated, in which a light sensor is located at a reference position. In the case of a flat panel display, the display plane 10 is formed by an array of pixels of light sources (such as LEDs, OLEDs, etc.) arranged in multiple holographic units. Each holographic unit has directional pixels 16 with directional rays 20 that emit light at various angles relative to the display plane 10, thereby forming a stereoscopic display. In this method, a light sensor 12 positioned at a distance from the display plane 10 receives light from the directional rays 20 of the pixel sources in the display plane 10 at a sensor origin 24.

[0177] All publications, patents, and patent applications mentioned in this specification reflect the skill level of a person skilled in the art to which this invention pertains and are incorporated herein by reference. Any reference to prior art in this specification does not constitute and should not be construed as an admission or in any way imply that such prior art is part of common general knowledge.

[0178] As will be apparent from the present invention as described herein, variations are possible in many respects. These variations should not be considered as departing from the scope of the invention, and all such modifications that will be apparent to those skilled in the art are intended to be included within the scope of the appended claims.

Claims

1. A pre-conditioning method for an input optical field, the pre-conditioning method comprising: The light sensor is positioned to receive light from the holographic unit in the light field display plane; A pixel illumination pattern centered on the first pixel position in the holographic unit is displayed on the light field display plane; The light intensity value at the reference pixel in the holographic unit is measured using the light sensor. The pixel illumination pattern is displayed on the light field display plane at an offset position relative to the first pixel position; When the pixel illumination pattern is at the offset position, the light intensity value at the reference pixel is measured; The brightness distribution of the reference pixel is generated based on the light received by the light sensor at the reference pixel; as well as Calculate the correction factor for the reference pixel.

2. The method according to claim 1, further comprising: The pixel illumination pattern is displayed at multiple offset positions relative to the first pixel position, and the light intensity value at the reference pixel is measured at each of the multiple offset positions.

3. The method according to claim 1 or 2, further comprising: The correction factor is applied to the input light field of at least one pixel in the holographic unit.

4. The method according to claim 3, further comprising: The correction factor is recalculated for each of the multiple input optical fields.

5. The method according to claim 3 or 4, further comprising: The correction factor is applied to the input light field of all pixels in the holographic unit.

6. The method according to any one of claims 3 to 5, wherein the correction factor adjusts the luminance of at least one pixel in the holographic unit.

7. The method according to any one of claims 1 to 6, wherein the pixel illumination pattern is a single pixel pattern, a linear pattern, a stepped linear pattern, a square pattern, a stepped square pattern, a rectangular pattern, a cross pattern, a stepped cross pattern, a puncture pattern, or a target pattern.

8. The method according to any one of claims 1 to 7, wherein the light intensity value at the reference pixel is measured in a plurality of color channels.

9. The method of claim 8, wherein the correction factor is calculated for each of the plurality of color channels.

10. The method of claim 8 or 9, wherein the correction factor for each color channel is combined and applied to the input light field.

11. The method according to any one of claims 1 to 10, wherein the offset position relative to the first pixel position is along one or both of the x-axis and y-axis of the holographic unit.

12. The method according to any one of claims 1 to 11, further comprising: Calculate the pixel-by-pixel blur kernel based on the brightness distribution.

13. The method according to any one of claims 1 to 12, wherein the light field display plane is formed by a light field display, the light field display comprising an oriented pixel array in a projector-based light field display or a flat panel light field display.

14. The method of claim 13, wherein the flat panel display is one of LED, OLED, LCD or MicroLED display.

15. The method according to any one of claims 1 to 14, wherein a correction factor is calculated such that one or more of the light field is sharpened and the light field is smoothed.

16. The method according to any one of claims 1 to 15, wherein the light sensor measures the light intensity from more than one holographic unit in the light field display plane.

17. The method according to any one of claims 1 to 16, wherein the light sensor simultaneously measures the light intensity values ​​in a plurality of color channels.

18. The method according to claim 17, further comprising: The additional light sensor is positioned to receive light from the additional reference pixel in the holographic unit in the light field display plane; A pixel illumination pattern centered on the additional pixel positions in the holographic unit is displayed on the light field display plane; The light intensity value at the additional reference pixel in the holographic unit is measured using the light sensor. The pixel illumination pattern is displayed on the light field display plane at an additional offset position relative to the additional pixel position; When the pixel illumination pattern is at the additional offset position, the light intensity value at the additional reference pixel is measured; as well as The brightness distribution of the additional reference pixel is generated based on the light received at the additional reference pixel by the additional light sensor.

19. The method of claim 18, wherein calculating the correction factor comprises: An additional correction factor for the additional reference pixel is incorporated into the correction factor.

20. A system for pre-adjusting a light field display, the system comprising: A light sensor for measuring the light intensity value in the direction of a reference pixel in a holographic unit in the display plane of a light field display; A frame for supporting the light sensor in front of the display plane; and A processor configured to perform the following methods: A pixel illumination pattern centered on the first pixel position in the holographic unit is displayed on the display plane; The pixel illumination pattern, which is offset relative to the first pixel position, is displayed on the display plane; A luminance distribution is generated based on the light intensity values ​​measured by the light sensor for the reference pixel located at the first pixel position and the offset position. The luminance distribution describes the light received from the light source on the display plane along the direction of the reference pixel.

21. The system of claim 20, wherein the processor calculates the correction factor of the reference pixel.

22. The system of claim 20 or 21, wherein the light field display comprises an oriented pixel array in a projector-based light field display or a flat panel light field display.

23. The system of claim 22, wherein the flat panel display is one of LED, OLED, LCD or MicroLED display.

24. The system according to any one of claims 20 to 23, wherein the processor is further configured to perform the following steps: Calculate the correction factor for the reference pixel; The correction factor is applied to the input optical field to generate a corrected optical field; and The corrected light field is displayed on the light field display.

25. The system according to any one of claims 20 to 24, wherein the optical sensor is a single-frequency optical sensor, a multi-frequency optical sensor, a photodetector, a charge-coupled device (CCD), a liquid crystal on silicon (LCOS) sensor, or a camera.

26. The system according to any one of claims 20 to 25, wherein the optical sensor includes one or more additional optical components for optical deflection, lensing, collimation, or guiding of light.

27. The system according to any one of claims 20 to 26, wherein the offset position relative to the first pixel position is along one or both of the x-axis and y-axis of the holographic unit.

28. The system according to any one of claims 20 to 27, wherein the system comprises more than one optical sensor.

Citation Information

Patent Citations

  • Improvement in step-spindles

    US101109A

  • Method and apparatus for correcting distortion

    US10523928B2