Viewing angle dependent color / brightness correction for display systems

By tracking the viewer's posture and correcting the pixel value for viewing angle, the problem of the display device's brightness and contrast decrease when the viewing angle changes is solved, and the consistency of the color saturation and contrast of the image at different viewing angles is achieved.

CN113272883BActive Publication Date: 2024-11-01GOOGLE LLC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201980044640.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2019-12-12
Publication Date
2024-11-01
Estimated Expiration
2039-12-12

AI Technical Summary

Technical Problem

Existing display devices can cause a decrease in brightness and contrast when viewing angles change, and the color response is uneven, resulting in undesirable color shifts and contrast loss of the image.

Method used

By tracking the viewer's head or eye posture, determining the viewing angle of each pixel and correcting the pixel values ​​based on the viewing angle, generating alternative corrected pixel values ​​to offset the viewing angle-dependent changes in the display panel's viewing angle-dependent brightness or color response, ensuring The image has consistent color saturation, brightness and contrast at different viewing angles.

Benefits of technology

It effectively reduces the impact of viewing angle changes on the image, improves the color saturation, brightness and contrast of the image at different angles, and provides a more stable visual experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113272883B_ABST
    Figure CN113272883B_ABST
Patent Text Reader

Abstract

A display system (100) includes a tracking system (104) configured to track one of a posture of a viewer's head or a posture of a viewer's eyes. The display system further includes an image correction module configured to determine, for each pixel in at least a subset of pixels of an input image (318), a pixel viewing angle of the pixel based on the posture or gaze direction, determine a correction pixel value based on an input value of the pixel in the input image and based on the pixel viewing angle; and provide the correction pixel value for a pixel in an angle-corrected image (322) corresponding to the input image. The display system further includes a display panel (106) to display the angle-corrected image.
Need to check novelty before this filing date? Find Prior Art

Description

Background Art

[0001] Due to radiation and physical dependencies, many display devices have brightness or color responses that are dependent on viewing angle. For example, for liquid crystal displays (LCDs), bright pixels tend to become darker as the viewing angle increases, while dark pixels become brighter as the viewing angle increases, largely because of the angle-dependent phase delay of the liquid crystal (LC) molecules. Similarly, many displays use polarizing films to reduce reflections. However, these films have some variations in extinction ratios based on wavelength and viewing angle. Brightness changes based on viewing angle are also often color-dependent, and therefore increased viewing angles can result in undesirable color cast desaturation and loss of contrast in the displayed image. BRIEF DESCRIPTION OF THE DRAWINGS

[0002] The present disclosure may be better understood, and its numerous features and advantages may be made apparent to those skilled in the art by referencing the accompanying drawings. The same reference numerals are used in different drawings to indicate similar or identical items.

[0003] Figure 1 is a block diagram of a display system that employs viewing angle dependent color / brightness correction processing in accordance with some embodiments.

[0004] Figure 2 is a diagram illustrating the relationship between pixel viewing angle and illumination for multiple pixel values ​​in accordance with some embodiments.

[0005] Figure 3 According to some embodiments Figure 1 A block diagram of the display system's hardware configuration.

[0006] Figure 4 is a flow chart illustrating a method of viewing angle dependent image correction in accordance with some embodiments.

[0007] Figure 5 is a diagram illustrating a technique for mitigating the effects of quantization during view-dependent image correction in accordance with some embodiments.

[0008] Figure 6 is a diagram illustrating a technique for mitigating the effects of clipping during view-dependent image correction in accordance with some embodiments.

[0009] Figure 7 is a diagram illustrating a display system employing viewing angle-dependent image correction for multiple simultaneous viewers in accordance with some embodiments.

[0010] Figure 8 is a diagram illustrating a display system that employs viewing angle dependent correction for one or more viewers of a light field display in accordance with some embodiments. DETAILED DESCRIPTION

[0011] The physical properties of the pixel matrix of many types of display panels or the polarizer film covered cause the color and brightness of each pixel to be perceived by the viewer as depending on the viewing angle of the pixel relative to the viewer. As a result, the farther the viewer is positioned from the central viewing axis of the display panel (or the farther the viewer's gaze direction extends from the edge in a near-eye display scenario), the more brightness and contrast are shifted or reduced. Various embodiments of a display system are disclosed herein, which mitigate the spectral / angular response of such a display panel by performing a viewing angle-dependent correction on its pixel values ​​before the image is displayed on the display panel. In at least one embodiment, the display system tracks the viewer's head (or the viewer's eyes in a near-eye display real-time mode (e.g., as a gaze direction or other eye gesture)), and thereby determines the current gesture of the viewer (head and / or eyes) relative to the display panel. For each pixel of the input image intended to be displayed at the display panel, the display system determines the pixel viewing angle of the pixel from the current gesture. As used herein, the term "gesture" may refer to one or both of position and orientation. Thus, for example, the gesture of the eye may indicate one or both of the position of the eye and the direction of the eye's line of sight. The display system applies corrections to pixel values ​​(e.g., red-blue-green (RGB) values) based on the pixel viewing angle of the pixel to produce angle-corrected pixel values ​​that are used in place of original input pixel values. The corrections applied to the pixel values ​​represent corrective adjustments to offset the viewing angle-dependent brightness or color response of the corresponding pixels of the display panel. The process is repeated for some or all of the pixels in the input image, and the resulting corrected image is provided to the display panel for display. In this manner, the brightness and contrast of each pixel in the displayed image are adjusted as a function of the viewing angle of the viewer of the corresponding pixel in the display panel.

[0012] For illustrative reasons, an example embodiment is described with respect to one display panel. The one display panel, for example, represents a single-panel display system, such as a television, a single-monitor computer system, a smart phone, a tablet computer, and the like. However, in other embodiments, the display system may include multiple display panels, in which case the image correction process described herein that depends on the viewing angle may be performed simultaneously for each display panel. For illustration, in some embodiments, the display system is a stereoscopic display system having one display panel for each of the left and right eyes of the viewer, and wherein the image provided to the left-eye display panel is subjected to the image processing described herein that depends on the viewing angle based on the gaze direction or posture of the left eye, while similar processing is performed for the image provided to the right-eye display panel based on the gaze direction or posture of the right eye. As another example, the display system may include a stereoscopic light field display system, wherein a lenticular lens or other mechanism causes a separate image to be provided for each of the left and right eyes of the viewer (or in some cases, a separate image is provided for each eye of each of the multiple viewers), and the image correction process that depends on the viewing angle may be applied to each image for each eye based on the gaze direction or other gesture indication of the corresponding eye of the viewer.

[0013] Figure 1 A display system 100 employing view-dependent image correction processing according to at least one embodiment is shown. The display system 100 includes any of a variety of electronic systems used to display video or other images, including smart phones, tablet computers, laptop computers, computer monitors, video game devices, televisions, and other flat panel displays, as well as near-eye display devices such as augmented reality (AR) or virtual reality (VR) head-mounted devices, and the like. The display system 100 includes a display device 102 and a tracking subsystem 104. The display device 102 includes at least one display panel 106 composed of a matrix of pixels for displaying a sequence of images to at least one viewer. In the illustrated embodiment, the display device 102 is a single display system, such as a television, a computer monitor, a smart phone, a tablet computer, and the like. However, in other embodiments, the display device 102 may, for example, include a virtual reality (VR) headset, an augmented reality (AR) headset, or other near-eye stereoscopic display system, and thus, for example, include two display panels, one for each eye of the viewer, or include a single display panel 106 that is used to present a separate image to each eye of the viewer, such as by using a light field display with a lenticular lens, by using a polarizer or shutter, etc.

[0014] The tracking subsystem 104 includes any of a variety of systems known in the art for viewer head tracking (e.g., head pose tracking in the case of non-near-eye embodiments) or eye gaze tracking (e.g., viewer gaze direction tracking in the case of near-eye embodiments). For example, the tracking subsystem 104 may include a stereo camera subsystem that utilizes reflected infrared (IR) light to detect the presence of a user's face, and further detects the position and orientation (i.e., "pose") of the user's face relative to the display panel 106. For ease of reference, the systems and techniques of the present disclosure are generally described in the context of head tracking, but unless otherwise noted, these descriptions apply equally to gaze tracking embodiments configured for near-eye displays. Further, for purposes hereinafter, the pose of a viewer's face (or more generally, a viewer's head) is described herein with reference to a central viewing axis 108, the point of which at the center of the display panel 106 is orthogonal to (i.e., perpendicular to) the XY plane defined by (i.e., extending along the Z axis) the display panel 106. However, in other embodiments, the pose may be defined relative to a different fixed reference (such as a center point between two cameras of the tracking subsystem 104, a specified corner of the display panel 106, etc.).

[0015] like Figure 1 As shown in 110 of the overall overview of the operation of the display system 100, at the iteration of box 112, the tracking subsystem 104 continuously monitors the presence of the viewer, and when detected, determines and periodically updates the current posture of the viewer's head relative to the central viewing axis 108 (or the gaze direction in the near-eye embodiment). At the same time, at the iteration of box 114, the display device 102 obtains an input image from a sequence of input images representing a video stream. The input image includes a plurality of pixel values, each pixel value corresponding to a pixel of the display panel 106. The input image can be a rendered image, an image captured from a live event, or a combination thereof. The video stream can be generated at the display device 102 itself (for example, a video stream generated by executing a video game at the display device 102, a video stream generated from playing a video file stored in a local storage drive or a digital versatile disk (DVD), etc.). In other embodiments, the video stream is remotely generated and transmitted to the display device 102, such as via a remote server or the like.

[0016] At a corresponding iteration of block 116, display device 102 uses the current pose (or gaze direction) of the viewer's head to determine a corresponding pixel viewing angle for each pixel (or each pixel in an identified subset thereof) of display panel 106. For example, when the viewer's head is aligned with central viewing axis 108, such as at head pose 117-1, the viewing angle is determined to be θ 1= 0 degrees (deg.), and the viewer's head has an angular component θ along the X axis at the head posture 117-2. 2X and the angular component θ along the Y axis 2Y The angle of view θ 2 (That is, θ 2 =[θ 2X ,θ 2Y ]). Similarly, the viewer's head has an angular component θ along the X-axis at head pose 117-3. 3X and the angular component θ along the Y axis 3Y The angle of view θ 3 (That is, θ 3 =[θ 3X ,θ 3Y ]). Therefore, the pixel viewing angle is a function of the viewing angle, the distance and distance of the viewer's head from the display panel 106, and the position (X, Y) of the pixel within the display panel 106.

[0017] The pixel viewing angle of each pixel and the pixel value of the pixel are then used to determine a corrected pixel value, which represents an adjustment of the pixel to compensate for the expected angle-dependent display behavior of the corresponding pixel when viewed by the viewer at the corresponding viewing angle. That is, PVcorr = f(PVinput, θpx, θpy), where PVinput is the input pixel value from the input image, θpx and θpy are the X-axis component and Y-axis component of the pixel viewing angle of the pixel, respectively, and PVcorr is the corrected pixel value of the pixel. As described in more detail herein, the correction to be applied given a particular pixel value and pixel viewing angle can be determined empirically, via simulation or modeling, etc., and can be implemented in any of a variety of transform structures such as a lookup table (LUT), analysis function, etc. Since the effect on the color and brightness of the displayed pixel is dependent on the viewing angle, for a given pixel value, the amount of correction applied generally increases with increasing pixel viewing angle. By way of illustration, when the viewer's head is in posture 117-2, an input pixel value for pixel 118-1 in the display panel 106 having a low or zero pixel viewing angle 119-1 will typically have a lower amount of correction applied than the same input pixel value for pixel 118-2 in the display panel 106 having a relatively sharp pixel viewing angle 119-2.

[0018] This process of perspective-based pixel value correction is repeated for some or all of the pixels in the input image, resulting in an angle-corrected display image, which is then output for display on the display panel 106 at a corresponding iteration of block 120. With the per-pixel angle correction applied, the displayed image exhibits enhanced color saturation, brightness, and contrast when viewed by a viewer at a greater perspective than a similar display of the original input image, particularly when the viewer's head is in a position that results in a relatively oblique perspective.

[0019] Figure 2 A graph 200 depicting an example relationship between illuminance and pixel viewing angle for various pixel values ​​is shown. In graph 200, the ordinate represents the illuminance of the pixel as observed by a viewer, the abscissa represents the pixel viewing angle of the pixel relative to the viewer, and each of the ten line graphs depicted represents the relationship between the illuminance observed by the pixel at the corresponding pixel viewing angle for a corresponding pixel value in a set of ten pixel values. As shown in graph 200, the angular response is different for different pixel values. For example, line graph 202-1 demonstrates that for a first pixel value, a pixel illuminated according to the first pixel value actually has a gain in perceived illuminance as the pixel viewing angle increases; while line graph 202-2 demonstrates the opposite relationship for a second pixel value - that is, when the pixel is illuminated according to the second pixel value, the observed illuminance decreases as the pixel viewing angle increases. Therefore, in at least one embodiment, the angle-dependent correction applied by display system 100 is configured to attempt to approximately maintain the same default illuminance (i.e., the illuminance exhibited at a pixel viewing angle of 0 degrees) even as the pixel viewing angle increases. As evidenced by bracket 204, this may involve a correction that increases the brightness of pixel values ​​for those pixel values ​​that experience illuminance loss as the pixel viewing angle increases, and as evidenced by bracket 206, this may also involve a correction that decreases the brightness of pixel values ​​for those pixel values ​​that experience illuminance gain as the pixel viewing angle increases.

[0020] Figure 3 According to some embodiments Figure 1 300 of a simplified hardware configuration of a display system 100. In the depicted example, the display device 102 includes one or more central processing units (CPUs) 302, one or more graphics processing units (GPUs) 304, at least one system memory 306, a frame buffer 308, and an angle-dependent image correction module 310 (referred to herein for simplicity as "image correction module 310"). The display panel 106 includes a display controller 312 and a pixel matrix 314 composed of a two-dimensional array of pixels, such as liquid crystal (LC)-based pixels, light emitting diode (LED)-based pixels, organic LED (OLED)-based pixels, or a combination thereof.

[0021] The illustrated configuration represents an embodiment in which the display device 102 is the source of the input video stream being displayed. Therefore, the CPU 302 operates to execute a video source application 316 stored in the system memory 306, which is used to instruct the GPU 304 to render a stream of input images (e.g., input images 318), each of which is temporarily stored in the frame buffer 308. For example, the video source application 316 may include a video game application, which causes a stream of video images representing a player's perspective to be rendered into a video game scene. As another example, the video source application 316 may include a video decoding application, which is executed to decode the encoded video data stored on a hard disk or other large storage device (not shown) to generate a sequence of decoded input images. In other embodiments, the video data originates from a remote device such as a remote video server, in which case the video data is received on the display device 102 via a wireless or wireless network, temporarily buffered, and decoded or otherwise processed to generate a video stream composed of input images.

[0022] The image correction module 310 is implemented as hard-coded or programmable (e.g., freely programmable, parameter programmable) logic, such as one or both of the CPU 302 or GPU 304, or a combination thereof, that executes correction software 324 representing the functions described herein. In instances where the input image is rendered at the display device 102, the correction software 324 can be implemented as part of the rendering software. In instances where the input image is decoded or otherwise recovered from previously encoded video data, the correction software 324 can be implemented as part of the decoding software. The image correction module 310 operates to receive current posture data 320 representing the current posture (or current gaze direction) of the viewer's head, and based on this information, the pixel values ​​of the input image 318 are modified based on the perspective represented by the viewer's current posture (or current gaze direction) to generate a corrected image 322. In embodiments where the input image 318 is being rendered by the GPU 304, the correction process may be part of the rendering process itself (e.g., as part of or following a pixel shader stage) such that the pixel values ​​are corrected as the image itself is being generated, and thus the input image 318 and the corrected image 322 are the same image. In other embodiments, such as when the input image 318 is generated elsewhere (such as when the display device 102 decodes encoded video data to obtain the input image 318), the correction process may be applied as each pixel value of the input image 318 is obtained or after the input image 318 is completely obtained.

[0023] In at least one embodiment, the image correction module 310 determines corrected pixel values ​​for input pixel values ​​of the input image 318 using a corrective transform component 326 that represents a specified relationship between the input pixel values, the pixel perspective, and the desired corrected pixel values ​​(typically on a per color channel basis, as described in more detail below). As such, the corrective transform component 326 may be implemented, for example, as a LUT that takes as input the input pixel values ​​and the pixel perspective and outputs corrected pixel values, or as a software-implemented or hardware-implemented analytical function that receives these inputs and provides corresponding corrected pixel values. The operation of the image correction module 310 and the corrective transform component 326 is described below with reference to Figure 4-8 Describe in more detail.

[0024] When the angle-dependent corrections have been applied to all applicable pixel values, the resulting corrected image 322 is transmitted to the display controller 312, which then operates to drive individual pixels in the pixel matrix 314 according to the corresponding corrected pixel values ​​in the corrected image 322, and thereby display to the viewer an image that has been corrected or otherwise adjusted to accommodate the viewer's particular viewing angle.

[0025] Figure 4 An example method 400 for generating and displaying a perspective-corrected image at a display device 102 is shown in accordance with some embodiments. For ease of illustration, reference is made to Figure 3 The method 400 is described with reference to the hardware configuration 300 of the display system 100 of FIG. 4. The illustrated method 400 consists of two subprocesses: an angle-dependent correction configuration subprocess 402 and an angle-dependent correction application subprocess 404. The angle-dependent correction configuration subprocess 402 represents a process for initializing or otherwise configuring an angle-dependent correction scheme to be applied by the display device 102. In general, the angle-dependent behavior exhibited by the display panel 106 depends on the technology or physical parameters of its pixel matrix 314 (e.g., LED, LCD, OLED) or overlying polarizing film, the manufacturing process used to employ the technology, and variations in the implementation of the manufacturing process for a specific instance of the display panel 106.

[0026] Thus, the illuminance / pixel viewing angle relationship for a given pixel value typically depends on one or more of the display technology, manufacturing process, panel design, emitter natural angle lighting profile, etc. Therefore, in box 406, the manufacturer, supplier, implementer, or user of the display device 102 determines the relationship between illuminance and pixel viewing angle for each applicable pixel value in the corresponding color space of the display device 102. In at least one embodiment, this is determined on the basis of each color channel. For illustration, assuming that the color space is an RGB color space, the illuminance / angle relationship is determined separately for each applicable red component value, each applicable green component value, and each applicable blue component value. In one embodiment, the illuminance / angle relationship is determined empirically via testing / measurement of the display panel 106 itself or a representative example of the display panel 106. To illustrate, a pixel block of display panel 16 may be configured to display a certain pixel value, and a luminance / color meter may then be positioned relative to the pixel block at different pixel viewing angles and luminance / color measurements taken at each location, and from these readings, a relationship between luminance and viewing angle may be determined for the corresponding pixel value (or its color component values), such as those represented in the line graph of chart 200.

[0027] With the determined illuminance / angle relationships, at block 408, the correction transform component 326 is configured to reflect these relationships. As described above, in at least one embodiment, the correction transform component 326 is implemented as a LUT (e.g., LUT 409) that takes pixel values ​​(e.g., color component values) and pixel viewing angles as inputs and provides correction pixel values ​​in response, wherein the correction pixel values ​​are determined or selected so that when a pixel of the pixel matrix 314 is driven by the display controller 312 based on the correction pixel values, the light emitted by the pixel and observed at the pixel viewing angle is substantially similar in brightness and color to the light emitted by the pixel when driven by the input pixel value and viewed head-on (i.e., at a viewing angle of 0 degrees). That is, the correction pixel values ​​in the LUT for a given input pixel value are configured for each applicable pixel viewing angle so as to provide a substantially constant illuminance that is substantially independent of the pixel viewing angle. In other embodiments, the correction transform component 326 is implemented using different structures implemented in software or hardware, such as an analysis function (e.g., analysis function 411), and is configured in a similar manner to determine a corresponding correction pixel value for each input pixel value / pixel view tuple so that the resulting correction pixel value, when used to drive a pixel in the pixel matrix 314, produces a substantially constant brightness or luminance for the corresponding pixel view. In at least one embodiment, the process is employed on a color basis by applying the above method independently to each color channel; thus, color is preserved when the luminance ratio of the color elements (e.g., RGB channels) is preserved as a function of view. If one of the correction pixel values ​​exceeds the maximum value of the display ("clipping"), the color can be preserved by reducing the non-clipped channel according to the gamma transform of the display so that the luminance ratio is preserved at the expense of local texture.

[0028] In the case where the display device 102 is so initialized, the method 400 proceeds to the angle-dependent correction application process 404 for providing angle-dependent correction to the input image of the video stream prior to display. Therefore, in box 410, the display device 102 obtains an input image to be processed (e.g., input image 318) for the video stream being displayed at the display device 102. As mentioned above, in some embodiments, the input image is obtained by local rendering of the input image at the display device 102, while in other embodiments, the input image is obtained by decoding or other such processing of video data representing the video stream, such as video data received from a remote server or other remote source via a network connection or video data obtained from a local large storage device. In box 412, the image correction module 310 selects a pixel and obtains a pixel value of the input image for the pixel. In the instance where the input image is rendered locally, the selected pixel can be the current pixel being rendered by the GPU 304. In instances where the input image is generated remotely, the image correction module 310 may select pixels from the input image in a predefined order, such as row by row, column by column, block by block, or as each pixel value is determined during a decoding process or other image recovery process. As shown in block 414, the viewing tracking subsystem 104 monitors the pose of the user's head (or gaze direction / eye pose for near-eye display implementations) and provides head pose data 320 (indicating the currently observed head pose (or gaze direction / eye pose)) to the image correction module 310. Figure 3 ). Therefore, at block 416, the image correction module 310 adjusts the image based on the current viewing angle (that is, the current posture of the viewer's head relative to the Figure 1 The pixel viewing angle of the pixel selected at block 412 is determined by the angle of the central viewing axis 108 or other fixed reference of the display panel 106 and the position of the pixel in the display panel 106 relative to the central viewing axis 108 or other reference point. As described above, in one embodiment, the pixel viewing angle can be expressed using the X-axis component θ relative to the XY plane formed by the display surface of the display panel 106. px and the Y-axis component θ py In other embodiments, the viewer's head may be assumed to be relatively fixed along a particular axis, and so instead the pixel viewing angle is represented by only a single angular component relative to another axis. To illustrate, in some instances, such as for a fixed desktop television or computer monitor, the viewer's head will remain at the same height relative to the display panel 106, but may move horizontally as the viewer shifts position, and so the Y-axis component of the pixel viewing angle may be fixed to 0 degrees or otherwise ignored.

[0029] With the pixel value and pixel perspective determined for the selected pixel, in box 418, the image correction module 310 determines the angle-corrected pixel value based on these parameters. As explained above, the determination process is performed by the configuration of the correction transform component 326 to adjust for the measured or theoretical angle-dependent illumination changes exhibited by the specified set of pixel values. For illustration, for LUT-based or analysis function-based implementations, the image correction module 310 provides the pixel value and pixel perspective as input and receives the angle-corrected pixel value as output. In some embodiments, the correction process is performed separately for each color component of the pixel value. For example, using the RGB color space, a range of red component values ​​is tested or simulated to generate a corresponding set of LUT entries or analysis function configurations representing the angle-corrected red values ​​to be used in place of each input red value, and the same process can be repeated for each of the green and blue value ranges.

[0030] In other embodiments, the entire color component tuple representing the complete pixel value is used to determine the corresponding corrected color component tuple of the corrected pixel value. To illustrate using a 24-bit RGB color space, for example, the RGB888 value RGB[r1,g1,b1] of the pixel can be converted to a corresponding YUV444 value YUV[y1,u1,v1] using a standard RGB888 to YUV444 conversion, and then the original luminance component value y1 can be replaced by the corrected luminance component value y2 based on the luminance and angle relationship of the RGB888 value to generate a corrected YUV444 value YUV[y2,u1,v1], which is then converted back to the corresponding RGB888 value RGB[r2,g2,b2], which serves as the corrected pixel value for the input pixel value and pixel viewing angle combination.

[0031] At block 420, the image correction module 310 converts the corresponding angle-corrected image 322 ( Figure 3 ) is replaced or otherwise substituted with the corrected pixel value. The processing of blocks 412, 416, 418, and 420 is repeated for each pixel (or a specified subset of pixels) of the input image 318. Once all applicable pixels have been corrected for perspective, the resulting angle-corrected image 322 is provided for display as described above, and the method 400 returns to block 410 for processing the next input image in the video sequence.

[0032] The process of adjusting pixel values ​​based on the corresponding pixel view angle to pre-compensate for the angle-dependent color / brightness shift introduced by viewing pixels at their corresponding pixel view angle can, if not prevented, introduce undesirable visual artifacts, including banding due to quantization, clipping, and color imbalance. Figure 5 and6 Various techniques are shown for alleviating such problems.

[0033] As is well known and reflected in Weber's law, changes in brightness are more easily detected by observers for dark, low spatial frequency textures than for brighter, higher spatial frequency textures. Darkening, therefore, the quantization effect of updating pixel values ​​based on pixel angle viewing correction can lead to banding artifacts, especially at lower illumination / color levels. Figure 5 Figure 500 depicts an example configuration of an image correction module 310 for mitigating these quantization effects in accordance with some embodiments. In one embodiment, the image correction module 310 employs one or both of spatial and temporal color mixing to reduce the quantization effects of angle-dependent pixel adjustments. This typically includes applying a spatial / temporal color mixing pattern 502 to the pixels in order to introduce "white noise" into the output and thereby smooth out gradient effects. Any of a variety of spatial and / or temporal color mixing patterns may be employed, such as the Poisson-Disc color mixing pattern, as well as any of a variety of known color mixing algorithms. In such a case, the image correction module 310 applies a correction pixel value from the correction transform component 326 ( Figure 3 ) are output, and thus randomized noise is introduced into the output pixel stream, which then forms the corrected image 322. Since banding effects are more likely to be detected at darker pixel values, in some embodiments, the image correction module 310 may have a specified pixel value threshold so that the blending mode is only applied to those output corrected pixel values ​​below the threshold.

[0034] In other embodiments, the image correction module 310 may employ an intensity weighted excitation function 504 to excite an angle-dependent correction when the pixel value exceeds a specified threshold. Examples of such excitation functions include a hyperbolic tangent (tanh) excitation function, an S-shaped excitation function, a linear excitation function, and any of a variety of asymmetric excitation functions. In such an example, the image correction module 310 calculates the corrected pixel value as:

[0035] PVcorr _final =PVinput+AW*(PVcorr _initial -PVinput)

[0036] Where PVinput represents the input pixel value of the selected pixel of the input image 218, PVcorr _initial represents the angle-corrected pixel value obtained from the correction transform component 326 for the input pixel value, AW represents the excitation weight obtained from the excitation function 504 based on the input pixel value, and PVcorr _finalrepresents the final angle-corrected pixel value for the selected pixel in the angle-corrected image 322 .

[0037] Another problem that arises with the angle-dependent correction process is the clipping of pixels when the output intensity level of the angle-corrected pixel value exceeds the maximum level supported by the display panel 106 . Figure 6 A diagram of various solutions that can be adopted by the image correction module 310 to solve such a clipping problem is shown. As an overall overview, these solutions relate to pre-darkening of pixel values ​​to reduce or eliminate the possibility of correcting pixel values ​​exceeding the maximum supported intensity level. In one embodiment, the pixel values ​​are statically pre-darkened according to the corresponding pixel position. For illustration, the image correction transformation can utilize a dish-shaped dimming pattern 602 with multiple concentric contours 604 of a display area 606 that logically covers the pixel matrix 314, wherein each contour 604 is associated with different levels of pre-darkening and the amount of pre-darkening that increases from the outermost contour 604 to the innermost contour 604. Therefore, the level of pre-darkening applied to the pixel value of a given pixel depends on the specific contour in which the pixel is located within the dimming pattern 602. The pre-darkening pattern can adopt other shapes, such as columns of contours, rows of contours, 2D grids of contours, etc.

[0038] The pre-dim can be a fixed value for a given pre-dim level. For example, for an RGB888 implementation in which the color component values ​​range from 0 to 255, the pre-dim applied to the pixels in the innermost contour 604 can be to reduce each color component by 20, while the pre-dim applied to the pixels in the outermost contour can be to reduce each color component by only 5. In other embodiments, the pre-dim is adopted as a scaling value, such as, for example, a scaling value between 0.8 and 1. In some embodiments, the pre-dim process is applied to the input pixel value so that the input pixel value is pre-dimmed and the resulting pre-dim input value is input to the correction transformation component 326 to obtain the corresponding angle-corrected pixel value. In other embodiments, the pre-dim process is applied to the angle-corrected pixel values ​​when they are output from the correction transformation component 326. In either instance, the pre-dim mode 602 can be implemented in a LUT or similar data structure, where the pixel position is provided as an input and the corresponding pre-dim value to be applied to the pixel value is provided as an output.

[0039] Instead of using a static position-based pre-darkening, in other embodiments, the image correction module 310 uses a uniform pre-darkening based on the viewer's viewing angle rather than the pixel position. For illustration, the image correction module 310 may use a specified viewing angle / pre-darkening relationship, which is Figure 6608, wherein the viewing angle is represented on the abscissa and the corresponding pre-dim scale factor is represented on the ordinate, and which can be implemented as a LUT or other similar structure. In this method, each pixel value is scaled down by the same scale factor, which is determined based on the viewing angle and can range between a minimum and a maximum scale factor (e.g., 0.8 to 1.0), wherein the pre-dim scale increases with increasing viewing angle. As with the position-based static pre-dim process described above, the pre-dim scale can be applied to the input pixels so that the pre-dim input pixels are used as the basis for determining the angle-corrected pixel values, or the pre-dim scale can be applied to the angle-corrected pixel values ​​determined for the original pixel values.

[0040] Some display devices provide segmented backlights, where different partitions such as columns, rows, or 2D blocks of backlight can be controlled independently, and therefore the amount of backlight can vary between different partitions. In instances where display device 102 supports such segmented backlights, display device 102 can utilize the segmented backlight capability to actually retrieve some of the intensity lost by the pre-dim process, so as to provide almost "flat" lighting after pre-dim. For illustration, in a configuration where the pre-dim mode is a column-based profile mode and the backlight 610 of display device 102 is also segmented into columns, the backlight intensity of each segment can be increased in proportion to the amount of pre-dim for the profile co-located with the backlight segment. As another example, when using angle-dependent static pre-dim, the backlight intensity can vary based on the angle of each backlight segment relative to the viewer's head posture, so that the less sharp the angle (that is, the closer to the viewer's head), the lower the backlight intensity applied. To illustrate, when the user's head is at the center of the display, backlight segments closer to the center are controlled to provide less backlight, while backlight segments farther from the center are controlled to provide more backlight, such as Figure 6 As the viewer's head goes to one side of the display, the backlight segments on that side are controlled to provide less backlight, while those on the other sides are controlled to provide more backlight. By varying the backlight intensity in this manner, and in fact inversely proportional to the pre-dim applied to the pixels in the corresponding partition, the display device 102 is able to maintain substantially flat illumination while mitigating the chance of clipping.

[0041] Clipping is particularly problematic when one color component of a pixel value is clipped while the other color components remain unclipped, and thus result in a changed luminance ratio between the color component values. In such instances, the image correction module 310 can reweight the color components of the corrected pixel value so as to maintain the same luminance ratio of the original pixel value. In doing so, changes in texture may be lost, but the color of the pixel itself will not change from the input to the corrected value.

[0042] Figure 7 An embodiment of a display system 100 for providing angle-dependent image correction when there are multiple simultaneous viewers is shown. In this method, the tracking subsystem 104 tracks the head poses of multiple viewers (e.g., viewers 702, 704), and the image correction module 310 determines a combined head pose 706, which is then used by the image correction module 310 to apply angle-dependent image correction to the input image 318 to generate an angle-corrected image 322 for display on the display device 102. The combined head pose 706 can be determined in various ways based on the individual head poses. For illustration, the individual head poses can be simply averaged. In other embodiments, a weighted average algorithm can be employed, whereby each head pose is weighted based on its viewing angle. Still further, certain thresholds can be employed to eliminate head poses at extreme viewing angles (to prevent corresponding corrections that rely on extreme angles from providing detracting results for more centrally located viewers), and so on.

[0043] Light field displays are particularly well suited for multi-viewer experiences because they are able to provide a 3D view of a scene to multiple viewers simultaneously (either a single image to each viewer, or, in some embodiments, separate images for each eye of the viewer, with different left-eye / right-eye image pairs provided to different viewers based on each viewer's head pose and / or gaze direction). Figure 8 As shown, the light field display device 802 ( Figure 1 An embodiment of a display device 102 of FIG. 100 ) employs angle-dependent image correction processing because the head poses of multiple simultaneous viewers (e.g., viewers 804, 806) are tracked via the tracking subsystem 104. The source system generates or obtains an input light field 818 ( Figure 3), and the display device 802 performs a separate angle-dependent correction for each viewer (or for each eye of each viewer if a separate image is provided to each eye of the viewer) using the viewer's corresponding head pose (e.g., head poses 808, 810 for viewers 804, 806, respectively) and / or eye pose / gaze direction on the light field pixel data representing light rays expected to be perceived by the viewer at the viewer's corresponding position, resulting in a corrected light field 822 with angle-dependent color correction employed separately for each tracked viewer ( Figure 3 ), which is then provided for display on the light field display 802.

[0044] In some embodiments, certain aspects of the techniques described above may be implemented by one or more processors executing software in a processing system. The software includes one or more executable instruction sets stored on a non-transient computer-readable storage medium or embodied in other tangible ways. The software may include instructions and certain data, which, when executed by one or more processors, manipulate the one or more processors to perform one or more aspects of the techniques described above. The non-transient computer-readable storage medium may, for example, include a disk or optical disk storage device, a solid-state storage device such as a flash memory, a cache, a random access memory (RAM), or one or more other non-volatile memory devices, and the like. The executable instructions stored on the non-transient computer-readable storage medium may be source code, assembly language code, object code, or other instruction formats interpreted by one or more processors or capable of being executed in other ways.

[0045] Computer-readable storage media may include any storage medium or combination of storage media that can be accessed by a computer system during use to provide instructions and / or data to the computer system. Such storage media may include, but are not limited to, optical media (e.g., compact disks (CDs), digital versatile disks (DVDs), Blu-ray disks), magnetic media (e.g., floppy disks, tapes, or magnetic hard disks), volatile memory (e.g., random access memory (RAM) or cache), non-volatile memory (e.g., read-only memory (ROM) or flash memory), or micro-electromechanical system (MEMS)-based storage media. The computer-readable storage medium may be embedded in a computing system (e.g., system RAM or ROM), fixedly attached to a computing system (e.g., a magnetic hard drive), removably attached to a computing system (e.g., an optical disk or flash memory based on a universal serial bus (USB)), or coupled to a computer system via a wired or wireless network (e.g., network accessible storage (NAS)).

[0046] Note that generally not all activities or elements described above in the general description are required, a portion of a particular activity or device may not be required, and one or more additional activities may be performed or one or more additional elements may be included in addition to those described. Further, the order in which the activities are listed is not necessarily the order in which they are performed. Moreover, the concepts have been described with reference to specific embodiments. However, it will be appreciated by those skilled in the art that various modifications and changes may be made without departing from the scope of the present disclosure as set forth below. Therefore, the specification and the drawings are to be considered in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of the present disclosure.

[0047] Some examples are described below:

[0048] Example 1: A display system comprising:

[0049] a tracking subsystem configured to track one of a viewer's head gesture or a viewer's eye gesture;

[0050] An image correction module is configured to:

[0051] For each pixel in at least a subset of the pixels of the input image:

[0052] Determine a pixel viewing angle of the pixel based on the posture or the sight direction;

[0053] determining a corrected pixel value based on an input pixel value for the pixel in the input image and based on the pixel viewing angle; and

[0054] providing corrected pixel values ​​for pixels in the angle-corrected image corresponding to the input image; and

[0055] The display panel is used to display the angle-corrected image.

[0056] Example 2: A display system according to Example 1, wherein the correction pixel value represents a corrective adjustment for offsetting the viewing angle-dependent brightness or color response of the corresponding pixel of the display panel.

[0057] Example 3: A display system according to Example 1 or 2, wherein the corrected pixel value is at least one of luminance or color.

[0058] Example 4: The display system according to at least one of the preceding examples, further comprising:

[0059] a correction transform component configured for a plurality of pixel values ​​based on a relationship between pixel illumination and a viewing angle of the pixel of the display panel; and

[0060] The image correction module uses the correction transformation to determine the correction pixel value.

[0061] Example 5: A display system according to Example 4, wherein the relationship is determined based on one of empirical measurement, simulation, or modeling of the display panel.

[0062] Example 6: A display system according to Example 4 or 5, wherein the correction transformation component includes one of a lookup table or an analysis function.

[0063] Example 7: According to at least one of the preceding examples, the display system, wherein the image correction module is configured to apply at least one of a spatial color mixing mode or a temporal color mixing mode to the corrected pixel values ​​of the angle-corrected image.

[0064] Example 8: A display system according to at least one of the preceding examples, wherein the image correction module is configured to selectively correct the input pixel value using an excitation function.

[0065] Example 9: A display system according to at least one of the preceding examples, wherein the image correction module is configured to pre-dim the pixel values ​​of one of the input image or the angle-corrected image using one of the following: uniform pre-dimming of the pixel values ​​based on posture or gaze direction; and static pre-dimming of the pixel values ​​based on the position of the pixel within the input image.

[0066] Example 10: The display system according to Example 9, further comprising:

[0067] A backlight comprising a plurality of independently controllable segmented backlights,

[0068] Therein, the backlight is controlled to compensate for pre-dimming of the pixels on a segment-by-segment basis.

[0069] Example 11: A display system according to at least one of the preceding examples, wherein the display panel comprises a display panel of one of the following: a television; a smart phone; a tablet computer; a netbook computer; a computer monitor; a near-eye display device; and a light field display device.

[0070] Example 12: A display system according to at least one of the preceding examples, wherein the tracking system includes a camera-based head tracking subsystem.

[0071] Example 13: A method for implementation in a display system, the method comprising:

[0072] Tracking one of a head pose of a viewer or an eye pose of an eye of a viewer;

[0073] For each pixel in at least a subset of the pixels of the input image:

[0074] determining a pixel viewing angle of the pixel based on the pose or gaze direction;

[0075] determining a corrected pixel value based on an input pixel value for the pixel in the input image and based on the pixel viewing angle; and

[0076] providing corrected pixel values ​​for pixels in the angle-corrected image corresponding to the input image; and

[0077] The angle-corrected image is provided for display on a display panel of the display system.

[0078] Example 14: The method of Example 13, wherein the correction pixel value represents a corrective adjustment for offsetting a viewing angle-dependent brightness or color response of a corresponding pixel of the display panel.

[0079] Example 15: A display system according to Example 13 or 14, wherein the corrected pixel value is at least one of a luminance value or a color value.

[0080] Example 16: The method according to at least one of Examples 13 to 15, further comprising:

[0081] For a plurality of pixel values, configuring a correction transform component based on a relationship between a pixel illumination of the display panel and a viewing angle of the pixel; and

[0082] Wherein, determining the corrected pixel value includes using the corrected transform to determine the corrected pixel value.

[0083] Example 17: The method according to Example 16 further comprises:

[0084] The relationship is determined based on one of empirical measurements, simulations, or modeling of the display panel.

[0085] Example 18: The method according to at least one of Examples 13 to 17, further comprising:

[0086] At least one of a spatial color mixing mode or a temporal color mixing mode is applied to the corrected pixel values ​​of the angle-corrected image.

[0087] Example 19: The method according to at least one of Examples 13 to 18, further comprising selectively correcting the input pixel value based on an excitation function.

[0088] Example 20: The method according to at least one of Examples 13 to 19, further comprising:

[0089] Pixel values ​​of one of the input image or the angle-corrected image are pre-darkened using at least one of: uniform pre-darkening of pixel values ​​based on a viewer's posture; and static pre-darkening of pixel values ​​based on a position of the pixel within the input image.

[0090] Example 21: The method according to Example 20 further comprises:

[0091] Independently controllable segments of the backlight are controlled to compensate for pre-dimming of pixel values ​​on a segment-by-segment basis.

[0092] Example 22: A method according to at least one of Examples 13 to 21, wherein the display panel includes a display panel of one of the following: a television; a smart phone; a tablet computer; a netbook computer; a computer monitor; a near-eye display device; and a light field display device.

[0093] The benefits, other advantages, and solutions to problems have been described above with respect to specific embodiments. However, the benefits, advantages, solutions to problems, and any features that may cause any benefit, advantage, or solution to the problem to occur or become more prominent are not to be understood as being key, required, or essential features in any or all of the claims. In addition, the specific embodiments disclosed above are merely illustrative, as the disclosed subject matter may be modified and practiced in different but equivalent ways that are obvious to those skilled in the art who have benefited from the teachings herein. Except as described in the following claims, it is not intended to limit the details of the construction or design shown herein. Therefore, it is apparent that the specific embodiments disclosed above may be changed or modified, and all such changes are considered to be within the scope of the disclosed subject matter. Therefore, protection is sought herein as set forth in the following claims.

Claims

1. A display system, include: a tracking subsystem configured to track one of a pose of a viewer's head or a pose of a viewer's eyes; An image correction module, wherein the image correction module is configured to: For each pixel in at least a subset of the pixels of the input image: determining a pixel viewing angle of the pixel based on the posture of the head or the posture of the eyes; determining a corrected pixel value based on an input pixel value of the pixel in the input image and based on the pixel viewing angle; as well as providing the corrected pixel values ​​for pixels in the angle-corrected image corresponding to the input image; as well as applying at least one of a spatial color mixing mode or a temporal color mixing mode to the corrected pixel values ​​of the angle-corrected image; and A display panel is used to display the angle-corrected image.

2. The display system according to claim 1, in, The corrected pixel value represents a corrective adjustment for counteracting a viewing angle-dependent brightness or color response of a corresponding pixel of the display panel.

3. The display system according to claim 1, in, The corrected pixel value is at least one of luminance or color.

4. The display system according to claim 1, further comprising: include: a correction transform component configured for a plurality of pixel values ​​based on a relationship between a pixel illumination of the display panel and a viewing angle of the pixel; and Wherein, the image correction module uses a correction transformation to determine the correction pixel value.

5. The display system according to claim 4, in, The relationship is determined based on one of empirical measurement, simulation, or modeling of the display panel.

6. The display system according to claim 4, in, The correction transformation component includes one of a lookup table or an analysis function.

7. The display system according to claim 1, in, The image correction module is configured to selectively correct the input pixel values ​​using an excitation function.

8. The display system according to claim 1, in, The image correction module is configured to pre-dim the pixel values ​​of one of the input image or the angle-corrected image using one of: uniform pre-dimming of the pixel values ​​based on the posture of the head or the posture of the eyes; and static pre-dimming of the pixel values ​​based on the position of the pixel within the input image.

9. The display system according to claim 8, further comprising: include: a backlight comprising a plurality of independently controllable segments, Therein, the backlight is controlled to compensate for the pre-dimming of the pixel values ​​on a segment-by-segment basis.

10. The display system according to claim 1, in, The display panel includes a display panel of one of the following: a television; a smart phone; a tablet computer; a netbook computer; a computer monitor; a near-eye display device; and a light field display device.

11. The display system according to claim 1, in, The tracking subsystem includes a camera-based head tracking subsystem.

12. A method for implementing in a display system, the method include: tracking at least one of a pose of a viewer's head or a pose of a viewer's eyes; For each pixel in at least a subset of the pixels of the input image: determining a pixel viewing angle of the pixel based on a posture of the viewer's head or a posture of the viewer's eyes; determining a corrected pixel value based on an input pixel value of the pixel in the input image and based on the pixel viewing angle; as well as providing the corrected pixel value for the pixel in the angle-corrected image corresponding to the input image; as well as applying at least one of a spatial color mixing mode or a temporal color mixing mode to the corrected pixel values ​​of the angle-corrected image; and The angle-corrected image is provided for display on a display panel of the display system.

13. The method according to claim 12, in, The corrected pixel value represents a corrective adjustment for counteracting a viewing angle-dependent brightness or color response of a corresponding pixel of the display panel.

14. The method according to claim 12, in, The corrected pixel value is at least one of a luminance value or a color value.

15. The method according to claim 12, further comprising: include: For a plurality of pixel values, configuring a correction transformation component based on a relationship between a pixel illumination of the display panel and a viewing angle of the pixel; and Wherein determining the corrected pixel value comprises using the corrected transform to determine the corrected pixel value.

16. The method according to claim 15, further comprising: include: The relationship is determined based on one of empirical measurement, simulation, or modeling of the display panel.

17. The method according to claim 12, further comprising: include: The input pixel value is selectively corrected based on an excitation function.

18. The method according to claim 12, further comprising: include: Pixel values ​​of one of the input image or the angle-corrected image are pre-darkened using at least one of: uniform pre-darkening of the pixel values ​​based on a posture of the viewer's head or a posture of the viewer's eyes; and static pre-darkening of the pixel values ​​based on a position of the pixel within the input image.

19. The method according to claim 18, further comprising: include: Independently controllable segments of a backlight are controlled to compensate for the pre-dimming of the pixel values ​​on a segment-by-segment basis.

20. The method according to claim 12, in, The display panel includes a display panel of one of the following: a television; a smart phone; a tablet computer; a netbook computer; a computer monitor; a near-eye display device; and a light field display device.