Stereoscopic image pair from monoscopic image blur

The autostereoscopic display system processes monoscopic images to generate a blur intensity map and select a final disparity map based on blur characteristics, improving 3D rendering accuracy by aligning convergence planes with image focus, enhancing the 3D experience.

WO2025159794A1PCT designated stage expired Publication Date: 2025-07-31LEIA INC
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
PCT/US2024/042085
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2024-01-22
Filing Date
2024-08-13
Publication Date
2025-07-31

AI Technical Summary

Technical Problem

Existing autostereoscopic displays struggle to effectively generate stereoscopic images from monoscopic inputs, lacking a method to accurately determine convergence planes based on blur and disparity characteristics, leading to suboptimal 3D rendering.

Method used

An autostereoscopic display system processes monoscopic images to generate a blur intensity map and multiple provisional disparity maps, selecting or interpolating a final disparity map based on blur characteristics, and uses this to create a stereoscopic image pair for accurate 3D rendering without requiring special eyewear.

Benefits of technology

The system enhances the accuracy of 3D image rendering by aligning convergence planes with blur characteristics, ensuring that focused objects appear at the display plane and blurred objects appear above or below, providing a more immersive 3D experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure US2024042085_31072025_PF_FP_ABST
    Figure US2024042085_31072025_PF_FP_ABST
Patent Text Reader

Abstract

An autostereoscopic display system can generate a stereoscopic image pair from a monoscopic image. Processing circuitry can receive data specifying a monoscopic image. The processing circuitry can generate multiple provisional disparity maps from the monoscopic image, the provisional disparity maps having different convergence planes. A final convergence plane can be based on blur and disparity characteristics of the monoscopic image. For example, the autostereoscopic display system can select, as a final disparity map, the provisional disparity map that is most similar to the blur intensity map. As another example, the autostereoscopic display system can interpolate between adjacent convergence planes to generate a final disparity map that is most similar to the blur intensity map. The processing circuitry can generate a stereoscopic image pair using the monoscopic image and the final disparity map. The autostereoscopic display system can include an autostereoscopic display to display the stereoscopic image pair.
Need to check novelty before this filing date? Find Prior Art

Description

STEREOSCOPIC IMAGE PAIR FROM MONOSCOPIC IMAGE BLURCROSS-REFERENCE TO RELATED APPLICATION

[0001] This application claims the benefit of U.S. Provisional Application No. 63 / 623,747, filed January 22, 2024, which is incorporated by reference herein in its entirety.FIELD OF THE DISCLOSURE

[0002] This document relates generally to display systems, and more specifically relates to multiview displays, three-dimensional displays, or autostereoscopic displays.BACKGROUND OF THE DISCLOSURE

[0003] A multiview display can provide different views of a multiview image to a viewer. A stereoscopic display can provide two different views of a three-dimensional image to the two eyes of a viewer. An autostereoscopic display can provide the two different views to the two eyes of the viewer without requiring the viewer to wear special glasses or eyewear. There is ongoing effort to improve autostereoscopic displays.SUMMARY

[0004] In an example, an autostereoscopic display system can comprise processing circuitry that can perform operations. The operations can comprise: receiving data specifying a monoscopic image; specifying an initial convergence plane; generating a final disparity map based on the monoscopic image and a final convergence plane spaced apart from the initial convergence plane, the final convergence plane being based on blur and disparity characteristics of the monoscopic image; and generating a stereoscopic image pair using the monoscopic image and the final disparity map.

[0005] In an example, a method can include generating a stereoscopic image pair from a monoscopic image. The stereoscopic image pair can optionally be displayed using an autostereoscopic system. The method can comprise: receiving, with processing circuitry, data specifying a monoscopic image; specifying, with the processing circuitry, an initial convergence plane; generating, with the processing circuitry, a final disparity map based on the monoscopic image and a final convergence plane spaced apart fromthe initial convergence plane, the final convergence plane being based on blur and disparity characteristics of the monoscopic image; and generating, with the processing circuitry, a stereoscopic image pair using the monoscopic image and the final disparity map.

[0006] This Summary is intended to provide an overview of subject matter of the present document. It is not intended to provide an exclusive or exhaustive explanation of the invention. The detailed description is included to provide further information about the present subject matter.BRIEF DESCRIPTION OF THE DRAWINGS

[0007] FIG. 1 shows an exploded, perspective-view schematic drawing of an example of a multi view display system.

[0008] FIG. 2 shows a front-view drawing of an example of a display panel that includes an array of light-emitting diodes.

[0009] FIG. 3 shows a front-view drawing of an example of a display panel that includes a backlight and a light valve array.

[0010] FIG. 4 shows a front-view drawing of an example of a parallaxgenerating optic that includes a lenticular lens.

[0011] FIG. 5 shows a cross-sectional view of the lenticular lens of FIG. 4.

[0012] FIG. 6 shows a front-view drawing of an example of a parallaxgenerating optic that includes a parallax barrier.

[0013] FIG. 7 shows a cross-sectional view of the parallax barrier of FIG. 6 having transmissive slits.

[0014] FIG. 8 shows an example of various quantities associated with generating a stereoscopic image pair from a monoscopic image.

[0015] FIG. 9 shows a flowchart of an example of a method for generating a stereoscopic image pair from a monoscopic image.

[0016] FIG. 10 shows a schematic drawing of an example of determining the final convergence plane based on blur and disparity characteristics of the monoscopic image.DETAILED DESCRIPTION

[0017] In capturing an image, a photographer may choose to draw attention to an object of interest in a scene by causing a plane that includes the object of interest to appear in sharp focus. Other objects in the scene, away from the plane, can appear blurry, with an amount of blur being proportional to a distance away from the plane. Blur generally refers to a lack of sharpness or clarity in all or part of an image. Blur occurs when boundaries between different elements or objects in the image become less distinct, resulting in a softening or smearing effect.

[0018] To generate a stereoscopic image pair from the image, an autostereoscopic display system can use blur information from the image to determine where to locate the convergence plane of the stereoscopic image pair. Doing so can cause the object of interest to appear at or near a plane of the autostereoscopic display.

[0019] The autostereoscopic display system can use one or more of the following techniques to identify blur characteristics in an image. In edge detection, the autostereoscopic display system can analyze the sharpness of edges in an image to determine blur levels. Blurred images can have less defined edges. In frequency domain analysis, the autostereoscopic display system can transform the image into the frequency domain (e.g., using Fourier transforms) to reveal blur characteristics. Blurred images can have fewer high-frequency components. In gradient magnitude, the autostereoscopic display system can calculate the gradient magnitude across the image to help identify areas of blur. Blurred regions can have lower gradient values. In Laplacian variance, the autostereoscopic display system can compute the variance of the Laplacian of the image to provide a measure of image focus. Lower variance can indicate more blur. In point spread function (PSF) estimation, the autostereoscopic display system can estimate the PSF to help characterize the type and extent of blur in an image. In blur kernels, the autostereoscopic display system can analyze the shape and size of blur kernels to help identify the nature of the blur (such as motion blur or defocus blur). In machine learning approaches, the autostereoscopic display system can use trained models to detect and classify blur characteristics based on various image features. In histogram analysis, the autostereoscopic display system can examine the histogram of pixel intensities to reveal blur characteristics. Blurred images can have narrower histograms. These techniques can be used individually or in combination to assess the presence, type, and extent of blur in an image.

[0020] A blur intensity map can quantify the blur of an image, such as a monoscopic image or one or both images of a stereoscopic image pair. For example, the blur intensity map can be an array, such as with a same size as the image. The value of the blur intensity map at a particular pixel can correspond to the blur value at the particular pixel of the image.

[0021] In general, the quantity disparity can represent a horizontal distance between left and right images of a stereoscopic image pair for the same point in a scene. In other words, the disparity can correspond to a distance between a pixel in the left (right) image and its horizontal match in the right (left) image. It is assumed that a viewer is oriented upright, such that the left and right eyes of the viewer are at the same height and that an axis connecting the left and right eyes is horizontal.

[0022] A disparity map can quantify the disparity of a stereoscopic image pair. For example, the disparity map can be an array, such as with a same size as the images of the stereoscopic image pair. The value of the disparity map at a particular pixel can correspond to the disparity value at the particular pixel of the image of the stereoscopic image pair.

[0023] A convergence plane is the plane in the scene at which the disparity equals zero. For objects in the scene on one side of the convergence plane, the disparity is positive. For objects in the scene on the other side of the convergence plane, the disparity is negative.

[0024] When an autostereoscopic display displays the stereoscopic image pair, objects in the scene at the convergence plane appear to be located at a plane of the autostereoscopic display. Objects in the scene that are closer to the viewer than the convergence plane appear to be located above the autostereoscopic display. Objects in the scene that are farther away from the viewer than the convergence plane appear to be located below the autostereoscopic display.

[0025] In general, a disparity map can include the same amount of information as a depth map, such that one can be created from the other. In addition, for three data structures that include a left image, a right image, and a disparity map, any two of the three data structures can be used to create the third of the three data structures.

[0026] Further, there exists image processing software, executable by processing circuitry, that can use artificial intelligence to generate an artificial or simulated disparity map from a (single) monoscopic image. For example, the processing circuitry can use afully convolutional neural network model. The neural network model can include an encoder based on a residual neural network (ResNet). The neural network model can include a decoder configuration having skip connection that are similar to those in a U- Net. The architecture of a U-Net can include an encoder path, also known as the contracting path, and a decoder path, also known as the expanding path. The encoder can extract features from an input image. The decoder can project those features onto pixel space to produce a dense classification. The encoder and decoder may be symmetrical and connected by paths, which can give the model a “U” shape. The convolutional neural network model can include millions (or more) of trainable parameters, such as can be trained on multiple (e.g., thousands, millions) of images that have known disparity characteristics. Processing circuitry can use the model to process a (single) monoscopic image and provide a corresponding disparity map.

[0027] The autostereoscopic display system can use blur characteristics in part to generate a stereoscopic image pair from a monoscopic image. For example, the autostereoscopic display system can generate a blur intensity map from the monoscopic image. The autostereoscopic display system can generate multiple provisional disparity maps from the monoscopic image. The provisional disparity maps can have different convergence planes. In some examples, the autostereoscopic display system can select, as a final disparity map, the provisional disparity map that is most similar to the blur intensity map. In some examples, the autostereoscopic display system can interpolate between adjacent convergence planes to generate a final disparity map that is most similar to the blur intensity map. The autostereoscopic display system can use the monoscopic image and the final disparity map to generate the stereoscopic image pair.

[0028] When the stereoscopic image pair is displayed on an autostereoscopic display, objects in the scene at or near the convergence plane (e.g. objects that are relatively sharply focused in the monoscopic image) will appear to be located at or near a plane of the autostereoscopic display. Objects in the scene away from the convergence plane (e.g., objects that are relatively blurry in the monoscopic image) will appear to be located above or below the plane of the autostereoscopic display.

[0029] The preceding paragraphs merely summarize some aspects of the image generation and display technique described in detail below, and should not be construed as limiting in any way.

[0030] FIG. 1 shows an exploded, perspective-view schematic drawing of an example of a multiview display system 100 that includes a multiview display 110. The configuration of FIG. 1 is but one example of a multiview display system 100; other configurations can be used.

[0031] The sign conventions shown in FIG. 1 and used below assume that the multiview display 110 extends in an (x, ) plane, and that a z-axis extends away from the multiview display 110 and generally toward a viewer 42, along a direction that is orthogonal to a plane of the multiview display 110. Other sign conventions can be used.

[0032] The multiview display system 100 can include a multiview display 110. The multiview display 110 can provide different views of a multiview image to the viewer 42. For example, as the viewer 42 moves in space, the multiview display 110 can direct different views of the multiview image to the left and right eyes of the viewer 42, so that the viewer 42 can observe the different views of the multiview image from different locations or orientations. In some configurations, the multiview display 110 can provide the multiple views at respective fixed location regions in space, so that the multiview display 110 can operate without using eye tracking. In other configurations, such as the autostereoscopic configurations described in detail below, the multiview display system 100 can use eye tracking to dynamically and continuously (or at relatively frequent discrete times) determine a location of the viewer 42, and in response, can dynamically and continuously control how the multiview display 110 displays the multiview image so that the multiple views follow the viewer 42 or follow the tracked eye location(s) of the viewer 42 as the viewer 42 moves in space relative to a position of the multi view display 110.

[0033] For configurations in which the multiview display 110 provides just two different views of the multiview image, the multiview display 110 can be an autostereoscopic display or three-dimensional (3D) display. The autostereoscopic display can provide a left image to a left eye of the viewer 42 and a right image to a right eye of the viewer 42. The left image and the right image can correspond to different views of an object or a scene, and can allow the viewer 42 to perceive the object or scene in 3D with just the viewer’s naked eyes, without the use of additional glasses or headgear.

[0034] The multiview display system 100 can include a viewer tracker 120 that can dynamically determine the location of the viewer 42. The multiview display system100 can use the determined location of the viewer 42 to direct the left image to the left eye of the viewer 42 and the right image to the right eye of the viewer 42. Because the viewer’s location can vary as the viewer 42 moves in space, using eye tracking can allow the multiview display system 100 to follow the viewer 42, so that the autostereoscopic display can automatically direct the left image to the left eye at the viewer 42’ s (dynamically varying) location and automatically direct the right image to the right eye at the viewer 42’ s (dynamically varying) location. The viewer tracker 120 can provide a tracked position of the viewer 42, such as a tracked position of a head of the viewer 42, of one or both eyes of the viewer 42, or of another anatomical feature of the viewer 42. The viewer tracker 120 can be coupled to the processing circuitry 130 (described below) or controller, such as by providing viewer 42 location data (shown in FIG. 1 as coordinates xv, yv, and zv) that represents a measured position or location of the viewer 42. The viewer tracker 120 can provide the viewer 42 location data at regular or irregular intervals to the processing circuitry 130. In a specific example of a viewer tracker 120, a camera can capture an image of the viewer 42. The viewer tracker 120 can further include an image processor (or general-purpose computer programmed as an image processor) that can determine a position of the viewer 42 within the captured image to provide the tracked position. In some examples, the processing circuitry 130 can include the image processor of the viewer tracker 120. In other examples, the processing circuitry 130 can be separate from the image processor of the viewer 42 tracker. Other suitable viewer trackers can be used, including viewer trackers based on lidar (e.g., using time-of-flight of reflected light over a scene to determine distances to one or more objects in the scene, such as a viewer’s head or a viewer’s eyes) or other technologies. The processing circuitry 130 can use an output of the viewer tracker 120, among other data, to perform one or more downstream calculations involved with providing the left view or left image to the left eye of the viewer 42 and the right view or right image to the right eye of the viewer 42.

[0035] The autostereoscopic display can be a lenticular autostereoscopic display. In a lenticular autostereoscopic display, a display panel 112 can display the multiview image, and a parallax-generating optic 118 can direct light from the display panel 112 to the viewer 42 such that a left image can be visible from the left eye of the viewer 42 and a right image can be visible from the right eye of the viewer 42. During use of the lenticular autostereoscopic display, the processing circuitry 130 can track the location ofthe viewer 42, and can use the tracked location to dynamically determine how to distribute content of the multiview image over a surface area of the display panel 112 (e.g., using pixels distributed over the display panel 112) such that a left image remains visible from the left eye of the viewer 42 and a right image remains visible from the right eye of the viewer 42, even as the viewer 42 changes location. In this manner, the location tracking and the image content distribution can be performed in software, such that following the location of the viewer 42 may not involve physically moving any components of the lenticular autostereoscopic display with respect to one another. Examples of suitable display panels and examples of suitable parallax-generating optics are described below.

[0036] In an example, a display panel 112 can display the multiview image. The display panel 112 can have an array of subpixels 114 that can display an image according to stereo mapping coordinates associated with the viewer 42. The subpixels 114 can be located at subpixel locations in a grid having grid axes (for example, the x-axis and y- axis). Each subpixel 114 can generate light having a specified color. For example, the subpixels 114 can include red subpixels, green subpixels, and blue subpixels, which generate red light, green light, and blue light, respectively. Other color / wavelength schemes can be used. The subpixels 114 can be grouped into pixels, with each pixel including at least two subpixels 114 that produce light of different colors. The display panel 112 can receive, from the processing circuitry 130 (described below), a display panel driving electrical signal 138 that can specify how the content of the multiview image is distributed over the pixels and / or subpixels 114 of the display panel 112. Two possible configurations for the display panel 112 are described below and shown in FIGS. 2 and 3. Other configurations can be used.

[0037] FIG. 2 shows a front-view drawing of an example of a display panel 112A that includes an array 202 of light-emitting diodes 204, such as an array 202 of organic light-emitting diodes. Each light-emitting diode 204 can correspond to a subpixel. The array 202 of light-emitting diodes 204 can include red light-emitting diodes 204R, green light-emitting diodes 204G, and blue light-emitting diodes 204B, which correspond to the red subpixels, green subpixels, and blue subpixels, respectively. Each light-emitting diode 204 can controllably generate light in response to an electrical signal provided by the processing circuitry 130, such as display panel-driving electrical signal 138, or by suitable light-emitting diode-driving circuitry in communication withthe processing circuitry 130. The processing circuitry 130 can cause a specified lightemitting diode 204 to be directly powered with a power that varies as a function of an intensity in a corresponding location in the image. The power delivered to a lightemitting diode 204 can optionally be pulse-width modulated at a modulation frequency that is greater than can be perceived by a human eye. Using pulse-width modulation can simplify a design of a light-emitting diode array controller, because it can generate an arbitrary average power level from a relatively small number of instantaneous power levels by varying a duty cycle of the power. In some examples, the array 202 of lightemitting diodes 204 can be arranged in a rectangular or square repeating pattern over a surface area 206 of the array 202. For example, the array 202 can have grid axes 208 that are orthogonal to each other. In some examples, the grid axes 208 can be parallel to edges 210 of the array 202 of light-emitting diodes 204.

[0038] FIG. 3 shows a front-view drawing of an example of a display panel 112B that includes a backlight 302 and a light valve array 304. Although FIG. 3 shows the backlight 302 and the light valve array 304 as being separated, in practice, the backlight 302 and the light valve array 304 may be in contact or may be located as close together as is practical. The backlight 302 can provide illumination having a uniform or substantially uniform intensity over a surface area of the backlight 302. The backlight 302 can provide illumination having a relatively broad spectrum, such as including most or all of the visible portion of the electromagnetic spectrum. The backlight 302 can provide the illumination into a continuum of propagation angles toward the light valve array 304. The backlight 302 can provide unmodulated illumination to the light valve array 304. The light valve array 304 can include light valves 306 that are individually controllable or controllable in one or more groups by the processing circuitry 130 (described below). Each light valve 306 can controllably attenuate the illumination from the backlight 302, such as in response to an electrical signal provided by the processing circuitry 130, such as display panel-driving electrical signal 138, or by suitable light valve driving circuitry in communication with the processing circuitry 130. Each light valve 306 can have a corresponding color filter that allows only a portion of the electromagnetic spectrum to pass through the light valve. For example, the light valves 306 can include red light valves 306R that have a red filter that allows only red light to pass through the red light valves 306R, green light valves 306G that have a green filter that allows only green light to pass through the green light valves 306G, and blue lightvalves 306B that have a blue filter that allows only blue light to pass through the blue light valves 306B. Other color schemes and numbers of colors can be used. Suitable light valves can include liquid crystal light valves, electrophoretic light valves, light valves based on electrowetting, and others. In some examples, the light valves 306 of the light valve array 304 can be arranged in a rectangular or square repeating pattern over a surface area 308 of the light valve array 304. For example, the light valve array 304 can have grid axes 208 that are orthogonal to each other. In some examples, the grid axes 208 can be parallel to edges 312 of the light valve array 304.

[0039] Referring again to FIG. 1, the autostereoscopic display can include a parallax-generating optic 118 that can direct light from the display panel 112 to the viewer 42, such that a left view or a left image can be visible from a left eye of the viewer 42 and a right view or a right image can be visible from a right eye of the viewer 42. Two possible configurations for the parallax-generating optic 118 are described below and shown in FIGS. 4 and 5 and in FIGS. 6 and 7. Other configurations can be used. Each of the configurations of FIGS. 4 and 5 and in FIGS. 6 and 7 can be used in combination with any of the configurations of the display panel 112 shown in FIGS. 2 and 3 (e.g., the array of light-emitting diodes 204 in FIG. 2 or the backlight 302 and light valve array 304 in FIG. 3).

[0040] FIG. 4 shows a front-view drawing of an example of a parallaxgenerating optic 118A that includes a lenticular lens 402. FIG. 5 shows a cross-sectional view of the lenticular lens 402 of FIG. 4. The lenticular lens 402 can include a plurality of cylindrical lenses 504 that are equally spaced apart. The lenticular lens 402 can have a focal plane coincident with the display panel 112. The lenticular lens 402 can be positioned to receive light from the display panel 112 and at least partially focus the received light to direct the light to specified regions proximate the viewer’s eyes.

[0041] FIG. 6 shows a front-view drawing of an example of a parallaxgenerating optic 118B that includes a parallax barrier 602. The parallax barrier can include a plurality of transmissive slits 704 that are equally spaced apart. FIG. 7 shows a cross-sectional view of the parallax barrier 602 (FIG. 6) having transmissive slits 704. The parallax barrier 602 can include an array of opaque strips 706 and thin transmissive slits 704 arranged to occlude portions of a displayed image in left and right viewing regions. The transmissive slits 704 can be spatially arranged to ensure that the left / right image portions are only visible in the corresponding left / right viewing regions for whichthey are intended. The parallax barrier 602 can be provided by a static physical layer in which the slits are precisely positioned, or electronically generated on an adaptive intermediate liquid crystal display layer.

[0042] The parallax-generating optic 118 can be invariant along an optical axis (OA) that is angled with respect to the grid axes (e.g., the x-axis and the -axis), such as at a slant angle (a) of 45 degrees or about 45 degrees with respect to the grid axes 208. For example, the parallax-generating optic 118 can have transmissive features, such as the cylindrical lenses or the transmissive slits, that are invariant along the optical axis (OA) and are periodic along an axis that is orthogonal to the optical axis (OA).

[0043] Referring again to FIG. 1, the autostereoscopic display can include a material 116 disposed between the display panel 112 and the parallax-generating optic 118. In some examples, the material 116 may extend fully between the display panel 112 and the parallax-generating optic 118, such that a light ray originating at the display panel 112 passes only through the material 116 (and does not pass through any air or unfilled volume) before arriving at the parallax-generating optic 118. In other examples, the material 116 may occupy only a portion of the volume between the display panel 112 and the parallax-generating optic 118, such that a light ray originating at the display panel 112 passes through at least some of the material 116 and passes through a volume of air before arriving at the parallax-generating optic 118. The material 116 may have a refractive index denoted by quantity n. The value of the refractive index n may be between about 1.3 and about 2, although other suitable values may be used. Suitable materials 116 can include glass, plastic, a transparent optical adhesive, and others. In some examples, the material 116 can be dispensed in a liquid form, then cured in place, such as by exposure to ultraviolet light or heat. In other examples, the material 116 can be manufactured as a solid unit and placed in its location in the autostereoscopic display. For example, the material 116 can function as a cover glass for the display panel 112. In some examples, the material 116 can function as a relatively precise spacing element. For example, the material 116 can be manufactured to have a specified thickness to within a specified thickness tolerance and can set the spacing between the display panel 112 and parallax-generating optic 118 to have a value equal to the specified thickness when the autostereoscopic display is assembled.

[0044] As illustrated in FIG. 1, the multiview display system 100 can include processing circuitry 130. The processing circuitry 130 can include a non-transitorycomputer-readable storage medium 132, such as a hard disk, a solid-state hard drive, memory, optical media, magnetic media, semiconductor media, punch cards, or others. The non-transitory computer-readable storage medium 132 can be included locally with the processing circuitry 130 or can be located remotely from the processing circuitry 130 and be accessible through a wired or wireless connection. The non-transitory computer- readable storage medium 132 can store instructions 134 for performing a particular task or series of tasks or executing some or all steps of a method.

[0045] The instructions 134, when executed by the processing circuitry 130, can cause the processing circuitry 130 to perform operations 136. For example, the instructions 134 may be for generating a stereoscopic image pair from a monoscopic image. Suitable operations 136 for generating a stereoscopic image pair from a monoscopic image can include, among other operations: receiving, with processing circuitry 130, data specifying a monoscopic image; generating, with the processing circuitry 130, a blur intensity map based on the monoscopic image; generating, with the processing circuitry 130, an initial disparity map based on the monoscopic image and an initial convergence plane; generating, with the processing circuitry 130, a final disparity map based on the monoscopic image and a final convergence plane spaced apart from the initial convergence plane, the final convergence plane being based on blur and disparity characteristics of the monoscopic image; and generating, with the processing circuitry 130, a stereoscopic image pair using the monoscopic image and the final disparity map. By executing these operations 136, the processing circuitry 130 can cause the multiview display system 100 to perform generating a stereoscopic image pair from a monoscopic image. The operations 136 can further include using the multiview display system 100 to display the stereoscopic image pair. These operations 136 are described in detail below.

[0046] FIG. 8 shows an example of various quantities 800 associated with the operations 136 that can generate a stereoscopic image pair from a monoscopic image. The quantities 800 can be data structures, such as arrays, matrices, or variables. The quantities 800 can be stored in memory. The processing circuitry 130 can read one or more of the quantities 800 from memory and write one or more of the quantities 800 to memory when the processing circuitry 130 executes the operations 136. The quantities 800 are but mere examples of quantities that can be used to generate a stereoscopic image pair from a monoscopic image. Other suitable quantities can be used.

[0047] The processing circuitry 130 can receive data that specifies a monoscopic image 802. The data can be received from memory, a local storage device, or a remote storage device. The data can be included as one or more frames of a video signal, which can specify the monoscopic image 802 as one of a series of sequential monoscopic images.

[0048] The processing circuitry 130 can generate a blur intensity map 804 based on the monoscopic image 802. In general, the quantity blur can correspond to measure of a relative fuzziness or lack of sharpness in an image. For example, for typical photography, an object that is in focus can have a relatively small blur, while an object that is out of focus can have a relatively large blur. A blur intensity map can be an array, with a same size as an accompanying image. The value of the blur intensity map at a particular pixel or pixel region can correspond to the amount of blur at the particular pixel or pixel region of the accompanying image. Objects in the scene that are in focus have relatively small values in the blur intensity map. Objects in the scene that are out of focus have relatively large values in the blur intensity map. Image processing software, such as the software executed by the processing circuitry 130, can generate a blur intensity map from a monoscopic image.

[0049] The processing circuitry 130 can generate a plurality 806 of provisional disparity maps 806A, 806B, 806C, from the monoscopic image 802. The provisional disparity maps 806A, 806B, 806C can have different convergence planes. In the example of FIG. 8, the plurality 806 of provisional disparity maps includes three provisional disparity maps 806A, 806B, 806C. Alternatively, the plurality 806 of provisional disparity maps can include two provisional disparity maps or more than three provisional disparity maps.

[0050] The processing circuitry 130 can compare the provisional disparity maps 806A, 806B, 806C to the blur intensity map 804. Based on the comparisons, the processing circuitry 130 can select the provisional disparity map that is most similar to the blur intensity map 804 to form a final disparity map 808. Alternatively, the processing circuitry 130 can interpolate between adjacent convergence planes of the provisional disparity maps 806A, 806B, 806C to determine a final convergence plane and use the final convergence plane to generate the final disparity map 808.

[0051] The processing circuitry 130 can generate a stereoscopic image pair 810 using the monoscopic image 802 and the final disparity map 808. The stereoscopicimage pair 810 can include a left image 812 and a right image 814. When an autostereoscopic display displays the stereoscopic image pair 810 to a viewer, the autostereoscopic display directs the left image 812 to a left eye of the viewer and directs the right image 814 to a right eye of the viewer.

[0052] For the method 900 discussed below, the monoscopic image 802 is the input and the stereoscopic image pair 810 is the output. The stereoscopic image pair 810 can be displayed on an autostereoscopic display, such as the multiview display system 100 of FIG. 1

[0053] FIG. 9 shows a flowchart of an example of a method 900 for generating a stereoscopic image pair from a monoscopic image. The method 900 can be executed on, apply to, or can be used by or used with, an autostereoscopic display system, such as the multiview display system 100 of FIG. 1. The method 900 can be stored as instructions 134 on a non-transitory computer-readable storage medium 132. The instructions 134, when executed by the processing circuitry 130, can cause the processing circuitry 130 to perform the operations 136. The operations 136 can include the operations detailed below for the method 900. The method 900 is but one method for generating a stereoscopic image pair from a monoscopic image; other suitable methods may be used.

[0054] At operation 902, the processing circuitry 130 can receive data specifying a monoscopic image 802.

[0055] At operation 904, the processing circuitry 130 can generate a blur intensity map 804 based on the monoscopic image 802.

[0056] At operation 906, the processing circuitry 130 can generate an initial disparity map based on the monoscopic image 802 and an initial convergence plane.

[0057] At operation 908, the processing circuitry 130 can generate a final disparity map based on the monoscopic image 802 and a final convergence plane spaced apart from the initial convergence plane. The final convergence plane can be based on blur and disparity characteristics of the monoscopic image 802. A similarity between the final disparity map and the blur intensity map can be greater than a similarity between the initial disparity map and the blur intensity map.

[0058] At operation 910, the processing circuitry 130 can generate a stereoscopic image pair 810 using the monoscopic image 802 and the final disparity map.

[0059] A similarity between the final disparity map and the blur intensity map 804 may be greater than a similarity between the initial disparity map and the blurintensity map 804. In other words, the particular disparity map that the processing circuitry 130 uses to generate the stereoscopic image pair 810 may not be the initial convergence plane.

[0060] The processing circuitry 130 can specify a plurality of provisional convergence planes that are spaced apart from one another. The plurality of provisional convergence planes can include the initial convergence plane. The processing circuitry 130 can generate a plurality 806 of provisional disparity maps 806A, 806B, 806C based on the monoscopic image 802 and the respective plurality of provisional convergence planes. The plurality 806 of provisional disparity maps 806A, 806B, 806C can include the initial disparity map.

[0061] The processing circuitry 130 can determine a plurality of similarity values. Each similarity value can represent a similarity between the blur intensity map 804 and a respective provisional disparity map of the plurality 806 of provisional disparity maps.

[0062] As a specific example of forming the similarity values, each similarity value of the plurality of similarity values can be determined by, for each pixel (or pixel group) of the monoscopic image: forming a numerator as a scalar product between the blur intensity map value at the pixel and the respective provisional disparity map value at the pixel; forming a denominator as a product of a magnitude of the blur intensity map value at the pixel and a magnitude of the respective provisional disparity map value at the pixel; forming a normalized value as the numerator divided by the denominator; and forming, as the similarity value, an image-wide similarity value from the normalized values of the pixels of the monoscopic image. Other techniques for forming the similarity values can be used, such as using various statistical measures or machine learning approaches.

[0063] There are several techniques that can combine the pixel-specific values into a single image-wide value. For example, the image-wide similarity value can include a sum of squares of the normalized values of the pixels of the monoscopic image. The image-wide similarity value can include a square root of the sum of the squares of the normalized values of the pixels of the monoscopic image. The image-wide similarity value can include a sum of absolute values of the normalized values of the pixels of the monoscopic image. These are but mere examples; other techniques for combining the pixel-specific values into a single image-wide value can be used.

[0064] FIG. 10 shows a schematic drawing of an example of determining the final convergence plane based on blur and disparity characteristics of the monoscopic image.

[0065] The top of FIG. 10 shows an example of the blur intensity map 804. The processing circuitry 130 can generate the blur intensity map 804 based on the monoscopic image 802 (FIG. 8). In the example of FIG. 10, the blur intensity map 804 shows a feature that is shaped as a plus sign (“+”). Other shapes can be used. In the example of FIG. 10, the plus sign is intended to be in focus, while a background to the plus sign is intended to be out of focus. More specifically, the blur intensity map 804 indicates the monoscopic image includes a first portion (e.g., is shaped as a plus sign (“+”)) that encloses an area characterized by a particular blur intensity, and a second portion that is outside of the enclosed area and characterized by a different blur intensity. Other shapes can be used, such as can correspond to various objects in the image. In other examples, more than two different blur intensities can be represented. The blur characteristic of the monoscopic image 802 is based on the blur intensity map 804.

[0066] Below the blur intensity map 804, FIG. 10 shows a plurality 806 of provisional disparity maps 806A, 806B, 806C. The processing circuitry 130 can generate the plurality 806 of provisional disparity maps 806A, 806B, 806C based on the monoscopic image 802 and a respective plurality of provisional convergence planes that are spaced apart from one another. In some examples, the processing circuitry 130 can specify the plurality of provisional convergence planes without input from a user. The plurality of provisional convergence planes can include the initial convergence plane.

[0067] The bottom of FIG. 10 shows a plot of a similarity value as a function of convergence plane location. For example, the plot shows the similarity between the blur intensity map 804 and corresponding respective provisional disparity maps. Each circle on the plot can correspond to a different convergence plane location corresponding to a respective different disparity map. Each linear or curved portion between the circles can correspond to interpolation between the convergence plane locations.

[0068] The processing circuitry 130 can determine, based on the plurality of similarity values, the final convergence plane. Two examples of making this determination follow; other techniques for making this determination can be used.

[0069] In a first example, the processing circuitry 130 can identify a greatest similarity value 1030 among the plurality of similarity values, where each similarityvalue represents a respective similarity between the blur intensity map 804 and a respective provisional disparity map of the plurality 806 of provisional disparity maps. The processing circuitry 130 can select, as the final disparity map, a provisional disparity map of the plurality of provisional disparity maps that corresponds to the greatest similarity value 1030.

[0070] In a second example, the processing circuitry 130 can fit a similarity value curve 1020 via interpolation to the plurality of similarity values. The processing circuitry 130 can determine a maximum value 1040 of the similarity value curve 1020. The processing circuitry 130 can set the final convergence plane to be disposed at a location that corresponds to the maximum value 1040 of the similarity value curve 1020. The maximum value 1040 of the similarity value curve 1020 can be between two adjacent similarity values of the plurality of similarity values. The final convergence plane can be located between two adjacent provisional convergence planes of the plurality of provisional convergence planes.

[0071] In an example, an autostereoscopic display, such as the multiview display 110, can display the stereoscopic image pair, such as the stereoscopic image pair 810, such that when the stereoscopic image pair is displayed on the autostereoscopic display, the final convergence plane can coincide with a plane of the autostereoscopic display. In this example, a first portion of the stereoscopic image pair can appear to be located above the plane of the autostereoscopic display. In this example, a second portion of the stereoscopic image pair can appear to be located below the plane of the autostereoscopic display. A viewer tracker, such as the viewer tracker 120, can determine a location of a viewer, such as the viewer 42.

[0072] The autostereoscopic display can include a display panel, such as the display panel 112. The autostereoscopic display can have an array of subpixels, such as the array of subpixels 114, that can display the stereoscopic image pair. A parallaxgenerating optic, such as the parallax-generating optic 118, can direct light from the display panel to the viewer. The parallax-generating optic can include a lenticular lens, such as the lenticular lens 402. The parallax-generating optic can include a parallax barrier, such as the parallax barrier 602, having transmissive slits, such as the transmissive slits 704. The parallax-generating optic can be invariant along an optical axis (OA) having a slant angle relative to the display panel. The slant angle being within a specified angular tolerance of forty -five degrees, such as five degrees, two degrees, onedegree, or another suitable value. The parallax-generating optic can be periodic along an axis orthogonal to the optical axis (OA). The processing circuitry 130 can arrange left and right images of the stereoscopic image pair, such as the left image 812 and the right image 814, on the array of subpixels such that the parallax-generating optic directs the left and right images to respective eyes of the viewer at the location of the viewer.

[0073] To further illustrate the system and method disclosed herein, a nonlimiting list of examples is provided below. Each of the following non limiting examples can stand on its own or can be combined in any permutation or combination with any one or more of the other examples.

[0074] In Example 1, an autostereoscopic display system can comprise processing circuitry configured to perform operations. The operations can comprise: receiving data specifying a monoscopic image; specifying an initial convergence plane; generating a final disparity map based on the monoscopic image and a final convergence plane spaced apart from the initial convergence plane, the final convergence plane being based on blur and disparity characteristics of the monoscopic image; and generating a stereoscopic image pair using the monoscopic image and the final disparity map.

[0075] In Example 2, the autostereoscopic display system of Example 1 can optionally be configured such that the operations further comprise: generating a blur intensity map based on the monoscopic image, the blur characteristic being based on the blur intensity map; and generating an initial disparity map based on the monoscopic image and the initial convergence plane, wherein a similarity between the final disparity map and the blur intensity map is greater than a similarity between the initial disparity map and the blur intensity map.

[0076] In Example 3, the autostereoscopic display system of any one of Examples 1-2 can optionally be configured such that the operations further comprise: specifying a plurality of provisional convergence planes that are spaced apart from one another, the plurality of provisional convergence planes including the initial convergence plane; and generating a plurality of provisional disparity maps based on the monoscopic image and the respective plurality of provisional convergence planes, the plurality of provisional disparity maps including the initial disparity map.

[0077] In Example 4, the autostereoscopic display system of any one of Examples 1-3 can optionally be configured such that the operations further comprise: determining a plurality of similarity values, each similarity value representing asimilarity between the blur intensity map and a respective provisional disparity map of the plurality of provisional disparity maps; and determining, based on the plurality of similarity values, the final convergence plane.

[0078] In Example 5, the autostereoscopic display system of any one of Examples 1-4 can optionally be configured such that the operations further comprise: selecting, as a selected similarity value, a greatest similarity value of the plurality of similarity values; and selecting, as the final disparity map, a provisional disparity map of the plurality of provisional disparity maps that corresponds to the selected similarity value.

[0079] In Example 6, the autostereoscopic display system of any one of Examples 1-5 can optionally be configured such that the operations further comprise: fitting a similarity value curve via interpolation to the plurality of similarity values; determining a maximum value of the similarity value curve; and setting the final convergence plane to be disposed at a location that corresponds to the maximum value of the similarity value curve, wherein: the maximum value of the similarity value curve is between two adjacent similarity values of the plurality of similarity values; and the final convergence plane is located between two adjacent provisional convergence planes of the plurality of provisional convergence planes.

[0080] In Example 7, the autostereoscopic display system of any one of Examples 1-6 can optionally be configured such that each similarity value of the plurality of similarity values is determined by, for each pixel of the monoscopic image: forming a numerator as a scalar product between the blur intensity map value at the pixel and the respective provisional disparity map value at the pixel; forming a denominator as a product of a magnitude of the blur intensity map value at the pixel and a magnitude of the respective provisional disparity map value at the pixel; forming a normalized value as the numerator divided by the denominator; and forming, as the similarity value, an image-wide similarity value from the normalized values of the pixels of the monoscopic image.

[0081] In Example 8, the autostereoscopic display system of any one of Examples 1-7 can optionally be configured such that the image-wide similarity value comprises at least one of: a sum of squares of the normalized values of the pixels of the monoscopic image; a square root of the sum of the squares of the normalized values ofthe pixels of the monoscopic image; or a sum of absolute values of the normalized values of the pixels of the monoscopic image.

[0082] In Example 9, the autostereoscopic display system of any one of Examples 1-8 can optionally further comprise: an autostereoscopic display configured to display the stereoscopic image pair such that when the stereoscopic image pair is displayed on the autostereoscopic display: the final convergence plane coincides with a plane of the autostereoscopic display; a first portion of the stereoscopic image pair appears to be located above the plane of the autostereoscopic display; and a second portion of the stereoscopic image pair appears to be located below the plane of the autostereoscopic display; and a viewer tracker configured to determine a location of a viewer.

[0083] In Example 10, the autostereoscopic display system of any one of Examples 1-9 can optionally be configured such that the autostereoscopic display comprises: a display panel having an array of subpixels configured to display the stereoscopic image pair; and a parallax-generating optic configured to direct light from the display panel to the viewer, the parallax-generating optic including one of a lenticular lens or a parallax barrier having transmissive slits, the parallax-generating optic being invariant along an optical axis having a slant angle relative to the display panel, the slant angle being within a specified angular tolerance of forty -five degrees, the parallaxgenerating optic being periodic along an axis orthogonal to the optical axis; and wherein the operations further comprise: arranging left and right images of the stereoscopic image pair on the array of subpixels such that the parallax-generating optic directs the left and right images to respective eyes of the viewer at the location of the viewer.

[0084] In Example 11, a method for generating a stereoscopic image pair from a monoscopic image can comprise: receiving, with processing circuitry, data specifying a monoscopic image; specifying, with the processing circuitry, an initial convergence plane; generating, with the processing circuitry, a final disparity map based on the monoscopic image and a final convergence plane spaced apart from the initial convergence plane, the final convergence plane being based on blur and disparity characteristics of the monoscopic image; and generating, with the processing circuitry, a stereoscopic image pair using the monoscopic image and the final disparity map.

[0085] In Example 12, the method of Example 11 can optionally further comprise: generating, with the processing circuitry, a blur intensity map based on themonoscopic image, the blur characteristic being based on the blur intensity map; and generating, with the processing circuitry, an initial disparity map based on the monoscopic image and the initial convergence plane, wherein a similarity between the final disparity map and the blur intensity map is greater than a similarity between the initial disparity map and the blur intensity map.

[0086] In Example 13, the method of any one of Examples 11-12 can optionally further comprise: specifying, with the processing circuitry, a plurality of provisional convergence planes that are spaced apart from one another, the plurality of provisional convergence planes including the initial convergence plane; generating, with the processing circuitry, a plurality of provisional disparity maps based on the monoscopic image and the respective plurality of provisional convergence planes, the plurality of provisional disparity maps including the initial disparity map; determining, with the processing circuitry, a plurality of similarity values, each similarity value representing a similarity between the blur intensity map and a respective provisional disparity map of the plurality of provisional disparity maps; and determining, with the processing circuitry, based on the plurality of similarity values, the final convergence plane.

[0087] In Example 14, the method of any one of Examples 11-13 can optionally further comprise: selecting, with the processing circuitry, as a selected similarity value, a greatest similarity value of the plurality of similarity values; and selecting, with the processing circuitry, as the final disparity map, a provisional disparity map of the plurality of provisional disparity maps that corresponds to the selected similarity value.

[0088] In Example 15, the method of any one of Examples 11-14 can optionally further comprise: fitting, with the processing circuitry, a similarity value curve via interpolation to the plurality of similarity values; determining, with the processing circuitry, a maximum value of the similarity value curve; and setting, with the processing circuitry, the final convergence plane to be disposed at a location that corresponds to the maximum value of the similarity value curve, wherein: the maximum value of the similarity value curve is between two adjacent similarity values of the plurality of similarity values; and the final convergence plane is located between two adjacent provisional convergence planes of the plurality of provisional convergence planes.

[0089] In Example 16, a non-transitory computer-readable storage medium can store instructions for generating a stereoscopic image pair from a monoscopic image. The instructions, when executed by processing circuitry, can cause the processingcircuitry to perform operations. The operations can comprise: receiving, with the processing circuitry, data specifying a monoscopic image; specifying, with the processing circuitry, an initial convergence plane; generating, with the processing circuitry, a final disparity map based on the monoscopic image and a final convergence plane spaced apart from the initial convergence plane, the final convergence plane being based on blur and disparity characteristics of the monoscopic image; and generating, with the processing circuitry, a stereoscopic image pair using the monoscopic image and the final disparity map.

[0090] In Example 17, the non-transitory computer-readable storage medium of Example 16 can optionally be configured such that the operations further comprise: generating, with the processing circuitry, a blur intensity map based on the monoscopic image; and generating, with the processing circuitry, an initial disparity map based on the monoscopic image and the initial convergence plane, wherein a similarity between the final disparity map and the blur intensity map is greater than a similarity between the initial disparity map and the blur intensity map.

[0091] In Example 18, the non-transitory computer-readable storage medium of any one of Examples 16-17 can optionally be configured such that the operations further comprise: specifying, with the processing circuitry, a plurality of provisional convergence planes that are spaced apart from one another, the plurality of provisional convergence planes including the initial convergence plane; generating, with the processing circuitry, a plurality of provisional disparity maps based on the monoscopic image and the respective plurality of provisional convergence planes, the plurality of provisional disparity maps including the initial disparity map; determining, with the processing circuitry, a plurality of similarity values, each similarity value representing a similarity between the blur intensity map and a respective provisional disparity map of the plurality of provisional disparity maps; and determining, with the processing circuitry, based on the plurality of similarity values, the final convergence plane.

[0092] In Example 19, the non-transitory computer-readable storage medium of any one of Examples 16-18 can optionally be configured such that the operations further comprise: selecting, with the processing circuitry, as a selected similarity value, a greatest similarity value of the plurality of similarity values; and selecting, with the processing circuitry, as the final disparity map, a provisional disparity map of the plurality of provisional disparity maps that corresponds to the selected similarity value.

[0093] In Example 20, the non-transitory computer-readable storage medium of any one of Examples 16-19 can optionally be configured such that the operations further comprise: fitting, with the processing circuitry, a similarity value curve via interpolation to the plurality of similarity values; determining, with the processing circuitry, a maximum value of the similarity value curve; and setting, with the processing circuitry, the final convergence plane to be disposed at a location that corresponds to the maximum value of the similarity value curve, wherein: the maximum value of the similarity value curve is between two adjacent similarity values of the plurality of similarity values; and the final convergence plane is located between two adjacent provisional convergence planes of the plurality of provisional convergence planes.

[0094] The above detailed description includes references to the accompanying drawings, which form a part of the detailed description. The drawings show, by way of illustration, specific embodiments in which the invention can be practiced. These embodiments are also referred to herein as "examples." Such examples can include elements in addition to those shown or described. However, examples are contemplated in which only those elements shown or described are provided. Moreover, other examples can any combination or permutation of those elements shown or described (or one or more aspects thereof), either with respect to a particular example (or one or more aspects thereof), or with respect to other examples (or one or more aspects thereof) shown or described herein.

[0095] In this document, the terms "a" or "an" are used, as is common in patent documents, to include one or more than one, independent of any other instances or usages of "at least one" or "one or more." In this document, the term "or" is used to refer to a nonexclusive or, such that "A or B" can include "A but not B," "B but not A," and "A and B," unless otherwise indicated. In the appended claims, the terms "including" and "in which" are used as the plain-English equivalents of the respective terms "comprising" and "wherein". Also, in the following claims, the terms "including" and "comprising" are open-ended, that is, a system, device, article, or process that includes elements in addition to those listed after such a term in a claim are still deemed to fall within the scope of that claim. Moreover, in the following claims, the terms "first," "second," and "third," etc. are used merely as labels, and are not intended to impose numerical requirements on their objects.

[0096] The above description is intended to be illustrative, and not restrictive. For example, the above-described examples (or one or more aspects thereof) can be used in combination with each other. Other embodiments can be used, such as by one of ordinary skill in the art upon reviewing the above description. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. Also, in the above Detailed Description, various features can be grouped together to streamline the disclosure. This should not be interpreted as intending that an unclaimed disclosed feature is essential to any claim. Rather, inventive subject matter can lie in less than all features of a particular disclosed embodiment. Thus, the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separate embodiment, and it is contemplated that such embodiments can be combined with each other in various combinations or permutations. The scope of the invention should be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.

Claims

CLAIMSWHAT IS CLAIMED IS:

1. An autostereoscopic display system, comprising: processing circuitry configured to perform operations, the operations comprising: receiving data specifying a monoscopic image; specifying an initial convergence plane; generating a final disparity map based on the monoscopic image and a final convergence plane spaced apart from the initial convergence plane, the final convergence plane being based on blur and disparity characteristics of the monoscopic image; and generating a stereoscopic image pair using the monoscopic image and the final disparity map.

2. The autostereoscopic display system of claim 1, wherein the operations further comprise: generating a blur intensity map based on the monoscopic image, the blur characteristic being based on the blur intensity map; and generating an initial disparity map based on the monoscopic image and the initial convergence plane, wherein a similarity between the final disparity map and the blur intensity map is greater than a similarity between the initial disparity map and the blur intensity map.

3. The autostereoscopic display system of claim 2, wherein the operations further comprise: specifying a plurality of provisional convergence planes that are spaced apart from one another, the plurality of provisional convergence planes including the initial convergence plane; and generating a plurality of provisional disparity maps based on the monoscopic image and the respective plurality of provisional convergence planes, the plurality of provisional disparity maps including the initial disparity map.

4. The autostereoscopic display system of claim 3, wherein the operations further comprise: determining a plurality of similarity values, each similarity value representing a similarity between the blur intensity map and a respective provisional disparity map of the plurality of provisional disparity maps; and determining, based on the plurality of similarity values, the final convergence plane.

5. The autostereoscopic display system of claim 4, wherein the operations further comprise: selecting, as a selected similarity value, a greatest similarity value of the plurality of similarity values; and selecting, as the final disparity map, a provisional disparity map of the plurality of provisional disparity maps that corresponds to the selected similarity value.

6. The autostereoscopic display system of claim 4, wherein the operations further comprise: fitting a similarity value curve via interpolation to the plurality of similarity values; determining a maximum value of the similarity value curve; and setting the final convergence plane to be disposed at a location that corresponds to the maximum value of the similarity value curve, wherein: the maximum value of the similarity value curve is between two adjacent similarity values of the plurality of similarity values; and the final convergence plane is located between two adjacent provisional convergence planes of the plurality of provisional convergence planes.

7. The autostereoscopic display system of claim 4, wherein each similarity value of the plurality of similarity values is determined by, for each pixel of the monoscopic image: forming a numerator as a scalar product between the blur intensity map value at the pixel and the respective provisional disparity map value at the pixel; forming a denominator as a product of a magnitude of the blur intensity map value at the pixel and a magnitude of the respective provisional disparity map value at the pixel; forming a normalized value as the numerator divided by the denominator; and forming, as the similarity value, an image-wide similarity value from the normalized values of the pixels of the monoscopic image.

8. The autostereoscopic display system of claim 7, wherein the image-wide similarity value comprises at least one of: a sum of squares of the normalized values of the pixels of the monoscopic image; a square root of the sum of the squares of the normalized values of the pixels of the monoscopic image; or a sum of absolute values of the normalized values of the pixels of the monoscopic image.

9. The autostereoscopic display system of claim 1, further comprising: an autostereoscopic display configured to display the stereoscopic image pair such that when the stereoscopic image pair is displayed on the autostereoscopic display: the final convergence plane coincides with a plane of the autostereoscopic display; a first portion of the stereoscopic image pair appears to be located above the plane of the autostereoscopic display; and a second portion of the stereoscopic image pair appears to be located below the plane of the autostereoscopic display; and a viewer tracker configured to determine a location of a viewer.

10. The autostereoscopic display system of claim 9, wherein the autostereoscopic display comprises: a display panel having an array of subpixels configured to display the stereoscopic image pair; and a parallax-generating optic configured to direct light from the display panel to the viewer, the parallax-generating optic including one of a lenticular lens or a parallax barrier having transmissive slits, the parallax-generating optic being invariant along an optical axis having a slant angle relative to the display panel, the slant angle being within a specified angular tolerance of forty -five degrees, the parallax-generating optic being periodic along an axis orthogonal to the optical axis; and wherein the operations further comprise: arranging left and right images of the stereoscopic image pair on the array of subpixels such that the parallax-generating optic directs the left and right images to respective eyes of the viewer at the location of the viewer.

11. A method for generating a stereoscopic image pair from a monoscopic image, the method comprising: receiving, with processing circuitry, data specifying a monoscopic image; specifying, with the processing circuitry, an initial convergence plane; generating, with the processing circuitry, a final disparity map based on the monoscopic image and a final convergence plane spaced apart from the initial convergence plane, the final convergence plane being based on blur and disparity characteristics of the monoscopic image; and generating, with the processing circuitry, a stereoscopic image pair using the monoscopic image and the final disparity map.

12. The method of claim 11, further comprising: generating, with the processing circuitry, a blur intensity map based on the monoscopic image, the blur characteristic being based on the blur intensity map; and generating, with the processing circuitry, an initial disparity map based on the monoscopic image and the initial convergence plane, wherein a similarity between the final disparity map and the blur intensity map is greater than a similarity between the initial disparity map and the blur intensity map.

13. The method of claim 12, further comprising: specifying, with the processing circuitry, a plurality of provisional convergence planes that are spaced apart from one another, the plurality of provisional convergence planes including the initial convergence plane; generating, with the processing circuitry, a plurality of provisional disparity maps based on the monoscopic image and the respective plurality of provisional convergence planes, the plurality of provisional disparity maps including the initial disparity map; determining, with the processing circuitry, a plurality of similarity values, each similarity value representing a similarity between the blur intensity map and a respective provisional disparity map of the plurality of provisional disparity maps; and determining, with the processing circuitry, based on the plurality of similarity values, the final convergence plane.

14. The method of claim 13, further comprising: selecting, with the processing circuitry, as a selected similarity value, a greatest similarity value of the plurality of similarity values; and selecting, with the processing circuitry, as the final disparity map, a provisional disparity map of the plurality of provisional disparity maps that corresponds to the selected similarity value.

15. The method of claim 13, further comprising: fitting, with the processing circuitry, a similarity value curve via interpolation to the plurality of similarity values; determining, with the processing circuitry, a maximum value of the similarity value curve; and setting, with the processing circuitry, the final convergence plane to be disposed at a location that corresponds to the maximum value of the similarity value curve, wherein: the maximum value of the similarity value curve is between two adjacent similarity values of the plurality of similarity values; and the final convergence plane is located between two adjacent provisional convergence planes of the plurality of provisional convergence planes.

16. A non-transitory computer-readable storage medium storing instructions for generating a stereoscopic image pair from a monoscopic image, the instructions, when executed by processing circuitry, cause the processing circuitry to perform operations, the operations comprising: receiving, with the processing circuitry, data specifying a monoscopic image; specifying, with the processing circuitry, an initial convergence plane; generating, with the processing circuitry, a final disparity map based on the monoscopic image and a final convergence plane spaced apart from the initial convergence plane, the final convergence plane being based on blur and disparity characteristics of the monoscopic image; and generating, with the processing circuitry, a stereoscopic image pair using the monoscopic image and the final disparity map.

17. The non-transitory computer-readable storage medium of claim 16, wherein the operations further comprise: generating, with the processing circuitry, a blur intensity map based on the monoscopic image; and generating, with the processing circuitry, an initial disparity map based on the monoscopic image and the initial convergence plane, wherein a similarity between the final disparity map and the blur intensity map is greater than a similarity between the initial disparity map and the blur intensity map.

18. The non-transitory computer-readable storage medium of claim 17, wherein the operations further comprise: specifying, with the processing circuitry, a plurality of provisional convergence planes that are spaced apart from one another, the plurality of provisional convergence planes including the initial convergence plane; generating, with the processing circuitry, a plurality of provisional disparity maps based on the monoscopic image and the respective plurality of provisional convergence planes, the plurality of provisional disparity maps including the initial disparity map; determining, with the processing circuitry, a plurality of similarity values, each similarity value representing a similarity between the blur intensity map and a respective provisional disparity map of the plurality of provisional disparity maps; and determining, with the processing circuitry, based on the plurality of similarity values, the final convergence plane.

19. The non-transitory computer-readable storage medium of claim 18, wherein the operations further comprise: selecting, with the processing circuitry, as a selected similarity value, a greatest similarity value of the plurality of similarity values; and selecting, with the processing circuitry, as the final disparity map, a provisional disparity map of the plurality of provisional disparity maps that corresponds to the selected similarity value.

20. The non-transitory computer-readable storage medium of claim 18, wherein the operations further comprise: fitting, with the processing circuitry, a similarity value curve via interpolation to the plurality of similarity values; determining, with the processing circuitry, a maximum value of the similarity value curve; and setting, with the processing circuitry, the final convergence plane to be disposed at a location that corresponds to the maximum value of the similarity value curve, wherein: the maximum value of the similarity value curve is between two adjacent similarity values of the plurality of similarity values; and the final convergence plane is located between two adjacent provisional convergence planes of the plurality of provisional convergence planes.

Citation Information

Patent Citations

  • Method for generating super-resolution video using multi-camera video triplets and device for the same

    KR1020240103974A

  • System and method for integrated visualization camera and optical coherence tomography

    US20210169324A1

  • Transferring of three-dimensional image data

    US20210211630A1

  • Multifocal plane based method to produce stereoscopic viewpoints in a DIBR system (MFP-DIBR)

    US20230027251A1

  • Multiview display system and method employing multiview image convergence plane tilt

    US20230290046A1