Method and apparatus for canceling distortion and displacement of a displayed three-dimensional image - Patents.com

Abstract

The invention relates to a method for cancelling distortions and displacements of a three-dimensional image perceived by an observer of an autostereoscopic display device, where the cancelling is performed by taking into account 1) the refractive properties of the display device, 2) the position of each pixel in the display device, and 3) the viewing position of the observer's eyes. In this way, the observer may have a more realistic perception of the three-dimensional virtual content. This includes, for example, an improved look-around effect for the observer and an improved interaction of the observer with the displayed three-dimensional virtual content. Preferably, the cancellation of the distortions and / or displacements is performed by using a refraction model for the display device.

Description

[Technical field]

[0001] The present invention relates to a method and apparatus for canceling distortion and displacement of a three-dimensional image displayed to an observer (or viewer). [Background technology]

[0002] <Background> Autostereoscopic displays have attracted a great deal of attention in the past two decades. One of their most outstanding features is that they allow the viewer to perceive three-dimensional images without the use of a dedicated eyewear device. The key to this technology is the presence of a lenticular lens or parallax barrier within the screen, which directs the left-eye image only to the viewer's left eye and the right-eye image only to the viewer's right eye. The resulting three-dimensional image then appears in front of the display, but also further away than the display ("behind" the display). In practice, a small percentage of light always reaches the eye for which it was not intended, resulting in a loss of the three-dimensional image that is to be viewed, a phenomenon known as "crosstalk". The process of determining which pixels in an autostereoscopic display generate pixel outputs for the left-eye image and which pixels generate pixel outputs for the right-eye image involves image interlacing, a process commonly called "weaving". A unit that does this for an autostereoscopic display is typically called a "3D-weaver."

[0003] Combined with eye / face tracking, this technology also enables the so-called "look-around" effect, which allows the viewer to view the displayed three-dimensional object (i.e., virtual object) from different angles and therefore different perspectives, with the virtual object being perceived in the same position in the real world. An even more realistic experience is provided when the viewer actually interacts with the displayed virtual content. For example, the viewer may simply reach out to the virtual object and interact with it as if it were real, such as pressing a virtual button with a finger or puncturing a virtual balloon with a real pin. The display then utilizes sensor technology to identify and track real objects, such as fingers or pins. Knowing the positions of such real (or real, real) objects, the display device can associate them with the positions of the virtual images it displays.

[0004] However, it has been experienced that interaction with virtual objects is hindered in some cases, especially when the screen (or parts of it) are viewed at angles that are not close to the screen normal. The three-dimensional image appears distorted and / or displaced. More importantly, when an observer wishes to interact with an object, the interaction does not occur at the time and place that the observer expects it to be. Thus, known autostereoscopic displays appear to have some deficiencies with respect to their ability to render three-dimensional images with the correct size, shape, and position.

[0005] A related consequence of this shortcoming is that the look-around effect is hindered: when a displayed three-dimensional object is viewed from different angles, the object deforms according to the angle and does not appear in a stationary position. Summary of the Invention [Problem to be solved by the invention]

[0006] <Summary of the Invention> It is an object of the present invention to provide means and / or methods for improving the rendering of virtual three-dimensional images displayed by an autostereoscopic display in that deviations in size, shape and position of the images are reduced or even cancelled. In particular, it is an object to improve the look-around effect produced by the display. A further object is to improve the observer's interaction with the displayed virtual content, in particular the observer's overall virtual experience.

[0007] It has now been found that one or more of these objectives may be reached by applying certain methods that counteract effects that cause an inaccurate display of three-dimensional content. [Means for solving the problem]

[0008] Thus, in a first aspect, the invention relates to a method for cancelling the perceived distortion and / or displacement of a three-dimensional image displayed to a viewer by an autostereoscopic display device, the autostereoscopic display device comprising: an eye tracking system for determining the position of an observer's eyes relative to the autostereoscopic display device; a display configured to display a composite three-dimensional image of a left image observed by an observer's left eye and a right image observed by an observer's right eye, an array of pixels for generating a display output, each pixel capable of generating a pixel output; A transparent plate, - a lenticular lens provided over the array and comprising a parallax barrier or lenticular element, the front side of the transparent plate facing the observer and the back side of the transparent plate facing the array of pixels; the transparent plate having light refraction properties that cause an observer to perceive a distortion and / or a displacement of a three-dimensional image displayed by the display device, The method comprises: - determining the viewing position of an observer's eyes relative to the autostereoscopic display device using an eye tracking system; - providing three-dimensional image data; - weaving three-dimensional image data into an array of pixels, the step including selecting pixels which generate a pixel output for a left image and pixels which generate a pixel output for a right image, taking into account the position of the observer's eyes relative to the autostereoscopic display device; - controlling the selected pixels to display a three-dimensional image to a viewer; The method is characterized in that it cancels the perceived distortion and / or displacement of the three-dimensional image displayed by the display device caused by the transparent plate, taking into account the refractive properties of the transparent plate, the position of each pixel relative to the transparent plate, and the position of the observer's eyes relative to the autostereoscopic display device.

[0009] In another aspect, the present invention provides a method for producing a composition comprising: an eye tracking system for determining the position of an observer's eyes relative to an autostereoscopic display device; a display unit configured to display a composite three-dimensional image of a left image observed by an observer's left eye and a right image observed by an observer's right eye, an array of pixels for generating a display output, each pixel capable of generating a pixel output; A transparent plate, - a lenticular lens provided over the array and comprising a parallax barrier or lenticular element, the front side of the transparent plate facing the observer and the back side of the transparent plate facing the array of pixels; - the transparent plate having light-refractive properties that cause an observer to perceive a distortion and / or a displacement of a three-dimensional image displayed by the display device; - means for controlling pixels to display a three-dimensional image from image data to an observer, taking into account the position of the observer's eyes relative to the autostereoscopic display device, The present invention relates to an autostereoscopic display device, characterized in that the means for controlling the pixels include means for cancelling a perceived distortion and / or displacement caused by the transparent plate of a three-dimensional image displayed by the display device, taking into account the refractive properties of the transparent plate, the position of each pixel relative to the transparent plate and the position of the observer's eyes relative to the autostereoscopic display device.

[0010] The invention further relates to a computer readable medium comprising transitory or non-transitory data representing instructions for causing a processor system to execute a method for cancelling the perceived distortion and / or displacement as described above.

[0011] The invention further relates to software arranged, when executed on a computer, to carry out the method for cancelling the perceived distortion and / or displacement as described above. [Brief description of the drawings]

[0012] [Figure 1]FIG. 1 shows a schematic of an autostereoscopic display, where an offset is shown for two viewing positions. [Diagram 2] FIG. 2 shows an enlargement of a portion of the autostereoscopic display of FIG. [Diagram 3] FIG. 3 shows a diagram of a first embodiment of the method according to the invention. [Figure 4] FIG. 4 shows a diagram of a second embodiment of the method according to the invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] The elements in the figures are illustrated simply and clearly and are not necessarily drawn to scale. For example, the dimensions of some elements in the figures may be exaggerated relative to other elements to facilitate understanding of various exemplary embodiments of the present invention. In particular, the relative dimensions of the lenticular lens (particularly its lenticular elements) and the layers above and below cannot be derived from the figures, nor can the refraction angles be derived. Furthermore, terms such as "first", "second", etc. in this specification, if any, are generally used to distinguish between similar elements and are not necessarily used to describe a sequential or chronological order.

[0014] In the context of the present invention, the term "pixel" refers to the smallest addressable (controllable) light-emitting element in a display device. In some cases, a group of pixels are treated as one pixel, for example a combination of a red pixel, a blue pixel, and a green pixel, which together are perceived as a specific single color when viewed from a distance. In such cases, the color pixels are called subpixels.

[0015] In the context of the present invention, the term "left image" refers to an image displayed by an autostereoscopic display device dedicated to the left eye. Correspondingly, the term "right image" refers to an image displayed by an autostereoscopic display device dedicated to the right eye. It is understood herein that exclusivity for a particular eye is often not achievable in practice, but in such cases the observer's three-dimensional viewing experience will nevertheless be satisfactory to the observer.

[0016] Throughout this specification and the claims, the terms "three-dimensional image" and "autostereoscopic image" are used interchangeably, recognizing that an autostereoscopic image is not strictly the same as a three-dimensional image, but is merely an image that is perceived by an observer to be three-dimensional. The same applies to the terms "three-dimensional view" and "autostereoscopic view."

[0017] Following this, it was recognised that the term "perceived distortion and displacement of a three-dimensional image" actually refers to the perceived distortion and displacement of the left and right images.

[0018] It will be appreciated that where a particular position (e.g., of the observer's eyes) is defined relative to an autostereoscopic display device, it is equivalent to that position being defined relative to an item forming part of the autostereoscopic display device, such as a display or transparent plate.

[0019] In the context of the present invention, the term "three-dimensional image data" refers to image information that is input in some way to the autostereoscopic display device of the method of the present invention and converted into a format processable by this device. It may be recorded in a memory portion associated with the device or may be captured by a camera associated with the device (such a camera may be a camera remote from the device that is capable of capturing images of a scene, for example a scene around a person participating in a video conference in a different environment). The three-dimensional image data includes information that represents a visible three-dimensional image, in that the three-dimensional image data may be used to generate a two-dimensional or three-dimensional image on the device. The three-dimensional image data consists of left image data and right image data, which include information that represents a visible left image or right image, respectively. The three-dimensional image is typically generated on the autostereoscopic display device using the left image data and the corresponding right image data.

[0020] In the context of the present invention, the term "observer" means a person who consumes the content presented to him according to the method of the present invention. In addition to seeing a three-dimensional image, the observer can also experience other sensory stimuli, such as sound or tactile stimuli. However, for convenience, said person is therefore called an "observer", although it is understood that at the same time he can also be, for example, a "listener".

[0021] Throughout the text, references to the observer will be made by masculine terms such as "he," "him," or "his." This is for purposes of clarity and brevity only, it being understood that feminine terms such as "she" and "her" apply equally.

[0022] The method of the present invention makes use of a conventional autostereoscopic display device, for example as described in WO2013120785A2, whose main components are an eye tracking system and a display unit.

[0023] Typically, an eye tracking system comprises means for tracking the position of a user's eyes relative to the autostereoscopic display device and is operatively connected to the autostereoscopic display device.

[0024] The display typically includes means for displaying a three-dimensional image to an observer whose eyes are tracked by an eye-tracking device, such means comprising an array of pixels for generating a display output, and a parallax barrier or a lenticular lens comprising a plurality of lenticular elements provided over the array for directing a left image to the observer's left eye and a right image to the observer's right eye.

[0025] Usually the parallax barrier or lenticular lens is part of a more transparent structure provided above the array, which in the context of the present invention is denoted by the term "transparent plate". For example, the transparent plate may include a support layer, a protective layer, a layer of adhesive, a layer of hardened resin or a means for switching between a two-dimensional view and a three-dimensional view of the display. In fact, the transparent plate is a transparent plate through which the display output is Egress Includes all transparent materials that convey the display output of the pixels to the observer until they enter the medium between the autostereoscopic display device and the observer's eyes (usually air).

[0026] Typically, a transparent plate includes at least one lenticular lens backed by a transparent cover layer that forms the front surface of the transparent plate, i.e., the surface that faces the viewer. The lenticular lens typically includes a transparent spacer plate that forms the back side of the transparent plate. When a transparent plate is present in an autostereoscopic display device, such a spacer plate creates a spacing between the lenticular elements of the lenticular lens and the array of pixels.

[0027] Preferably, the transparent plate is a plate with uniform thickness, which simplifies the calculations required to apply the correct cancellation of the perceived distortions and displacements caused by the transparent plate, or, when considering the refractive properties of the transparent plate, the transparent plate is approximated by a plate with uniform thickness.

[0028] In general, the display comprises means for controlling pixels to display a three-dimensional image from the image data to a viewer. In particular, the means for controlling pixels includes a so-called 3D weaver, which receives left and right image data and weaves the left and right images over an array of pixels. In doing so, the weaver determines which pixels should generate pixel outputs corresponding to the respective images. This allows a three-dimensional image to be displayed from the left and right image data to a viewer at a particular position.

[0029] The inventors have recognized that deviations in size, shape, and position of a displayed three-dimensional image may have their origin in the refraction of light caused by a transparent plate between the pixel and the viewer. Such deviations are small for light incidence angles close to normal and large for more oblique incidence. It is essential that different regions of an image are viewed at different angles, and therefore the display of different regions of an image may be subject to different deviations. This may lead to the viewer perceiving a distortion and / or displacement of the three-dimensional image displayed by the display device.

[0030] It was envisaged that this problem could be solved by creating an offset in the pixel output for each pixel by attributing a particular pixel output of a first pixel at a first location in the array to a second pixel at a second location in the array. In this way, the viewer perceives a particular pixel output as coming from a first location in the array, when in fact it comes from a second location in the array. The first location in the array is then the apparent location of the pixel as experienced by the viewer.

[0031] In the method of the present invention, when a particular three-dimensional picture is displayed, the range of offsets is adjusted relative to the observer's position such that for each pixel, the offset precisely cancels the undesired refraction that the light from each pixel experiences as it passes through the transparent plate. The particular offset for each pixel is obtained by a calculation that takes into account 1) the position of each pixel relative to the transparent plate, 2) the position of the observer's eye relative to the transparent plate, and 3) the refractive properties of the transparent plate (which also includes the refraction at all lenticular elements). In this way, the undesired optical effects of the transparent plate are compensated for (i.e., canceled out) for each pixel or group of pixels.

[0032] This is shown in Figure 1, which is a cross-sectional view of a display (1) that can be applied in the method of the present invention, comprising a lenticular lens (2) backed by a transparent cover layer (3). The other side of the lenticular lens (2) faces towards an array of pixels (4). This array is perpendicular to the plane of the cross section of Figure 1. Figure 1 shows light emitted by a first pixel (4a) and a second pixel (4b), where the first pixel (4a) emits at a first viewing position (5a) and the second pixel (4b) emits at a second viewing position (5b). The light travels in the plane of the cross section of Figure 1. Due to the optical properties of the display (1), the light perceived to be emitted by the third pixel (4c) is actually emitted by the first pixel (4a) for the viewing position (5a) and the second pixel (4b) for the viewing position (5b). Thus, the apparent origin of the perceived light is offset from the actual origin of the light. In FIG. 1, the pixels and viewing positions are selected such that the light of both pixels (4a, 4b) appears to the viewer to originate from the position of a third pixel (4c).

[0033] FIG. 2 is an enlarged view of the display of FIG. 1, showing the surroundings of the positions from which light is emitted. The difference between the first pixel (4a) and the third pixel (4c) is the first offset (6a), and the difference between the second pixel (4b) and the third pixel (4c) is the second offset (6b). In FIG. 2 it can be seen that each of the two viewing positions experiences a different offset with respect to the perceived origin of the light (i.e. position A). This demonstrates not only that the apparent origin of the perceived light is different (i.e. there is an offset) from the actual origin of the light, but also that the offset depends on the viewing angle relative to the display. This has the consequence that the image is displayed in a distorted manner, with some positions of the image being more offset than other positions of the image.

[0034] As mentioned above, an eye tracking system is used. This system determines the position of the observer's eyes relative to the autostereoscopic display device (and therefore relative to the transparent plate). However, it is not necessary to use the positions of both eyes to calculate the correction. It is also possible to approximate the position of the observer's eyes by using a single value for the positions of both eyes, typically a central position between the eyes. The distortion experienced by the left eye differs only slightly from the distortion experienced by the right eye, which justifies the approximation of taking a position intermediate between the eyes.

[0035] The key to the method of the present invention is to cancel the perceived distortion and displacement caused by the transparent plate when a three-dimensional image is displayed by the display device. The cancellation is performed by taking into account the refractive properties of the transparent plate, the position of each pixel relative to the transparent plate, and the position of the observer's eyes relative to the autostereoscopic display device.

[0036] There are different ways to implement this method. The first method relies on modifying the 3D image data before 3D weaving is applied. The second method relies on pixel selection during the 3D weaving process.

[0037] The first method intervenes at a very early stage of the image generation process. Here, the three-dimensional image data is modified to generate a distorted and displaced image in such a way that it is cancelled by the actual distortion and / or displacement caused by the display. As a result, the displayed image corresponds exactly to the image contained in the original (i.e. unmodified) image data. More specifically, in such an embodiment, the three-dimensional image data (unmodified or original image data) is first modified into modified three-dimensional image data, taking into account the refractive properties of the transparent plate, the position of each pixel relative to the transparent plate, and the viewing position of the observer's eyes, so that the weaving is performed using the modified three-dimensional image data and the observer may perceive a view of the three-dimensional image with proportions and positions corresponding to the intended one as contained in the unmodified three-dimensional image data. Figure 3 shows a schematic diagram of this method according to the invention.

[0038] The second method intervenes somewhat later in the process of image generation, namely in the selection of pixels during 3D weaving. A pixel output at a certain location (defined by image data) is shifted to a pixel at a different location. This misalignment cancels the offset caused by the transparent plate (for example, the offset as shown in FIG. 2). More specifically, in such an embodiment, a correction is made to the selection of pixels during weaving, which includes attributing a specific pixel output of a selected pixel at a first location in the array to a different pixel at a second location in the array, taking into account the refractive properties of the transparent plate, the position of each pixel relative to the transparent plate, and the viewing position of the observer's eyes, so that the observer perceives a specific pixel output coming from a first location in the array, while in reality it comes from a second location in the array, thereby allowing the observer to perceive a view of a three-dimensional image with a proportion and position corresponding to the intended one as contained in the (uncorrected) three-dimensional image data. FIG. 4 shows this method according to the invention in a schematic manner.

[0039] The cancellation of the perceived distortion and displacement caused by the transparent plate may be calculated by using a refraction model for the display of the autostereoscopic display device. In such a case, the cancellation is - defining a refraction model for the display, the refraction model comprising at least one variable for the position of each pixel relative to a transparent plate, at least one variable for the viewing position of an observer's eyes relative to the autostereoscopic display device, at least one variable for characterizing the transparent plate and parameters related to these variables; - determining at least one variable for the position of each pixel relative to a transparent plate and at least one variable for characterizing the transparent plate; - using the model to cancel the perceived distortion and / or displacement of the three-dimensional image, the step including supplying the model with position data obtained using an eye tracking system (i.e. the viewing position of the observer's eyes relative to the autostereoscopic display device).

[0040] Here, it is assumed that the position of each pixel relative to the transparent plate is already included in the refraction model and does not need to be fed into the model.

[0041] In particular, when the method of the present invention is implemented by modifying the three-dimensional image data before the 3D weaving is applied, the use of the refraction model: - defining a refraction model for the display, the refraction model comprising at least one variable for the position of each pixel relative to the transparent plate, at least one variable for the viewing position of the observer's eyes relative to the autostereoscopic display device, at least one variable for characterizing the transparent plate and parameters related to these variables; - determining at least one variable for the position of each pixel relative to a transparent plate and at least one variable for characterizing the transparent plate; - using a refraction model to correct the three-dimensional image data to generate corrected three-dimensional image data, the step including supplying to the model the observation positions of the observer's eyes obtained using an eye tracking system.

[0042] Here, it is assumed that the position of each pixel relative to the transparent plate is already included in the refraction model and does not need to be fed into the model.

[0043] In particular, when the method of the present invention is implemented by modifying the selection of pixels during weaving, the use of the refraction model: - defining a refraction model for the display, the refraction model comprising at least one variable for the position of each pixel relative to a transparent plate, at least one variable for the viewing position of an observer's eyes relative to the autostereoscopic display device, at least one variable for characterizing the transparent plate and parameters related to these variables; - determining at least one variable for the position of each pixel relative to a transparent plate and at least one variable for characterizing the transparent plate; - applying corrections to the selection of pixels using a refraction model, the step including feeding the model with the viewing positions of the observer's eyes obtained using an eye tracking system.

[0044] Here, it is assumed that the position of each pixel relative to the transparent plate is already included in the refraction model and does not need to be fed into the model.

[0045] A transparent plate can be most conveniently characterized as a flat homogeneous plate with a homogeneous refractive index and a uniform thickness. For this purpose, an average total thickness and an average refractive index can be assumed. This does not accurately reflect its true identity (e.g., due to the presence of lenticular elements inside, which by definition are responsible for the inhomogeneous refractive index), but usually serves as a good approximation. A more accurate characterization of the transparent plate can take into account the shape of the transparent plate at the interface with air and / or with the array of pixels. It can also take into account the internal composition of the transparent plate (e.g., layers of adhesive, spacer plates facing the pixel array, lenticular elements). Knowing the specifications of the manufactured transparent plate allows the properties of the transparent plate to be calculated. However, transparent plates manufactured by the same process may still exhibit different properties, but within the manufacturing tolerances of the process. This also applies to the manufacture of the entire display or autostereoscopic display device of which the transparent plate forms a part, to ensure properties such as the curvature of the screen and the position of the pixels relative to the transparent plate. In this respect, it should be noted that even a small sloping surface relief at the air interface can affect the direction of light emitted from the display. In order to take into account any concomitant variations between different transparent plates, each transparent plate must be characterized separately. This can be done by performing a calibration of the transparent plate, preferably when it is incorporated into the display or into the final autostereoscopic display device.

[0046] Therefore, in one embodiment, before using the refraction model, the display is calibrated by repeating the following steps for every pixel of the display: - obtaining calibration data, observing the visibility of a particular pixel through a transparent plate in at least two viewing directions, the particular pixel then being assigned a second pixel qualification for each of the at least two viewing directions; determining which pixels are visible in each of at least two viewing directions as if there was no transparent plate between the pixel and the viewing position (and only one medium, such as air, was between them), and then the pixels are assigned a first pixel qualification for each of the at least two viewing directions; determining for each viewing direction an offset of the second pixel from the first pixel; - fitting the calibration data to a refraction model for each display to obtain a parameter related to at least one variable for characterizing the transparent plate; and - storing parameters for each display unit.

[0047] The cancellation of the perceived distortion and displacement is preferably performed separately for each image data point or pixel, more preferably for each color component in each pixel due to the different refractions per color, as this results in the best quality of the displayed image with the least distortion and displacement. However, it is also possible to determine how the distortion and displacement needs to be cancelled for a portion of the image data points or for a portion of the pixels, and to perform an interpolation for the remaining image data points or pixels, respectively. Thus, the cancellation of the distortion and displacement can be first performed for a portion of the image, and then completed for the remaining portion by performing the interpolation.

[0048] When the method of the present invention is implemented by modifying the three-dimensional image data before 3D weaving is applied, the interpolation is performed as follows: the modifications are calculated for a selection of the image data, followed by an interpolation to determine the modifications for the image data that are not part of the selection.

[0049] When the method of the present invention is implemented by modifying a selection of pixels during weaving, the interpolation is performed as follows: the corrections are calculated for the selection of pixels, followed by an interpolation to determine the corrections for the pixels that are not part of the selection.

[0050] An advantage of the present invention is that the observer is provided with a more realistic perception of three-dimensional virtual content. This includes, for example, an improved look-around effect for the observer. Looking around an object implies a change of the viewing position relative to the autostereoscopic display device, which usually causes a distortion and / or displacement of the displayed three-dimensional image. The method of the present invention cancels this distortion and / or displacement.

[0051] Another advantage is that the perceived interaction of the viewer with the displayed three-dimensional virtual content can be made much more accurate than with conventional autostereoscopic display devices, since the perceived location of the content more closely matches the actual location of the surrounded image data, and it is this actual location that is used to display the viewer's interaction with the three-dimensional virtual content.

[0052] The beneficial effects of the present invention To put into context (or to explain),It is necessary to distinguish between two different phenomena that can occur when displaying three-dimensional images by an autostereoscopic display device, both of which can lead to impairments in image perception. These are 1) the occurrence of crosstalk, and 2) the occurrence of a perceived distortion and / or displacement of the displayed three-dimensional image (e.g. virtual objects in the image are not perceived where they are intended). ,Although both can lead to impairments in image perception, they are of a fundamentally different nature.

[0053] Crosstalk is a physical, measurable phenomenon that occurs when an image intended for a particular eye is also seen by the other eye. This effect does not manifest itself as (or result in) any deviation in the perceived size, shape, and position of the displayed elements in the image (i.e., crosstalk is not associated with the occurrence of distortions and / or displacements of the displayed three-dimensional image).

[0054] In contrast to the phenomenon of crosstalk, the phenomenon of perceived distortion and / or displacement of a displayed three-dimensional image is due to the fact that the light of a pixel of the display, which is intended to be seen as originating from the respective pixel location on the display, is perceived as originating from another location due to refraction in the transparent plate. This results in a perceived distortion (warping / displacing) of the intended three-dimensional object represented by the (three-dimensional) stereoscopic image. The occurrence of distortion and / or displacement does not mean that crosstalk is also manifested. Thus, both phenomena occur independently of each other.

[0055] However, their independent occurrence makes it impossible to argue that both phenomena may have (at least in part) a common cause, which is light refraction in the transparent plate. However, this does not mean that canceling the crosstalk will simultaneously cancel the distortion and / or displacement of the displayed three-dimensional image, since the transparent plate itself typically does not participate in solving either of both phenomena (the present invention only corrects, for example, the left and right parts of the stereo input image independently of each other to cancel the distortion / displacement effect of the transparent plate).

[0056] Crosstalk can be addressed by combining pixels from the left and right parts of the stereoscopic image input to largely cancel the optical mixing of the display, a conventional process that may rely on previously performed calibration of the display. When properly performed, conventional crosstalk correction (particularly crosstalk cancellation) makes it possible to view a three-dimensional image in which crosstalk is virtually absent. However, any such conventional method (or more advanced conventional methods) still does not cancel the perceived distortion and / or displacement of the image.

[0057] Thus, in one embodiment, a method of the invention may comprise performing crosstalk correction in respect of a three-dimensional image displayed by an autostereoscopic display device to an observer of the autostereoscopic display device, where the phenomenon of crosstalk is addressed independently from the phenomenon of perceived distortion and / or displacement as performed in accordance with the method of the invention.

[0058] The present invention further comprises: an eye tracking system for determining the position of an observer's eyes relative to an autostereoscopic display device; a display unit configured to display a composite three-dimensional image of a left image observed by an observer's left eye and a right image observed by an observer's right eye, an array of pixels for generating a display output, each pixel capable of generating a pixel output; A transparent plate, - a lenticular lens provided over the array and comprising a parallax barrier or lenticular element, the front side of the transparent plate facing the observer and the back side of the transparent plate facing the array of pixels; - the transparent plate having light-refractive properties that cause an observer to perceive a distortion and / or a displacement of a three-dimensional image displayed by the display device; - means for controlling pixels to display a three-dimensional image from image data to an observer, taking into account the position of the observer's eyes relative to the autostereoscopic display device, The present invention relates to an autostereoscopic display device, characterized in that the means for controlling the pixels include means for cancelling a perceived distortion and / or displacement caused by the transparent plate of a three-dimensional image displayed by the display device, taking into account the refractive properties of the transparent plate, the position of each pixel relative to the transparent plate and the position of the observer's eyes relative to the autostereoscopic display device.

[0059] Preferably, the autostereoscopic display device comprises a memory containing a refraction model for the display, in particular for the transparent plate.

[0060] In one embodiment, an autostereoscopic display device comprises a processor and a memory, the memory comprising computer executable code that, when executed by the processor, causes the processor to perform a method for canceling perceived distortion and / or displacement as described above.

[0061] The invention further relates to a computer readable medium comprising transitory or non-transitory data representing instructions for causing a processor system to execute a method for cancelling the perceived distortion and / or displacement as described above.

[0062] The invention further relates to software arranged, when executed on a computer, to carry out the method for cancelling the perceived distortion and / or displacement as described above.

Claims

1. 1. A method of canceling distortion and displacement of a three-dimensional image displayed to an observer by an autostereoscopic display device, said distortion and displacement being perceived by said observer, said autostereoscopic display device comprising: an eye-tracking system for determining the position of the observer's eyes relative to the autostereoscopic display device; a display configured to display a composite three-dimensional image consisting of a left image observed by the left eye of the observer and a right image observed by the right eye of the observer, an array of pixels for generating a display output, each pixel being capable of generating a pixel output; - A transparent plate, - a lenticular lens provided above said array and comprising a parallax barrier or lenticular element, the front side of said transparent plate facing the viewer and the back side of said transparent plate facing said array of pixels, the transparent plate having light-refractive properties that cause the observer to perceive distortions and / or displacements of a three-dimensional image displayed by the display device, and the method comprising: - determining the viewing positions of the observer's eyes relative to the autostereoscopic display device using the eye tracking system; - providing three-dimensional image data; - weaving the three-dimensional image data into the array of pixels, the weaving comprising selecting pixels which generate pixel outputs for the left image and pixels which generate pixel outputs for the right image taking into account the viewing positions of the observer's eyes relative to the autostereoscopic display device; - controlling the selected pixels to display the three-dimensional image to the observer, The method is characterized in that it cancels the perceived distortion and / or displacement caused by the transparent plate of the three-dimensional image displayed by the display device, taking into account the refractive properties of the transparent plate, the position of each pixel relative to the transparent plate, and the viewing position of the observer's eyes relative to the autostereoscopic display device.

2. 2. The method of claim 1, wherein the three-dimensional image data is first modified into modified three-dimensional image data taking into account the refractive properties of the transparent plate, the position of each pixel relative to the transparent plate, and the viewing position of the observer's eyes, so that the weaving is performed using the modified three-dimensional image data and the observer can perceive the three-dimensional image having proportions and positions corresponding to those contained in the unmodified three-dimensional image data.

3. 2. The method of claim 1, wherein a modification is made to the selection of pixels during the weaving, the modification including attributing a particular pixel output of a selected pixel at a first position in the array to a different pixel at a second position in the array, taking into account the refractive properties of the transparent plate, the position of each pixel relative to the transparent plate, and the viewing position of the observer's eyes, so that the observer perceives the particular pixel output as coming from the first position in the array, while in fact it comes from the second position in the array, thereby allowing the observer to perceive the three-dimensional image having proportions and positions corresponding to those contained in the three-dimensional image data.

4. 3. The method of claim 2, wherein the modification of the three-dimensional image data is performed by calculating the modification for a selection of the image data and then performing interpolation to determine the modification for the image data that is not part of the selection.

5. 4. The method of claim 3, wherein interpolation is performed to determine the corrections to a selection of pixels by calculating the corrections to a selection of pixels for a group of pixels and then performing interpolation to determine the corrections for pixels that are not part of the selection.

6. Cancelling the perceived distortion and / or displacement of the three-dimensional image may include: - defining a refraction model for the display, the refraction model having at least one variable for the position of each pixel relative to the transparent plate, at least one variable for the viewing position of the observer's eyes relative to an autostereoscopic display device, at least one variable for characterizing the transparent plate and parameters related to these variables; and - determining said at least one variable for the position of each pixel relative to said transparent plate and said at least one variable for characterizing said transparent plate, and then - using the refraction model to cancel the perceived distortion and / or displacement of the three-dimensional image, the step comprising feeding the model with the observation positions of the observer's eyes obtained using the eye tracking system.

7. Cancelling the perceived distortion and / or displacement of the three-dimensional image may include: - defining a refraction model for the display, the refraction model having at least one variable for the position of each pixel relative to the transparent plate, at least one variable for the viewing position of the observer's eyes relative to an autostereoscopic display device, at least one variable for characterizing the transparent plate and parameters related to these variables; and - determining said at least one variable for the position of each pixel relative to said transparent plate and said at least one variable for characterizing said transparent plate, and then - using the refraction model to correct the three-dimensional image data to generate corrected three-dimensional image data, the step comprising supplying the model with observation positions of the observer's eyes obtained using the eye tracking system.

8. Cancelling the perceived distortion and / or displacement of the three-dimensional image may include: - defining a refraction model for the display, the refraction model having at least one variable for the position of each pixel relative to the transparent plate, at least one variable for the viewing position of the observer's eyes relative to an autostereoscopic display device, at least one variable for characterizing the transparent plate and parameters related to these variables; and - determining said at least one variable for the position of each pixel relative to said transparent plate and said at least one variable for characterizing said transparent plate, and then - performing said corrections to said selection of pixels using said refraction model, said step comprising feeding said model with observation positions of the observer's eyes obtained using said eye tracking system.

9. 7. The method of claim 6, wherein, before using the refraction model, the display is calibrated by repeating the following steps for every pixel of the display: - obtaining calibration data, - observing the visibility of a particular pixel through the transparent plate in at least two viewing directions, the particular pixel then being assigned a second pixel qualification for each of the at least two viewing directions; - determining which pixels are visible in each of the at least two viewing directions as if no transparent plate were present between said pixel and said viewing position, said pixels then being assigned a first pixel qualification for each of the at least two viewing directions; - determining the offset of the second pixel from the first pixel for each viewing direction; - fitting the calibration data to the refraction model for each of the displays to obtain the parameters related to the at least one variable for characterizing the transparent plate; and - storing said parameters for said respective display.

10. The method of claim 1 , wherein the transparent plate is a plate having a uniform thickness or is approximated by a plate having a uniform thickness when considering the refractive properties of the transparent plate.

11. 10. The method of claim 1, wherein the transparent plate comprises a lenticular lens backed by a transparent cover layer forming a front surface of the transparent plate, the lenticular lens comprising a transparent spacer plate forming a back surface of the transparent plate.

12. The method of claim 1 , wherein the viewing position of the observer's eyes is approximated by using a single value for both eye positions, typically a position centered between the eyes.

13. The method of claim 1 , wherein the method includes performing crosstalk correction.

14. 1. An autostereoscopic display device, comprising: an eye-tracking system for determining the position of the observer's eyes relative to said autostereoscopic display device; a display configured to display a composite three-dimensional image composed of a left image observed by the left eye of the observer and a right image observed by the right eye of the observer, an array of pixels for generating a display output, each pixel being capable of generating a pixel output; - A transparent plate, - a lenticular lens provided above said array and comprising a parallax barrier or lenticular element, the front side of said transparent plate facing the viewer and the back side of said transparent plate facing said array of pixels, - the transparent plate having light-refractive properties that cause a viewer to perceive distortions and / or displacements of the three-dimensional image displayed by the display device; means for controlling said pixels to display said three-dimensional image from three-dimensional image data to said observer, taking into account the viewing positions of said observer's eyes relative to said autostereoscopic display device, 10. An autostereoscopic display device, characterized in that the means for controlling the pixels includes means for canceling the perceived distortion and / or displacement caused by the transparent plate of the three-dimensional image displayed by the display device, taking into account the refractive properties of the transparent plate, the position of each pixel relative to the transparent plate, and the viewing position of the observer's eyes relative to the autostereoscopic display device.

15. 15. An autostereoscopic display device as claimed in claim 14, comprising a memory containing a refraction model for the display.

16. 16. An autostereoscopic display device according to claim 14 or 15, comprising a processor and a memory, said memory comprising computer executable code which, when executed by said processor, causes said processor to perform the method of claim 1.

17. 10. A computer-readable medium containing transitory or non-transitory data representing instructions that cause a processor system to perform the method of claim 1.

18. Software configured to perform the method of claim 1 when executed on a computer.

Images