Method and apparatus for remapping pixel positions

Abstract

Apparatus and methods for optically remapping projection pixels to maximize utilization and optimize the distribution of remapped projection pixels to achieve optimal visual performance (typically uniform resolution and brightness). A device placed between a projector and an imaging surface for optically remapping projection pixel positions with minimal aberrations. When the device is placed between the projector and the imaging surface, it changes the terminal position of each focused pixel so that it maximally coincides with the imaging surface, which is typically a surface of complex curvature and very different from the natural focusing surface of the projector. One embodiment of the technology includes an apparatus using multiple optical surfaces.

Description

[0001] Cross-references to related applications

[0002] This application claims the benefit of and priority to U.S. Provisional Patent Application Serial No. 62 / 769,368, filed on November 19, 2018, entitled DEVICE FOR OPTICALLY REMAPPING PROJECTED PIXEL LOCATIONS WITH MINIMAL ABERRATION, the entire contents of which are incorporated herein. Technical Field

[0003] The technology disclosed herein generally relates to optical remapping of projected pixel positions, and more particularly, to remapping with minimal aberrations. Background Art

[0004] The cost and performance efficiency of projector-based visual display systems depends primarily on the utilization of available projector pixels. Commercial projectors are designed to provide a rectangular projection frustum with square pixels and negligible field curvature. Visual display systems are typically designed to achieve a desired field of view (FOV), resolution, and brightness. Since the most efficient pixel distribution for human observation is that of the visual sphere, and rectangles cannot be efficiently tessellated onto a sphere, these visual requirements typically result in poor utilization of a rectangular, square pixel, projection frustum.

[0005] There are some special lens systems, namely fisheye ("f-θ") and anamorphic lenses, which allow some adjustment of the rectangular projection frustum, but they are not sufficient. Fisheye lenses have a limited tolerance for the position of the origin of the projected light, which forces the projector to be located within the viewing area or other physical obstacles. In addition, the tolerance of fisheye lenses for adjusting the local distribution of pixels is also low. Afocal anamorphic cylindrical lens systems are arranged on the side of the longer distance between the rear lens and the image in the projection path of the light rays of the spherical projection and the camera objective and are used to obtain a sharp image of all image points. An anamorphic cylindrical lens system is already known, which consists of a lens component with positive cylindrical refractive power located in front of the objective lens and another lens component, which is separated from the first by air and has negative cylindrical refractive power, the cylindrical axes of these components are parallel to each other, each of the two components is formed by a converging lens, which is bonded to a diverging lens, the cylindrical bonding surface in the diverging component having a converging effect, its concave surface facing the converging component, and the bonding surface of the converging component having a diverging effect, its converging surface facing the diverging component. The axes of the two components lie in the vertical plane, allowing the system to reduce the focal length of the objective in the horizontal plane while keeping the focal length unchanged in the vertical plane, i.e., to achieve a change in the image ratio in the horizontal plane while keeping the image ratio unchanged in the vertical plane. However, anamorphic lenses are also relatively intolerant of adjustments in the local distribution of pixels. Commercial anamorphic lenses are also limited to small changes in aspect ratio.

[0006] Another objective known hitherto consists of at least five lenses, the diverging part of which consists of three lenses. Despite the number of lenses and the arrangement of these known systems, the projection quality is not satisfactory, in particular with regard to coma and distortion.

[0007] What is needed is a better apparatus and / or method to improve optical remapping of projected pixel locations, and more particularly, to remap with minimal aberrations. Summary of the invention

[0008] The technology disclosed herein includes a method and apparatus for optically remapping projection pixels to maximize utilization and optimize the distribution of remapped projection pixels to achieve optimal visual performance (roughly uniform resolution and brightness). For example, fisheye lenses are used to try to meet this need, but fisheye lenses have a small tolerance for adjusting the local distribution of pixels, while the slightly flexible mirror elements (e.g., thin glass) proposed herein for the present technology and its various embodiments and implementations allow such adjustment without the need to remanufacture optical elements.

[0009] One embodiment of the technology is a device between a projector and an imaging surface for optically remapping projected pixel positions with minimal aberrations. When the device is between the projector and the imaging surface, it changes the terminal position of each focused pixel so that it coincides maximally with the imaging surface, which is typically a surface of complex curvature and very different from the natural focusing surface of the projector. One embodiment of the technology includes a device that uses multiple optical surfaces. However, one embodiment of the technology includes a single optical surface that is effective when the aberration (focus blur) caused by the remapping is less than the resolution required by the visual display system. Another embodiment of the technology includes a device that uses a combination of reflective and refractive optical surfaces, however, the visual display system performance is improved by a purely reflective or refractive group of optical surfaces. For one embodiment of the present invention, one optical element is used to bend light and another is used to correct focus.

[0010] One embodiment of the technology includes a projector that provides a frustum of projected light incident on a mostly cylindrical refractive lens. The center of curvature of the lens is approximately coincident with the optical origin of the projector. The element is capable of correcting anisotropic focus, which is a product of the degree of angular deviation affected by downstream elements in order to achieve remapping. This is because the principal ray (central ray) from the optical origin is largely perpendicular to the refractive optical surface and therefore experiences minimal deviation, but the marginal rays (edges of the defocused focus from the optical origin) experience significant deviation, which changes the depth of focus (but not the position) of a given pixel. For one embodiment of the technology disclosed and claimed herein, the lens uses multiple elements to reduce dispersion (e.g., achromatic doublets). The light leaving the above-mentioned lens is then incident on one or more reflective optical surfaces, which remap the projected pixel position through non-planar surface geometry.

[0011] For one embodiment of the technology disclosed and claimed herein, a single reflective surface is used, and the reflective surface is provided by a thin glass substrate coated with a first surface (for example, for one embodiment with a thickness of about 200 microns). The thin mirror is forced to become a nearly cylindrical hyperbolic parabola. When the pixel is finally incident on the imaging surface, the thin glass mirror provides significant anisotropic magnification, resulting in an elongated aspect ratio. It also produces significant field curvature on the same axis. In "cross-cockpit collimation" visual display systems and "dome" visual display systems, spherical, annular or other secondary imaging surfaces are usually used, which are much wider than their height. The aforementioned thin glass mirror adjusts the pixel position and the focusing surface to more optimally converge on the imaging surface. This enables a single higher resolution projector to replace two or more lower resolution projectors. The thin mirror material can be made of any material that can be polished to the desired optical smoothness and / or a reflective coating applied thereto. The mirror has a reflective coating or a reflective portion, and has another portion that holds the structure itself (substrate). For one embodiment, the substrate is made of one or more of glass, plastic, metal, metal composite material, acrylic, ceramic and carbon fiber. One embodiment of the reflective coating includes one or more of aluminum, silver or similar materials on glass. The reflective coating can be applied to plastic film, hard plastic or acrylic. Various manufacturing processes for substrates and reflective coatings can be used without departing from the scope of the present invention.

[0012] Through mechanical pressure, the surface geometry of the thin glass mirror can be adjusted to achieve a free-form optical shape that more optimally distributes the pixels required by each visual display system. This enables adaptation to deviations in screen surface geometry from nominal, or deviations in the mapping of FOV angles onto the imaging surface. For one embodiment of the technology, a proprietary software analysis package (MATLAB code and GUI) is used, which enables empirical (human in the loop) determination of free-form and aspheric geometries to optimally remap pixels through mechanical adjustment of the mirror.

[0013] Visual display systems for aviation simulation training for regulatory certification use multiple projectors with overlapping images to provide certifiable visual performance. The technology disclosed and claimed herein provides the ability to replace these projectors with a single projector. This greatly reduces the cost and complexity of maintaining the "alignment" (color, brightness, gamma and geometric performance) of adjacent channels, and eliminates the need for expensive, error-prone adjacent channel "mixing". The low utilization of projector pixels brings millions of dollars of costs to the audio-visual (A / V) and visual simulation (VizSim) markets every year. If pixels can be optically remapped, fewer projectors are needed, and fewer multi-projector alignment systems are needed. Brightness, dynamic range and system MTBF are all increased.

[0014] For one embodiment of the technology, a visual system design tool is utilized, which can then be modified to simulate the lens behavior of the proposed system. For the technology disclosed and claimed herein, the technology disclosed herein provides a higher level of control over pixel position through mirror shape, and produces curves and local pixel redistribution, rather than just angular uniform stretching. In particular, visual display system applications often require a combination of deformation and local pixel redistribution. If a fisheye lens is used, a large number of pixels will be lost. For one embodiment of the technology, the solution can be adjusted for various imaging surface geometries. As the degree of remapping increases, focus aberrations quickly become apparent. The technology disclosed and claimed herein seeks to solve the focus problem. One embodiment of the technology disclosed and claimed is a device for remapping projector pixels with substantially maximum pixel utilization and substantially optimal resolution for one or more observers of the display system. The device is interposed in the optical path between image formation in the projector and subsequent image formation on the curved screen. The device includes at least a refractive element and a reflective element located in an optical sequence. The focusing surfaces of the refractive and reflective elements each have a longest dimension, and the focusing surface longest dimensions are substantially orthogonal to each other. For one embodiment, the device first uses refraction to substantially astigmatize the focus of the projected image to a degree that is substantially opposite to the focusing astigmatism introduced by the optical subsequent elements. For one embodiment, the device then uses reflection to supplement optically redirect (also known as remap pixels) the light from the projector to an optimal position on the curved screen. The device is adjustable in its optical effects to support any of the following: various projectors, various projector configurations, various display systems, various display system configurations, changes in display system components, changes in observer position, and changes in required field of view and / or resolution. Fewer projectors are required, and fewer multi-projector alignment systems are required. Brightness, dynamic range, and system MTBF are all increased. The device can be adjustable if necessary. The thin mirror configuration with thin glass minimizes chromatic aberration and enables large magnification / large remapping. The features, functions, and advantages that have been discussed can be implemented independently in various embodiments, or can be combined in other embodiments, further details of which can be seen with reference to the following description and accompanying drawings.

[0015] The technology disclosed and claimed herein is particularly suitable for applications with larger remapping at higher pixel densities. Higher pixel density requires larger remapping, which necessitates newer solutions and is one of the reasons for the technology described. In addition, the technology provides corrected resolution from the DEP. The prior art does not remap high pixel density on a large scale to a remapping position that is optimal for a DEP with uniform resolution across the field of view. In addition, the technology is adjustable to a free form that the prior art does not teach. As an illustration, for a collimation system, one embodiment of the technology provides eye points for at least two observers, so the technology disclosed and claimed optimizes the system to achieve the best balance between multiple observers, so when the term "design eye position" is used herein, for one embodiment, the DEP is a balance between the eye positions of multiple observers. In this case, the DEP represents the positions of many observers, so when the term observer is used herein, it can refer to multiple observers. The adjustment parameters of the system can be adjusted relative to one or more observer positions. These and other advantageous features of the disclosed technology will be partially obvious and partially pointed out below. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to better understand the disclosed technology, reference may be made to the accompanying drawings, in which:

[0017] Figure 1A and 1B is a diagram of the imaging surface of the simulator;

[0018] Figure 1C and 1D is a diagram of a thin mirror between the projector and the imaging surface and the designed eye point;

[0019] Figure 2 is a graphical representation of the mapping of pixels to the imaging surface;

[0020] Figures 3A to 3C is a diagram of the application of a "cross-cockpit collimation" visual display system or a "dome" visual display system;

[0021] Figure 4 is an illustration of a set of observation vectors generated from the DEP to represent the projection of the field of view; and

[0022] Figure 5 It is a free-form graphic illustration.

[0023] Although the disclosed technology is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will be described in detail herein. However, it should be understood that the drawings and detailed description presented herein are not intended to limit the disclosure to the specific embodiments disclosed, but rather, the present invention is intended to cover all modifications, equivalents and alternatives within the scope of the technology disclosed and defined by the appended claims. DETAILED DESCRIPTION

[0024] According to the disclosed embodiments of the present technology, various views are shown in Figures 1-5, and the same reference numerals are always used to refer to the same and corresponding parts of the technology in the various views and figures of all drawings. In addition, please note that the first digit of the reference number of a given item or part of the technology should correspond to the figure number in which the item or part is first identified. References in the specification to "one embodiment" or "an embodiment"; "an implementation" or "an implementation" mean that the specific features, structures, or characteristics described in conjunction with the embodiment or implementation are included in at least one embodiment or implementation of the present invention. The phrases "in one embodiment" or "in one implementation" appearing in different places in the specification do not necessarily refer to the same embodiment or the same implementation, nor are they necessarily separate or alternative embodiments or implementations that are mutually exclusive with other embodiments or implementations.

[0025] The technology disclosed and claimed herein is a method and apparatus for optically remapping projection pixels to maximize utilization and optimize the distribution of remapped projection pixels to achieve optimal visual performance (approximately uniform resolution and brightness). One embodiment of the technology is a device between a projector and an imaging surface for optically remapping projection pixel positions with minimal aberrations. When the device is between the projector and the imaging surface, it changes the terminal position of each focused pixel so that it coincides with the imaging surface to the greatest extent, which is typically a surface of complex curvature and is very different from the natural focusing surface of the projector. One embodiment of the technology includes a device using multiple optical surfaces. However, one embodiment of the technology includes a single optical surface that is effective when the aberration (focus blur) caused by the remapping is less than the resolution required by the visual display system. Another embodiment of the technology includes a device using a combination of reflective and refractive optical surfaces, however, the visual display system performance is improved by a set of optical surfaces that are purely reflective or refractive. The technology disclosed and claimed herein is particularly suitable for applications with larger remapping with higher pixel density. Higher pixel density requires larger remapping, which makes newer solutions necessary and is one of the reasons for the described technology. In addition, the technology provides corrected resolution from DEP. The prior art does not massively remap high pixel density to remapped positions that are optimal for DEP with uniform resolution across the field of view. In addition, the present technology is adjustable to freeforms that the prior art does not teach.

[0026] refer to Figure 1A and 1B , provides an illustration of the imaging surface of the simulator. Figure 1A shows a surface onto which an image is projected, and Figure 1B An image projected onto surface 101 is shown. The imaging surface is shown as a surface of complex curvature and is very different from the natural focusing surface of the projector, which teaches the apparatus and method disclosed and claimed herein for optimizing the distribution of remapped projection pixels to achieve optimal visual performance (substantially uniform resolution and brightness). Figure 1B The illustrated visual display system, such as a flight simulator, is one application of the technology disclosed and claimed herein.

[0027] refer to Figure 1C and 1D , shows a diagram of a thin mirror between the projector and the imaging surface and the designed eye point. One embodiment of providing a thin film polyester film mirror is in Figure 1D , where a thin film polyester film mirror is formed on a substrate to set a shape, and a tension frame is used to form the shape of the substrate to the polyester film.

[0028] refer to Figure 2, shows a diagram of the mapping of pixels to the imaging surface. The on-screen field of view as seen from the designed eye point and the pixel mapping is shown. Figures 3A-3C , shows the application in a "cross cockpit collimation" visual display system or "dome" visual display system. Light leaves the projector, passes through a refractive element, then reflects from a reflective element before being incident on a screen.

[0029] refer to Figure 4 , shows an illustration of a set of observation vectors generated from the DEP to represent the projection of the field of view. Figure 5 , provides an illustration of a freeform. This type of mirror will be referred to herein as a "freeform" folding mirror. The freeform folding mirror can be shaped to reduce or eliminate the resolution loss of image components near the boundaries of the projected light cone. A freeform optical surface is defined as any non-rotationally symmetric surface or a symmetric surface rotated around any axis of asymmetry. These surfaces can result in a smaller system size compared to rotationally symmetric surfaces. When illuminated by a point light source, the freeform mirror produces a given illumination pattern on a target surface, which can be flat, spherical, or other shapes. For one method of completing the design, the light mapping can be modeled by a second-order partial differential equation. For another method of implementing the design, an approximation of the optical surface can be modeled and verified by ray tracing and the design of the optical surface, and in particular, the freeform mirror can be determined. In addition to the first curved portion, the freeform mirror can also include a second curved portion separated from the first curved portion. In this arrangement, the first flat portion is located between the first curved portion and the second curved portion. Additional curved portions can be added as needed or necessary.

[0030] One goal of freeform mirror technology is to equalize the size and spacing of projector pixels to create uniform resolution so that the appearance of the image is always clear. Any geometric correction, such as pre-deforming a square into a "barrel" shape so that it appears square to the observer rather than making the corners appear elongated, will be done by the image generator that creates the image, not by the mirror. The prior art does not address uniform resolution. Another goal of the technology is to provide a folding mirror that produces uniform pixel density (and therefore uniform resolution and more uniform brightness) on any screen surface. The freeform folding mirror 30 described herein will adjust the distribution of the projector's light on the screen 16 to equalize the resolution on the image and produce more uniform resolution and brightness. For example, for one embodiment of designing, manufacturing and providing a freeform mirror, a ray tracing optimizer is utilized, which includes a computer-based software tool for modeling ray tracing and the freeform shape of the final folding mirror. The freeform mirror can be pre-manufactured using this computer-based tool. However, for one embodiment, the freeform mirror is dynamically adjusted with a mechanical push / pull system that mechanically deforms the reflective surface of the mirror to the appropriate curvature. For one embodiment, the push / pull mechanism is computer controlled to adjust the curvature of the mirror based on other system parameters to reduce aberrations and improve resolution.

[0031] The details and various embodiments of the disclosed technology may be better understood by referring to the accompanying drawings. Referring to FIGS. 1-5 , one embodiment of the disclosed and claimed technology includes a device 102 interposed between a projector 104 and an imaging surface 106 for optically remapping projection pixel positions with minimal aberrations, whereby when the device is interposed between the projector and the imaging surface, it changes the terminal position of each focused pixel (see FIG. 1 ). Figure 2 ) so that it maximally coincides with the imaging surface, which is typically a surface of complex curvature and very different from the natural focusing surface of the projector, which teaches an apparatus and method as disclosed and claimed herein for optimizing the distribution of remapped projection pixels to achieve optimal visual performance (approximately uniform resolution and brightness).

[0032] One embodiment of the technology disclosed and claimed herein is a method that includes determining a display system observer position, also referred to as a design eye point 108, as well as a desired field of view 110 and a desired resolution for each portion of the field of view. For one embodiment, the method includes generating a set of observation vectors 402 generated from the DEP 108 to represent the projection of the field of view 110 on an image forming surface. The observation vectors 402 intersect a curved screen that forms an image. For each portion of the field of view, the screen intersection position determines the location to which the projector pixels will be substantially mapped. The resolution required for a given portion of the field of view determines the optimal pixel density within that portion of the field of view.

[0033] One embodiment of the method includes inserting a device 102 in an optical path 404 between image formation in a projector and subsequent image formation 101 on a curved screen. For one embodiment, the device includes at least a refractive element and a reflective element, which are positioned in an optically consecutive order. The focusing surfaces of the refractive and reflective elements each have a longest dimension, and the focusing surface longest dimensions are substantially orthogonal to each other. For one embodiment, the method includes adjusting the optical effect of the adjustable device to support any of the following: various projectors, various projector configurations, various display systems, various display system configurations, changes in display system components, changes in observer position, changes in required field of view and / or resolution. One embodiment of the method substantially astigmatizes the focus of the projected image by refraction, and then uses reflection to supplement optically redirect light from the projector to the screen intersection position having the optimal pixel density. The method includes optimizing the device's utilization of projector pixels by using the fewest projector pixels to meet the resolution required for a given field of view.

[0034] One embodiment of the technology disclosed and claimed herein includes an apparatus for remapping projector pixels at maximum pixel utilization and optimal resolution for one or more observers of a display system (see the illustration of item 202). The apparatus is interposed in an optical path between image formation in the projector and subsequent image formation on a curved screen. For one embodiment of the technology, the apparatus includes at least a refractive element and a reflective element, which are positioned in an optical sequence. The focusing surfaces of the refractive and reflective elements each have a longest dimension, and the focusing surface longest dimensions are oriented substantially orthogonal to each other. The apparatus is configured to use refraction to substantially astigmatize the projected image focus to a degree that is substantially opposite to the focusing astigmatism introduced by the optical subsequent elements. For one embodiment, the apparatus is also configured to use reflection to supplement optically redirect light from the projector to an optimal location on the curved screen. For one embodiment, the apparatus is configured such that its optical effect is adjustable to support any of the following: various projectors, various projector configurations, various display systems, various display system configurations, changes in display system components, changes in observer position, changes in desired field of view and / or resolution.

[0035] For another embodiment, as an application, when used as a "cross cockpit collimated" visual display system or a "dome" visual display system, the curved screen is viewed through a collimator (see the illustration of item 302), whereby the required total horizontal field of view exceeds 180 degrees and the resolution is less than 6.1 arc minutes per optical line pair and at least the reflective element is adjustable. For an embodiment, substantially orthogonal is between 70 degrees and 110 degrees.

[0036] For a projection system, an image is formed within a projector and is projected to be formed on a screen. For an embodiment, the projector may include a projection lens. In the case of two or more observers, a compromise design eye position (DEP) is uniquely determined for each portion of the field of view.

[0037] For one application of the technology, the curved screen is viewed through a collimator, in which case the viewing vector is reflected from the collimator and intersects at the intersection with the curved screen. For one embodiment, the curved screen is a portion of an ellipsoid or a torus, or a slight variation thereof. The slight variation does not exceed about 0.25 times the maximum curved screen radius of curvature. For one embodiment, the curved screen includes a radius of curvature that is approximately infinite.

[0038] Pixel density refers to the number of pixels per unit screen area. By pixel shifting, the resolution of a projector may exceed its pixel count. For one embodiment, the refractive element is positioned before, after, or within the projection lens along the optical path. For one embodiment, a flat fold mirror may be used around or between any optical element in the entire optical path from image formation in the projector to image observation at the designed eye position (DEP).

[0039] For one embodiment, the refractive element includes a plurality of sub-elements and a plurality of refractive optical surfaces. For one embodiment, the refractive element has one or more optical surfaces having a shape substantially having a generalized cylinder, wherein the focusing surface is substantially close to or surrounds the vertex position of the projection light frustum of the projector. For one embodiment, the refractive element has one or more optical surfaces having a shape substantially having a free form, wherein the focusing surface is substantially close to or surrounds the vertex position of the projection light frustum of the projector. (Free form as defined in ISO standard 17450-1:2011). For one embodiment, the refractive element has one or more optical surfaces having a Gaussian curvature substantially having about zero, wherein the focusing surface is substantially close to or surrounds the vertex position of the projection light frustum of the projector.

[0040] For one embodiment, the refractive element is adjustable in its optical effect, so that the refractive element can be adjusted to a free-form shape (free-form as defined in ISO standard 17450-1:2011). For one embodiment, the refractive element adjustment is achieved by mechanical and / or thermal deformation of any refractive surface. For one embodiment, the refractive element adjustment is achieved by adjusting the relative position of the sub-elements. The reflective element is positioned after the refractive element and the projection lens along the optical path. For one embodiment, the reflective element can combine multiple sub-elements and reflective optical surfaces. For one embodiment, the reflective element basically has the shape of a generalized cylinder. For one embodiment, the reflective element basically has a free-form shape (free-form as defined in ISO standard 17450-1:2011). The reflective element surface basically has a Gaussian curvature that is approximately zero.

[0041] For one embodiment, the reflective element is adjustable in its optical effect. The reflective element is adjustable to a substantially free form shape (free form as defined in ISO standard 17450-1:2011). The reflective element adjustment is achieved by mechanical and / or thermal deformation of the reflective surface. For one embodiment, the reflective element adjustment is achieved by adjusting its position relative to the projector, and / or its position relative to the refractive element, and / or the relative position of the sub-elements of the reflective element. The refractive element is close enough to the reflective element so that the projection light passes through the refractive element before and after reflection from the reflective element.

[0042] The various embodiments and examples shown above illustrate methods and systems for utilizing a software analysis package (MATLAB code and GUI) that enables empirical (human in the loop) determination of free-form and aspheric geometries to optimally remap pixels through mechanical adjustment of mirrors.

[0043] Depending on the desired application, the user of the present method and system can select any one of the above embodiments or its equivalent. In this regard, it is recognized that various forms of the subject software analysis method and system for determining free forms can be utilized without departing from the scope of the disclosed present technology and various embodiments.

[0044] As will be apparent from the foregoing description, certain aspects of the present embodiment are not limited to the specific details of the examples shown herein, and it is therefore contemplated that other modifications and applications or their equivalents will occur to those skilled in the art. Therefore, the claims will cover all such modifications and applications that do not depart from the scope of the present embodiment. Therefore, the description and drawings are to be regarded as illustrative rather than restrictive.

[0045] Certain systems, devices, applications or processes are described herein as including multiple modules. A module can be a unit of different functions presented in software, hardware or a combination thereof. When the functions of an analysis module are performed in any part by software, the module includes a computer-readable medium. The analysis module can be considered as a communication coupling. The subject matter of the present invention can be represented in a variety of different embodiments, wherein there are many possible arrangements.

[0046] The methods described herein do not have to be performed in the order described or in any particular order. In addition, the various activities described for the methods identified herein can be performed in serial or parallel manner. In the foregoing detailed description, it can be seen that various features are grouped together in a single embodiment in order to simplify the present disclosure. This disclosure method should not be interpreted as reflecting an intention that the claimed embodiments require more features than those expressly stated in each claim. On the contrary, as reflected in the following claims, the subject matter of the invention may lie in less than all the features of a single disclosed embodiment. Therefore, the following claims are hereby incorporated into the detailed description, each claim independently as a separate embodiment.

[0047] In example embodiments, the machine operates as a standalone device, or can be connected (e.g., networked) to other machines. In a networked deployment, the machine can run as a server or client machine in a server-client network environment, or as a peer machine in a peer (or distributed) network environment. The machine can be a server computer, a client computer, a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a cellular phone, a network device, a network router, a switch or a bridge, or any machine capable of executing a set of instructions (sequential or otherwise) specifying the actions to be taken by the machine or computing device. In addition, although only a single machine is shown, the term "machine" should also be understood to include any machine collection that executes a set (or multiple sets) of instructions individually or together to perform any one or more methods discussed herein.

[0048] The example computer system and client computer may include a processor (e.g., a central processing unit (CPU), a graphics processing unit (GPU), or both), a main memory, and a static memory, which communicate with each other via a bus. The computer system may also include a video / graphics display unit (e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)). The computer system and client computing device may also include an alphanumeric input device (e.g., a keyboard), a cursor control device (e.g., a mouse), a drive unit, a signal generating device (e.g., a speaker), and a network interface device.

[0049] The drive unit includes a computer-readable medium having one or more sets of instructions (e.g., software) stored thereon, which embody any one or more methods or systems described herein. The software may also reside completely or at least partially in a main memory and / or a processor during execution by a computer system, the main memory and the processor also constituting a computer-readable medium. The software may also be transmitted or received over a network via a network interface device.

[0050] The term "computer-readable medium" should be understood to include a single medium or multiple media (such as centralized or distributed databases, and / or related caches and servers) that store one or more sets of instructions. The term "computer-readable medium" should also be understood to include any medium that can store or encode a set of instructions executed by a machine and cause the machine to perform any one or more methods of the present embodiment. Therefore, the term "computer-readable medium" should be understood to include, but is not limited to, solid-state memories, optical media, and magnetic media.

[0051] The various optical remapping configurations and embodiments shown above illustrate remapping of projected pixel locations with minimal aberrations. Depending on the desired application, users of the disclosed technology may select any of the above embodiments or their equivalents. In this regard, it is recognized that various forms of the subject optical remapping methods and apparatuses may be utilized without departing from the scope of the present invention.

[0052] It is apparent from the foregoing description that certain aspects of the disclosed technology are not limited to the specific details of the examples shown herein, and it is therefore conceivable that other modifications and applications or their equivalents will occur to those skilled in the art. Therefore, the claims will cover all such modifications and applications that do not depart from the scope of the disclosed and claimed technology.

[0053] Other aspects, objects, and advantages of the disclosed technology can be obtained from a study of the drawings, the disclosure, and the appended claims.

Claims

1. A method for optically remapping a projected pixel position, comprising: defining a designed eye point (DEP), a field of view corresponding to the DEP, and image resolutions for multiple portions of the field of view when viewing a projected image; generating a set of observation vectors from the DEP representing the projection of the field of view onto the image forming surface, whereby the observation vectors intersect the curved screen on which the projected image is to be formed; determining, for each of a plurality of portions of the field of view, a screen intersection location to which a projector pixel will be substantially mapped; determining an optimal pixel density within each of a plurality of portions of the field of view based on an image resolution for a given portion of the field of view; inserting an optical device in an optical path between an image source forming point in the projector and a projected image forming point on the curved screen, wherein the optical device comprises at least a refractive element and a reflective element located in an optical subsequence, wherein the focusing surfaces of the refractive element and the reflective element each have a longest dimension, and the focusing surface longest dimensions are oriented substantially orthogonal to each other; refracting and reflecting the projected image with the optical device; Adjusting the optical effect of the optical device to support one or more of various projectors, various projector configurations, various display systems, various display system configurations, changes in display system components, changes in observer positions, changes in field of view, and changes in image resolution; and The focus of the projected image is substantially astigmatized by refraction, using refraction to substantially astigmatize the focus of the projected image to a degree substantially opposite to the focusing astigmatism introduced by optical subsequent elements; and then using reflection to supplement optically redirect light from the projector to the screen intersection location having the optimal pixel density, whereby the optical device optimizes the utilization of the projector pixels by satisfying the image resolution of a given portion of the field of view with the fewest projector pixels.

2. The method for optically remapping a projected pixel position as claimed in claim 1, comprising: A collimator is inserted between the DEP and the curved screen, whereby an observer at the DEP observes the curved screen through the collimator.

3. The method for optically remapping projection pixel positions of claim 2, wherein: The viewing vector reflects from the collimating mirror before intersecting the curved screen.

4. The method for optically remapping projection pixel positions of claim 1, wherein: The horizontal field of view exceeds 180 degrees, the image resolution is less than 6.1 arc minutes per optical line pair, and one or more of the reflective elements and the refractive elements are adjustable.

5. The method for optically remapping projected pixel positions of claim 1, wherein: Substantially orthogonal means that the angles therebetween are between 70 and 110 degrees.

6. The method for optically remapping projection pixel positions of claim 1, wherein: The projector includes a projection lens through which the projector projects an image.

7. The method for optically remapping projection pixel positions of claim 6, wherein: The refractive element is positioned along the optical path one of before, after, and within the projection lens.

8. The method for optically remapping projected pixel positions of claim 1, further comprising: When there are two or more observers, a compromise DEP is determined for each of the multiple portions of the field of view.

9. The method for optically remapping projected pixel positions of claim 1, wherein: The curved screen is a portion of one of an ellipsoid, a torus, and a slight variation from an ellipsoid or a torus, wherein the slight variation does not exceed 0.25 times the maximum radius of curvature of the curved screen.

10. The method for optically remapping projected pixel positions of claim 1, wherein: The curved screen includes an infinite radius of curvature.

11. The method for optically remapping projected pixel positions of claim 1, wherein: Pixel density is the number of pixels per unit screen area, and wherein the resolution of the projector can exceed the pixel count of the projector by pixel shifting.

12. The method for optically remapping projected pixel positions of claim 1, wherein: One or more planar folding mirrors are used around or between any optical elements in the entire optical path from the image source formation point within the projector to the image observation at the DEP.

13. The method for optically remapping projected pixel positions of claim 1, wherein: The refractive element includes a plurality of sub-elements and a plurality of refractive optical surfaces.

14. The method for optically remapping projection pixel positions of claim 13, wherein: The refractive element includes one or more optical surfaces substantially having the shape of a generalized cylinder, wherein a focusing surface is substantially close to or surrounds the vertex position of a projection light frustum of the projector.

15. The method for optically remapping projected pixel positions of claim 13, wherein: The refractive element includes one or more optical surfaces having a substantially free-form shape, wherein a focusing surface is substantially close to or surrounds a vertex position of a projection light frustum of the projector.

16. The method for optically remapping projected pixel positions of claim 13, wherein: The refractive element includes one or more optical surfaces having substantially zero Gaussian curvature, wherein the focusing surface is substantially close to or surrounds the vertex position of the projection light frustum of the projector.

17. The method for optically remapping projected pixel positions of claim 13, wherein: Adjusting the optical effect of the optical device comprises one or more of mechanical deformation and thermal deformation of any refractive optical surface of the refractive element.

18. The method for optically remapping projected pixel positions of claim 13, wherein: Adjusting the optical effect of the optical device includes adjusting the relative positions of sub-elements of the refractive element.

19. The method for optically remapping projected pixel positions of claim 1, further comprising: The refractive element is adjusted by changing the optical effect of the refractive element.

20. The method for optically remapping projected pixel positions of claim 19, wherein: The refractive element is adjusted to a free-form shape.

21. The method for optically remapping projected pixel positions of claim 1, wherein: The reflective element is positioned along the optical path after the refractive element and the projection lens.

22. The method for optically remapping projected pixel positions of claim 1, wherein: The reflective element includes one or more of a plurality of sub-elements and a plurality of reflective optical surfaces.

23. The method for optically remapping projected pixel positions of claim 1, wherein: The reflective element has a generalized cylindrical shape.

24. The method for optically remapping projected pixel positions of claim 1, wherein: The reflective element has a free-form shape.

25. The method for optically remapping projected pixel positions of claim 1, wherein: The reflective element has a Gaussian curvature of zero.

26. The method for optically remapping projected pixel positions of claim 1, wherein: The reflective element is adjustable with respect to its optical effect.

27. The method for optically remapping projected pixel positions of claim 1, wherein: The reflective element is adjustable to a substantially free form shape.

28. The method for optically remapping projected pixel positions of claim 1, wherein: Adjusting the optical effect of the optical device includes one or more of mechanical deformation and thermal deformation of the reflective surface of the reflective element.

29. The method for optically remapping projected pixel positions of claim 13, wherein: Adjusting the optical effect of the optical device includes adjusting one or more of: the position of the reflective element relative to the projector, the position of the reflective element relative to the refractive element, and the relative positions of sub-elements of the reflective element.

30. The method for optically remapping projected pixel positions of claim 1, wherein: The refractive element is sufficiently close to the reflective element so that the projection light passes through the refractive element before and after being reflected from the reflective element.

31. An apparatus for optically remapping projected pixels for an observer of a display system, comprising: a projector and a curved screen oriented and positioned for a designed eye point (DEP) and corresponding field of view, and for projecting an image from the projector onto the curved screen; an optical device in an optical path between an image source forming point in the projector and a projected image forming point on the curved screen, wherein the optical device comprises at least a refractive element and a reflective element in an optical sequence, and wherein a focusing surface of the refractive element and a focusing surface of the reflective element each have a longest dimension, and the longest dimensions are oriented substantially orthogonal to one another; The refractive element is configured to substantially astigmatize the projected image focus to an extent that is substantially opposite to the focusing astigmatism introduced by the optical subsequent element using refraction through the refractive element; The reflective element is configured to optically redirect supplemental light from the projector to an optimal location on the curved screen to optically remap pixels, wherein the reflective element optically redirects the light using reflection; and The optical device is configured to be adjustable in its optical effect to support one or more of various projectors, various projector configurations, various display systems, various display system configurations, changes in display system components, changes in observer positions, and changes in field of view and changes in image resolution, Thereby, a substantially maximum pixel utilization and a substantially optimal image resolution for one or more observers of the display system is achieved.

32. The apparatus for optically remapping projected pixels for an observer of a display system as recited in claim 31, wherein: A collimator is located between the DEP and the curved screen, such that the curved screen is viewed through the collimator, and a set of viewing vectors from the DEP are reflected from the collimator before intersecting the curved screen.

33. The apparatus for optically remapping projected pixels for an observer of a display system as recited in claim 31 , wherein: The projector and curved screen are configured relative to the DEP such that the horizontal field of view exceeds 180 degrees, the image resolution is less than 6.1 arc minutes per optical line pair, and at least the reflective element is adjustable.

34. An apparatus for optically remapping projected pixels for an observer of a display system as recited in claim 31, wherein substantially orthogonal means between 70 degrees and 110 degrees.

35. An apparatus for optically remapping projected pixels for an observer of a display system as recited in claim 31, wherein the projector comprises a projection lens.

36. An apparatus for optically remapping projection pixels for observers of a display system as described in claim 31, wherein when there are two or more observers, the projector and curved screen are oriented and positioned for a compromise DEP for each portion of the field of view.

37. The apparatus for optically remapping projected pixels for an observer of a display system as recited in claim 31, wherein: The curved screen is a portion of one of an ellipsoid, a torus, a slight variation from an ellipsoid, and a slight variation from a torus, wherein the slight variation does not exceed 0.25 times a maximum radius of curvature of the curved screen.

38. The apparatus for optically remapping projected pixels for an observer of a display system as recited in claim 37, wherein: The curved screen has an infinite radius of curvature.

39. The apparatus for optically remapping projected pixels for an observer of a display system as recited in claim 35, wherein: The refractive element is positioned along the optical path one of before and after the projection lens.

40. The apparatus for optically remapping projected pixels for an observer of a display system as recited in claim 31, wherein: Planar folding mirrors are used around or between any optical elements in the entire optical path from the image source formation point within the projector to the image observation at the DEP.

41. The apparatus for optically remapping projected pixels for an observer of a display system of claim 31 , wherein: The refractive element includes a plurality of refractive sub-elements and a plurality of refractive optical surfaces.

42. The apparatus for optically remapping projected pixels for an observer of a display system as recited in claim 41, wherein: The refractive element includes one or more optical surfaces substantially having the shape of a generalized cylinder, wherein a focusing surface is substantially close to or surrounds the vertex position of a projection light frustum of the projector.

43. The apparatus for optically remapping projected pixels for an observer of a display system as recited in claim 41, wherein: The refractive element includes one or more optical surfaces having a substantially free-form shape, wherein a focusing surface is substantially close to or surrounds a vertex position of a projection light frustum of the projector.

44. The apparatus for optically remapping projected pixels for an observer of a display system as recited in claim 41, wherein: The refractive element includes one or more optical surfaces having substantially zero Gaussian curvature, wherein the focusing surface is substantially close to or surrounds the vertex position of the projection light frustum of the projector.

45. The apparatus for optically remapping projected pixels for an observer of a display system as recited in claim 31, wherein: The refractive element is configured to be adjustable in its optical effect.

46. ​​An apparatus for optically remapping projected pixels for an observer of a display system as claimed in claim 45, wherein: The refractive element is configured to be adjustable to a free-form shape.

47. An apparatus for optically remapping projected pixels for an observer of a display system as claimed in claim 46, wherein: The adjustment of the refractive element is achieved by one or more of mechanical deformation and thermal deformation of the refractive optical surface of the refractive element.

48. The apparatus for optically remapping projected pixels for an observer of a display system of claim 41, wherein: The optical effect of the optical device can be adjusted by adjusting the relative positions of the sub-elements of the refractive element.

49. The apparatus for optically remapping projected pixels for an observer of a display system as recited in claim 35, wherein: The reflective element is positioned along the optical path after the refractive element and the projection lens.

50. The apparatus for optically remapping projected pixels for an observer of a display system as recited in claim 31, wherein: The reflective element includes one or more of a plurality of reflective sub-elements and a plurality of reflective optical surfaces.

51. The apparatus for optically remapping projected pixels for an observer of a display system of claim 31 , wherein: The reflective element has a generalized cylindrical shape.

52. The apparatus for optically remapping projected pixels for an observer of a display system of claim 31 , wherein: The reflective element has a free-form shape.

53. The apparatus for optically remapping projected pixels for an observer of a display system of claim 31 , wherein: The reflective element has an optical surface with zero Gaussian curvature.

54. The apparatus for optically remapping projected pixels for an observer of a display system as recited in claim 31, wherein: The reflective element is adjustable with respect to its optical effect.

55. The apparatus for optically remapping projected pixels for an observer of a display system as recited in claim 54, wherein: The reflective element is adjustable to a substantially free form shape.

56. The apparatus for optically remapping projected pixels for an observer of a display system as recited in claim 54, wherein: The adjustment of the reflective element is achieved by one or more of mechanical deformation and thermal deformation of the reflective surface of the reflective element.

57. The apparatus for optically remapping projected pixels for an observer of a display system as recited in claim 54, wherein: The adjustment of the reflective element is achieved by adjusting one or more of the following: its position relative to the projector, its position relative to the refractive element, and the relative position of the reflective sub-element of the reflective element.

58. The apparatus for optically remapping projected pixels for an observer of a display system of claim 31, wherein: The refractive element is positioned sufficiently close to the reflective element so that light projected by the projector passes through the refractive element before and after reflecting from the reflective element.

Images