Systems, methods, and devices for image processing

Abstract

A method of reducing light pollution in a scene rendered on a display includes receiving a portion of the scene to be displayed on the display at an initial brightness, adjusting the initial brightness of the portion of the scene to a final brightness based on a light pollution model associated with the display, and rendering the scene to a display with the portion of the scene having the final brightness.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to and the benefit of U.S. Provisional Patent Application No. 63 / 357,279, filed on June 30, 2022, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure generally relates to systems, methods, and apparatus for image processing, for example, to reduce light pollution in a displayed image or scene. Background Art

[0004] The quality of an image displayed on a display is subject to the hardware and / or software capabilities of the display system. Some characteristics that affect the quality of an image displayed on a display include resolution, contrast, brightness, etc. Image processing techniques can be used to adjust the characteristics of an image to improve the quality of the image when displayed on a display. Summary of the invention

[0005] Additional features and advantages are described herein, and will be apparent from the following description and accompanying drawings.

[0006] At least one embodiment of the present disclosure is directed to a method for reducing light pollution in a displayed image. The method may include: receiving a portion of a scene to be displayed on a display at an initial luminance; adjusting the initial luminance of the portion of the scene to a final luminance based on a light pollution model associated with the display; and rendering the scene to the display with the portion of the scene having the final luminance.

[0007] At least one embodiment of the present disclosure is directed to an apparatus for reducing light pollution in a displayed image. The apparatus may include a processing circuit for: receiving a portion of a scene to be displayed on a display at an initial brightness; adjusting the initial brightness of the portion of the scene to a final brightness based on a light pollution model associated with the display; and rendering the scene to the display with the portion of the scene having the final brightness.

[0008] At least one embodiment of the present disclosure relates to a system for reducing light pollution in a displayed image. The system may include a processing circuit for: determining an initial brightness of a pixel in each of a plurality of frames of a video signal to be displayed on a display; adjusting the initial brightness of the pixel in each frame to a final brightness frame by frame based on a light pollution model, the light pollution model compensating for light pollution occurring when each frame is displayed on the display; and rendering each frame to the display using pixels having the final brightness.

[0009] At least one embodiment of the present disclosure relates to a method for calibrating a display system. The method includes: displaying a calibration image on a display; capturing the calibration image displayed on the display using one or more image sensors; determining light pollution of a region of the calibration image based on the captured calibration image; and generating a light pollution model based on the determined light pollution of the region.

[0010] At least one embodiment of the present disclosure relates to a system comprising: a screen; one or more projectors that project images onto the screen; one or more image sensors; and an image processing circuit for: presenting a calibration image to the screen using the one or more projectors; controlling the one or more image sensors to capture the calibration image displayed on the screen; determining light pollution of an area of ​​the calibration image based on the captured calibration image; and generating a light pollution model based on the light pollution of the determined area. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The present disclosure is described in conjunction with the accompanying drawings, which are not necessarily drawn to scale.

[0012] Figure 1 A display system according to at least one example embodiment is shown.

[0013] Figure 2 Shows Figure 1 Various example implementations of a display system in FIG.

[0014] Figure 3 Example patterns of calibration images according to at least one example embodiment are described.

[0015] Figure 4 A method according to at least one example embodiment is shown.

[0016] Figure 5 A method according to at least one example embodiment is shown. DETAILED DESCRIPTION

[0017] The following description provides only embodiments and is not intended to limit the scope, applicability or configuration of the claims. Instead, the following description will provide an enabling description for implementing the described embodiments to those skilled in the art. It should be understood that various changes may be made to the function and arrangement of the elements without departing from the spirit and scope of the appended claims.

[0018] It will be appreciated from the following description, and for reasons of computational efficiency, the components of the system may be arranged at any suitable location within a distributed network of components without affecting the operation of the system.

[0019] In addition, it should be understood that the various links connecting the elements can be wired, trace or wireless links, or any suitable combination thereof, or any other suitable known or later developed elements capable of providing data to and / or transmitting data from the connected elements. For example, the transmission medium used as a link can be any suitable carrier for electrical signals, including coaxial cables, copper wires and optical fibers, electrical traces on PCBs, etc.

[0020] As used herein, the phrases "at least one," "one or more," "or," and "and / or" are open-ended expressions that are both conjunctive and disjunctive in operation. For example, each of the expressions "at least one of A, B, and C," "at least one of A, B, or C," "one or more of A, B, and C," "one or more of A, B, or C," "A, B and / or C," and "A, B, or C" means A alone, B alone, C alone, A and B together, A and C together, B and C together, or A, B, and C together.

[0021] As used herein, the terms "determine," "compute," and "calculate," and variations thereof, are used interchangeably and include any suitable type of method, process, operation, or technique.

[0022] Various aspects of the disclosure are described herein with reference to the figures, which may be schematic illustrations of idealized configurations.

[0023] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present disclosure belongs. It will be further understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure.

[0024] As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "include", "including", "includes", "comprise", "comprises" and / or "comprising" when used in this specification specify the presence of stated features, integers, phases, operations, elements and / or components, but do not preclude the presence or addition of one or more other features, integers, phases, operations, elements, components and / or groups thereof. The term "and / or" includes any and all combinations of one or more of the relevant listed items.

[0025] Where it is appropriate to refer to a general element or set of elements rather than a specific element, the description may refer to the element or set of elements by its root term. For example, when reference to a specific element Xa, Xb, Xc, etc. is not required, the description may refer to the element generally as "X".

[0026] Example embodiments generally relate to predicting and overcoming fill factor-related contrast losses in displayed images. The instantaneous range of displayable brightness provided to the eye point (EP) of a collimated D-level flight simulator, also known as system contrast (CR), is determined by a variety of optical and geometric constraints. Although these constraints are generally necessary compromises in order to achieve the desired field of view (FOV), peak brightness (also known as white level), and brightness uniformity, the magnitude of the CR reduction and the magnitude of the subsequent grayscale distortion (also known as "washout") is determined by the relative brightness of the displayed FOV (also known as the fill factor (FF)). Example embodiments provide a process for estimating the effect of FF on the system CR and the corresponding washout of grayscale near the black level of the system for a given input image, which can be obtained using a simulated washed-out version of the input image. The just noticeable difference (JND) between the displayed brightness level and the system black level is then estimated, and the washed-out image is analyzed. Washout produces significant visually indistinguishable regions and regions of significant grayscale nonlinearity. A system-level solution is proposed to recapture and re-tune these lost and distorted gray levels. The system approach introduces a dynamic FF-dependent tone mapping (TM) operator that is uniquely enabled by the image generator (IG) or image processor.

[0027] Projector contrast CR ( ) ranges from 1000:1 to 500000:1, while system CR measures approximately 10:1. The difference in these measurements is the fill factor (FF) of the test pattern applied. Projector CR is usually quoted in terms of its "sequential" CR, where the luminance of an all-white and all-black pattern is measured sequentially and their results are divided. However, system CR is measured using a checkerboard of white and black squares. For a given image that is commanded by the IG and then converted to grayscale, the FF is defined as the sum of the commanded grayscale values ​​from all pixels divided by the same sum for the all-white image. Depending on the test pattern, the FF can range from 0.0 to 1.0. Given this definition, sequential CR measurements represent the limiting case, where the FF approaches zero and CR is maximized. However, a standard checkerboard pattern (an image of half black and half white blocks of equal size) has a relative command luminance of 0.5 and equivalently 0.5 (or ). This higher fill factor provides a significantly lower CR.

[0028] Many factors limit the CR efficiency of a collimated display system. For a typical rear-projection Class D flight simulation system, the limiting factors from image creation to image observation may include image generator capability, alignment system capability, projector capability and native contrast, efficiency of the imaging optics (lenses), projection screen related parameters (such as specular and diffuse reflections), ambient light, and human visual capabilities.

[0029] As FF increases, the amount of light in the system increases, which in turn increases the amplitude of light scattered and reflected from components between the IG (e.g., the imaging panel of the projector) and the eyepoint. While some of this scattered and reflected light can be reduced or eliminated through careful system layout and optically absorptive baffles around the projection screen and lens, a significant portion cannot be captured without degrading key system performance parameters such as FOV and brightness uniformity. A portion of this unblocked light strikes the imaging surface of the projection screen and produces undesirable light leakage. .

[0030] In order to provide adequate brightness uniformity for images composed of light projected from a variety of angles, projection screens in typical collimated systems have relatively wide scattering and reflection profiles. These profiles tend to widely distribute and receive both wanted and unwanted light. As a result, it is reasonable to assume that unwanted light from a given pixel is distributed almost uniformly across all other pixels within the FOV. This uniformly improves the system black level by .

[0031] The assumption of uniform scattering greatly simplifies the calculation of the system CR for a given FF. This is shown in the following equation, where: and represent the projector white and black levels respectively. Then, for example Figure 3 For the test pattern shown in , the white and black levels measured at the eye point are given by ( and( )express.

[0032]

[0033] In practice, the numerator and denominator The terms are not exactly equal, and their ratio determines how uniformly the unwanted scattered light is distributed. Therefore, the ratio of these terms should provide a relevant measure that indicates how well the uniform scattering model successfully represents the system contrast in a given application.

[0034] The wide scattering and reflection profiles used in collimated display systems demonstrate that unwanted light is scattered uniformly to produce a uniform The example embodiments associated with generating the light pollution model reasonably assume that for a given FOV in a collimated display system, the calculated FF will produce approximately the same amount of light as a black and white test pattern with the same FF. This then enables the straightforward collection of calibration data in order to verify the suitability of the model, such as that provided in a subsequent section.

[0035] The formula for FF-related system CR is as follows, where: is the contrast factor, which is obtained by adjusting The contribution of the ambient brightness ( ) does not come from Lw or Lb. For a collimated display system, the ambient brightness Usually negligible. In general, ambient brightness is probably more important for large FOV systems.

[0036]

[0037]

[0038] Sequential contrast ratio of the projector ( ) can be assumed to represent the relationship between the input Lw and Lb values.

[0039]

[0040] Contrast coefficient So it can be related to the system board CR, This is shown below. This is very useful because the checkerboard CR of the display system is known or measured at regular intervals.

[0041]

[0042] Therefore, within the approximation, the system CR as a function of the FF legal entity for a particular system can be given by and when necessary The only certainty.

[0043] The model confirms the expectation that system CR is significantly affected by FF, but the values ​​of FF for common flight training tasks practiced in flight simulators remain largely unknown. The relationship between FF and the time of day (TOD) for a given scenario is useful for understanding the impact of FF on flight training tasks.

[0044] The system CR calculation reasonably assumes a FF representing the entire system FOV. The subsequent description sometimes describes a partial FOV provided by a single projection channel (single projector). The examples described herein were performed using a 180° horizontal FOV and a 40° vertical FOV representing the Level D standard for flight simulators.

[0045] The pilot's ability to discern and ultimately identify mission-critical surfaces and objects is largely determined by system CR. During near-ground maneuvers, such as takeoff and landing, adequate system contrast is especially critical for identification of blips, runway markings, and related indications. At the decision height (DH), adequate system contrast can mean the difference between a safe landing and a go-around. The table below shows estimates of the FF range per TOD at DH for 64 combinations of different brightness, celestial, and weather controls.

[0046]

[0047] The corresponding system CR estimates provided in the table above imply that night scenes benefit from the relatively high sequential CR provided by the projector. For other TODs, the unwanted light added to the imaging surface (i.e., projection screen) significantly exceeds the initial black level of the projector. As a result, an increase in the projector sequential CR is likely to be observable only for night scenes. For DH applications, the system CR for dawn, dusk, and daytime scenes can be significantly improved by an increase in the display system CR, most likely by enhancing the CR of the projection screen.

[0048] As FF increases, the system CR decreases, and the system black level increases. If the image generator does not compensate for the dynamic CR of the system, and assumes a high static CR, such as the sequential CR of a projector, this increased black level has the potential to render content of very dark scenes invisible. Content just above this level will also suffer from unintended gray level compression, which distorts the intended tone mapping (TM) operator and produces visual non-linearities in the perceived grey levels. This effect, i.e., loss of dark grey levels and non-linear distortion of grey levels just above, is known as system washout. System washout is driven by the system CR and depends largely on the FF of a given image. System washout has a significant impact on the range and accuracy of displayed luminance, especially if the image generator assumes a static system CR.

[0049] In addition to losing scene content, the compression of brightness differences between adjacent gray levels near the system black level can significantly reduce the discernibility of visual cues in the affected areas. The visibility of textures, and to some extent critical runway markings, is also reduced due to system washout.

[0050] The FF-related TM operator conceived in accordance with the present invention can restore linearity to grayscale levels that are otherwise unnaturally compressed or completely invisible. Since the image generator has instantaneous image data for the entire FOV, and the output of the image generator is tuned to the brightness and chrominance of a specific simulator through a manual or automatic alignment system, the image generator can be uniquely able to perform the FF-related TM operator to restore visible brightness previously lost due to system washout. Representations of the system black level, the observer's adaptive brightness, and the brightness JND of any particular image can be provided within the IG. Therefore, nonlinear corrections can be applied to the TM operator, which effectively amplifies the intensity of lost and distorted grayscale levels until a visually linear output is achieved. The commandable bit depth may still be lost to system washout, but the expected discernibility of darker brightness areas is largely restored by the re-linearization provided by the FF-related TM.

[0051] Real-time application of FF-related TM operators has the potential to introduce visual artifacts for rapidly changing scene content. Furthermore, any significant changes in TM should be taken within a visually indiscernible time period, perhaps several seconds. Therefore, the proposed method is best used as a means to provide deterministic, psychophysically based recovery of otherwise lost or distorted dark gray levels over a truly continuous TOD range.

[0052] Example embodiments relate to a system-level approach to recovering grayscale levels lost by system washout, where an alignment system identifies the sequential CRs of the projectors and an image generator employs a FF correlation model to produce a continuous representation of the entire visual FOV. This enables the image generator to provide a customized TM operator for the instantaneous image.

[0053] In order to produce a sufficiently large FOV and / or high resolution images, a visual display system may employ multiple projectors configured so that their projected images overlap. In order for the resulting image to appear uniform, each component image must have corrections for spatial and radiometric non-uniformities. In applications with relatively large FOVs (e.g., between 40 and 360 degrees wide), such as flight simulator visual display systems, these corrections are calculated by an alignment system (AS) and executed by the AS and / or an image generator (IG) (e.g., an image processor).

[0054] Correction of static non-uniformity is well known and practiced, but correction of dynamic non-uniformity (such as light pollution between adjacent pixels) is not. As the scene displayed on the projection screen changes, the amount and location of light pollution also changes, producing a constantly changing washout brightness, which reduces scene contrast and the displayable brightness range. This is the performance limiting factor for daytime, dusk and dawn scenes, as the overall brightness is appreciable.

[0055] By increasing the contrast of the projection screen, the instantaneous displayable brightness range can be significantly improved. However, increasing contrast has a negative impact on brightness by shortening the life of the projector light source, and in many systems has a negative impact on life cycle costs. Fortunately, in flight simulation applications, the reduction in instantaneous brightness range usually has little effect on scene fidelity. This is because the distance between the pilot and the objects outside the window is large. Even in the best weather conditions (such as ceiling and visibility unlimited (CAVU)), atmospheric attenuation is perceptible within the scene, and therefore, black objects will be dark gray due to atmospheric scattering. This means that within these scenes, pixels are rarely (if ever) commanded to display black, and therefore brightness range is not usually implemented for this content and is not a significant performance limiting factor.

[0056] The reduction in scene contrast due to washout luminance is perceptible. Each scene employs multiple gray-to-gray contrasts, many of which are on the order of washout luminance, which may shift the perceived position of grayscale gradients within an image by a negligible amount (e.g., by a fraction of a pixel), but it may also result in loss of fine detail or premature loss of out-of-range objects due to reduced gray-to-gray modulation transfer function (MTF). This is well-recognized in the industry, and contrast of visual display systems is a tuning parameter.

[0057] The inventive concept employs both software and hardware to improve scene contrast. Since contrast is typically measured as the ratio of black and white, it is not affected by light pollution correction (LPC). However, direct measurement of contrast within a scene quantifies the benefit and shows it to be equivalent to at least a doubling of contrast. To maintain the scene contrast benefit, LPC must be recalculated for each scene, typically at 120Hz. LPC also performs best with a complete light pollution characterization in order to best predict the per-pixel instantaneous light pollution of the scene.

[0058] Figure 1 A system 100 is shown according to at least one example embodiment. The system 100 includes an image processing device (also referred to herein as a device or image generator or image processor) 104, a display system 108, and a database 112.

[0059] In at least one example embodiment, device 104 corresponds to one or more of a personal computer (PC), a laptop computer, a tablet computer, a smart phone, a server, a collection of servers, etc. In some embodiments, device 104 may correspond to any suitable type of device that communicates with other devices that are also connected to a common type of communication network. As another specific but non-limiting example, device 104 may correspond to a server that provides information resources, services, and / or applications to user devices, client devices, or other hosts in system 100. In at least one example embodiment, device 104 includes any suitable device that generates, receives, stores, and / or processes moving and / or still images.

[0060] The display system 108 includes one or more displays to display moving and / or still images received from the image processing device 104. The display system 108 may include any suitable type of display, such as a liquid crystal display (LCD), a light emitting diode (LED) display, etc. The display system 108 may include one or more stand-alone devices or devices integrated as part of another device (such as a smart phone, a laptop, a tablet computer, etc.). The display system 108 may be integrated with the device 104. In one non-limiting embodiment, the display system 108 is integrated with the flight simulator and includes one or more screens and one or more projectors (e.g., see FIG. 1 ) to project images onto the one or more screens. Figure 2 ). In this case, one or more screens may include a full or partial dome or dome-like structure (e.g., made of acrylic material) illuminated by one or more projector backlights. For example, a screen according to at least one embodiment implements a horizontal field of view of 360 degrees or nearly 360 degrees and a vertical field of view of 135 degrees or nearly 135 degrees. In another example, the screen implements a horizontal field of view of 180 degrees or nearly 180 degrees and a vertical field of view of 40 degrees or nearly 40 degrees. Example embodiments are not limited to the above-described types of displays and / or the above-described fields of view, and any suitable display with any suitable field of view may be used for display system 108.

[0061] The database 112 may include any suitable type of memory device or collection of memory devices configured to store data. Non-limiting examples of suitable memory devices that may be used include flash memory, random access memory (RAM), read-only memory (ROM), variations thereof, combinations thereof, and the like. The database 112 may be remote from the device 104 and / or the display system 108 or local to the device 104 and / or the display system 108. In at least one embodiment, the database 112 stores a light pollution model and information related thereto. The light pollution model is discussed in more detail below with reference to the remaining figures, but the light pollution model may include a data set having a light pollution value for each pixel displayed on a display of the display system 108. The light pollution value of a pixel may indicate the amount of light pollution experienced by the pixel (e.g., light pollution from ambient light and / or other pixels). The light pollution model may be used to generate a set of correction factors that may be applied when generating an image so that light pollution is reduced or eliminated from the image when displayed on the display. A correction factor may be generated for each pixel of the display so that light pollution for each pixel is reduced or eliminated. In at least one example, the light pollution model includes a scattering coefficient calculated for each pixel of the display, which can later be used to determine a correction factor for the pixel.

[0062] The device 104, the display system 108, and the database 112 may be communicatively coupled to each other via a communication network. The communication network may include a wired network and / or a wireless network that implements wired and / or wireless communications within the system 100. Examples of communication networks that may be used include Internet Protocol (IP) networks, Ethernet networks, InfiniBand (IB) networks, Fiber Channel networks, the Internet, cellular communication networks, wireless communication networks, combinations thereof (e.g., Fiber Channel over Ethernet), variants thereof, and the like. The communication network may use one or more protocols in the 802.11 protocol suite, near field communication (NFC) protocols, Bluetooth protocols, LTE protocols, 5G protocols, and the like to implement wireless communications. The device 104, the display system 108, and the database 112 may include one or more communication interfaces to facilitate wired and / or wireless communications on the communication network. The device 104 and the display system 108 may be connected to each other using any suitable connection (e.g., HDMI connection, display port connection, RS232 connection, BNC connection, RJ45 connection, etc.) for carrying video and / or audio signals.

[0063] Although device 104, display system 108, and database 112 are shown as separate entities that communicate over a communication network, it should be understood that these elements may be combined into a single device (eg, a server, personal computer, etc.).

[0064] The processing circuit 116 may include suitable software, hardware, or a combination thereof for processing images from the source device 104 and performing other types of computing tasks. The processing circuit 116 may perform various image processing operations and algorithms described herein. The memory 120 may include executable instructions, and the processing circuit 116 may execute the instructions on the memory 120. Therefore, the processing circuit 116 may include a microprocessor, a microcontroller, etc. to execute the instructions on the memory 120. The memory 120 may correspond to any suitable type of memory device or a collection of memory devices configured to store instructions. Non-limiting examples of suitable memory devices that can be used include flash memory, random access memory (RAM), read-only memory (ROM), variants thereof, combinations thereof, etc. In some embodiments, the memory 120 and the processing circuit 116 may be integrated into a common device (e.g., a microprocessor may include an integrated memory). Additionally or alternatively, the processing circuit 116 may include hardware, such as an application-specific integrated circuit (ASIC). Other non-limiting examples of processing circuit 116 include an integrated circuit (IC) chip, a central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), a digital signal processor (DSP), a collection of logic gates or transistors, resistors, capacitors, inductors, diodes, etc. Some or all of processing circuit 116 may be provided on a printed circuit board (PCB) or a collection of PCBs. It should be understood that any suitable type of electrical component or collection of electrical components may be suitable for inclusion in processing circuit 116. Processing circuit 116 may send signals to and / or receive signals from other elements of system 100 to control various operations of system 100.

[0065] The input device 124 includes suitable hardware and / or software that enables input (e.g., user input) to the system 100 through the device 104. The input device 124 may include a keyboard, a mouse, a touch-sensitive pad, touch-sensitive buttons, a touch-sensitive portion of a display, mechanical buttons, switches, and / or other control elements for providing user input to the system 100 to enable a user to control certain functions of the system 100.

[0066] The output device 128 may include suitable hardware and / or software that generates visual, audio, and / or tactile feedback for a user or other interested party based on one or more inputs from the processing circuit 116. In at least one example embodiment, the output device 128 includes one or more displays to display an output image and / or one or more characteristics of an input image after the device 104 processes the input image. The input image may be based on a source image received from the source device 104 via a communication network. The display may include any suitable type of display, such as a liquid crystal display (LCD), a light emitting diode (LED) display, and / or the like. The output device 128 may be a stand-alone device or a device integrated as part of another device (such as a smart phone, a laptop computer, a tablet computer, etc.).

[0067] Although the input device 124 and the output device 128 are illustrated as being part of the image processing device 104 , the input device 124 and / or the output device 128 may be implemented separately from the device 104 depending on the design preferences of the system 100 .

[0068] Additionally, it should be appreciated that device 104, display system 108, and / or database 112 may include other processing devices, storage devices, and / or communication interfaces typically associated with computing tasks, such as sending and receiving data over wired and / or wireless connections.

[0069] Figure 2 Shows Figure 1 Various example implementations of the display system 108 in FIG. In more detail, Figure 2 Two embodiments of a display system 108 that may be useful in a flight simulator are shown. As such, the display system 108 in both embodiments includes one or more projectors 200a, 200b, and 200c and a display or screen 204 having a dome structure. In operation, the image processing device 104 controls the projectors 200a to 200c to display moving and / or still images on the rear side of the screen 204 for viewing by a user sitting in one or more seats 208 on the front side of the screen 204 within the dome. In the flight simulator example, the seat 208 may be a pilot's seat, and although not explicitly shown, a control panel and other flight instruments associated with a cockpit may be located within the dome near the seat 208 to allow the user sitting in the seat 208 to control the simulated flight, as depicted on the screen 204.

[0070] Figure 2The bottom implementation of 200a shows one or more image sensors (e.g., cameras) 212 that can be located in the center area of ​​screen 204. As discussed in more detail below, image sensor 212 can be used to capture one or more calibration images displayed on screen 204 for the purpose of building a light pollution model that can be stored in database 112. Sensor 212 can operate at a different resolution than one or more of projectors 200a, 200b, and 200c. For example, in some embodiments, sensor 212 has a lower resolution than at least one of projectors 200a, 200b, and 200c.

[0071] Figure 3 Example patterns 300 and 304 of calibration images that may be displayed by projectors 200a to 200c on screen 204 are shown. As can be appreciated, pattern 300 includes alternating white and black stripes of equal width extending in a horizontal direction on screen 204, while pattern 304 includes alternating white and black stripes of equal width extending in a vertical direction on screen 204. Both patterns 300 and 304 include black and white images having a repeating pattern. Here, it should be appreciated that except for Figure 3 Patterns other than those shown in can be used as calibration images for constructing a light pollution model (e.g., checkered patterns, dot patterns, diagonal patterns, etc.). Furthermore, the patterns used for calibration images are not limited to black and white patterns, and any suitable color combination can be used.

[0072] Figure 4 A method 400 is shown according to at least one example embodiment. Figure 4 The general order of the steps of method 400 is shown in FIG. 4 , but method 400 may include more or fewer steps, or may be combined with Figure 4 The order of the steps may be arranged differently from the order shown in . The method 400 may be performed as a set of computer executable instructions encoded or stored on the memory 120 and executed by the processing circuit 116. In another embodiment, one or more operations of the method 400 are implemented by the processing circuit 116 in the form of an ASIC. Figures 1 to 3 The described systems, components, assemblies, devices, user interfaces, environments, software, etc. are used to explain the method 400. In general, Figure 4 The method shows the calibration operation performed to generate a light pollution model according to the inventive concept.

[0073] Typically, light pollution models view light pollution as unwanted light from a given pixel or group of pixels that is distributed to other pixels. In at least one embodiment, it is assumed that this unwanted light is substantially uniformly distributed across all other pixels in the field of view (e.g., the field of view of a projector). Figure 1As noted in the previous discussion of the discussion, the fill factor of an image to be displayed may be determined by converting pixel values ​​of the image to grayscale, summing the grayscale values ​​of all pixels from the image, and dividing the sum by the sum of the grayscale values ​​of all pixels from the image in an all-white image, as commanded by the image processor 104. A light pollution model may be used to generate correction factors (e.g., tone mapping operators or brightness adjustments) that may be applied to pixels of a scene during an image generation process, wherein the correction factors differ based on an expected fill factor of the scene. Then, as part of the correction process, the fill factor of a scene (e.g., a frame) may be determined by converting pixels of the scene to grayscale and dividing the sum of the grayscale levels by the grayscale of an all-white image displayed on the same display as the display used to ultimately display the scene. The correction factors for the pixels of the scene may then be retrieved based on the calculated fill factor of the scene. As described above and below, the correction factors may correspond to tone mapping operators that adjust the brightness of the pixels in a manner that eliminates or reduces light pollution associated with each pixel, thereby increasing system contrast without increasing the sequential contrast of the display or projector.

[0074] Operation 404 includes displaying a calibration image on a display (eg, on screen 204). As described above, the calibration image may be a portion of the image from the image displayed simultaneously on screen 204 by projectors 200a through 200c. Figure 3 304. As can be appreciated, each projector 200a to 200c can correspond to a different "display," even though the projectors cooperate to display a single continuous scene.

[0075] Operation 408 includes capturing a calibration image displayed on the display using one or more image sensors 212. For example, one or more image sensors 212 may be implemented using a camera ball with multiple integrated cameras and / or using multiple individual cameras arranged on the back of the screen 204. In one non-limiting example, the number of cameras for the 360-degree screen 204 may be equal to eight, each camera having the same or similar field of view (e.g., a 120-degree field of view) directed toward a corresponding portion of the screen 204. However, the multiple cameras and the corresponding fields of view may vary according to design preferences. The camera may be located at a design eye point, which may be a position corresponding to the eye level of a user sitting in the seat 208. In some embodiments, one or more of the cameras are aligned with the light of the design eye point. The camera may capture the calibration image under ambient light conditions that are typically present when the screen 204 is used for flight simulation or other purposes. In another embodiment, the camera captures the calibration image under dark conditions.

[0076] Operation 412 includes determining light contamination of a zone of the calibration image based on the captured calibration image. For example, the image processing device 104 determines a scattering coefficient of a zone of the captured calibration image, wherein each zone of the captured calibration image corresponds to a single pixel or a group of adjacent pixels of the screen 204. In doing so, the image processing device 104 may collect information about the brightest white and the brightest black in the calibration image displayed on the screen 204. As can be appreciated, the image processing device 104 may perform a mapping of pixels in the captured calibration image to pixels of the screen 204 in order to accurately determine the scattering coefficient of the pixels of the screen 204. In at least one embodiment, the scattering coefficient of each pixel of the screen 204 may be determined using information related to a known brightness of a pixel of the screen (e.g., a brightness value of a pixel commanded by the image processing device 104 of the projector 200) and a captured brightness value of the same pixel captured by one or more image sensors 212 (the difference between the known brightness value of a pixel and the captured brightness value may indicate the scattering coefficient of the pixel). Some examples employ the following pseudo code to determine the scattering coefficient of a pixel using two calibration images: a first image that is 50% white and 50% black, such as one of the examples of image patterns 300 or 304, as captured by image sensor 212, and a second image that is the inverse of the first image (i.e., all white areas in the first image are black in the second image, and all black areas in the first image are white in the second image). The darkest and whitest brightness values ​​of each pixel are measured in each captured image. Such brightness measurements can be linear in nature (e.g., ranging from 0.0 to 1.0), and the resulting scattering coefficient for each pixel is unitless. The pseudo code for calculating the scattering coefficient in the two captured images is as follows:

[0077] for each display d

[0078] for each pixel i

[0079] scattering_coefficient[d][i] = 2 * black[d][i] / (white[d][i] - black[d][i])

[0080] The translation is as follows:

[0081] For each display d

[0082] For each pixel i

[0083] Scattering coefficient [d][i] = 2*black[d][i] / (white[d][i]-black[d][i])

[0084] Since the scattering coefficient typically varies slowly across the display system, the scattering coefficient can be interpolated from lower resolution measurements (e.g., across the display system). For example, if a single calibration image is used rather than two images as in the example above, white brightness measurements in black areas of the single image are estimated or interpolated from white brightness measurements in white areas of the image near the black areas, and vice versa, e.g., a black pixel in the image is assigned the same white brightness value as the white pixel closest to the black pixel (and vice versa for white pixels assigned black brightness values). A single measurement for scattering coefficient determination can be used to generate the correction factor.

[0085] Determining the scattering coefficient for each pixel of screen 204 based on the captured calibration images is a useful measure of light pollution because the scattering coefficient is substantially independent of the color displayed by the pixel within the scene. Therefore, calculating the scattering coefficient using a relatively simple black and white calibration image (such as image 300 or image 304) is an effective method for collecting information about light pollution between adjacent pixels of a color scene displayed on screen 204. As discussed in more detail below, the scattering coefficient can be used to construct a light pollution model that is used to predict and correct for instantaneous light pollution of a scene for each pixel (or each area other than pixels) of screen 204.

[0086] Operation 416 includes generating a light pollution model based on the determined light pollution from operation 412. For example, operation 416 generates the light pollution model based on the calculated scattering coefficient. In at least one embodiment and as discussed in more detail below, the light pollution model can be used to generate a correction factor (e.g., a brightness adjustment) for each region (e.g., each pixel) of the displayed image. In at least one example, the light pollution model includes an estimate of light pollution within a range of possible fill factors. In other words, the fill factor of the display changes as the image being displayed changes, and therefore, the light pollution model includes a light pollution value for each possible fill factor value. In the example where the fill factor ranges from 0.00 to 1.00, the light pollution model may include 101 light pollution values ​​- one value for each possible fill factor 0.00, 0.01, 0.02, etc., up to 1.00. In at least one embodiment, each light pollution value is a vector "pollution.rgb" and is defined for each pixel i of the display d as follows: pollution.rgb=FillFactor.rgb*scattering_coefficient[d][i]. The light pollution model may be calculated according to the following in Figure 5 The FillFactor.rgb of the scene to be displayed by projector 200 on screen 204 is calculated in real time based on the discussion in .

[0087] As described above, each region of the calibration image may correspond to a pixel of the screen 204, and thus, the light pollution model may be used to control brightness adjustments for that pixel, thereby allowing the system to predict and correct light pollution on a per-pixel basis. The tone mapping operator may be static for each pixel, or variable based on certain conditions.

[0088] Figure 5 A method 500 is shown according to at least one example embodiment. Figure 5 The general order of the steps of method 500 is shown in FIG. 1 , but method 500 may include more or fewer steps, or may be combined with Figure 5 The order of the steps may be arranged differently from the order shown in . The method 500 may be performed as a set of computer executable instructions encoded or stored on the memory 120 and executed by the processing circuit 116. In another embodiment, one or more operations of the method 500 are implemented by the processing circuit 116 in the form of an ASIC. Figures 1 to 4 The described systems, components, assemblies, devices, user interfaces, environments, software, etc. are used to explain the method 500. In general, Figure 5 The method illustrates operations performed to generate a scene that includes brightness correction on a per-pixel basis.

[0089] Operation 504 includes generating a light pollution model based on at least one calibration operation. Thus, operation 504 may correspond to Figure 4 4 , which illustrates details of at least one calibration operation. As described above and below, the light pollution model may include light pollution values ​​calculated from the scattering coefficients of pixels p of display d, and may be used to adjust the initial brightness of pixels of a scene to a final brightness in real time so as to reduce light pollution such as image washout, which in turn increases the effective contrast of the scene when displayed on screen 204 (recall that screen 204 may include multiple displays, each of which corresponds to an area of ​​screen 204 illuminated by a respective projector 200). Here, it should be understood that operation 504 may be skipped or omitted from method 500 if, for example, the light pollution model has already been generated (e.g., during a previous iteration of method 500 or a separate instance of method 400). In general, the light pollution model may be updated (regenerated) when one or more of the components in the system are changed (e.g., after projector maintenance, such as lamp replacement, projector replacement, projector repositioning, etc.).

[0090] Operation 508 includes the image processing device 104 receiving a portion of a scene having an initial brightness, for example, as commanded by a video signal or a still image signal having the scene. In the case of a video signal (e.g., a 60 Hz or 120 Hz video signal), the scene may correspond to a frame of the video signal. In this case, operations 508 to 516 may be performed iteratively in real time so that each frame of the video signal rendered to the display system 108 is analyzed and the brightness is adjusted according to the light pollution model. In one embodiment, the scene corresponds to a still image to be rendered to the display system 108. When the scene is displayed on the display system 108, the portion of the scene may correspond to a single pixel or a group of pixels. For example, the portion of the scene may be mapped to a single pixel on the screen 204, or to a group of adjacent pixels on the screen 204. In some cases, operation 508 may include converting image information of the portion of the scene into an initial brightness value and / or determining the initial brightness value of the portion of the scene in a suitable manner.

[0091] The portion of the scene may have one or more brightness values, such as linear brightness values ​​ranging from 0.0 to 1.0 as discussed herein (which may have corresponding RGB brightness values). For example, when the portion of the scene corresponds to a single pixel, the initial brightness of the pixel may correspond to a preset brightness value of the pixel determined by the image processing device 104 for rendering the scene to the display system 108. In another example, when the portion of the scene corresponds to a group of pixels, the initial brightness may be an average brightness value of the preset brightness values ​​of the pixels, a median brightness value of the preset values ​​of the pixels, and / or another suitable brightness value that takes into account the preset brightness values ​​of one or more pixels.

[0092] Operation 512 includes adjusting the initial brightness of the portion of the scene to a final brightness based on the light pollution model generated in operation 504. Here, the brightness adjustment may compensate for light pollution on a per-pixel basis that would otherwise occur when the scene is rendered to display system 108 without the brightness adjustment. As described above, the light pollution model may be associated with a display, which in some examples corresponds to projector 200. In a system with multiple displays or projectors, the light pollution model may be determined separately for each display or projector, or determined for one display or projector and applied to the remaining displays or projectors.

[0093] Operation 516 includes rendering the scene to a display, wherein portions of the scene have final brightness. In at least one example embodiment, a plurality of pixels or all pixels within the scene may have their brightness adjusted in operation 512 such that the entire displayed scene has reduced or eliminated light pollution compared to the initial signal.

[0094] The following is a more detailed discussion of example embodiments related to operations 512 and 516, beginning with a discussion of various equations for adjusting the initial brightness value to the final brightness value. When all displays (i.e., projector 200) have the same or nearly the same display characteristics (e.g., number of pixels, FOV, color range, etc.), which is usually the case, the peak (or maximum) display brightness can be assumed to be 1.0 for each display. X.rgb represents the color vector of the pixel, which is a linearized rgb brightness value ranging from the minimum possible value to the maximum possible value (such as 0.0 to 1.0). The following discussion avoids the use of non-linear sRGB encoded integer values ​​(e.g., in the range of 0 to 255 or 0 to 1023) because the following expressions do not support the use of non-linear encoded data. The use of linear 0 to 1 values ​​is common in graphics rendering documents and GPU programming because the conversion from linear 0 to 1 values ​​to display-specific encodings (e.g., 0 to 255) is not usually part of the GPU programming interface. However, it should be understood that a linear brightness rgb value can have a corresponding non-linear brightness sRGB value.

[0095] Luminance value averaging may be implemented in one or more GPUs of processing circuitry 116, and the average may be calculated at the frame rate of the video or image being displayed. Real-time fill factor calculation (FillFactor.rgb) for a scene may be performed according to the following pseudo code, where the "+=" sign is equivalent to the sigma sign indicating a summation operation:

[0096] for each display d, for each pixel i

[0097] average_fill_factor[d].rgb += input_pixel[d][i].rgb / #display_pixels[d]

[0098] FillFactorSum.rgb += peak_display_luminance[d] * average_fill_factor[d].rgbFillFactorMax.rgb += peak_display_luminance[d]

[0099] FillFactor.rgb = FillFactorSum.rgb / FillFactorMax.rgb

[0100] The translation is as follows:

[0101] For each display d, for each pixel i

[0102] average_fill_factor[d].rgb += input_pixel[d][i].rgb / #display_pixels[d]

[0103] FillFactorSum.rgb += peak_display_luminance[d] * average_fill_factor[d].rgbFillFactorMax.rgb += peak_display_luminance[d]

[0104] FillFactor.rgb = FillFactorSum.rgb / FillFactorMax.rgb

[0105] In the equation above:

[0106] --input_pixel[d][i].rgb is the initial brightness of pixel i in display d that is initially commanded to be displayed on screen 204,

[0107] --#display_pixels[d] is the total number of pixels in display d,

[0108] --FillFactorSum.rgb is a measure of the overall fill factor of all displays d, and

[0109] --FillFactorMax.rgb is the maximum fill factor value for the system (assumed to be 1.0 in this example).

[0110] As can be appreciated, FillFactor.rgb is a real-time fill factor that has been normalized and ranges from 0.0 to 1.0. The real-time fill factor.rgb is then used to apply brightness correction on a per-pixel (or per-region) basis.

[0111] Per-pixel correction can be implemented in the GPU at frame rate according to the following pseudo-code, which uses the Figure 4 The pollution value of the light pollution model pollution.rgb:

[0112] for each display d, for each pixel i in a scene to be displayed, light pollution is determined as:

[0113] pollution.rgb = FillFactor.rgb * scattering_coefficient[d][i]

[0114] The translation is as follows:

[0115] For each display d, for each pixel i in the scene to be displayed, the light pollution is determined as:

[0116] pollution.rgb = FillFactor.rgb * scattering_coefficient[d][i]

[0117] The final brightness value of the pixel can be calculated as:

[0118] pixel.rgb = input_pixel[d][i].rgb * (1.rgb + pollution.rgb) -pollution.rgb

[0119] As described above, pollution.rgb is a value included in the light pollution model and represents the amount of light pollution of pixel i, and is obtained by multiplying the real-time fill factor value FillFactor.rgb by the scattering coefficient of pixel i. Thereafter, the final brightness pixel.rgb of pixel i is determined by multiplying the initial brightness value input_pixel[d][i].rgb by (1.rgb+pollution.rgb) and then subtracting pollution.rgb from the result, where 1.rgb is the peak brightness color vector of (1, 1, 1).

[0120] In some examples, the system implements tone mapping to prevent or mitigate detail loss in dark areas (darkarea) according to the following pseudo code:

[0121] tone_map.rgb = min(pixel.rgb - pollution.rgb, 0.rgb), where 0.rgb is the minimum brightness color vector of (0,0,0) and tone_map.rgb is an intermediate value used as input in the following expression:

[0122] output_pixel[d][i].rgb = pixel.rgb + tone_map.rgb * tone_map.rgb / (4* pollution.rgb)

[0123] Here, output.pixel[d][i].rgb is a tone mapping operation that restores contrast to pixels with final luminance values ​​close to zero or less than zero by mapping the final luminance values ​​to higher values. In other words, the tone mapping operation output.pixel[d][i].rgb may be applied to those pixels with final luminance values ​​pixel.rgb less than zero or so close to zero.

[0124] In at least one example embodiment, instead of or in addition to the brightness adjustment described in the above equation, operation 512 includes selecting a correction factor based on the brightness threshold and the initial brightness from operation 508, which correction factor may correspond to a weight applied to the rgb brightness value or other suitable means for adjusting the brightness. Then, operation 512 may further include applying the selected correction factor to the portion of the scene to adjust the initial brightness of the portion to the final brightness of the portion. Typically, the brightness threshold may be a set of design parameters based on empirical evidence and / or preferences, and may be stored in a memory or database and accessible by an image processing device. For example, the brightness threshold is based on or equal to the leakage level of the projector 200 that projects the portion of the scene onto the screen 204. The brightness threshold may be the same or different for multiple or all pixels of the screen 204. Selecting the correction factor as part of operation 512 may include: when the initial brightness is above the brightness threshold, selecting a first correction factor that reduces the initial brightness of the portion of the scene to the final brightness by a first amount, the first amount being substantially equal to the amount of light pollution predicted by the light pollution model for the portion of the scene. For example, the light pollution model may predict that a brightness value of a first pixel above a brightness threshold causes a known amount of light pollution on one or more neighboring pixels based on the scattering coefficients calculated in method 400. A first correction factor may then be selected to reduce the initial brightness of the first pixel above the brightness threshold by an amount that offsets or substantially reduces the light pollution caused by the first pixel on the neighboring pixels.

[0125] Selecting a correction factor as part of operation 512 may include selecting a second correction factor that reduces the initial brightness by a second amount that is less than the amount of light pollution predicted by the light pollution model for the portion of the scene when the initial brightness is at or below the brightness threshold. Thus, the second amount may be less than the first amount of the first correction factor. In at least one embodiment, the second amount becomes lower as the initial brightness moves further below the brightness threshold. The second amount (the amount of brightness reduction from the initial brightness) may be inversely proportional to the difference between the initial brightness and the brightness threshold. That is, as the difference between the initial brightness and the brightness threshold increases, the amount of brightness reduction by the second amount becomes smaller. Here, it should be understood that when the initial brightness is at the minimum possible brightness achievable by the display system 108, the second amount may be equal to zero (i.e., final brightness = initial brightness). Using a second correction factor that gradually reduces the amount of correction as the difference between the brightness threshold and the initial brightness increases enables the system to make a smaller brightness correction than the first correction factor to avoid excessively reducing the initial brightness of the pixel as the initial brightness of the pixel approaches the minimum possible brightness of the display system 108 (where the minimum possible brightness is based on the capabilities and noise of the projector 200).

[0126] In at least one embodiment, operation 512 may consider an additional brightness threshold that is higher than the above brightness threshold. If it is determined that the initial brightness of the pixel is at or above the additional brightness threshold, a third correction factor may be selected to adjust the initial brightness of the pixel. In this case, the amount of brightness reduction caused by the third correction factor may gradually decrease as the difference between the initial brightness and the additional brightness threshold increases (e.g., the amount of brightness reduction becomes smaller until there is zero modification when the initial brightness of the pixel is at the maximum possible brightness of the display system 108). The additional brightness threshold may also be a set of design parameters based on empirical evidence and / or preference.

[0127] Although not explicitly shown, in at least one embodiment, method 500 determines whether the expected color of a pixel within the scene changes by more than a color difference threshold when the first correction factor or the second correction factor is applied. If so, the first correction factor or the second correction factor can be modified (e.g., the brightness is reduced) so that the color change of the pixel is less than the color difference threshold. If not, the first correction factor or the second correction factor can be applied to the pixel without further modification.

[0128] In at least one embodiment, the correction factors described above may vary based on the time of day setting of the flight simulation. For example, the tone mapping operator for each pixel may be globally adjusted up or down based on the time of day or ambient light conditions being simulated (e.g., dusk, dawn, clear daytime, nighttime, cloudy, raining, snowing, etc.). In this way, the image processing systems, methods, and devices of the present disclosure may adjust the initial brightness of a scene at any time of day. In contrast, some related art methods of adjusting brightness work adequately for one time of day (such as for scenes during daytime), but are inadequate for other times of day (such as dusk, dawn, and / or nighttime).

[0129] Here, it should be understood that operation 512 may be performed on a global basis for display system 108. That is, a correction factor (e.g., a tone mapping operator) applied to a pixel of a scene may be modified based on the instantaneous light pollution at that pixel, as predicted by a light pollution model responsive to the expected brightness of all pixels of the scene.

[0130] Although the correction factors have been described as being selected, it should be understood that the correction factors mentioned herein can be calculated or determined using one or more suitable equations in addition or alternatively. In addition, although the brightness adjustment and correction factors have been discussed with reference to projector-type displays, the same concepts can be applied to other displays, such as LEDs, OLEDs, LCDs, etc.

[0131] As can be appreciated from the foregoing description, example embodiments provide improved contrast perception for viewers (e.g., the contrast improvement of the above adjustments implemented by software). In addition, example embodiments can reduce the intensity of light emitted from a projector, thereby extending the life of the projector system (e.g., extending lamp life). Example embodiments can also enable a lower quality projector to produce a high quality image on a screen.

[0132] In view of the present disclosure, an example embodiment is directed to a method, the method comprising displaying a calibration image on a display, capturing the calibration image displayed on the display using one or more image sensors, and determining light pollution of a region of the calibration image displayed on the display. In one embodiment, equal-sized black / white latitude stripes propagate from pole to pole, and one or more industrial cameras collect per-pixel linear peak white / black signal levels. The operation is repeated for unequal-sized black / white latitude stripes. The method may include generating a correction factor for each region of the calibration image based on the determined light pollution, and generating a light pollution model based on at least one calibration operation. In at least one embodiment, a camera pixel to display pixel mapping is used to transform data from the camera to the display space. In addition, a per-pixel light pollution model is derived from the data and recorded for later use. The method may also include determining an initial brightness of a portion of a scene to be displayed on the display, and adjusting the initial brightness of the portion of the scene to a final brightness based on a light pollution model to compensate for light pollution when the scene is displayed on the display. In one embodiment, an initial image to be displayed is obtained. For the initial image, a light pollution correction is calculated for the color value (red, green, blue) of each pixel using the light pollution model. For very low color values ​​(e.g., "very low color values" may contribute less than 1% of the peak white intensity), the light pollution correction decays smoothly to zero, which can prevent or mitigate detail loss in very dark domains of the image by preventing the correction from subtracting the amplitude of pollution from smaller amplitude color values. In addition, when the corrected color value deviates from the initial color value by a color difference threshold (e.g., the "color difference threshold" may be a value of 1.0 in CIELAB delta E color coordinates, such as https: / / en.wikipedia.org / wiki / Color_difference), the light pollution correction decays smoothly to zero, which can ensure that the image color is not significantly changed. The method may finally include rendering the scene to a display with the portion of the scene having a final luminance. In one embodiment, each image generator channel applies a per-pixel, per-color light pollution correction to an output frame buffer in a post-processing step just prior to transmission to a display device. The above method is repeated for each subsequent image at the frame rate of the display system.

[0133] It should be understood that the inventive concept covers any embodiment combined with any one or more other embodiments, any one or more features disclosed herein, any one or more features as substantially disclosed herein in combination with any one or more other features as substantially disclosed herein, any one of the aspects / features / embodiments in combination with any one or more other aspects / features / embodiments, and the use of any one or more embodiments or features as disclosed herein. It should be understood that any feature described herein can be claimed in combination with any other feature described herein, regardless of whether these features are from the same described embodiment. A system on a chip (SoC) includes any one or more of the above aspects.

[0134] The example embodiment may be configured as follows:

[0135] (1) A method for reducing light pollution in a displayed image, the method comprising:

[0136] receiving a portion of a scene to be displayed on a display at an initial brightness;

[0137] adjusting the initial brightness of the portion of the scene to a final brightness based on a light pollution model associated with the display; and

[0138] The scene is rendered to the display with the portion of the scene having the final brightness.

[0139] (2) The method according to (1), further comprising:

[0140] The light pollution model is generated based on at least one calibration operation.

[0141] (3) The method according to one or more of (1) to (2), wherein the at least one calibration operation comprises:

[0142] displaying a calibration image on the display;

[0143] capturing the calibration image displayed on the display using one or more image sensors; and

[0144] Light contamination of one or more regions of the calibration image is determined based on the captured calibration image.

[0145] (4) The method according to one or more of (1) to (3), wherein the calibration image comprises a black and white image having a repeating pattern.

[0146] (5) The method of one or more of (1) to (4), wherein determining light contamination of the one or more regions of the calibration image comprises:

[0147] A scatter coefficient is determined for the one or more regions of the captured calibration image.

[0148] (6) The method of one or more of (1) to (5), wherein each region of the captured calibration image corresponds to a single pixel of the display.

[0149] (7) The method according to one or more of (1) to (6), further comprising:

[0150] A correction factor is generated for each region of the calibration image based on the determined scatter coefficient.

[0151] (8) The method according to one or more of (1) to (7), wherein the correction factor comprises a tone mapping operator for brightness adjustment.

[0152] (9) The method of one or more of (1) to (8), wherein adjusting the initial brightness of the portion of the scene to the final brightness based on the light pollution model comprises:

[0153] selecting a correction factor based on a brightness threshold and the initial brightness; and

[0154] The selected correction factor is applied to the portion of the scene to adjust the initial brightness to the final brightness.

[0155] (10) A method according to one or more of (1) to (9), wherein selecting the correction factor includes: when the initial brightness is above the brightness threshold, selecting a first correction factor that reduces the initial brightness of the portion of the scene by a first amount to the final brightness, the first amount being substantially equal to an amount of light pollution predicted by the light pollution model for the portion of the scene.

[0156] (11) A method according to one or more of (1) to (10), wherein selecting the correction factor includes: when the initial brightness is at or below the brightness threshold, selecting a second correction factor that reduces the initial brightness by a second amount that is less than the amount of light pollution predicted by the light pollution model for the portion of the scene, the second correction factor being different from the first correction factor.

[0157] (12) The method of one or more of (1) to (11), wherein the second amount becomes lower as the initial brightness moves further below a brightness threshold.

[0158] (13) The method of one or more of (1) to (12), wherein the portion of the scene corresponds to a single pixel.

[0159] (14) A device for reducing light pollution in a displayed image, the device comprising:

[0160] A processing circuit, the processing circuit being configured to:

[0161] receiving a portion of a scene to be displayed on a display at an initial brightness;

[0162] adjusting the initial brightness of the portion of the scene to a final brightness based on a light pollution model associated with the display; and

[0163] The scene is rendered to the display with the portion of the scene having the final brightness.

[0164] (15) The apparatus of (14), wherein the processing circuit adjusts the initial brightness of the portion of the scene to the final brightness based on the light pollution model by:

[0165] selecting a correction factor based on a brightness threshold and the initial brightness; and

[0166] The selected correction factor is applied to the portion of the scene to adjust the initial brightness to the final brightness.

[0167] (16) An apparatus according to one or more of (14) to (15), wherein selecting the correction factor includes: when the initial brightness is at or below the brightness threshold, selecting a first correction factor that reduces the initial brightness by a first amount that is less than the amount of light pollution predicted by the light pollution model for the portion of the scene.

[0168] (17) The apparatus of one or more of (14) to (16), wherein the first amount becomes lower as the initial brightness moves further below the brightness threshold.

[0169] (18) An apparatus according to one or more of (14) to (17), wherein selecting the correction factor includes: when the initial brightness is above the brightness threshold, selecting a second correction factor, the second correction factor reducing the initial brightness of the portion of the scene by a second amount to the final brightness, the second amount being substantially equal to the amount of light pollution predicted by the light pollution model for the portion of the scene.

[0170] (19) The apparatus of one or more of (14) to (18), wherein the portion of the scene corresponds to a single pixel of the display.

[0171] (20) A system for reducing light pollution in a displayed image, the system comprising:

[0172] A processing circuit, the processing circuit being configured to:

[0173] determining an initial brightness of a pixel in each of a plurality of frames of a video signal to be displayed on a display;

[0174] adjusting the initial brightness of pixels in each frame to a final brightness based on a light pollution model on a frame-by-frame basis, the light pollution model compensating for light pollution that occurs when each frame is displayed on a display; and

[0175] Each frame is rendered to the display with the pixels having the final brightness.

[0176] (21) A method for calibrating a display system, the method comprising:

[0177] displaying the calibration image on a display;

[0178] capturing the calibration image displayed on the display using one or more image sensors; and

[0179] determining light contamination of a region of the calibration image based on the captured calibration image; and

[0180] A light pollution model is generated based on the determined light pollution of the area.

[0181] (22) The method according to (21), wherein the calibration image comprises a black and white image having a repeating pattern.

[0182] (23) The method of one or more of (21) to (22), wherein determining the light pollution of the area of ​​the calibration image comprises:

[0183] The scattering coefficient of the region of the captured calibration image is determined.

[0184] (24) The method of one or more of (21) to (23), wherein each region of the captured calibration image corresponds to a single pixel of the display.

[0185] (25) A system comprising:

[0186] Screen;

[0187] one or more projectors that project images onto the screen;

[0188] one or more image sensors; and

[0189] Image processing circuit for:

[0190] rendering a calibration image to the screen using the one or more projectors;

[0191] controlling the one or more image sensors to capture the calibration image displayed on the screen; and

[0192] determining light contamination of a region of the calibration image based on the captured calibration image; and

[0193] A light pollution model is generated based on the determined light pollution of the area.

[0194] (26) The system of (25), wherein the calibration image comprises a black and white image having a repeating pattern.

[0195] (27) The system of one or more of (25) to (26), wherein determining the light contamination of the region of the calibration image comprises:

[0196] The scattering coefficient of the region of the captured calibration image is determined.

[0197] (28) The system of one or more of (25) to (27), wherein each region of the captured calibration image corresponds to a single pixel of the screen.

Claims

1. A method for reducing light pollution in a displayed image, the method comprising: receiving a portion of a scene to be displayed on a display at an initial brightness; adjusting the initial brightness of the portion of the scene to a final brightness based on a light pollution model associated with the display; as well as The scene is rendered to the display with the portion of the scene having the final brightness.

2. The method according to claim 1, further comprising: The light pollution model is generated based on at least one calibration operation.

3. The method according to claim 2, wherein: The at least one calibration operation comprises: displaying a calibration image on the display; capturing the calibration image displayed on the display using one or more image sensors; and Light contamination of one or more regions of the calibration image is determined based on the captured calibration image.

4. The method according to claim 3, wherein: The calibration image includes a black and white image having a repeating pattern.

5. The method according to claim 3, wherein: Determining the light contamination of the one or more regions of the calibration image comprises: A scatter coefficient is determined for the one or more regions of the captured calibration image.

6. The method according to any one of claims 4 to 5, wherein: Each region of the captured calibration image corresponds to a single pixel of the display.

7. The method according to claim 5, further comprising: A correction factor is generated for each region of the calibration image based on the determined scatter coefficient.

8. The method according to claim 7, wherein: The correction factors include a tone mapping operator for brightness adjustment.

9. The method according to any one of claims 1 to 5, wherein: Adjusting the initial brightness of the portion of the scene to the final brightness based on the light pollution model includes: selecting a correction factor based on a brightness threshold and the initial brightness; and The selected correction factor is applied to the portion of the scene to adjust the initial brightness to the final brightness.

10. The method according to claim 9, wherein: Selecting the correction factor includes: when the initial brightness is above the brightness threshold, selecting a first correction factor that reduces the initial brightness of the portion of the scene by a first amount to the final brightness, the first amount being substantially equal to an amount of light pollution predicted by the light pollution model for the portion of the scene.

11. The method according to claim 10, wherein: Selecting the correction factor includes: when the initial brightness is at or below the brightness threshold, selecting a second correction factor that reduces the initial brightness by a second amount that is less than the amount of light pollution predicted by the light pollution model for the portion of the scene, the second correction factor being different from the first correction factor.

12. The method according to claim 11, wherein: As the initial brightness moves further below the brightness threshold, the second amount becomes lower.

13. The method according to any one of claims 1 to 12, wherein: The portion of the scene corresponds to a single pixel.

14. An apparatus for reducing light pollution in a displayed image, the apparatus comprising: A processing circuit, the processing circuit being configured to: receiving a portion of a scene to be displayed on a display at an initial brightness; adjusting the initial brightness of the portion of the scene to a final brightness based on a light pollution model associated with the display; as well as The scene is rendered to the display with the portion of the scene having the final brightness.

15. The device according to claim 14, wherein: The processing circuit adjusts the initial brightness of the portion of the scene to the final brightness based on the light pollution model by: selecting a correction factor based on a brightness threshold and the initial brightness; as well as The selected correction factor is applied to the portion of the scene to adjust the initial brightness to the final brightness.

16. The device according to claim 15, wherein: Selecting the correction factor includes, when the initial brightness is at or below the brightness threshold, selecting a first correction factor that reduces the initial brightness by a first amount that is less than an amount of light pollution predicted by the light pollution model for the portion of the scene.

17. The device according to claim 16, wherein: As the initial brightness moves further below the brightness threshold, the first amount becomes lower.

18. The device according to claim 16, wherein: Selecting the correction factor includes: when the initial brightness is above the brightness threshold, selecting a second correction factor that reduces the initial brightness of the portion of the scene by a second amount to the final brightness, the second amount being substantially equal to an amount of light pollution predicted by the light pollution model for the portion of the scene.

19. The device according to any one of claims 14 to 18, wherein: The portion of the scene corresponds to a single pixel of the display.

20. A system for reducing light pollution in a displayed image, the system comprising: A processing circuit, the processing circuit being configured to: determining an initial brightness of a pixel in each of a plurality of frames of a video signal to be displayed on a display; adjusting an initial brightness of pixels in each frame to a final brightness based on a light pollution model frame by frame, the light pollution model compensating for light pollution that occurs when each frame is displayed on a display; as well as Each frame is rendered to the display with the pixels having the final brightness.

Images