LED panel control method and system

Abstract

A computer-implemented method for converting received red green blue (RGB) video data into red green blue and fourth color (RGBF) LED drive signals is disclosed. The method may include receiving RGB video data representing a video including a plurality of frames. Each frame may include a plurality of pixels. For each pixel of each frame of the video, the RGB video data may define R, G, and B brightnesses for representing the color and brightness of the pixel. The method may also include generating an RGBF LED drive signal according to the received RGB video data. The RGBF LED drive signal may be configured for driving one or more LED modules including a plurality of LED units. Each LED unit may represent a pixel. Each LED unit may include a red LED, a green LED, a blue LED, and a fourth color LED different in color from the red LED, the green LED, or the blue LED. The generating may also include applying a control parameter to adjust a characteristic of the RGBF LED drive signal.

Description

Technical Field

[0001] This disclosure relates to methods and systems for controlling LED panels. More specifically, but not exclusively, methods for controlling the characteristics of video panels are disclosed. For example, methods for increasing brightness and changing spectral characteristics are disclosed. Background Technology

[0002] A standard LED video screen consists of a tightly packed array of pixels, where each pixel comprises red, green, and blue (RGB) LEDs. The standard video signal also contains information about the brightness of each red, green, and blue element for each pixel, thus providing a simple mapping of video data to the screen. More complex LED video screens may also include a processing system that adjusts the input video signal to allow for calibration, temperature, and other variable parameters specific to the LEDs and screen structure. Such a system helps to display colors faithfully to the input signal and matched to standards for those signals.

[0003] While LED screens have historically been used for large-scale live outdoor events (such as at concerts or music festivals) and for advertising (such as at trade shows), their use as set backdrops in film and television productions has become increasingly popular in recent years. In such applications, LED screens can serve a dual purpose, especially when the screen extends beyond the field of view of the camera and, in some cases, encloses the set. First, one or more LED screens act as the primary backdrop, allowing dynamic background content to be displayed behind the subjects or actors being filmed. Second, the LED screens illuminate the subjects or people being filmed, including various lighting aspects of the physical set.

[0004] While RGB LEDs are the best choice for most direct-view applications, their spectral output is "sharp." Specifically, the radiation from each LED is focused near specific red, green, and blue wavelengths. This is undesirable if the screen is used for lighting purposes, as the gaps in the spectral output result in poor color rendering. For example, skin tones under RGB lighting often appear with a slightly reddish tint. The darker the skin tone of the illuminated subject, the more pronounced this problem may be. Therefore, in cases where LED screens are used for background and lighting purposes, these issues can lead to poor color quality in recorded footage. Furthermore, most LED screens used in film and television production are not specifically designed for such applications. Therefore, they are not designed to provide on-site engineers with control over the LED panels in a way that provides preferred technical control in film and television production environments. Summary of the Invention

[0005] A computer-implemented method is disclosed for converting received RGB video data into RGBF LED drive signals of a fourth color. The method may include receiving RGB video data representing a video comprising multiple frames. Each frame may include multiple pixels. For each pixel in each frame of the video, the RGB video data may define red luminance, green luminance, and blue luminance to represent the color and brightness of the pixel. The method may also include generating RGBF LED drive signals based on the received RGB video data. The RGBF LED drive signals may be configured to drive one or more LED modules comprising multiple LED units. Each LED unit may represent a pixel. Each LED unit may include a red LED, a green LED, a blue LED, and a fourth color LED of a different color than the red, green, or blue LEDs. The generation may also include applying control parameters to adjust the characteristics of the RGBF LED drive signals. The method enables more adaptive control of the LED panel. For example, color and / or brightness characteristics can be better controlled.

[0006] This characteristic can be a spectral characteristic. The control parameters can alter the spectral characteristic by controlling the relative brightness levels of two or more of the red, green, blue, and / or fourth-color LEDs. The control parameters can change the spectral characteristic while maintaining the color and brightness of the pixels as defined by RGB video data. This allows for the use of different ratios of LEDs to produce colors, which in turn improves control over the spectrum of emitted light.

[0007] When generating an RGBF drive signal, multiple different combinations of brightness levels for the red, green, blue, and fourth-color LEDs can be available. These combinations essentially produce the color and brightness for the corresponding pixels as defined by the received RGB video data. Control parameters allow selection of a single combination from these multiple options. These control parameters provide a simple method for controlling a complex range of color combinations.

[0008] The control parameters may indicate one of the following: (i) an instruction to cause one or more LED units of a plurality of LED units to prioritize the use of a fourth color LED over red, green, and blue LEDs to substantially produce colors and brightness defined by RGB video data; or (ii) an instruction to cause one or more LED units of a plurality of LED units to prioritize the use of red, green, and blue LEDs over a fourth color LED to substantially produce colors and brightness defined by RGB video data; or (iii) an instruction to cause one or more LED units of a plurality of LED units to balance the use of red, green, and blue LEDs relative to a fourth color LED with specific weights to substantially produce colors and brightness defined by RGB video data.

[0009] Control parameters may include or originate from information indicating one or more of the following: the content of the video represented by RGB video data; one or more objects to be illuminated by one or more LED units; one or more characteristics of the image capture device; and one or more preferences of the user operating the image capture device.

[0010] When the control parameters include information indicating one or more characteristics of the image capture device, the method may further include determining whether light emitted by one or more LED units should be directly captured by the image capture device. The method may also include generating control parameters to: prioritize the use of a fourth color LED over red, green, and blue LEDs to produce color and brightness defined by RGB video data if light emitted by one or more LED units will not be directly captured by the image capture device; and prioritize the use of red, green, and blue LEDs over the fourth color LED to produce color and brightness defined by RGB video data if light emitted by one or more LED units is to be directly captured by the image capture device. This allows selection of the light emitted by the LEDs to be designed for capture on a camera device or for providing background illumination. Some LEDs may be captured by the camera device across the entire screen, while others may provide background illumination. These roles may change as the camera device moves.

[0011] The method may also include receiving information indicating one or more of the following: the position, orientation, movement, or field of view of the image capturing device. Determining whether light emitted by one or more LEDs should be captured by the image capturing device can be based on the received information.

[0012] The generation process may also include determining whether the balance between (i) the combination of red, green, and blue LEDs and / or (ii) the use of a fourth color LED is sufficient to produce color and brightness for the corresponding pixel. If color and brightness cannot be produced, the balance between (i) the combination of red, green, and blue LEDs and / or (ii) the use of a fourth color LED can be adjusted to substantially produce color and brightness for the corresponding pixel.

[0013] When adjusting the balance between (i) the combination of red, green, and blue LEDs and / or (ii) the use of a fourth color LED, the balance can be adjusted by making it possible to produce essentially the minimum possible amount of color and brightness for the respective pixel.

[0014] The method may include determining a minimum drive signal for a fourth-color LED that achieves color and brightness for the corresponding LED cell while minimizing the drive level used by the fourth-color LED. The method may also include determining a maximum drive signal for the fourth-color LED that achieves color and brightness for the corresponding LED cell while maximizing the drive level used by the fourth-color LED. The minimum and maximum drive signals may define a range including a maximum and a minimum value. The method may include selecting a value from this range as the RGBF LED drive signal. The selection may involve performing interpolation between the minimum and maximum drive signals. The selection may correspond to control parameters.

[0015] The control parameters can indicate adjustments corresponding to one or more of the red, green, blue, or fourth color components of the RGBF drive signal. This adjustment can compensate for one or more characteristics of a non-standard observer viewing or recording the output of one or more LED units. These characteristics can be variations in perceived color and / or brightness caused by one or more of the observer's characteristics.

[0016] The method may include driving an LED unit using a first RGBF LED drive signal. The method may also include driving an LED unit using a second RGBF drive signal. Alternatively, the method may include driving a second LED unit using a second RGBF LED drive signal. The method may further include observing an LED unit driven by the first RGBF LED drive signal and the second RGBF LED drive signal, or observing an LED unit and a second LED unit driven by the first RGBF LED drive signal and the second RGBF LED drive signal, respectively. The method may further include determining an adjustment in response to observing the LED unit or the second LED unit while driven by the first RGBF LED drive signal and the second RGBF LED drive signal. The LED unit or the second LED unit may be observed by a human user via a screen forming an image from an image capture device monitoring the LED unit or the second LED unit, and the human user may provide input via a user interface to change the adjustment in response to a comparison of the LED unit driven by the first RGBF LED drive signal and the second RGBF LED drive signal, or a comparison of the LED unit driven by the first RGBF LED drive signal and the second LED unit driven by the second RGBF LED drive signal. The LED unit or the second LED unit can be observed by a computing device using data captured by an image capture device that monitors the LED unit or the second LED unit. The computing device can adjust its settings in response to a comparison of the LED unit when driven by the first RGBF LED drive signal and the second RGBF LED drive signal, or a comparison of the LED unit driven by the first RGBF drive signal and the second LED unit when driven by the second RGBF LED drive signal.

[0017] The method may further include driving the LED unit using a first RGBF LED driving signal. The method may further include driving the LED unit using a second RGBF LED driving signal. The method may further include observing the LED unit while it is driven by the first RGBF LED driving signal and the second RGBF LED driving signal. The method may further include determining an adjustment in response to observing the LED unit while it is driven by the first RGBF LED driving signal and the second RGBF LED driving signal. Therefore, the balance of different LEDs can be improved. In another arrangement, the first RGBF LED driving signal can drive a first LED unit, and the second RGBF LED driving signal can drive a second LED unit. Therefore, two different pixels can be driven simultaneously, rather than driving the same pixel sequentially in two different ways.

[0018] The LED unit can be observed by a human user when driven by a first RGBF LED drive signal and a second RGBF LED drive signal. This can be achieved via a screen that forms an image from an image capture device monitoring the LED unit. The human user can provide input via a user interface to adjust the LED unit in response to a comparison of the LED unit driven by the first RGBF LED drive signal and the second RGBF LED drive signal.

[0019] The LED unit can be observed by a computing device when driven by a first RGBF LED drive signal and a second RGBF LED drive signal. The computing device can use data captured by an image capture device monitoring the LED unit. The computing device can adjust the settings in response to a comparison of the LED unit driven by the first RGBF LED drive signal and the second RGBF LED drive signal. The computing device can adjust the settings via an iterative process.

[0020] The control parameters can be defined to increase the maximum brightness of one or more LED units represented in the RGBF LED drive signal when compared with the maximum achievable brightness when the LED unit only includes red, green, and blue LEDs.

[0021] Generating an RGBF LED drive signal may further include increasing the brightness of one or more LED units represented in the drive signal based on control parameters. The brightness of one or more LED units can be increased by combining red, green, and blue LEDs with a fourth color LED, while still substantially producing colors defined by the RGB video data. If the resulting brightness represented in the RGBF LED drive signal exceeds the achievable brightness level of the one or more LED units of the RGBF LEDs, the brightness in the RGBF LED drive signal can be set to substantially the maximum achievable brightness, which enables substantially the color of the corresponding pixel in the received RGB video data to be achieved.

[0022] Improved control of brightness can be achieved through some of the methods disclosed in this paper.

[0023] The fourth color LED can be a white LED. The method may also include a fifth LED, wherein the fourth and fifth LEDs are selected from: white, cool white, warm white, stone gray, cyan, indigo, yellow, amber, and magenta. Control parameters can be changed in real time. Control parameters can be changed per frame and / or per pixel. Control parameters can be adjusted by the user via one or more graphical user interface elements. Generation may also include consideration of calibration data. Calibration data can be one or more of the following: per-pixel calibration data, per-region calibration data, and RGBF calibration data. Control parameters can be used to adjust the calibration data.

[0024] A computer program product comprising computer-readable code arranged to implement any of the methods disclosed herein is disclosed. The computer program product may be transient or non-transient.

[0025] A system comprising one or more processors arranged to perform any of the methods disclosed herein is disclosed. The one or more processors of the system may include at least one central processing unit (CPU). The one or more processors of the system may include multiple distributed processors. Each distributed processor may be associated with one or more LED units. Any methods disclosed herein can be performed by multiple distributed processors, and at least one CPU can distribute RGB video data to be processed to the multiple distributed processors.

[0026] The system may also include an array of LED modules, each LED module including one or more LED drivers and multiple RGBF LED units, each RGBF LED unit may include red LEDs, green LEDs, blue LEDs and a fourth color LED.

[0027] A computer-implemented method is disclosed for providing a preference between using the red-green-blue components and using the fourth color component when converting received RGB video data into RGB and fourth-color RGBF LED drive signals. The method may include receiving RGB video data representing a video comprising multiple frames. Each frame may include multiple pixels. For each pixel of each frame of the video, the RGB video data may define red luminance, green luminance, and blue luminance to represent colors having brightness. The method may further include generating RGBF LED drive signals based on the received RGB video data, wherein the RGBF LED drive signals may be configured to drive one or more LED modules comprising multiple LED units. Each LED unit may represent a pixel and include a red LED, a green LED, a blue LED, and a fourth-color LED of a different color than the red, green, or blue LED. The generation may further include balancing between (i) a combination of red, green, and blue LEDs and / or (ii) the brightness level of the fourth-color LED while generating the color and brightness for the corresponding pixel defined by the received RGB video data.

[0028] Alternatively, in a computer-implemented method for adjusting the color characteristics of video when viewed by a non-standard observer, the generation may also include adjusting one or more of the red, green, blue, or fourth color components of the RGBF drive signal. Such adjustment can compensate for one or more characteristics of a non-standard observer who may be viewing or recording the output of one or more LED units.

[0029] In another alternative computer-implemented method for enhancing the brightness of one or more aspects of a video displayed on one or more LED units, the generation may further include allowing a brightness enhancement. The brightness enhancement can be achieved by increasing the brightness of an equivalent pixel from a plurality of pixels in RGB video data by the brightness of a pixel represented in an RGBF LED drive signal.

[0030] A method for altering the spectral characteristics of light emitted by an LED unit is also disclosed. The method may include determining a preference between the spectral characteristics of one or more of a red, green, and blue LED, and the spectral characteristics of a preferred fourth emitter, for producing or substantially producing color. The preference may be determined based on whether the light emitted by the LED unit is directly captured by an image capturing device. Alternatively, the preference may be determined based on whether light reflected from one or more objects emitted by the LED unit is to be captured. One or more of the field of view, position, movement, or orientation of the image capturing device may form a defined component.

[0031] A method for adjusting the spectral characteristics of light emitted by an LED unit to adjust how an image capture device perceives the LED unit is also disclosed. The LED unit may include a red LED, a green LED, a blue LED, and a fourth-color LED. The method may further include adjusting one of the red LED, green LED, blue LED, or fourth-color LED. The adjustment may be performed in response to how brightness and / or color is perceived at the image capture device. The method may further include correcting how brightness and / or color is perceived at the image capture device by providing an adjustment to the spectral characteristics, and capturing the adjusted perceived brightness and / or color. A computer or a human may compare the captured adjusted perceived brightness and / or color. When compared by a human, the adjusted perceived brightness and / or color captured by the image capture device may be displayed on a screen.

[0032] A method for increasing the brightness of one or more portions or components of an LED panel is also disclosed. The method may include receiving an indication of a desired maximum brightness. This indication may be directed to one or more portions or components of the LED panel. The method then further includes adjusting the brightness of one or more portions or components of one or more drive signals driving the equivalent portion or component of the LED panel.

[0033] Another method for increasing the brightness of one or more portions or components of an LED panel is disclosed. The LED panel uses red LEDs, green LEDs, blue LEDs, and a fourth-color LED, wherein the fourth-color LED is a different color from the other LEDs. The method involves increasing the brightness of one or more portions, components, or pixels of the LED panel by using a combination of red LEDs, green LEDs, blue LEDs, and the fourth-color LED to produce a desired output color. The method may involve receiving an indication of a desired brightness increase from a user. The desired brightness increase may be set to a maximum possible brightness increase. The method may also include determining possible brightness increases by considering the maximum brightness increase and the given desired output color. Attached Figure Description

[0034] An exemplary arrangement of this disclosure will now be described with reference to the accompanying drawings, in which:

[0035] Figure 1 An RGBF LED array and associated processing equipment are shown;

[0036] Figure 2 A GUI slider for controlling spectral preferences is shown;

[0037] Figure 3 This demonstrates a first example of converting an RGB video input into an RGBF LED output incorporating spectral preferences;

[0038] Figure 4 A second example of converting an RGB video input into an RGBF LED output incorporating spectral preferences is shown;

[0039] Figure 5 A third example of converting an RGB video input into an RGBF LED output incorporating spectral preferences is shown;

[0040] Figure 6 An RGBF processing unit for incorporating spectral preferences is shown when generating RGBF drive levels;

[0041] Figure 7 An alternative RGBF processing unit for incorporating spectral preference is shown when generating RGBF drive levels;

[0042] Figure 8 A GUI slider for controlling spectral enhancement is shown;

[0043] Figure 9 An example of converting HDR RGB video input to RGB LED output is shown;

[0044] Figure 10 This demonstrates a first example of converting an HDR RGB video input into an incorporated spectral-enhanced RGBF LED output;

[0045] Figure 11 A second example of converting an HDR RGB video input into an incorporated spectral-enhanced RGBF LED output is shown;

[0046] Figure 12 A third example of converting HDR RGB video input into an incorporated spectral-enhanced RGBF LED output is shown;

[0047] Figure 13 The process for applying spectral enhancement while calibrating each pixel is illustrated;

[0048] Figure 14 This demonstrates the difference in perception of the color "orange" between standard and non-standard observers;

[0049] Figure 15 The difference in perception of the color "orange" between standard and non-standard observers is shown when using RGB and RGBF LEDs;

[0050] Figure 16 This demonstrates how to control spectral balance using a GUI slider;

[0051] Figure 17This demonstrates the effect of spectral balance on the perception of orange by non-standard observers, produced by RGB LEDs and RGBF LEDs.

[0052] Figure 18 This illustrates the process of applying spectral balance control to calibration data when generating RGBF drive levels from RGB video data; and

[0053] Figure 19 A system on which the processes disclosed herein can be operated is shown.

[0054] Throughout the description and accompanying drawings, the same reference numerals refer to the same parts. Detailed Implementation

[0055] It has been found that many of the technical problems and drawbacks associated with the use of RGB LED panels in film and television production environments can be substantially reduced or eliminated by adding additional non-RGB emitters, such as white emitters with a much wider spectral output. Alternatively, narrowband emitters such as amber or cyan can be selected to help “fill” the gaps in the spectral output. Therefore, throughout this disclosure, references to such systems are referred to as “RGBF,” where “F” stands for the “fourth” channel, which can be white or virtually any other color. In fact, in some arrangements, additional emitters can also be combined to provide even wider spectral coverage from said emitters, so “F” can also be considered as one or more additional emitters.

[0056] Regardless of the exact choice of the fourth transmitter, its addition introduces several problems that must be overcome. These problems are unknown, but are first identified herein. Therefore, this disclosure not only relates to the introduction of the fourth transmitter, but also provides solutions to the problems associated with its addition. Furthermore, improved methods and systems are disclosed for controlling such LED panels in a way that improves their use in a wide range of commercial applications, including film and television production.

[0057] One technical challenge with a fourth emitter is maintaining system calibration so that using the fourth emitter does not degrade the achievable color accuracy of the output (e.g., when measuring the CIE XYZ color coordinates of the output). Furthermore, even with the same CIE color point, viewing the screen with a "non-standard" viewer, such as a camera, can result in different colors being observed, depending on how much the viewer's spectral sensitivity deviates from the CIE standard observer color matching function (from which the XYZ colors are derived). This variation in observed color is undesirable and preferably compensated for.

[0058] For a conventional RGB system, a three-channel (RGB) video input signal is ultimately converted into a single set of RGB LED drive levels. These drive levels can take the form of one drive signal or multiple drive signals. It should be understood that multiple drive signals can be provided for each LED, or a single drive signal with multiple components for driving each LED can be provided. There is always only one correct solution for this conversion, so existing systems perform a simple mapping from the video input signal to RGB drive levels. However, for a three-channel (RGB) video input signal and a four-channel (RGBF) output, the choice of output level is an unconstrained problem. That is, there are many sets of RGBF drive levels (“metachromatic mixing”) that can produce the same perceived output color. For example, if the fourth emitter is a white LED, a white output can be achieved with 100% red, 100% green, 100% blue, and 0% white, or similarly with 50% red, 50% green, 50% blue, and 50% white, or alternatively with 0% red, 0% green, 0% blue, and 100% white. Different metamerism blends achieve the same color output but with different spectral contents. The optimal blend choice will depend on a range of factors, including the video content, the object to be illuminated (if any), the viewing device, and the user's preferences. Therefore, there is no single choice that will always produce the best blend for the four outputs. While this example focuses on the specific case of RGBW with the fourth emitter being white, the same issue applies to any color with the fourth emitter, as well as to a greater number of emitters. Therefore, high-quality RGBF video display systems must determine the optimal metamerism blend based on these additional factors.

[0059] Finally, the addition of a fourth emitter allows the system to achieve a brighter output for certain colors than it could without the fourth emitter. For example, an RGBW panel can output 100% red, 100% green, 100% blue, and 100% white to produce white light that is twice as bright as an equivalent RGB-only panel. In some cases, this would be appealing (e.g., for lighting purposes seeking maximum achievable brightness), but for video display devices, it conflicts with the color space definition of video signals, which requires peak white output to equal the sum of peak red, peak green, and peak blue outputs. Ignoring this imbalance and always displaying the brightest possible output results in an image where some areas (e.g., light colors for an RGBW panel) are rendered brighter than expected, while other areas (e.g., highly saturated colors for an RGBW panel) are rendered darker than expected, resulting in extremely poor overall image quality.

[0060] Figure 1 A simplified schematic representation of an RGBF-based video display system is shown. In this system, RGB video data 1 to be displayed is received at the RGBF processing unit 10. The RGB video data comprises video data from multiple frames, each frame represented by multiple pixels. Each pixel has associated red, green, and blue luminance levels. In some arrangements, compression techniques may be used to reduce the repetition of data per frame or per pixel across multiple frames. Additionally, the RGBF processing unit 10 receives spectral control data 2 (sometimes referred to as spectral control settings) and per-pixel RGBF calibration data 3, which calibrates the LEDs to compensate for the drive levels output by the RGBF processing unit 10 for the specific performance of each LED. The spectral control data 2 (also referred to as control parameters) can control aspects of the spectral content, such as aspects of color and luminance, as will be discussed in more detail. The calibration data 3 may contain information related to the luminance and / or color of the LEDs. It should be understood that the calibration data 3 may contain both direct measurements (e.g., the LED has a luminance of 1250 nits) and derived values ​​(e.g., the LED needs to be driven at 80% to achieve a target luminance). Furthermore, it should be understood that the derived values ​​of calibration data 3 may not directly imply actual brightness (or color), especially if the target value is unknown. However, calibration data 3 may be derived from some knowledge about brightness (or color), even if it is only a relative (not absolute) measurement. It should be understood that the RGBF calibration data contains information related to the brightness and / or color of each of the R, G, B, and F LEDs. The RGBF processing unit 10 converts the input RGB video data 1 into RGBF data to be output, while also applying spectral control data 2 and calibration data 3 to the input RGB video data 1. The RGBF processing unit 10 then outputs an RGBF LED drive level 4, which is sent to the LED array 20. The RGBF LED drive level 4 can be one drive signal or multiple drive signals. The LED array 20 comprises an array of multiple RGBF LED modules, such as... Figure 1 The description in Figure 19 This is discussed in more detail below. Multiple emitter colors forming a single LED unit can be contained within a single LED package, or a subset of emitters can be contained in a separate package, or multiple pixels with multiple emitter colors can be contained within a single package.

[0061] The RGBF processing unit 10 can support any fourth color emitter, maintain calibrated color accuracy when viewed through the eye or a camera, and provide real-time control over metamerism mixing for each pixel. Each of these technical advantages will now be discussed in terms of how they are implemented in the RGBF processing system disclosed herein.

[0062] To determine how to convert RGB video data to RGBF LED drive levels, parameters that can be controlled by the user or automatically need to be considered. These parameters can therefore be based on a specific application or even current factors related to specific timing within the application. Such parameters can be referred to as one or more control parameters. Therefore, the methods disclosed herein obtain one or more control parameters to inform the conversion of input RGB video data to the appropriate RGBF LED drive levels. These parameters can be changed in real time or kept static, depending on user needs per frame or even per pixel. Furthermore, one or more control parameters can represent spectral parameters, the purpose of which may be to adjust the spectral content of the light emitted from the RGBF LED in a way that cannot be described solely by the RGB video signal. Control parameters can identify a preferred balance between emitters or a priority order of certain emitters relative to others. While various parameters are disclosed individually, it should be understood that they can be combined in more complex implementations. In summary, the spectral control data parameters considered herein are:

[0063] Spectral preference - A parameter indicating the desired degree of use of a fourth emitter or additional emitters (not just RGB).

[0064] Spectral enhancement - A parameter indicating how much (if any) additional brightness is provided due to the use of four or more emitters (instead of three).

[0065] Spectral balance - A parameter that indicates how to adjust the RGBF drive level to maintain color accuracy when using a fourth or additional emitter and the screen is viewed by a non-standard or non-human observer (such as a camera or other optical inspection device).

[0066] Each of these parameters affects the characteristics of the LED drive signal that drives the LED to produce emitted light. More specifically, but not exclusively, each of these characteristics is a spectral characteristic. Therefore, each characteristic will change one aspect of the light spectrum. Some characteristics change the spectrum to increase brightness without changing color. Some characteristics change the spectrum to change a spectral aspect of color (e.g., energy distribution across the spectrum) without changing the perceived aspect of color. The operation of each of these control parameters will now be described in detail.

[0067] Spectral preference

[0068] As mentioned above, spectral preference is a parameter indicating whether a fourth emitter or additional emitters (not just RGB) should be used, or to what extent a fourth or additional emitter should be used with RGB. Therefore, spectral preference can be thought of as a way of prioritizing certain emitters to achieve the desired output. Spectral preference can also be thought of as a way of balancing each emitter to achieve the desired output. This parameter will now be discussed in detail. To simplify the explanation, it should be noted that the following description only describes the use of a fourth emitter; however, it will be understood that additional emitters can also be incorporated into alternative implementations.

[0069] For any given color and luminance specified by pixels in the RGB video input, multiple metamerism blends can exist using four output emitters capable of achieving the requested color and luminance. The spectral preference input consists of values ​​between 0% (preferring RGB) and 100% (preferring F), which can be different for each pixel. Figure 2 The illustration shows graphical user interface elements that allow users to control spectral preferences using sliders. (See reference...) Figure 2 Users can manually adjust the slider-type graphical user interface element between 0% (preferring RGB) and 100% (preferring F). It should be understood that other forms of user interface elements (from rotary motion elements to digital data input) can be provided to offer the desired functionality. Furthermore, although it is possible, for example, through... Figure 2 The slider controls the spectral preference input based on pixels, but it can also be set for the entire screen, a panel, a set of panels, or a set of pixels. Furthermore, while a scale from 0 to 100% is provided, it should be understood that any suitable scale can be used.

[0070] A value close to 0% (preferably RGB) might be suitable for situations where the LEDs are not providing illumination, as this maintains performance similar to a regular RGB-only panel (e.g., avoiding any unexpected color shifts when viewed by a camera). When a wider spectral output from the panel is needed (e.g., when used for illumination), a value close to 100% (preferably F) might be appropriate. For situations where both predictable behavior on a camera and a wider spectral output are desired, any value between these two can be chosen to achieve a mid-level of performance.

[0071] For example, such as Figure 3 As shown, if the fourth color F is white, a regular RGB video signal might require pixels to display a light orange hue. With a spectral preference of 0% (preferring RGB), the RGB signal can (despite calibration) be passed unaffected to the RGB-only emitter of the pixel, as this achieves the desired output without using the F emitter at all.

[0072] In the other extreme case of spectral preference, such as Figure 4 As shown, the video processor will add as many fourth white emitters as possible and adjust the levels of the RGB emitters accordingly to produce the same light orange color, but using as many broad-spectrum white emitters as possible and as few other narrow-spectrum RGB emitters as possible.

[0073] As a compromise, with a spectral preference of 50%, an F emitter should be used, but at a reduced level, requiring slightly more R, G, and B. This is in Figure 5 As shown in the figure. It should be understood that the values ​​given here for the spectral preference signal are merely exemplary and do not in any way limit how the signal should be formatted.

[0074] Outside of this specific example, the choice of the fourth color F is arbitrary and does not involve the simple assumption of R+G+B=W as in the prior art. Instead, a comprehensive analysis of the four colors of the transmitter, the desired target color from the RGB video signal, and the spectral preference signal must be performed to calculate the required metamerism for the four transmitters. This process is repeated individually for each pixel in the video display screen. In some implementations, the process is repeated at full video rate for each frame of the input video.

[0075] The requested color from the video input may be outside the color gamut achievable using only red, green, and blue emitters. This could be because the color space of the video input is wider than the achievable gamut of RGB LEDs, or because one or more video input levels are negative, indicating that the color is outside the input's RGB-only gamut. If a fourth color F (e.g., a white fourth emitter) is within the gamut of red, green, and blue LEDs, the overall achievable color gamut is not expanded by adding a fourth emitter. In this case, existing methods (e.g., clipping the negative video level to zero) would be needed to restore the color to the gamut.

[0076] However, if the fourth color F is outside the color gamut of red, green, and blue LEDs (e.g., for other narrow-spectrum emitters including but not limited to cyan), the requested color outside the RGB color gamut can be achieved, but only by using the fourth emitter. In this case, the system may have to partially or completely ignore the spectral preference to achieve the requested output color (therefore it is a preference, not a requirement). In this case, 0% spectral preference will use the minimum (but not zero) amount of fourth emitter necessary to achieve the requested color, while 100% spectral preference will increase the amount of fourth emitter as much as possible while still achieving the requested color. In some cases, the minimum and maximum amount of fourth emitter can be the same (e.g., if the requested color exactly matches the color of a fourth emitter outside the RGB color gamut, such that adding any red, green, or blue will cause the color output to deviate from the desired color shift), in which case the setting of the spectral preference control will have no effect on the RGBF drive level used by that pixel to display that particular color. In some cases, a subset of pixels may be characterized by a fourth color emitter within the RGB color gamut, while different subsets may be characterized by a fourth color emitter outside the RGB color gamut. The system needs to support both scenarios. In other words, if the spectral preference indicates that RGB LEDs are preferred, but the desired color is achieved only by using a fourth LED, then the minimum number of fourth LEDs will be used.

[0077] To achieve the above behavior, deploy Figure 6 The RGBF processing unit 10 is depicted as a functional component. Each pixel of the input RGB video data 1 is used at calculation unit 101 to calculate (non-negative) RGBFmin (to achieve the requested RGBF output of color and luminance while minimizing the F-drive level), taking into account per-pixel RGBF calibration data 2 to ensure colorimetric accuracy of the solution. Similarly, each pixel of the input RGB video data 1 is also used at calculation unit 102 to calculate (non-negative) RGBFmax (to achieve the requested RGBF output of color and luminance while maximizing the F-drive level). Per-pixel RGBF calibration data 2 can be taken into account to ensure colorimetric accuracy of the solution. Then, spectral preference control input 3 is used at calculation unit 103 to notify interpolation (e.g., linear interpolation) between two possible solutions to produce the final RGBF drive level. Other methods can be used to determine the final RGBF drive level based on the maximum and minimum values.

[0078] RGB video data 1, RGBF calibration data 2, and spectral preference 3 can vary from one pixel to the next. Compared to basic 2D, 2.5D, or 3D LUT-based implementations, this results in significantly higher computational complexity, but provides dynamic per-pixel control and color accuracy that such LUT-based implementations cannot achieve. Therefore, the quality of the light ultimately output from the LED panel is significantly improved.

[0079] A possible improvement to the above method is to calculate only the minimum and maximum (non-negative) F drive levels (excluding R, G, and B drive levels), then interpolate between these two F values, and only later calculate the necessary RGB drive levels for the final interpolated F values, such as... Figure 7 As shown in the diagram, the output of the interpolation calculation unit 103 is then input to the calculation unit 104, which calculates the RGB values ​​for the selected F based on the per-pixel calibration data 2. This method, which considers only the F-driving level instead of the RGBF-driving level, provides similar functionality while reducing the computational burden.

[0080] In other arrangements, spectral preferences can be automatically set and changed based on additional inputs. For example, spectral preferences can be set based on camera field-of-view data, panel position, input video pixel content, or by using mechanisms that analyze multiple regions of image video pixel content and differentiate between these regions (such as keying, masking, material ID, or object ID signals).

[0081] While the maximum and minimum RGBF signals have been discussed above, it should be understood that similar functionality can be achieved using any two data points. By employing both maximum and minimum values ​​to utilize the maximum range, the quality of the final result is improved. It should be understood that the maximum value can refer to the upper limit data point, and the minimum value can refer to the lower limit data point. Furthermore, the terms maximum and minimum do not need to be absolute maximum and absolute minimum values, but rather maximum and minimum values ​​chosen for the purposes of this processing.

[0082] Even when an LED unit includes more than four emitters, a single spectral control can still be provided. Such spectral control can implicitly have the same preference for all additional emitters. Alternatively, multiple spectral preference controls can exist, each for a subset of other emitters. For example, a panel with both warm white and cool white LEDs could have two separate spectral preference sliders: one for "prefer RGB <-> prefer white" and another for "prefer warm white <-> prefer cool white". Multiple RGB-to-RGBF conversions can then be performed, and the resulting mixtures can be created between these potential outputs using multiple interpolations.

[0083] Per-pixel RGBF calibration data can typically describe the performance of each individual LED pixel independently. However, to reduce processing complexity or the benefit of smaller data size, the granularity of the RGBF calibration data can alternatively be made "coarser" by describing the performance of multiple LED pixels together. For example, pixels can be grouped into pairs to halve the data size by storing each pair of RGBF calibration data, or all pixels within a rectangular area of ​​the display can be grouped to generate per-area RGBF calibration data. In these cases, the same RGBF calibration data is applied to each pair or all pixels within the area.

[0084] Spectral Enhancement

[0085] As previously mentioned, spectral boost is a parameter indicating how much (if any) additional brightness is provided due to the use of four emitters instead of three. In other words, spectral boost can be referred to as a brightness increase factor, or brightness boost, or brightness amplifier. This parameter will now be discussed in detail. To simplify the explanation, it should be noted that the following description only describes the use of a fourth emitter; however, it will be understood that additional emitters can be incorporated into alternative implementations.

[0086] The pixels that input RGB video data can describe brightness levels exceeding the capabilities of any given display. For example, high dynamic range content in PQ (Perceptual Quantizer) format can contain pixels with brightness up to 10,000 nits, while LED panels typically offer lower achievable brightness. For instance, typical brightness for LED panels is 500 to 2000 nits for indoor applications and 4000 to 8000 nits for outdoor applications. Furthermore, additional adjustments within the LED processing pipeline can increase any input level to even higher brightness levels, again exceeding the display's achievable brightness.

[0087] At least for some areas of the color gamut, the addition of a fourth emitter can provide additional brightness to the display. For example, adding a white LED would allow for the achievement of lighter colors with greater brightness by utilizing white LEDs along with red, green, and blue LEDs. Similarly, adding a cyan LED would allow for the achievement of near-cyan colors with greater brightness by utilizing cyan LEDs along with green and blue LEDs. The spectral boost input consists of values ​​between ×1 (no boost) and some higher limit (e.g., ×5, to allow for five times higher brightness), which can be different for each pixel.

[0088] Spectral enhancement can be achieved by implementing graphical user interface elements in software, such as... Figure 8The slider shown is used for operation. The user can then move the slider from the minimum spectral boost (×1 in this case) upwards to the maximum achievable spectral boost (×5 in this case). It should be understood that various other user controls can be provided to adjust this parameter, and the slider illustrates one way to achieve this functionality.

[0089] If the brightness of a quad-emitter RGBF monitor is intended to perform similarly to that of a standard triple-emitter RGB monitor, then a ×1 (no boost) spectral boost value might be appropriate. This has the benefit of maintaining a technically correct brightness relationship between red, green, blue, and white (and virtually all other possible colors), which is generally expected for RGB video monitors that receive RGB video signals. Therefore, when comparing areas with different colors and saturations in an image, the overall brightness level of the image will appear correct. If the output brightness of the RGBF monitor is paramount (perhaps because the RGBF monitor is projecting light onto a foreground object), then a substantially higher spectral boost value might be preferred, provided that relaxing the technically correct relationship between the maximum brightness of each color is acceptable. This might be the case for high dynamic range (HDR) video content, where typically the vast majority of pixels are at a substantially low brightness level (perhaps a few hundred nits), and only a small fraction of pixels are at extreme brightness levels (e.g., several thousand nits) to represent extremely bright objects (e.g., the sun, fire, a light bulb, or specular highlights). In each of these cases, extremely bright objects are typically relatively light colors (in most content, it is unusual for pixels to be both extremely bright and extremely saturated), which means that it is acceptable to relax the relationship between the maximum brightness of each color, since more saturated colors (where RGBW screens can achieve only lower brightness) always only need to be displayed at lower brightness, while lighter colors (where RGBW screens can achieve disproportionately higher brightness by using all four emitters simultaneously) effectively benefit from being able to achieve disproportionately higher brightness.

[0090] For example, such as Figure 9 As shown, the HDR RGB video signal can describe pixels that are light orange and exceptionally bright (exceeding the achievable brightness of an RGB panel). In this case, the expected behavior is likely that the RGB panel will limit the brightness of the pixels to the maximum achievable value while maintaining the same color point. Note that this maximum brightness has at least one of the three output channels at 100% maximum brightness.

[0091] For RGBF panels with a spectral boost setting of ×1 (no boost), while the panel can achieve the desired color in any number of ways (depending on spectral preference), in all cases, the brightness should match the brightness achievable on an RGB panel. For example, as Figure 10 As shown, if the fourth emitter is white, the RGB drive level can be significantly reduced and the F drive level can be increased to achieve the exact same output color and brightness as the RGB panel.

[0092] Note that in this case, none of the output drive levels are at 100%, so there is a considerable potential to achieve a brighter output. Therefore, if the spectral boost is set to a higher value (e.g., ×2), the drive levels can be increased to achieve the same color at a brightness level that is possible with an RGB-only panel. Figure 11 In this context, with the maximum video input level at 150%, any spectral boost value of ×1.5 or higher will achieve the same result because spectral boost effectively sets a limit on how much brightness the output is allowed to achieve. Therefore, if the input "only" requires 150% brightness, any limit corresponding to that brightness or higher will make the input brightness achievable.

[0093] A particularly bright video input combined with a sufficiently high spectral boost value can result in RGBF drive levels exceeding 100%, meaning the panel cannot achieve the desired brightness even using all four emitters. In this case, the expected behavior is for the RGBF panel to limit the brightness of each pixel to its maximum achievable value while maintaining consistent color. Note that this maximum brightness has at least one of the four output channels at 100% maximum brightness, such as... Figure 12 As shown in the figure.

[0094] To achieve the spectral enhancement function described above, it is possible to implement... Figure 13The process involves inputting RGB video levels in "extended range," meaning each channel may exceed 100%. In the first step 131, these levels are compared pixel-by-pixel with spectral boost values ​​to see if the red, green, or blue levels exceed the boost value. In this description, video levels are given as percentages, while spectral boost values ​​are given as coefficients; therefore, for the purposes of this comparison, 100% is equivalent to ×1, 150% is equivalent to ×1.5, and so on. If any of the video levels does exceed the boost value for a given pixel, all three video levels for that pixel are scaled by a coefficient of boost / Max(R, G, B) such that the highest level in the resulting extended range RGB video data cannot exceed the boost value. For example, with a boost value of ×1.5, the extended range RGB video data level cannot exceed 150%, which is one and a half times brighter than what a regular RGB panel would be able to achieve, matching the specified boost value.

[0095] In the next step 132, these extended-range RGB video levels are converted to extended-range RGBF video levels, such that the resulting extended-range RGBF levels achieve the same color and brightness as the extended-range RGB levels. This conversion can select any metamerism blend that meets this requirement, for example, optionally using a spectral preference input to determine which specific blend to use. The nature of extended-range conversion is similar to standard range conversion; the only difference is that it allows input and output values ​​(and therefore intermediate values) to exceed 100%.

[0096] Finally, at step 133, since it is impossible to drive the RGBF LED at a level exceeding 100%, a check must be performed to test whether any of the four levels exceeds 100%. If so, then all four channels must be scaled by a factor of 1 / Max(R, G, B, F) so that the highest level in the resulting RGBF output cannot exceed 100%, and thus the final value is now in the standard range rather than the extended range.

[0097] Note that the extended-range RGB video data input and spectral boost values ​​can vary from one pixel to the next. Compared to basic 2D, 2.5D, or 3D LUT-based implementations, this results in significantly higher computational complexity, but provides dynamic per-pixel control over the maximum permissible brightness, which is not achievable with such LUT-based implementations.

[0098] While the above description of spectral enhancement considers an LED unit with four emitters, spectral enhancement can also be applied to LED units with five or more emitters using a similar principle. This allows the brightness obtained when using at least one of a fourth, fifth, or additional emitter, possibly in combination with red, green, or blue emitters, to exceed the brightness obtained from using only red, green, and blue emitters. This can be achieved by applying... Figure 13 The same process is used, but step 132 uses per-pixel RGBFX calibration data (where X represents the fifth and any other emitter) to perform extended-range RGB to extended-range RGBFX conversion to produce extended-range RGBFX video data for step 133, where the maximum value of all five or more channels is used to reduce all five or more channels, ultimately producing standard-range RGBFX video data.

[0099] Spectral balance

[0100] As already mentioned, spectral balance is one (or more) parameters that indicate how the RGBF drive levels should be adjusted to maintain color accuracy when using a fourth emitter. This can be particularly important when the screen is viewed by non-standard or non-human observers, such as camera devices or other optical inspection equipment. This is because image capture devices, such as cameras, can distort characteristics such as color and brightness. Therefore, color and brightness compensation may be necessary to balance or calibrate for variations in color and / or brightness caused by the image capture device.

[0101] The processing of RGBF panels can employ an algorithmic approach that allows the use of a fourth LED while maintaining color accuracy for a specific type of observer (e.g., a standard observer defined by the CIE color matching function (or some similarly well-defined observer spectral sensitivity)). For such a standard observer, metamerism for the same color will be perceived as having a similar appearance. However, non-standard observers (e.g., camera systems) can be used, where the viewer's spectral sensitivity differs from that of any standard observer used by the LED processing in determining metamerism. In this case, the non-standard observer will perceive different colors compared to the standard observer. For example, if the fourth emitter is white, these principles apply... Figure 14 As shown in the example, the output is perceived as the correct color by a standard observer, while a non-standard observer perceives it as a different color containing more red.

[0102] In many applications, it is desirable to prioritize the perception of a non-standard observer over that of a standard observer. For example, prioritizing color accuracy on a camera device over color accuracy when viewed by the eye. This can thus be considered non-standard observer calibration.

[0103] In this situation, a mechanism is needed to quantify the non-standard relationship between the perception of the fourth emitter and the RGB emitter. This could be achieved by directly measuring the spectral output of the LED and the spectral sensitivity of the non-standard observer, but the tools required to do so are often unavailable in the "field" where such a problem occurs. Alternatively, two or more metameris mixtures for the same color could be displayed on the LED panel, and the user could be provided with some controls to adjust the appearance of one or two mixtures until they are perceived as identical by a non-standard observer. Figure 15 The diagram illustrates the colors perceived by a non-standard observer (e.g., a camera device) as RGB metamerism and RGBF metamerism. While any two mixtures with different spectral outputs for the same color can be used, it is desirable that the two mixtures be as spectrally distinct as possible, as this maximizes the visibility of perceived color errors.

[0104] Then, users can be provided with controls to adjust the R / G / B balance when the fourth transmitter is in use. For example, such as Figure 16 As shown, three graphical user interface sliders can be provided to the user, one for each of red, green, and blue. The sliders can be adjusted from a center zero position (no adjustment provided) upwards to a positive 100% position and downwards to a negative -100% position. For a given channel (red, green, or blue), a positive spectral balance value typically adjusts the output to include a higher drive value for that channel, while a negative value typically results in a lower drive value for that channel, all provided that a fourth emitter is in use. The reverse can also be true if the F-emitter color is outside the RGB emitter gamut. The greater the contribution of the fourth emitter to the output, the more adaptable these adjustments become. For example, in... Figure 16 As shown in the controls, setting the red spectral balance to -30% may alter the panel's RGBF output, such as... Figure 17 As shown in the diagram.

[0105] for Figure 17Metamerism Mix 1, applying a -30% spectral balance to red does not change the output because the fourth channel is unused. However, for Metamerism Mix 2, the fourth channel is active, and by default (without any spectral balance), the red drive level is reduced by an amount to maintain the overall color and brightness expected by a standard observer. However, for a specific non-standard observer—who perceives the spectral content as too "reddish" compared to Metamerism Mix 1—specifying a -30% spectral balance for red instructs the system to reduce the perceived amount of red. Therefore, the brightness of the red LED is further reduced by an appropriate amount so that non-standard observers can correctly perceive the same color and brightness for both metamerism mixes. This can be achieved by scaling the red reduction by (100% - Rsb), where Rsb is the red spectral balance. For example, if for a standard observer, increasing the F by a certain amount requires the red channel to decrease from 0.75 to 0.35 (i.e., a decrease of 0.4), then for a non-standard observer, the -30% spectral balance for red would result in a decrease of 0.4 × (100% - (-30%)) (which is 0.52). Therefore, the final result for the red channel is 0.75 - 0.52 = 0.23.

[0106] While the example above illustrates a potential mapping from spectral balance parameter values ​​to the desired scaling of adjustments to the R / G / B channels, different mappings can be used to achieve similar effects.

[0107] Although the example above only considers the red channel, similar spectral balance adjustments can be performed on the green and blue channels.

[0108] To determine the correct values ​​for the three spectral balance controls, changes to these controls should typically be updated in real time with the metamerism from the panel, allowing users to check the output from non-standard observers (e.g., watching camera output on a monitor) and adjust the controls until the two mixtures look identical.

[0109] While the example here uses RGB controls to adjust the spectral balance, any other set of controls that provides similar functionality (such as hue, saturation, and brightness controls) can be used for adjustments, where the resulting values ​​are converted back to R, G, and B control values. Although the example here uses white as the fourth emitter, the fourth emitter does not need to be any particular color or brightness.

[0110] Additionally, more than one metamerism mixture can be displayed for a single color, and / or multiple colors (with two or more metamerism mixtures for each color) can be displayed to provide a large number of visual comparisons. The user can then manually adjust the spectral balance for all these colors simultaneously, or adjust the spectral balance independently for each color, where known methods such as averaging or least squares fitting are used to determine the final spectral balance from these multiple spectral balance inputs.

[0111] Alternatively, a closed-loop automation system can examine outputs from non-standard observers (e.g., measuring color and brightness levels in different regions of a video signal from a camera device) and automatically adjust spectral balance controls iteratively to minimize perceptual differences between metamerism mixtures without further user input.

[0112] To achieve the application of spectral balance adjustment, it is possible to realize... Figure 18 The process is illustrated below. First, at step 181, red, green, and blue spectral balance control values ​​are obtained and used to adjust the per-pixel RGBF calibration data. Depending on the nature of the calibration data format, this adjustment may take the form of scaling a subset of the calibration data using coefficients derived from the red spectral balance values, scaling a different subset using coefficients derived from the green spectral balance values, and scaling yet another subset using coefficients derived from the blue spectral balance values. While various adjustments may be appropriate depending on the format of the calibration data, all such adjustments are intended to adapt the per-pixel calibration to achieve better performance for a particular non-standard observer. Then, at step 182, the resulting “spectrally balanced” per-pixel RGBF calibration data can be used to inform a calibration conversion from the RGB video input level to the appropriate RGBF drive level, which will achieve the desired color and brightness when viewed by a non-standard observer.

[0113] Equivalently, the two stages of this process can be combined (e.g., in an optimized implementation with reduced computational complexity) so that the spectral balance value is directly applied to the conversion stage. However, the resulting effect on the output remains unchanged, because the spectral balance value allows the system to achieve better color accuracy when the four-emitter LEDs are viewed by a non-standard observer. Note that the RGB video input and the spectral balance value can vary from pixel to pixel, providing dynamic per-pixel control over how the output from each pixel on each video frame should be adjusted to achieve better color accuracy for a non-standard observer, and this observer can also vary over time or for different areas of the screen.

[0114] While the goal is to ensure that the colors output by the LED panel are exactly the same as those in the received RGB video data, it should be understood that producing similar colors is considered acceptable in other arrangements. For example, the aim may be to achieve colors that are as close as possible to the intended colors.

[0115] While the previous description of spectral balancing considered LED units with four emitters, spectral balancing can also be applied to LED units with five or more emitters using similar principles, ensuring color consistency for non-standard observers when using any combination of five or more emitters. A set of red, green, and blue (or equivalent) spectral balancing controls can be provided for each of the fourth, fifth, and any other emitters, allowing for independent, manual or automatic adjustment of the perceived output when using each of these emitters. The required corrections can then be calculated independently for each of the fourth, fifth, and any other emitters (in the same manner as previously described for the fourth emitter), where the results are applied to correct for each of the independent (standard observer-based) adjustments of the red, green, and blue drive levels required when using each of the fourth, fifth, or any other emitters.

[0116] System-related hardware

[0117] Now refer to Figure 19 Describe the hardware used to implement the various processes described above. Figure 19 An LED panel driving system is shown. The system includes two main components: a central controller 1910 and an LED panel 1920.

[0118] The central controller 1910 is operated by a user via a local user interface (UI) 1931 and / or a remote UI 1932 for control and monitoring. The user interface functions are operated via the CPU 1911 and memory 1912 of the central controller 1910. The central controller 1910 receives video in various standard formats (e.g., HDMI or SDI), and the FPGA 1913 is used to perform "standard" video processing (e.g., scaling to resize, basic color adjustments, etc.) and then performs some LED-specific "pixel processing" (e.g., splitting the raster into panel-sized blocks, rotating each panel, etc.). These "blocks" are then transmitted via Ethernet 1940 to an array of LED panels, including the illustrated LED panel 1920. The FPGA 1913 can buffer the video pixels and associated data in local memory 1914.

[0119] Each LED panel 1920 consists of a receiver card 1921 and one or more LED modules 1922. Only one LED panel is shown; however, it should be understood that the central controller 1910 will drive multiple LED panels. The receiver card can utilize an FPGA, which also includes an embedded CPU and memory, such as... Figure 19 As shown. In Figure 19 The diagram also illustrates the functional components of the FPGA. For example, within the FPGA, "blocks" of video data are received for use by the panel associated with receiver card 1921, and input pipeline processing is performed. This input processing may include, for example, displaying test patterns or on-screen displays, converting signal formats from gamma-encoded SDR or HDR encoding to the "linear light" signals required by the LED driver chips, etc. Per-pixel calibration is then applied based on calibration data read from the LED modules and provided to the FPGA modules via the CPU. The CPU 1911 in the central controller 1910 communicates with the embedded CPU in each FPGA receiver card associated with the respective panel via the same Ethernet used to send the video data blocks. This can therefore be used for management, status, and control. After per-pixel calibration, corrections are applied (e.g., for correcting nonlinearities in the driver chips and / or LEDs), the signal is buffered so that the entire screen can be updated simultaneously, and then reformatted as needed according to the specific brand and model of the LED driver chips in use before the data is transferred to the LED modules (e.g., LED module 1922).

[0120] Each LED module (e.g., LED module 1922) has a chain of LED driver chips 1923a, 1923b, each of which drives multiple LED units 1924a, 1924b, which may be in the form of a rectangular grid (e.g., 16×8 pixels). Each unit may include multiple LEDs, each of which has a different color. For example, an RGBF LED unit may include a red LED, a green LED, a blue LED, and a fourth LED (e.g., white). An LED may be a single plastic package containing four (R, G, B, and F) LED dies, or a combination of multiple packages each containing LED dies of one or more colors. Such a combination of LEDs (whether in a single package or multiple packages) may be referred to as an LED unit. While an LED module is shown as having a chain consisting of two LED driver chips, it should be understood that each panel may contain more modules, each module may contain more chains, and each chain may contain more LED driver chips. The flash memory 1925 contained within the LED module 1922 stores data for the LED units 1924a and 1924b on the module, so that if the LED modules are interchanged in the field, the calibration data “travels” with the LED unit to which the data belongs.

[0121] Each panel also transmits content via Ethernet signals, allowing multiple panels to be chained together to fully utilize Ethernet bandwidth.

[0122] Although most of the processing disclosed in this document is shown as being in Figure 19 This is implemented within the receiver card, but it should be understood that it is not mandatory. For example, in other arrangements, processing can be distributed between the central controller and the receiver card, or all processing can be performed by the central controller. However, for many reasons, Figure 19 The implementation method is advantageous. It minimizes the bandwidth requirements from the processor to the panel, and all calibration data is stored in flash memory chips within each panel, allowing the panels to be moved around and making the calibration data easy to maintain. Furthermore, by pushing processing to each receiver card, parallel processing is maximized, which increases the achievable processing speed.

[0123] The various methods described above can be implemented by a computer program product. The computer program product may include computer code arranged to instruct a computer to perform one or more of the various methods described above. The computer program and / or code for performing such methods may be provided on a computer-readable medium or computer program product to a device such as a computer. For example, such computer code may be implemented within a receiver card 1921 and / or a central controller 1910. The computer-readable medium may be transient or non-transient. The computer-readable medium may be, for example, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, or a propagation medium for data transmission (e.g., for downloading code via the Internet). Alternatively, the computer-readable medium may take the form of a physical computer-readable medium, such as semiconductor or solid-state memory, magnetic tape, removable computer disk, random access memory (RAM), read-only memory (ROM), hard disk, and optical disk such as CD-ROM, CD-R / W, or DVD.

[0124] Devices such as computers can be configured according to such code to perform one or more processes according to the various methods discussed in this article.

[0125] Although LED primarily refers to light-emitting diodes, it should be understood that any suitable light emitter can be used, and therefore in some arrangements, LED can be considered as referring to a light-emitting device.

[0126] An LED driver signal is a signal, message, or control means that instructs one or more LEDs on how to operate. For example, a driver signal may indicate the brightness for a specific color of LED, or the brightness for each color of an LED unit. It should be understood that the term "driver signal" can refer to a driver signal that includes multiple driving components. For example, each driving component may drive a different LED or group of LEDs. Therefore, a single driver signal can also be referred to as multiple driver signals, where each driver signal drives a different LED or group of LEDs.

[0127] Referring to the essentially produced color and / or brightness means that the produced color and / or brightness, when viewed individually, may not be significantly different from the color and / or brightness expected by the human eye. In some cases, small differences in color and / or brightness may be noticeable, but the color and / or brightness can still be substantially produced. In such cases, the produced color and / or brightness may be relevant relative to the production of other colors and / or brightness. In some cases, a more significant difference in one of the brightness or color may be noticeable, while the other may be visually equivalent.

Claims

1. A computer-implemented method for converting received red-green-blue, RGB, video data into red-green-blue and fourth color, RGBF, LED drive signals, the method comprising: receiving RGB video data representing a video comprising a plurality of frames, each frame comprising a plurality of pixels, wherein, for each pixel of each frame of the video, the RGB video data defines a red luminance, a green luminance, and a blue luminance for representing a color and a luminance of the pixel; and generating, from the received RGB video data, an RGBF LED drive signal, wherein the RGBF LED drive signal is configured for driving one or more LED modules comprising a plurality of LED units, wherein each LED unit represents a pixel and comprises a red LED, a green LED, a blue LED, and a fourth color LED that is not the same color as the red LED, the green LED, or the blue LED; wherein the generating further comprises: applying a control parameter to adjust a characteristic of the RGBF LED drive signal.

2. The method of claim 1, wherein, the characteristic is a spectral characteristic.

3. The method of claim 2, wherein, the control parameter changes the spectral characteristic by controlling relative luminance levels of two or more of the red LED, the green LED, the blue LED, and / or the fourth color LED.

4. The method of claim 3, wherein, the control parameter changes the spectral characteristic while substantially maintaining the color and luminance of the pixel as defined by the RGB video data.

5. The method of any preceding claim, wherein: a plurality of different combinations of luminance levels of the red LED, the green LED, the blue LED, and the fourth color LED are available when generating the RGBF drive signal, the plurality of different combinations substantially producing the color and the luminance for a respective pixel as defined by the received RGB video data; and the control parameter enables selection of a single combination from the plurality of different combinations.

6. The method of any preceding claim, wherein, the control parameter indicates one of: (i) an instruction for one or more LED units of the plurality of LED units to prioritize use of the fourth color LED over the red LED, the green LED, and the blue LED to substantially produce the color and the luminance defined by the RGB video data; or (ii) an instruction for the one or more LED units of the plurality of LED units to prioritize use of the red LED, the green LED, and the blue LED over the fourth color LED to substantially produce the color and the luminance defined by the RGB video data; or (iii) an instruction for the one or more LED units of the plurality of LED units to balance use of the red LED, the green LED, and the blue LED relative to the fourth color LED with specific weights to substantially produce the color and the luminance defined by the RGB video data.

7. The method of any preceding claim, wherein, the control parameter comprises or is derived from information indicating one or more of: (i) a color of the fourth color LED; content of the video represented by the RGB video data; one or more objects to be illuminated by the one or more LED units; one or more characteristics of the image capture device; and one or more preferences of a user operating the image capture device.

8. The method of claim 7, wherein, when the control parameter includes information indicative of one or more characteristics of the image capture device, the method further comprises: determining whether light emitted by the one or more LED units is to be directly captured by the image capture device; and generating the control parameter to: (i) prioritize use of the fourth color LED over the red, green, and blue LEDs to produce the color and the brightness defined by the RGB video data if light emitted by the one or more LED units is not to be directly captured by the image capture device; and (ii) prioritize use of the red, green, and blue LEDs over the fourth color LED to produce the color and the brightness defined by the RGB video data if light emitted by the one or more LED units is to be directly captured by the image capture device.

9. The method of claim 8, further comprising: receiving information indicative of one or more of: a position, an orientation, a movement, or a field of view of the image capture device, wherein determining whether light emitted by the one or more LEDs is to be captured by the image capture device is based on the received information.

10. The method of any preceding claim, wherein, the generating further comprises: determining whether a balance between (i) the combination of the red, green, and blue LEDs and / or (ii) use of the fourth color LED can produce the color and the brightness for a respective pixel, and if not, adjusting the balance between (i) the combination of the red, green, and blue LEDs and / or (ii) use of the fourth color LED to substantially produce the color and the brightness for the respective pixel.

11. The method of claim 10, wherein, when adjusting the balance between (i) the combination of the red, green, and blue LEDs and / or (ii) use of the fourth color LED, the balance is adjusted by an amount that is possible to substantially produce the color and the brightness for the respective pixel.

12. The method of claim 10 or 11, wherein, the method comprises: determining a minimum drive signal for the fourth color LED that achieves the color and the brightness for a respective LED unit while minimizing a drive level for the fourth color LED; determining a maximum drive signal for the fourth color LED that achieves the color and the brightness for the respective LED unit while maximizing the drive level for the fourth color LED, wherein the minimum drive signal and the maximum drive signal define a range including a maximum value and a minimum value; and selecting a value from the range as the RGBF LED drive signal.

13. The method of claim 12, wherein, The selection involves performing interpolation between the minimum drive signal and the maximum drive signal.

14. The method of claim 12 or 13, wherein, The selection corresponds to the control parameter.

15. The method of claim 1 or claim 2, wherein, The control parameter indicates an adjustment corresponding to one or more of a red component, a green component, a blue component, or a fourth color component of the RGBF drive signal.

16. The method of claim 15, wherein, The adjustment compensates for one or more characteristics of a non-standard observer viewing or recording an output of the one or more LED units.

17. The method of claim 16, wherein, The characteristics are changes in perceived color and / or luminance due to one or more characteristics of the observer.

18. The method of claim 15, 16, or 17, further comprising: driving an LED unit using a first RGBF LED drive signal; driving the LED unit using a second RGBF LED drive signal or driving a second LED unit using the second RGBF LED drive signal; observing the LED unit when driven by the first RGBF LED drive signal and the second RGBF LED drive signal or observing the LED unit and the second LED unit when driven by the first RGBF LED drive signal and the second RGBF LED drive signal, respectively; and determining the adjustment in response to the observation of the LED unit or the observation of the LED unit and the second LED unit when driven by the first RGBF LED drive signal and the second RGBF LED drive signal.

19. The method of claim 18, wherein, The LED unit or the LED unit and the second LED unit are observed by a human user via a screen that forms an image from an image capture device monitoring the LED unit or the LED unit and the second LED unit, and the human user provides input via a user interface to change the adjustment in response to a comparison of the LED unit when driven by the first RGBF LED drive signal and the second RGBF LED drive signal or a comparison of the LED unit when driven by the first RGBF LED drive signal and the second LED unit.

20. The method of claim 18, wherein, The LED unit or the LED unit and the second LED unit are observed by a computing device using data captured by an image capture device monitoring the LED unit or the LED unit and the second LED unit, wherein the computing device changes the adjustment in response to a comparison of the LED unit when driven by the first RGBF LED drive signal and the second RGBF LED drive signal or a comparison of the LED unit when driven by the first RGBF LED drive signal and the second LED unit.

21. The method of claim 20, wherein, The computing device changes the adjustment via an iterative process.

22. The method of claim 1 or 2, wherein, The control parameter defines a maximum luminance increase to be applied to one or more of the plurality of LED units represented in the RGBF LED drive signal when compared to a maximum luminance achievable as if the LED units only comprised red, green and blue LEDs.

23. The method of claim 22, wherein, Generating the RGBF LED drive signal further comprises increasing the luminance of the one or more LED units represented in the drive signal based on the control parameter.

24. The method of claim 22 or 23, wherein, The luminance of the one or more LED units is increased by combining the red, green and blue LEDs with the fourth color LED while still substantially producing the color defined by the RGB video data.

25. The method of claim 22, 23, or 24, wherein, If the resulting luminance represented in the RGBF LED drive signal exceeds the achievable luminance level of the RGBF LED of the one or more LED units, the luminance in the RGBF LED drive signal is set to substantially the maximum achievable luminance that enables substantially achieving the color of the corresponding pixel in the received RGB video data.

26. The method of any preceding claim, wherein, The fourth color LED is a white LED.

27. The method of any of claims 1-25, further comprising a fifth LED, wherein, The fourth and fifth LEDs are selected from the group consisting of: white, cool white, warm white, lime, cyan, indigo, yellow, amber and magenta.

28. The method of any preceding claim, wherein, The control parameter is changed in real-time.

29. The method of any preceding claim, wherein, The control parameter is changed per frame and / or per pixel.

30. The method of any preceding claim, wherein, The control parameter is adjustable by a user via one or more graphical user interface elements.

31. The method of any preceding claim, wherein, The generating further comprises taking into account calibration data.

32. The method of claim 31, wherein, The calibration data is one or more of: per-pixel calibration data, per-region calibration data and RGBF calibration data.

33. The method of claim 31 or 32, wherein, The control parameter is used to adjust the calibration data.

34. A computer program product comprising computer readable code arranged to implement the method of any preceding claim.

35. A system comprising one or more processors arranged to perform the method of any of claims 1 to 33.

36. The system of claim 35, wherein, The one or more processors of the system comprise: at least one central processor; and a plurality of distributed processors each arranged to be associated with one or more of the plurality of LED units.

37. The system of claim 36, wherein, The method of claims 1 to 33 is performed by the plurality of distributed processors and the at least one central processor distributes the RGB video data to be processed to the plurality of distributed processors.

38. The system of claim 36 or 37, wherein, The system further comprises an array of LED modules each comprising one or more LED drivers and a plurality of RGBF LED units each comprising a red, green, blue and fourth color LED. The control parameter defines a maximum luminance increase to be applied to one or more of the plurality of LED units represented in the RGBF LED drive signal when compared to a maximum luminance achievable as if the LED units only comprised red, green and blue LEDs. Generating the RGBF LED drive signal further comprises increasing the luminance of the one or more LED units represented in the drive signal based on the control parameter. The luminance of the one or more LED units is increased by combining the red, green and blue LEDs with the fourth color LED while still substantially producing the color defined by the RGB video data. If the resulting luminance represented in the RGBF LED drive signal exceeds the achievable luminance level of the RGBF LED of the one or more LED units, the luminance in the RGBF LED drive signal is set to substantially the maximum achievable luminance that enables substantially achieving the color of the corresponding pixel in the received RGB video data. The fourth color LED is a white LED. The fourth and fifth LEDs are selected from the group consisting of: white, cool white, warm white, lime, cyan, indigo, yellow, amber and magenta. The control parameter is changed in real-time. The control parameter is changed per frame and / or per pixel. The control parameter is adjustable by a user via one or more graphical user interface elements. The generating further comprises taking into account calibration data. The calibration data is one or more of: per-pixel calibration data, per-region calibration data and RGBF calibration data. The control parameter is used to adjust the calibration data.

34. A computer program product comprising computer readable code arranged to implement the method of any preceding claim.

35. A system comprising one or more processors arranged to perform the method of any of claims 1 to 33. The one or more processors of the system comprise: at least one central processor; and a plurality of distributed processors each arranged to be associated with one or more of the plurality of LED units. The method of claims 1 to 33 is performed by the plurality of distributed processors and the at least one central processor distributes the RGB video data to be processed to the plurality of distributed processors. The system further comprises an array of LED modules each comprising one or more LED drivers and a plurality of RGBF LED units each comprising a red, green, blue and fourth color LED.

Images