Method and system for operating an active display - Patents.com

Abstract

The present invention relates to a method for operating an active display having an array of active light emitting elements, the method comprising: The first image data sequence (F0 k providing a first feed of A second image data sequence (F1 k ,C1 k ;F2 k ,C2 k ;F3 k ,C3 k ) providing at least one second feed of each image combination (F1 k and C1 k ;F2 k and C2 k ;F3 k and C3 k ) results in a homogenous gray image. k ;F2 k ;F3 k ) and a second complementary image data sequence (C1 k ;C2 k ;C3 k ) in at least a second feed; selecting a reference frame rate (SFR) at which the first and at least second sequences of image data are presented to an active display; Operate an active display at a display high frame rate (HDFR) containing nd HDFR image slots / slice in a reference frame rate time interval ΔT=1 / SFR of a reference frame rate (SFR), each HDFR image slot being: [0080] Duration τ according to TIFF2024521230000009.tif54166 i and presenting image data of the first feed and at least the second feed in a time slice multiplexed manner in nd HDFR image slots of each reference frame rate time interval to an active display; At least two feeds, the first feed and at least one second feed, are a combination of image data and complementary image data (F1 k and C1 k ;F2 k and C2 k ;F3 k and C3 k ) and the gray images are presented to the active display with equal luminance and uniformly distributed within the nd HDFR image slots in such a way that luminance changes in the active display occur at a frequency greater than or equal to twice the reference frame rate (SFR).

Description

[Technical field]

[0001] The present invention relates to a method and system for operating an active display, particularly an LED display, in which two or more image data sequences are presented to the active display in a time slice multiplexed manner. [Background technology]

[0002] In recent years, active displays such as light-emitting diode (LED) displays have found a variety of applications, ranging from computer displays or television screens in office or home environments, where typically a single display or two or three independent displays are used, to large-scale applications in the advertising or entertainment industry, where multiple LED panels are essentially seamlessly joined to form an LED row or LED wall. In the context of this application, LED displays are primarily mentioned as a general example of active displays, but it will be apparent that the techniques described herein may also be used for other types of active displays, particularly pixel-based active displays such as active matrix organic light-emitting diode (AMOLED) displays, plasma displays, or quantum dot displays. Moreover, in the context of this application, horizontal or vertical LED display rows or LED panels refer to individual LED displays / panels joined in one dimension (e.g., as horizontal rows or vertical columns), whereas an LED wall refers to multiple LED panels joined in two dimensions. Besides showing the same image data / video content on each panel, appropriate control devices are typically used to distribute each image frame to the available number of LED panels, so that the viewer gets the impression of a single display with gigantic dimensions. Moreover, in the context of the present invention, a sequence of image data shown on an active display is also referred to as a "video stream". In general, the term "video stream" refers to any type of image content shown on an LED display, such as an actual video stream of a sequence of image frames shown in succession, or even a still image, a monochromatic still image. In this sense, "video stream" also encompasses the case where a single still image is presented to an LED display only once, i.e., at a certain frame rate, without the need to change the image content or retransmit image content.

[0003] LED or OLED (AMOLED) displays are a common display technology found in a wide variety of applications ranging from TV or computer screens to media displays in vehicles and mobile phones. LED displays in the form of walls, rows or columns are widely used as billboards or signs to convey information or advertisements to viewers. Moreover, LED walls have recently become increasingly popular in the entertainment industry to provide backgrounds and lighting in virtual studio installations.

[0004] In the past, a single dedicated image data sequence (or a single video stream) was shown on an active display. However, in recent years, new technological developments have necessitated the insertion of additional image content into an image data sequence, with individual images (or frames) of different image data sequences being interleaved with one another. In these cases, only one image data sequence (herein referred to as the "first image data sequence") is intended to be viewed in the sense of being consciously recognized by a direct viewer, while the additional interleaved image data sequences are typically hidden so as not to be seen or recognized by the direct viewer.

[0005] Typical applications include those in which the active display is part of a scene that is captured / filmed by a camera, such as a video camera or photographic still camera.

[0006] One of these applications relates to virtual studios, where a wall made of LED displays, especially fine pitch displays, replaces conventional green or blue screen background panels / screens. This allows a background scene to be presented on the LED wall while the foreground is being shot with the background (i.e. as a first processing of image data), which significantly reduces the amount of video post-production. As described in the applicant's US Pat. No. 6,233,633, it may be beneficial to present an additional image data sequence, such as chromakey image data and / or tracking pattern image data, to the LED display, interleaved with the first image data sequence to identify the position and relative orientation of the LED display and the camera capturing the scene within the scene.

[0007] Other applications may require that the direct viewer be able to see features or information that should not be captured by the camera, for example text prompting or some kind of locator that is not necessary in the camera recording of the scene.

[0008] Another application concerns the so-called "virtual advertising". At sporting events such as football matches, usually many LED displays are installed in the stadium on which advertising content is presented during the sporting event. When the video of the sporting event is broadcasted worldwide, certain advertising content at the event will also be seen in the video. However, some advertising is only relevant for spectators at certain locations and, more importantly, some advertising permitted in one country may even be prohibited by law in other countries. In virtual advertising, a solution to these problems is obtained by identifying LED displays in the captured video and replacing the real content shown in the stadium with alternative content depending on the respective target spectator. To this effect, it is necessary not only to identify the location of the LED displays in the image but also to identify the position and orientation of the camera with respect to a particular LED display. Therefore, in the context of virtual advertising, a similar problem as in a video studio arises at sporting events, i.e. the location of the LED displays and the orientation of the camera can be identified by introducing additional image content, e.g. chromakey images and tracking patterns.

[0009] As an alternative solution to the virtual advertising technique described above, the applicant has developed a method that allows different image contents targeted at different audiences to be shown on an active display, such as an LED sign, in a time-sliced ​​multiplexed manner. Images of a scene are generated by one or more video cameras that are synchronized with different image data sequences shown on the active display, such that multiple images of the scene are created that are identical as far as the overall scene is concerned, but show different images on the LED sign. This technique is described, for example, in the applicant's US Pat. No. 5,399,341.

[0010] The above techniques demonstrate that there are many applications where active displays such as LED panels are part of a scene recorded by a video camera, a TV camera, or a photographic still camera or a mobile phone / tablet camera, and these active displays show different image content in a time-slice multiplexed manner. In all of these applications, there is not only a camera recording the scene, but also a human being (hereinafter referred to as "person" or "direct viewer") who is part of the scene, for example as a spectator at a sporting or entertainment event, an actor or a studio staff at a movie studio, and therefore watches the scene including the active display present in the scene. When operating these active displays in a manner in which different image data sequences are presented in a time-slice multiplexed manner, typically only one of these image sequences is intended to be seen or recognized by a person present in or near the scene. In Patent Document 3, it is proposed that the additional image data sequence is presented only for a small portion of the time interval of the recording frame rate, so that the person does not see or recognize the related image content. In US Pat. No. 6,299,333 by the present applicant, this concept is developed further by proposing that the additional image data sequence be presented as a sequence of images and corresponding inverse / complementary images. [Prior art documents] [Patent documents]

[0011] [Patent Document 1] European Patent Application Publication No. 21169258.7 [Patent Document 2] International Publication No. 2018 / 138366 [Patent Document 3] US Patent Application Publication No. 2009 / 102957 Summary of the Invention [Problem to be solved by the invention]

[0012] However, when two or more unrelated image data sequences are interleaved with each other, several problems may arise. Even a very short introduction of additional image data may lead to noticeable flickering of the active display, even if the image data itself cannot be perceived. Moreover, since the human eye performs integration even within the time resolution of the eye (which is typically in the range of 40 milliseconds), the introduction of the additional image data will lead to a noticeable increase in the black level of the first image data sequence as seen or perceived by the direct viewer, especially in dark areas of the image. [Means for solving the problem]

[0013] It is therefore an object of the present invention to provide a method and system for presenting different image data sequences in a time slice multiplexed manner to one or more active displays that are part of a scene in such a way that persons present in or near the scene get an improved viewing experience of the first image data sequence that they are intended to view, in particular with respect to minimizing flicker and avoiding significant increases in black levels, while allowing recording of the scene synchronized with one or more image data sequences in a wide range of ambient lighting conditions. Moreover, modern video or movie studios employ control systems that can bring all light sources to the same level through one control device. All of these light sources need to be synchronized to avoid flicker, vibration, fading effects, etc.

[0014] This technical problem is solved by the method defined in claim 1 of the present application. Further embodiments of the invention are subject of the dependent claims.

[0015] The present invention therefore relates to a method for operating an active display comprising an array of active light emitting elements, the method comprising: The first image data sequence (F0 k providing a first feed of A second image data sequence (F1 k ,C1 k ;F2 k ,C2 k ;F3 k ,C3 k ) providing at least one second feed of each image combination (F1 k and C1 k , F2 k and C2 k , F3 k and C3 k ) so that a homogeneous gray image results. k ;F2 k ;F3 k ) and a second complementary image data sequence (C1 k ;C2 k ;C3 k ) the second feed including at least selecting a reference frame rate (SFR) at which the first and at least second sequences of image data are to be presented on an active display; The active display is operated at a display high frame rate (HDFR) that contains nd HDFR image slots / slice in a reference frame rate time interval ΔT=1 / SFR of a reference frame rate (SFR), and each HDFR image slot is

number

[0016] According to the present invention, an "active display" is an array of active light elements, such as LEDs, an AMOLED array, an array of plasma cells, or an array of quantum dots, capable of emitting light when a current is passed through the elements. The term "light" in the sense of the present invention refers to electromagnetic radiation, which is normally described using the term "light", in particular visible light having a wavelength in the range of 400 nm to 700 nm, but also longer wavelength infrared light and shorter wavelength ultraviolet light.

[0017] In the context of the present invention, a "feed" refers to a sequence of image data that are related to each other, for example a sequence of image data that may be frames of a video. A feed is characterized by a rate at which new data content instances are presented, which are described in the present invention as "image data", but which may also be considered as individual images or "frames". In the context of video, the frame rate of the feed defines the "reference frame rate". In the context of the present invention, the minimum reference frame rate is around the fusion rate of the human eye, i.e. the rate at which the human eye can no longer distinguish the individual images and views them as a video stream. Typically, the minimum reference frame rate in this context is around 24Hz or 25Hz. However, higher reference frame rates such as 50Hz or 60Hz are more common. "Image data sequence" Fx in the sense of the present invention kis a sequence x (x=1,2,3,...) of two-dimensional image information transmitted to or stored in an active display. One element k of an image data sequence generally represents one individual image shown on the active display, and subsequent images (k+1,k+2,...) of a given image data sequence are presented on the physical display at subsequent reference frame rate time intervals ΔT=1 / SFR. During a reference frame rate time interval ΔT, a given image of each image data sequence may be presented one or more times. Each individual image data sequence generally represents a video stream shown on the active display, where the image content may change from one image of the sequence to a subsequent image of the sequence. The image content from one image to a subsequent image, however, may remain unchanged, for example, if no motion is present in the video stream. In other embodiments, the image data sequence may also include still images shown over several reference frame rate time intervals ΔT. In other embodiments, the image data sequence may include only a single type of image, such as a monochromatic image or a tracking pattern image, that does not change at all throughout the image data sequence.

[0018] The term "direct viewer" in the context of the present invention refers to humans who are present in the venue / scene in which the active display is operating and who are able to view the active display directly.

[0019] The term "intended for direct viewing" refers to the presentation of the first image data sequence F0 in such a way that the image data sequence is viewable / perceivable by a direct viewer as part of a video, for example in the form of a still image or still image sequence, or when the image data sequence is presented in such a way that the individual images are not resolved in time. k Therefore, the first image data sequence F0 k This feed is also referred to as the "venue feed."

[0020] The term "not intended for direct viewing by a viewer" refers to an additional image data sequence F1 where the image data is presented in such a way that light emitted from the active display corresponding to an image in the image data sequence reaches directly to the viewer's eye, but the corresponding image cannot be seen / recognized by the human eye. k ,C1 k ;F2 k ,C2 k ;F3 k ,C3 k This not only refers to the temporal resolution of the video presentation, but more broadly to the fact that the image data is presented in such a way that neither the individual images nor the image sequence can be seen or recognized. k ,F2 k ,F3 k is noted as "parallel feed."

[0021] In order to effectively "hide" the at least one additional image data sequence from being recognized or perceived by the direct viewer, it is preferred that the time-integrated luminance at which the images of the first image data sequence are presented during a reference frame rate time interval ΔT=1 / SFR is higher than the time-integrated luminance of the images of the at least one additional image data sequence presented during that time interval. In the case where the additional image data sequences are presented multiple times, the time-integrated luminance of the first image data sequence intended for the direct viewer is preferably higher than the sum of the time-integrated luminance of all the additional image data sequences not intended for viewing by the direct viewer. Preferably, the time-integrated luminance of the first image data sequence is at least two times higher, more preferably at least four times higher.

[0022] In order to further reduce the influence of the additional image data sequence on the first image data sequence, the at least one additional image data sequence is at least one image data sequence that is different from the second image data sequence (F1 k ;F2 k ;F3 k ) and at least a second complementary image data sequence (C1k ;C2 k ;C3 k ) Thus, each image F1 of the second image data sequence k is its inverse / complementary image C1 within the same reference frame rate time interval ΔT=1 / SFR. k The introduction of inverse images to reduce flicker of active displays for a direct viewer has already been described in detail in the applicant's US Pat. No. 5,399,363. In one embodiment, the additional image data sequence comprises a pair sequence of image data and inverse / complementary images, the time-integrated luminance of an image of the first image data sequence when presented in a reference frame rate time interval ΔT=1 / SFR is lower than the time-integrated luminance of an image of at least one additional image data sequence presented in said time interval. Despite the generally noticeable increase in grey levels resulting from the combination of an image of the additional image data sequence with an inverse image, the first image data sequence can still be seen / perceived by a direct viewer. For example, the first image data sequence may be presented only in a portion, for example half or a third of the reference frame rate time interval.

[0023] According to the method of the present invention, a suitable reference frame rate at which the image data is presented to the active display is selected based on the first and at least second feeds of the image data sequence, where if the feeds represent video data, the suitable reference frame rate corresponds to a reference frame rate for recording the video.

[0024] The active display operates at a display high frame rate (HDFR) higher than the reference frame rate so that a certain number of nd HDFR image slots / slices can be presented in each reference frame rate time interval. As mentioned above, at least a first feed intended for the direct viewer and a second feed not intended for the direct viewer are presented in each reference frame rate time interval. As a result, the number of nd HDFR image slots / slices per reference frame rate time interval is at least 2, where the complementary image data of the second feed is mixed with the images intended for the direct viewer, and preferably nd is much higher, at least nd is 4, 8, 12, 16, 24. Thus, each reference frame rate time interval is subdivided into P nd slots / slices in which different image data can be presented. Within the nd image slots / slice of a given reference frame time interval, only image data corresponding to this frame at the reference frame rate of the original feed is presented, but each image content instance can be presented multiple times. Preferably, each image data content instance of the first feed intended for the direct viewer is presented in multiple available nd HDFR image slots / slices.

[0025] According to the present invention, image data of the first feed and at least the second feed, i.e., corresponding image data content instances for each reference frame rate time interval, are presented in a time slice multiplexed manner and distributed within nd HDFR image slots of each reference frame rate time interval.

[0026] To further reduce flicker, the invention proposes that at least two of the feeds, the at least first feed and the at least one second feed, contain the grey images obtained from a combination of image data and complementary / inverse image data, the grey images of the at least two feeds being presented to the active display with equal luminance and being uniformly distributed within nd HDFR image slots in such a way that luminance changes of the active display occur with a frequency equal to or greater than twice the reference frame rate.

[0027] According to the invention, one feed intended for viewing by the direct viewer is always provided, and the insertion of image data not intended for viewing by the direct viewer results in a flickering impression of the first feed. In a first step already described in the prior art, the flickering is reduced by presenting these additional feeds as a combination of image data and complementary image data. Nevertheless, there is some noticeable flicker, and the invention proposes to further reduce the flickering for the direct viewer by ensuring the provision of at least two feeds including a grey image resulting from the combination of an image with a respective inverse / complementary image. When presenting grey images to the active display, the invention proposes to ensure that each grey image is presented to the active display with equal luminance, even if they originate from different feeds. Furthermore, according to the invention, the grey images are distributed uniformly within the nd HDFR image slots / slices in such a way that the luminance changes of the active display occur with a frequency of at least twice the reference frame rate SFR. Uniform distribution means that for the grey image distribution within each reference frame rate time interval, it occurs with a certain frequency of at least twice the reference frame rate, even taking into account the subsequent reference frame rate time intervals. This means that the resulting grey image does not end up with multi-frequency components, but is distributed only at a single frequency that is an integer multiple of the reference frame rate.

[0028] The term "luminance" refers to the unit of candela per square meter (cd / m), also known as "nit." 2) refers to a photometric measure of luminous intensity per unit area of ​​the active display. In contrast, "brightness" refers to the subjective impression of objective brightness, but is therefore closely related to luminance. The term "luminance" in the context of the present invention also has a time component, taking into account the duration of the individual slots / slices within the reference frame rate time interval. The time component can be ignored when each reference frame rate time interval is subdivided into slots / slices of equal duration, but when slots / slices of different duration are compared with each other, "luminance" in the sense of the present invention refers to the luminous intensity per unit area of ​​the active display multiplied by the individual length of each slot / slice.

[0029] It is known that in the physiology of the human eye, the critical fusion frequency (i.e. the lowest frequency at which flicker disappears) is proportional to the logarithm of the light intensity (Ferry-Porter law). In the context of the present invention, the applicant has found that the flicker experience depends on the brightness of both the content intended for direct viewer viewing (first image data sequence) as well as the "hidden content" (subsequent image data sequence) that is captured by the human eye but is not intended for direct viewer viewing / perception. The applicant has found that at frequencies of 100 Hz or higher, flicker is not perceived by the human eye of a direct viewer. However, in dark light conditions and / or when displaying dark content, lower frequencies are also generally accepted by the human eye without normally introducing flicker. Therefore, preferably, the luminance changes of the active display occur at frequencies higher than 100 Hz, preferably higher than 120 Hz, particularly preferably higher than 200 Hz.

[0030] In many embodiments of the invention, there will be a first feed intended for viewing by a direct viewer, and multiple second feeds not intended for a direct viewer, e.g., a second feed and a third feed. Since the second feeds (i.e., second feed, third feed, fourth feed, etc.) are always presented as a combination of an image and an inverted image, the above requirement that at least two feeds containing a gray image are provided can be met.

[0031] However, in some embodiments of the present invention, especially in those embodiments with only a first feed and one second feed, i.e., no additional second feeds (no additional third, fourth, etc. feeds), the above requirement of providing two feeds containing an image and an inverse image creating a gray image can only be met when the first feed also contains an image and its respective complementary / inverse image. Since a mere combination of an image and an inverse image would "hide" the image itself from being perceived by a direct viewer, in such an embodiment, the image data of the first feed is presented in nd HDFR image slots more than the complementary image, such that a net visible image remains for a direct viewer of the first feed. However, the combination of an image of the first feed with its respective complementary / inverse image ensures that the criterion of uniformly distributing the gray image within the HDFR slots such that luminance changes occur at a rate greater than or equal to twice the reference frame rate can be met.

[0032] According to a preferred embodiment of the present invention, the second image data sequence (F1 k ;F2 k ;F3 k ) and the second complementary image data sequence (C1 k ;C2 k ;C3 k ) with the reverse / complementary image of each pair (F1 k ,C1 k ;F2 k ,C2 k ;F3 k ,C3 k) are presented within a time interval of 3.3 milliseconds or less. Flicker and subject motion of the active display for the direct viewer can be minimized particularly effectively if the corresponding images and the complementary / inverse images are presented within said 3.3 millisecond time interval. In many applications of the inventive concept, the second image data sequence contains images intended to be captured by a properly synchronized video camera (see US Pat. No. 5,399,323). To allow the capture of bright images, the duration during which the images of the second image data sequence are presented is preferably as long as possible. To meet the 3.3 millisecond constraint, complementary images can also be presented in time slots immediately before and after the HDFR time slots of the second image data sequence.

[0033] In some embodiments of the present invention, for each additional image data sequence, a corresponding complementary image data sequence is provided.

[0034] The additional image data sequence may include various images or video streams. For example, the additional image data sequence may include alternative advertising content captured by a suitably synchronized video camera. Typically, the video camera captures an image (F1 k ;F2 k ;F3 k ), while the inverse / complementary image (C1 k ;C2 k ;C3 k ) images will not be recorded.

[0035] In one embodiment, each of the nd HDFR slots is

number

[0036] In another embodiment, the nd HDFR slots of the reference frame rate time interval ΔT=1 / SFR are independent of each other, and therefore the duration τ icorresponds to the duration of the reference frame rate time interval 1 / SFR, i.e.,

number

[0037] In a preferred practical implementation of the method of the present invention, each presentation of an image in an HDFR image slot consists of one or more basic pulse width modulation frames (PWM frames, also referred to as "scrambled"), during which no new image data is presented, but the brightness of the image presented in the HDFR slot is controlled by activating individual active elements of the active display, e.g. LED elements, according to a pulse width modulation scheme.

[0038] Preferably, the duration τ of each of the nd HDFR slots i, τ are generated via a pulse counter fed by the G clock (GCLK). Thus, instead of employing a conventional LED driver that receives an external PWM signal via the driver's OE (output enable) pin, the present invention preferably uses an active display provided with an LED control circuit (LED driver) that generates the PWM frame (scramble) from an internally generated G clock (GCLK) signal that is fed to the circuit's pulse counter. The actual brightness of the LEDs is controlled by the PWM duty cycle, i.e., the ratio of the LEDs in "on" mode to "off" mode in each PWM time interval. In addition to or as an alternative to pulse width modulation, the brightness of the HDFR image slots can also be controlled via the current at which individual active elements, such as LED elements, are activated. As described in more detail below, by controlling the PWM frame via the G clock, it is possible to generate HDFR slots of equal duration as well as HDFR slots of variable duration. The PWM signal controls the color gradation and brightness of the LEDs. External PWM signals are distorted and attenuated over long distance transmission, resulting in color and brightness variations. In contrast, using an internally generated G clock and PWM signal to determine the duration and brightness of an HDFR slot results in a higher accuracy of the corresponding image frame.

[0039] This is particularly useful when the image and complementary / inverse image pairs have different durations than the duration varying schemes mentioned above. In a preferred embodiment of the present invention, different durations τ of HDFR image slots are obtained by varying the frequency of the G clock while counting the same predetermined number of pulses via the pulse counter. i Thus, our method ensures that both images are combined into an accurate homogenous gray image.

[0040] When synchronizing a camera capturing a scene that includes one or more active displays, either the leading or trailing edge of the HDFR slot can be used as a trigger point, and the variable length can be adjusted to the left or right of the trigger point.

[0041] When a video camera intended to capture one of the additional image data sequences is used, a synchronization procedure between the camera and the display must usually be implemented to ensure that only the desired HDFR image slot is captured, without capturing, for example, parts of adjacent image slots where the complementary / inverse image is shown. A preferred procedure involves setting the camera to a shutter time that is much shorter than that required for the actual video capture, so that only a portion of the HDFR image slot is captured. Given the short shutter time, the captured image will be fairly dark, but will be highly sensitive to interference with adjacent HDFR image slots, so that proper synchronization is quickly achieved. The desired shutter speed is then selected according to the lighting conditions, and video capture can begin.

[0042] In one embodiment of the present invention, at least one of the nd HDFR image slots includes a black phase. The term "black phase" refers to a time interval during which the physical display is dark, e.g. in the case of an LED display all LEDs are turned off. This can be achieved by turning off the G clock (CLK) for the duration of the black phase. The black phase has a typical HDFR image slot duration, but preferably the black phase has a duration up to 50% of the HDFR image slot. More preferably, the black phase is longer than the duration τ of the respective HDFR image slot. i The term "substantially short" means that the duration of the black phase is shorter than 20%, preferably shorter than 10%, particularly preferably shorter than 5% of the duration of the respective nd HDFR image slot. In general, when a reference frame time interval of, for example, 20 ms (50 Hz) or 16.7 ms (60 Hz) is adopted, the HDFR image slots have a duration in the millisecond range and the inserted black phases have a duration in the 0.1 ms range. The additional image data sequence (F1 k ;F2 k ;F3 kThe insertion of a black phase is particularly preferred in the initial part of the HDFR image slot associated with the first image data sequence (F0). Thus, during the transient oscillations of the amplifiers of the control circuit of the active display (during the settling time of the components of the control circuit), the active display is black, and thus the video camera is able to capture a clear image already exhibiting the desired color and luminance values. The black phase is inserted in the first image data sequence (F0 k ) image, but in this case the inverse / complementary image of the image in the first image data sequence generally does not exist, so slight deviations in color and brightness are less important.

[0043] Active display flicker can be further reduced if at least 6 HDFR image slots are provided in the reference frame rate time interval ΔT=1 / SFT. Preferred numbers of slots in the reference frame rate time interval ΔT are 12, 24, and 36 HDFR slots.

[0044] As mentioned above, the additional image data sequence (F1k, C1 k ;F2 k ,C2 k ;F3 k ,C3 k ) is the first image data sequence (F0 k ) tends to increase the black level of the image. This necessary increase is particularly problematic in bright environments, such as outdoor sporting events, where the sun shines directly on LED advertising signs. Under these conditions, the additional image data sequence intended to be captured by the video camera, i.e., image data sequence F1 k ;F2 k ;F3 kThe brightness of the first image data sequence is increased, exacerbating the problem of the black level increase of the first image data sequence. As a result, the direct viewers attending the event perceive the first image data sequence as a video feed with a sort of grayish overlay. In order to minimize the black level increase of the first image data sequence, the present invention provides a method for minimizing the black level increase of at least the second complementary image data sequence C1. k ;C2 k ;C3 k The HDFR image slot presenting the image of the first image data sequence (F0 k ) image data. This allows to increase the ratio of images from the first image data sequence to images from the additional image data sequence, improving the viewing experience of the direct viewer. Similarly, recording of a desynchronized camera when capturing two or more HDFR image slots, e.g. an entire reference frame rate time interval, is improved to essentially correspond to the viewing experience of the direct viewer.

[0045] Active displays usually operate at a certain brightness, for example 8 bits (256 brightness levels), 9 bits (512 brightness levels), 10 bits (1024 brightness levels), 12 bits, or even 16 bits or 24 bits. For example, due to a bright environment, an additional image data sequence F1 k ;F2 k ;F3 k When the image must be presented at high brightness, the combined image F1 k ,C1 k ;F2 k ,C2 k ;F3 k ,C3 k In order to form a homogeneous gray image in total, the corresponding complementary image data sequence C1 k ;C2 k ;C3 k must also be presented at high brightness. Usually, a corresponding additional image data sequence F1 k ;F2 k ;F3 kThe complementary image data is presented in a shorter HDFR image slot, and therefore a higher brightness is required for the complementary image data sequence. It is therefore difficult to include the additional image data from the first image data sequence without saturating the corresponding HDFR image slot. Therefore, in a preferred embodiment of the present invention, the active light emitting elements of the active display are operated at increased current while the brightness (bit-based brightness level) of the complementary image components is proportionally reduced. In one embodiment, the driver chip of the active element (e.g., LED or OLED) can reduce the nominal current (i.e., 100% current value) of the active light emitting element, e.g., via an adjustable resistor, so that the LED can be safely operated at 100-200% of its nominal current.

[0046] Typically, active elements such as LEDs have a non-linear power-current relationship. Therefore, when current adjustment is performed according to the present invention, a correction for this non-linear behavior must be applied to ensure that color changes that would otherwise affect the image directly (first image data sequence) or indirectly directly intended for the viewer are not affected by insufficient offset between the image of the additional image data sequence and the inverse / complementary image. The respective power-current relationship is usually provided by the manufacturer of the active element and can therefore be implemented in the hardware for controlling the display. Generally, interpolation / extrapolation over the operating range using two (linear approximation), three or four data points will give sufficient results.

[0047] In another embodiment, at least a second complementary image data sequence (C1 k ;C2 k ;C3 k An image slot presenting an image of ) includes the image component immediately before or after the HDFR image slot.

[0048] Preferably, the display used in the method of the present invention is an LED or OLED (AMOLED) display.

[0049] The invention also relates to a system for operating an active display comprising an array of active light-emitting elements, the system comprising a control unit adapted to implement the above method. [Brief description of the drawings]

[0050] The invention will now be described in more detail with reference to the accompanying drawings. [Figure 1] FIG. 1 is a schematic diagram of a studio environment in which the method of the present invention may be practiced. [Diagram 2] FIG. 2 is a schematic diagram of a sports stadium in which the method of the present invention may be practiced. [Diagram 3] FIG. 3 is a schematic diagram of the process of the present invention employing two feeds. [Figure 4] FIG. 4 is a schematic diagram of a process of the present invention employing three feeds. [Diagram 5] FIG. 5 is a schematic diagram of a frame sequence in which the individual slots have variable lengths. [Figure 6] FIG. 6 is a frame sequence similar to that of FIG. 5 with black phase insertion. [Figure 7] FIG. 7 is a frame sequence illustrating a proportional increase of the first image data sequence. [Figure 8] FIG. 8 is an alternative embodiment of the scheme of FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0051] FIG. 1 shows a schematic diagram of a digital imaging studio 10 with an LED scenic wall 11 made of a number of individual LED panels 12. The LED wall 11 is essentially a seamless wall when viewed from the front side 13, but consists of individual panels 12 fixed to a suitable mounting structure 15 when viewed from the rear side 14 of the wall 11, of which a portion is visible. The studio 10 further includes an LED ceiling 17 also made of individual LED panels, as well as an LED floor 16 also made of individual LED panels. The studio 10 further includes conventional lighting equipment such as overhead lights 18 and floor lights 19, and one or more digital cameras, represented diagrammatically in FIG. 1 by camera 20. FIG. 1 also shows the field of view of the camera 20, marked by a frustum 21. An actor 22 is shown in front of the camera 20 within the field of view of the frustum camera. Mounted on camera 22 is an auxiliary camera 23, which in the example of Figure 1 is pointed towards the LED ceiling, and a tracking pattern captured by the auxiliary camera 23 mounted on main camera 20 can be presented from which the position and orientation of camera 20 can be derived.

[0052] Figure 2 shows a sports stadium, in the case of figure 2 a soccer stadium 30, with a soccer field 31 and a stand structure 32 surrounding the soccer field 31. Around the periphery 33 of the soccer field 31, LED rows 34 consisting of individual LED 35 panels are provided to show advertisements. In the method of the invention, the advertisements seen by the audience at the stadium may be presented as a first image data sequence, while a chromakey image may be presented as a second image data sequence. To reduce disturbances by the on-site audience at the stadium, in the present invention a third image data sequence is provided consisting of image frames representing a complement / inverse image of the chromakey image of the second image data sequence. In one embodiment, the first image data sequence may also include a tracking pattern, while the third image data sequence may include a complement image of said tracking pattern.

[0053] In the following, the method of the invention is explained in more detail with reference to a typical sequence of HDFR image slots. In the example presented below, it is assumed that the reference frame rate corresponds to 50 Hz, and consequently the reference frame rate time interval ΔT=1 / SFR corresponds to 20 ms. Although only one reference frame rate time interval k is shown in Fig. 3 to Fig. 8, it should be understood that to the left of the depicted time interval, corresponding time intervals k-1, k-2, ... extend, and to the right of the depicted time interval, corresponding time intervals k+1, k+2, ... extend. Moreover, in many applications, the number of HDFR image slots is larger than the number of image slots depicted in the present example for simplicity's sake.

[0054] FIG. 3 is a schematic diagram of an embodiment of the method of the present invention in which two different feeds are presented to the active display in a time slice multiplexed manner. In the embodiment of FIG. 3, a reference frame rate time interval of 20 ms (50 Hz) is subdivided into 12 slots / slices, each having a duration of 1.67 ms. In the first column, e refers to the number of slots and L refers to the relative luminance at which each slot is presented. F0 refers to the first feed intended for the direct viewer, and F1 refers to the feed not intended for the direct viewer. In the embodiment of FIG. 3, mainly only two feeds are provided (in slots 1, 2, 3, 6, 7, 8, 9, 10, 12) in each reference frame rate time interval, and image data content instances intended for viewing by the direct viewer are displayed on the active display. Image data of the second feed F1 is shown only in slots 4 and 5 as a combination of image F1 and complementary / inverse image C1, so that a gray image is perceived (albeit not subjectively) as a result of the combined effect of slots 4 and 5. To further conceal the insertion of image content F1, the luminance of slots 4 and 5 is only 30 percent of the luminance of the slot in which image data F0 is presented. According to the present invention, to ensure that a gray image is presented in both feeds, slots 10 and 11 are not populated with a "canonical" representation of image data F0 of the first feed, but slots 10 and 11 are used to present an image and an inverse image of the image data of the first feed. Moreover, to ensure that the luminance changes occur with a defined frequency that is greater than the reference frame rate, the relative luminance of slots 10 and 11 is also reduced to 30 percent of the luminance of slots 1, 2, 3, 6, 7, 8, 9, 12. The fact of "hiding" one slot of image data intended for direct viewing by the viewer, i.e. "hiding" slot 10 by presenting a complementary / inverse image corresponding to slot 11, is counter-intuitive, but effectively reduces the flicker resulting from the insertion of the second feed of image data.

[0055] Figure 4 shows a similar example to Figure 3, but in addition to the first feed intended for viewing by a direct viewer, two additional "second" feeds, namely the second feed F1 and the third feed F2, are presented as combinations of images F1, F2 and their respective complementary / inverted images C1, C2, respectively. As with the embodiment of Figure 3, the combinations of images and inverted images are presented at a lower relative luminance than the images intended for viewing by a direct viewer, such that there is a defined frequency of luminance changes greater than the reference frame rate of 50 Hz, and the resulting grey images are evenly distributed among the 12 HDFR image slots.

[0056] The concept of having variable length HDFR image slots is described in detail below. The minimum duration of an image slot is equal to the minimum transmission time plus the vertical sync signal. The GCLK frequency should be changed to show the entire image within a given time. Assuming we have a maximum of 12 image slots: F1 is t F1 = Gain G for 2 ms F1 = 0.5. And t C1 = 1.67ms minimum time

number

[0057] FIG. 5 shows a schematic diagram of a frame sequence in which the individual slots have variable duration. FIG. 5 a) shows the actual sequence with 9 HDFR image slots, where a large percentage of the frame time interval is due to a first image data sequence F0, which is intended for direct viewing by the viewer. Three additional image data sequences F1, F2, F3 are provided, each with its complementary / inverse image data sequence C1, C2, C3. FIG. 5 b) shows the data clock (DCLK) sequence governing the transmission of the image data. As can be seen, during the presentation of a slot, for example F1, the image data of the following slot C1 is transmitted to the active display, etc. The image data contains the brightness values ​​of each LED of the active display. As an example, an LED display with a 10-bit resolution (1024 brightness levels) comprises LEDs which shall be operated at 50% of their maximum intensity according to the data clock information. The transmitted brightness information (ranging from 0 to 1023) therefore corresponds to 511 values. FIG. 5 c) shows how this value is transmitted to an appropriate pulse width modulation PWM. A G-Clock (CLK) is generated at a normal frequency, for example 10 MHz. In a simple embodiment, a pulse counter counts the number of pulses until it reaches the desired value for this image slot (511 in this example). During the remaining pulses of the G-Clock, the PWM signal is turned off. Thus, the PWM signal operates at a 50% duty cycle. However, in a preferred embodiment, the active pulses are evenly distributed over the time interval of the slot.

[0058] Figure 6 shows a frame sequence similar to that of Figure 5 with a black phase inserted. As can be seen from Figure 6a), a black phase having a duration of 0.1 ms is inserted at the beginning of each of the first image data sequences F1, F2, F3. As can be seen from Figure 6b), the black phase is generated by switching off the G clock during the desired black phase.

[0059] The concept of increasing the brightness of the venue feed, i.e. the first image data sequence F0, intended for direct viewing, is described in detail below. The inverted image is displayed with a higher current, but therefore a reduced brightness level. In this way, headroom is left in the color / brightness space and content for the direct viewer's human eye can be added to the image. Figure 7 shows an example of this concept, assuming a PWM-controlled 10-bit brightness range, i.e. an intensity from 0 (black) to 1023 (maximum). The image gain factor is the ratio of the PWM-controlled image brightness within that range, i.e. gain=0 corresponds to a 10-bit value of "0" and gain=1 corresponds to a 10-bit value of "1023". The actual brightness of an active light-emitting element (e.g. LED) in an active display is determined by multiplying the gain factor by the current at which the element is activated. In the basic embodiment of the method of the invention shown in FIG. 7a), F1 is displayed with a gain of 0.5 (i.e., 511-bit PWM level) and a current of 0.2, and the complement / inverse image C1 is displayed with the same gain and current, so that the sum of F1 and C1 is a monotonous gray image. In the embodiment shown in FIG. 7b), F1 is also displayed with a gain of 0.5 and a current of 0.2, and C1 is displayed with a gain of 0.25 and a current of 0.4, so that the sum of F1 and C1 is also a monotonous gray image, but a gain of 0.75 can be added to the venue feed F0 for the direct audience so that the overall proportion of the venue feed is increased. However, it must be noted that the current-intensity relationship of the R, G, and B LEDs changes. Therefore, color correction must be performed. It must be ensured that the brightness level is constant when the image content is complemented. Whatever current setting is selected for the inverse image slot. Here it is useful to note that the amount of light (essentially the number of photons) is proportional to the luminance / brightness level B and gain G at current c (coefficients determined by the driver chip current settings, so c ∈ [0,2]) and time t, and the image will appear as follows:

number

[0060] Figure 8 shows an extension of the scheme of figure 7, where two inverted images immediately before and after a given additional image data sequence F0 (venue feed), F1 (first parallel feed), F2 (second parallel feed) and additional image content of the venue feed are combined in one HDFR image slot to obtain a frame sequence with an increased proportion of the first image data sequence F0. This allows more flexibility and performance, especially in low slot systems such as the LED floor system "Black Marble" sold by ROE Visual.

[0061] Furthermore, this approach of HDFR image data slots with variable duration can be combined with the concept of boosting the brightness of the first image data sequence (venue feed). Thus, the gain of the inverse image is:

number

[0062] 10 Digital Video Studio 11 LED Background Wall 12 LED Panels 13 Front side 14 Rear side 15 Mounting structure 16 LED floor 17 LED ceiling 18 Upper Lighting 19 Floor lighting 20 Camera 21 Cone Frustum 22 Actor 23 Auxiliary Camera 30. Football Stadium 31 Soccer Field 32 Stand Structure 33 Around the soccer field 34 LED rows 35 LED Panel

Claims

1. 1. A method of operating an active display comprising an array of active light emitting elements, comprising the steps of: The method comprises: A first image data sequence (F0 ｋ providing a first feed of A second image data sequence (F1 ｋ , C1 ｋ ; F2 ｋ , C2 ｋ ;F3 ｋ , C3 ｋ ), wherein the at least one second feed comprises a second image data sequence (F1 ｋ ; F2 ｋ ;F3 ｋ ) and a second complementary image data sequence (C1) consisting of inverse or complementary image data of the second image data sequence. ｋ ; C2 ｋ ; C3 ｋ ) so that a combination of the images of the second image data sequence and the second complementary image data sequence (F1k and C1k; F2k and C2k; F3k and C3k) results in a second homogenous grey image; selecting a reference frame rate (SFR) at which the first sequence of image data and the at least second sequence of image data will be presented to the active display; operating the active display at a display high frame rate (HDFR) that includes nd HDFR image slots at a reference frame rate time interval ΔT=1 / SFR of the reference frame rate (SFR), each HDFR image slot being [0070] Duration by τ ｉ and presenting image data of the first feed and the at least second feed to the active display in a time slice multiplexed manner in the nd HDFR image slots of each reference frame rate time interval, wherein a time integrated luminance of the first image data sequence presented in a reference frame rate time interval is higher than a sum of the time integrated luminance of the second image data sequence; Inclusive of At least two of the first feed and the at least one second feed are the combination of the second image data sequence and the second complementary image data sequence (F1 ｋ and C1 ｋ ; F2 ｋ and C2 ｋ ;F3 ｋ and C3 ｋ ) wherein the second homogenous gray image is presented to the active display with equal luminance and uniformly distributed within the nd HDFR image slots in such a way that luminance changes of the active display occur at a frequency equal to or greater than twice the reference frame rate (SFR); Where: The first image data sequence (F0 ｋ ) comprises a first complementary image data sequence (C0) consisting of an inverse / complementary image of the first image data sequence. ｋ ) such that a combination of each image (F0 k and C0 k ) of the second image data sequence and the second complementary image data sequence results in a homogenous grey image; The gray images of the at least two feeds are ｋ and C0 ｋ ) and the second feed (F1 ｋ and C1 ｋ and a second homogenous grey image from said nd HDFR image slots, wherein at least one of said nd HDFR image slots shows a regular representation of said first image data sequence at a first luminance, one of said nd HDFR image slots shows a representation of said second image data sequence, one of said nd HDFR image slots shows a representation of said second complementary image data sequence, one of said nd HDFR image data slots shows a representation of said first image data sequence at a luminance lower than said first luminance, and one of said nd HDFR image slots shows said first complementary image data sequence at a luminance lower than said first luminance. A method comprising:

2. The method of claim 1 , wherein the luminance variations of the active display occur at a frequency greater than 100 Hz.

3. The second image data sequence (F1 ｋ ; F2 ｋ ;F3 ｋ ) and the second complementary image data sequence (C1 ｋ ; C2 ｋ ; C3 ｋ ) and the inverse / complementary image of each pair (F0 ｋ , C0 ｋ ; F1 ｋ , C1 ｋ ; F2 ｋ , C2 ｋ ;F3 ｋ , C3 ｋ 3. The method of claim 1, wherein the first and second images are presented within a time interval of 3.3 milliseconds or less.

4. 3. The method of claim 1 or 2, wherein the HDFR image slots of a reference frame rate time interval ΔT have the same length τ.

5. The HDFR image slots of the reference frame rate time interval ΔT have a variable length τ ｉ 3. The method of claim 1 or claim 2, comprising:

6. The length τ of each of the HDFR image slots ｉ 3. The method according to claim 1, wherein τ, . . . are generated via a pulse counter fed by a G clock (GCLK).

7. By varying the frequency of the G clock (GCLK) while counting the same predetermined number of pulses via a pulse counter, different durations τ of the HDFR image slots can be obtained. ｉ The method of claim 6, wherein

8. At least one of the HDFR image slots has a respective HDFR image slot duration τ ｉ 3. The method of claim 1 or claim 2, comprising a black phase having a shorter duration.

9. The method of claim 8 , wherein the duration of the black phase is less than 20% of the duration of the HDFR image slot.

10. 3. The method of claim 1 or 2, wherein at least 6 HDFR image slots are provided in a reference frame rate time interval ΔT=1 / SFR.

11. The at least second complementary image data sequence (C1 ｋ ; C2 ｋ ; C3 ｋ ) is a HDFR image slot presenting an image of the first image data sequence (F0 ｋ 3. The method of claim 1 or 2, comprising image data of 12. The method of claim 11, comprising operating the active light emitting elements of the active display at a high current while proportionally reducing the luminance of complementary image components.

13. The at least second complementary image data sequence (C1 ｋ ; C2 ｋ ; C3 ｋ 12. The method of claim 11, wherein the HDFR image slots presenting an image of a preceding or succeeding HDFR image slot include image components of the preceding and succeeding HDFR image slots.

14. The method of claim 1 or claim 2, wherein the active display is an LED or OLED display.

15. A system for operating an active display comprising an array of active light emitting elements, the system comprising a control unit arranged to perform the method of claim 1 or 2.

Images