Artifact removal in captured images of LED brick displays

Abstract

Systems and methods for capturing moving images from a video display using pulse width modulation (PWM) control are disclosed. Specifically, systems and methods for synchronizing display timing with camera timing to reduce or eliminate artifacts in the captured moving images caused by asynchronous interaction between camera shutter / exposure timing and the timing of the display's PWM control signals are disclosed.

Description

[0001] Cross-reference of related applications

[0002] This application claims the benefit of priority to U.S. Provisional Patent Application Serial No. 62 / 706,500, filed August 20, 2020, entitled “Elimination of Artifacts in Captured Images of LED Tile Displays,” which is incorporated herein by reference in its entirety. Technical Field

[0003] This disclosure generally relates to the fields of LED displays and motion picture capture. In particular, this disclosure relates to systems and methods for motion picture capture of pulse width modulation (PWM) driven displays while minimizing or eliminating artifacts caused by shutter timing in the capture device. Background Technology

[0004] Motion capture is frequently performed using cameras employing a "rolling shutter," where the shutter scans the scene in a single frame. This type of image capture can be electronically simulated in digital cameras. Depending on the shutter speed setting, a global shutter can also produce a similar effect to a rolling shutter. A common shutter speed in motion capture is 180 degrees, meaning that 180 degrees of the 360-degree rotation cycle exposes the camera sensor.

[0005] When capturing moving images with this type of camera equipment, LED display bricks may struggle to capture them properly without artifacts because LED displays use pulse-width modulation (PWM) control, where the LEDs generate pulses at a rate very different from the camera's exposure cycle. LED pulses are extremely rapid and may often only partially refresh when the shutter of a scan-type camera is fully open. Other types of PWM-driven displays face the same challenge.

[0006] Typically, LEDs are grouped into multiplexed scan groups, with as few as four (4) LEDs or as many as thirty-two (32) or more in a group. This means that, due to PWM control, at any given moment, only one-quarter (1 / 4) to one-thirty-second (1 / 32) of the LEDs are on. Misalignment between the camera shutter and the LED scan groups can therefore produce pulse artifacts or banding artifacts on the camera output, as described below. Figure 11 As shown. (Note that in) Figure 11 In the images captured, the camera was set to a shutter angle of approximately 11 degrees, solely for the purpose of demonstrating sharpness, in order to make the artifacts more noticeable.

[0007] With the increasing use of LED display walls as backdrops for television and film production, new solutions are needed that offer improved image capture and reduce the computational resources required to achieve high-quality image capture. Summary of the Invention

[0008] In one embodiment, this disclosure relates to a method for capturing an image of a scene including at least one display driven by a PWM control signal. The method includes: initiating image capture with an image capture device at an initial time, wherein the PWM control signal is set to off at the initial time; setting the PWM control signal to on after a first portion of the shutter time of the image capture device; maintaining the PWM control signal on during an open shutter time of the image capture device; setting the PWM control signal to off at or before a second portion of the shutter time of the image capture device; and repeating each of the setting and maintenance of the PWM control signal on a series of image capture frames.

[0009] In another embodiment, this disclosure relates to a method for capturing images of a scene including at least one display driven by a PWM control signal using at least one image capture device, wherein the image capture device captures images in frames, wherein each frame includes at least one partial shutter duration, an open shutter duration, and a closed shutter duration. The method includes dividing each frame into multiple time slices; setting the PWM control signal to off during time slices that are wholly or partially aligned with the partial shutter duration; setting the PWM control signal to on during time slices that are wholly or partially aligned with the open shutter duration; initiating image capture under the PWM control signal setting; and maintaining the PWM control setting over a series of image capture frames.

[0010] In another embodiment, this disclosure relates to a method for capturing images of a scene including at least one display using at least one image capture device. The method includes displaying a first static color on the at least one display during a synchronization delay time starting at an initial time; displaying a second static color on the at least one display during a synchronization on-time starting after the synchronization delay time; displaying a third static color on the at least one display after the synchronization on-time; capturing an image of the at least one display device using the image capture device; adjusting the synchronization delay time and the synchronization on-time until the first static color and the third static color no longer appear in the captured image of the at least one display; changing the second static color of a desired video feed; and capturing the scene including the at least one display showing the desired video feed.

[0011] In another embodiment, this disclosure relates to an image capture system. The system includes: an image capture device configured to capture moving images of a scene, the image capture device including a shutter mechanism for capturing images in a series of frames, each frame including an open shutter period, preceded and followed by partial shutter periods; a system display device including: a light-emitting display surface configured to display an image within the scene captured by the image capture device; a driver configured to drive the light-emitting display surface with a PWM control signal; and a control device that: sets the PWM control signal to off during a partial shutter period of the image capture device, and sets the PWM control signal to on during an open shutter period of the image capture device.

[0012] In yet another embodiment, this disclosure relates to a method for identifying an open shutter period of an image capture device during image capture of a scene including at least one PWM-driven display device. The method includes: displaying a first static color on the at least one display during the initial synchronization delay time starting at an initial time; displaying a second static color on the at least one display during the initial synchronization on time starting after the initial synchronization delay time; displaying a third static color on the at least one display after the initial synchronization on time; capturing an image of the at least one display device using the image capture device; adjusting the initial synchronization delay time and the initial synchronization on time until the first static color and the third static color no longer appear in the captured image of the at least one display; and identifying the time period during which the second static color appears on the display after adjustment as the open shutter period of the image capture device. Attached Figure Description

[0013] For the purpose of illustrating this disclosure, the accompanying drawings illustrate aspects of one or more embodiments of the disclosure. However, it should be understood that this disclosure is not limited to the precise arrangements and means shown in the drawings, in which:

[0014] Figure 1 A system according to an embodiment disclosed herein is illustrated schematically.

[0015] Figure 1A Alternative systems based on the embodiments disclosed herein are schematically depicted.

[0016] Figure 2 This is a timing diagram illustrating the shutter timing relative to the PWM control timing of an LED brick display according to an embodiment of this disclosure.

[0017] Figure 3A , Figure 3B and Figure 3C This is a timing diagram illustrating an example of a PWM timing and synchronization control method according to an embodiment of this disclosure.

[0018] Figure 4 This is a timing diagram illustrating the timing of an electronic roller shutter relative to the PWM control timing of an LED brick display, according to another embodiment of this disclosure.

[0019] Figure 5 An embodiment of a graphical user interface according to this disclosure is shown.

[0020] Figure 6 Another embodiment of a user interface configured to allow direct time control of synchronization delay parameters is shown.

[0021] Figure 7 This is a schematic side view of an LED brick.

[0022] Figure 8 This is a timing diagram illustrating the synchronization of multiple cameras with PWM control signals in another embodiment of this disclosure.

[0023] Figure 9 This is a schematic depiction of another alternative embodiment of a three-dimensional light-emitting display and two image capture devices according to this disclosure.

[0024] Figure 10 This is a block diagram illustrating an example of a system, camera, or display controller according to this disclosure.

[0025] Figure 11 The image capture from the LED brick video display exhibits artifacts caused by the use of an uncompensated rolling shutter. Detailed Implementation

[0026] This disclosure describes systems and methods for capturing moving images from a video display using pulse width modulation (PWM) control. In the disclosed systems and methods, artifacts appearing in the captured images due to asynchronous interaction between the camera shutter / exposure timing and the timing of the display's PWM control signals are reduced or eliminated by precisely controlling the display's on and off times. The systems and methods disclosed herein divide each frame of video into, or logically "slice" it into, multiple subframe partitions ("time zones"), and each partition can then be individually controlled relative to the shutter / exposure timing of the image capture device to allow precise timing control of the PWM signals relative to the shutter timing of the image capture device.

[0027] While LED displays have been mentioned in exemplary embodiments for illustrative purposes herein, it will be understood by those skilled in the art that the principles of this disclosure are equally applicable to any type of display employing PWM control. Examples of other display types include, but are not limited to, organic light-emitting diodes (OLEDs), polymer light-emitting diodes (PLEDs), active-matrix light-emitting diodes (AMOLEDs), liquid crystal displays (LCDs), or light-emitting electrochemical cells (LECs). Therefore, the scope of this disclosure and the appended claims is not limited to the illustrative examples of LED displays.

[0028] Figure 1 Basic components of an example system according to this disclosure are shown. As shown therein, system 100 includes an image capture device 110, such as a camera with a digital image sensor. The image capture device 110 is used to capture a moving scene that includes an LED display 112 within the camera's field of view (B) and may optionally include a presenter (126) or other live-action scenes appearing in front of the display. The LED display 112 may include a large video display wall formed by assembling panels comprising an array of LED pixels. Basic knowledge of constructing and controlling such large LED video display walls is generally understood in the art. International application PCT / US2020 / 057385, filed by the applicant on October 26, 2020, entitled "Unlimited PixelCanvas For LED Video Walls," describes embodiments of a next-generation large LED video display wall with improved structure and control, and that international application is incorporated herein by reference in its entirety. The image capture device 110 and the display 112 are controlled by a camera controller 114 and a display controller 116, respectively. Each controller includes one or more processors, memory, software, communication, APIs, and other components described in more detail below. The bidirectional communication links 120 and 122 between the controller and the controlled device can be wireless or wired. Optionally, communication link 124 can be provided between controllers 114 and 116.

[0029] Figure 1A An alternative system embodiment 100A is shown, in which image capture device 110 and display 112 are controlled by a common controller 118. Controller 118 can be configured in a substantially similar manner to controllers 114 and 116 and is further described below. Furthermore, wireless or wired communication links 120 and 122 provide bidirectional communication between the controller and the controlled device.

[0030] Table 1 below provides a basic illustration of the shutter timing of image capture devices such as Camera 110. This basic illustration describes the shutter timing of both mechanical "rolling shutter" type capture devices, such as traditional action picture cameras, and various types of digital capture devices employing digital image sensors. As shown in Table 1, there are three different camera shutter states during the time period of a single frame: the "C" state represents the time window when the shutter is fully closed (closed shutter period), the "O" state represents the time window when the shutter is fully open (open shutter period), and the "P" state represents the time window when the shutter is opening or closing such that the amount of light allowed to reach the sensor during these time fields will change with shutter movement (partial shutter period).

[0031]

[0032] The duration and timing of each of these time windows are specific to a particular image capture device and may vary from device to device. Therefore, the teachings of this disclosure apply to any image capture device (whether mechanical or digital analog) that employs a shutter mechanism in the form of shutter states shown in Table 1.

[0033] During the P-time field (also referred to in this paper as "partial shutter"), not all photons emitted by the source reach the image sensor, whether it is a film sensor or a digital sensor. To reduce or eliminate this... Figure 11 The artifacts generated by the pulses of the PWM-controlled LED display shown are evident; the LED tile should not output any light during the P-time field. During the O-time field, the LED tile should complete all PWM refresh cycles, and no output from the LED tile is required during the C-time field. However, if the LED tile is also being observed in person by a human viewer (in addition to the image capture device), a comfortable level of light output can be emitted during the C-time field to avoid visible flicker. It should be further noted that the time interval between the P-time field and the O-time field can be varied based on fixed camera parameters such as the characteristics of the digital sensor, and selectable camera parameters such as the shutter angle setting used to generate the image for the LED display.

[0034] Figure 2This illustration demonstrates the timing control of LED PWM control signals relative to the digital camera sensor charging state across two camera shutter periods for some embodiments disclosed herein. Top row box 10 indicates the digital sensor charging states (1 to 8) starting at a relative time reference (T0) (i.e., the start of a single camera shutter period). The camera shutter period begins at T0 and ends at T5, which is also the T0 of the next shutter period. A single shutter period corresponds to a single video capture frame. Wider black lines 20 indicate analog shutter open to closed states, where solid black line portion 22 represents fully closed (C in Table 1), dashed line portion 24 represents partially open (P in Table 1), and gap portion 26 represents a fully open state (O in Table 1). Line 30 represents the PWM control signal for the LED display, and various arrows (A to C) indicate different time control windows across shutter periods.

[0035] Based on the digital sensor's charging state, the shutter speed occurs during the following periods:

[0036] (1) At the start of each exposure, the sensor resets all photosensitive points to clear any charge from them. This has been conventionally taken as the start of exposure time from the camera's perspective, without regard to the teachings of this disclosure.

[0037] (2) The photosensitive points are reset one by one, for example, starting with the photosensitive point at the top left, creating a partial charge state. The reset process scans down line by line to the bottom right. During this partial charge state, only a portion of the light generated by the image source is received by the camera sensor. (Some sensors scan in different directions, but the effect is the same).

[0038] (3) The exposure time is fully opened when the full sensor reset occurs. This is the start of time window B.

[0039] (4) During the fully open exposure time, the photosensitive spots accumulate charge based on how much light they receive from the image source. The more light falls on each photosensitive spot, the higher the charge. Constrained by other camera settings such as aperture size, all light emitted by the source is received by the sensor during this time.

[0040] (5) Once the exposure time is fully open and the exposure time is over, measure and read the charge in each photosensitive spot. This is the end of time window B.

[0041] (6) Readout also begins from the upper left. In addition, this is a partial charging state, during which only a portion of the light generated by the image source is received by the camera sensor.

[0042] (7) Read out the scan line by line down to the lower right until all photosensitive points are read, thus ending the partial charging state.

[0043] (8) At the end of the shutter speed period, after the photosensitive point has been read out, there is usually a period of time during which the photosensitive point converts light into electrical charge. Typically, during this period, any captured charge is ignored until the next reset. The length of this period will depend on the specific camera / sensor design, settings, and controls.

[0044] During time window B, when the shutter is fully open with the digital sensor receiving all charge (sensor states (3) to (5) corresponding to time field O in Table 1), the LED PWM control signal 30 is transmitted in its fully on state 32, which is a pulse control configured to present the desired tile brightness, as generally understood for PWM control of LED tile displays. During partial shutter periods, such as sensor states (1) to (2) corresponding to time window A2 and sensor states (6) to (7) corresponding to time window C1, the LED PWM control signal 30 must be set to off 34 to minimize or eliminate artifacts in the moving image captured from the LED display. During the time when the shutter is fully closed (e.g., digital sensor state (8) corresponding to time windows C2 and A1), no PWM signal is required from the perspective of image capture alone. However, if visual appearance is a problem for the on-site viewer of the display, an optional PWM signal 36 can be passed to the display. This optional PWM signal can correspond to the overall video stream or can present momentary static colors or other still images to smooth out or eliminate any visually perceptible flicker on the display.

[0045] Usually about Figure 2 Time T0 represents the start of the shutter duration for a single frame. Time T1 represents the start of the fully open shutter window, during which the LED tiles will be at their full display value, and time T2 represents the end of the fully open shutter window. Time T3 represents the end of the shutter-closed state and the start of the shutter-closed duration. Time T4 represents the midpoint of the camera sensor off time within the shutter duration. Time T5 represents the end of the shutter duration and corresponds to time T0 of the next subsequent shutter duration. Therefore, the length of time window B (time field O in Table 1) when PWM control is set to on corresponds to T2 to T1. As explained further below, the minimum PWM off time for artifact removal corresponds to time windows A2 (T0 to T1) and C1 (T2 to T3).

[0046] While the LED display should be on as much as possible when the digital sensor is fully exposed (i.e., time window B) (this provides a good image on the camera without artifacts), a noticeable flicker effect can occur, as mentioned, if the display or a large segment of it remains off for the remainder of the shutter speed. However, balancing artifact elimination with flicker suppression introduces additional complexity, as the display's LEDs must still cycle off at least during the partial shutter speed states of time windows A2 and C1. Therefore, optimally, there exists a very short and precise time window during which the PWM control signal should be off to present a good on-off camera appearance.

[0047] To further explain the principles behind this disclosure, Figure 3A , Figure 3B and Figure 3C An example of synchronizing camera settings with PWM signal timing is presented based on a camera set to a 180-degree shutter angle and 60 frames per second (fps). As is known in the art, under these settings, the camera frame time is 16.6 milliseconds (ms) and the nominal exposure time is 8.3 ms. As explained above, the nominal exposure time includes the camera sensor charging and discharging time. Therefore, if the camera sensor has a total charging / discharging time of 5.0 ms, the time window available for clear video capture (i.e., time window B) starting after the initial 2.5 ms sensor charging time is 3.3 ms. However, if the camera manufacturer has not published the sensor's timing specifications, the operator may not know how to allocate the 8.3 ms total exposure time between charging / discharging and full opening, and therefore when to set the PWM signal to on and off, as described above.

[0048] To address this issue, in some embodiments, the operator can use visual feedback from the captured images to determine the correct timing settings, such as in... Figure 3A and 3B As shown in [the diagram]. Timing synchronization begins at T0, at which point the camera is turned on or otherwise begins capturing the initial frame. Before T0, the display PWM signal 30 (represented here as a 1000Hz PWM signal at 50% duty cycle) can initially be turned on or off at 38 before the first camera frame. The portion of the PWM signal at 34 is off, and the portion at 32 is on.

[0049] Given the selected frame rate and camera shutter angle, and based on familiarity with the system equipment and its performance, the operator can make an initial estimate of the synchronization delay time and synchronization on-time. The synchronization delay time is the time before shutter window B is fully open, i.e., in... Figure 2Within time window A2 between times T0 and T1, the PWM signal should be turned off to avoid capturing artifacts in the image. The synchronization turn-on time is the time after the synchronization delay corresponding to the fully open shutter window B, that is, in... Figure 2 During the time interval between T1 and T2, the PWM signal should be turned on for optimal image capture. For the example camera system described in the preceding paragraphs, Figure 3A The results are shown with an initial synchronization delay time estimate of 2.0 ms and an initial synchronization on-time estimate of 5.0 ms. After initiating image capture under these initial synchronization time settings, the operator observes the captured image to determine if there are any partial shutter states in the captured image due to the display being in A2 and C1 (e.g., in...). Figure 2 The artifact is caused by the activation of one or two states during the period identified in the image.

[0050] Figure 3A The diagram illustrates artifacts that an observer would see at both the leading and trailing edges of the fully open shutter window B in the captured image. The artifact at the leading edge of the fully open shutter window B is caused by the first PWM pulse 32 falling at the end of the 2.0ms synchronization delay but before time T1, because the light emitted in this first pulse will only be partially captured by the image sensor in a partially charged state. Similarly, the artifact at the trailing edge of the fully open shutter window B is caused by the last PWM pulse 32 falling after the closing of the fully open shutter window B, after time T2, which again occurs between times T2 and T3 in a partially charged (discharged) state of the camera sensor. By identifying these artifacts in the captured image, the operator knows that the synchronization delay is too short and the synchronization on-time is too long for that particular camera setting.

[0051] Using the knowledge gained from the initial synchronization time estimate, the operator can make a corrective estimate, increasing the synchronization delay time to further postpone the PWM turn-on signal 32 (to eliminate leading edge artifacts) and decreasing the synchronization turn-on time to further advance the PWM turn-off signal 34 when the shutter window B is fully open (to eliminate trailing edge artifacts). Therefore, as in Figure 3B As shown, the synchronization delay time can be set to 2.5ms and the synchronization on-time is set to 3.5ms. A 2.5ms synchronization delay activates the PWM, ensuring the first on-pulse 32 falls at time T1 (the start of fully opening shutter window B). With this setting, the operator will not observe leading-edge artifacts in the captured image and therefore knows that time T1 (the start of fully opening shutter window B) has been correctly identified. Also, as in... Figure 3B As shown, the 3.5ms synchronization turn-on time causes the last turn-on pulse 32A of the PWM signal 30 to span time T2, where the time portion X of pulse 32A falls within the subsequent shutter speed period ( Figure 2Within window C1). In this particular example, the overlap time portion X of the final pulse 32A is 0.2ms. Depending on various other factors in the camera setup and display parameters, a pulse 32A falling 0.2ms after time T2 may or may not produce trailing edge artifacts significant enough to unacceptably degrade the appearance of the captured image.

[0052] In cases where trailing edge artifacts caused by pulse 32A partially falling after time T2 unacceptably degrade the captured image, several control options are available in alternative embodiments. Typically, depending on the PWM frequency and duty cycle, the pulse may not be precisely aligned with the available shutter window B. For example, as in Figure 3B In this example, with the PWM signal driven at 1000Hz and 50% duty cycle, the leading edges of the pulses fall 1.0ms apart and each has a 0.5ms on-time. With the camera sensor timing parameters fixed as in this example, the system's physical constraints mean that the PWM signal cannot be turned on at T1 and then turned off exactly 3.3ms later at T2, since the PWM signal can only be controlled in 0.5ms segments. One solution is to further reduce the synchronous on-time sufficiently to shift the last pulse so that it falls entirely within the fully open shutter window B. In this specific example, this means reducing the synchronous on-time to less than 3.0ms, so that only three complete pulses occur during the fully open shutter window B between times T1 and T2. However, in some cases, such a solution may degrade aspects of the captured image, such as brightness, because it may not maximize the use of the available fully open shutter time.

[0053] In some instances, another possible solution is to adjust the PWM signal so that the pulses are more precisely aligned with the fully open shutter window B. In one option, this can be done by changing the duty cycle of the PWM signal while maintaining the frequency. Reducing the duty cycle increases or decreases the on-time of each pulse without changing the leading edge spacing. Therefore, in Figure 3B As shown, with the same synchronization delay and synchronization turn-on time, a 1000Hz PWM signal at 30% duty cycle instead of 50% will place four fully-on pulses within the fully open shutter window B, with no part of the final pulse falling after time T2. However, reducing the duty cycle may not be an option in some cases, as it also affects the displayed image. If hardware constraints, such as the capabilities of the LED tile controller, permit, another option is to adjust the PWM frequency to align the pulses with the time window. Figure 3CAn example of such an adjustment is shown. In this case, increasing the PWM frequency from 1000Hz to 2000Hz (maintaining 50% duty cycle) allows for the same 2.5ms synchronization delay and 3.3ms synchronization turn-on time without placing the pulse after time T2, because the trailing edge of the final pulse 32 falls at 3.25ms when the shutter window B is fully opened at 3.3ms. As those skilled in the art will note, the 1000Hz and 2000Hz PWM signals used in these examples facilitate the explanation of the principles of this disclosure in a simplified manner. In practice, high-resolution video walls can operate at frequencies of 8000Hz or higher. The principles of this disclosure are equally applicable to systems operating at such higher frequencies.

[0054] In a further alternative embodiment, a convenient way to implement the control method of this disclosure in a processor-controlled system is to create multiple regular time control slices within each camera frame. Since each camera model and manufacturer will vary the timing of shutter and / or charging states, it may be desirable to create these time control slices as virtual “time zones” within each shutter duration to individually select which time slices should display video (PWM on) and which should not (PWM off). Theoretically, the larger the number of time slices, the finer the control can be. However, the number of time slices can be limited based on the capabilities of the LED chips (a combination of processing power, driver ICs, etc.).

[0055] exist Figure 4 The diagram illustrates the use of this virtual time control slice, where nine equal time control slices are superimposed on a single shutter speed timing diagram. Each time control slice is designated as (1) through (9). As shown therein, baseline 40 indicates the start / end of each time control slice. In this example, to avoid unwanted artifacts in the captured video, the PWM signal 30 is set to off 34 during at least time control slices (1), (5), and (6). This solution is compatible with LEDPWM methods and typical camera systems and can be tuned across a variety of products. Alternatively, during time control slices (7) through (9) aligned with the time when the camera shutter is fully closed, the PWM signal can optionally be turned on 36 to display video or still images / colors to smooth out flicker effects without affecting the captured image.

[0056] Utilizing higher-performance LED display chips allows for the creation of more (smaller) timing control chips, thereby enabling more precise control over the on / off switching of PWM control signals. For example, those skilled in the art will recognize that... Figure 4The end (T2) of the fully open shutter window B falls approximately at the midpoint of the time control slice (5). Using nine time control slices as shown, a decision can be made regarding whether to turn the PWM control signal on during the time control slice (5), which may produce artifacts during partial sensor charging, or to turn the PWM control signal off, which would then not fully utilize the open shutter window B and may impair the characteristics of the captured image. To avoid the necessity of such a trade-off, in some cases it may be desirable to double the number of time control slices to more precisely align the PWM control with the shutter timing. In this example, if eighteen time control slices are used instead of nine as shown, the end of the seventh slice will be aligned with time T2, and thus allow at least one additional on pulse of the PWM control signal to be within the fully open shutter window B.

[0057] In a further alternative embodiment, a synchronization phase parameter can be incorporated to allow adjustment of the phase of the camera and LED timing. It may be preferable to "lock" both the camera timing and the PWM timing together, where the rate of camera exposure is the same as the rate of the PWM on / off cycle. However, while these systems can update at the same rate, they will typically be out of phase, but they will have a consistent phase offset. Therefore, to ensure that the time zones described above are aligned as well as possible with the fully on / positive partially on / positive off periods of the camera sensor, the synchronization phase parameter can be slightly shifted in phase of the LED on / off cycle. Figure 4 An embodiment of such synchronization phase parameters is illustrated graphically, where line 40 indicates the start of an independent time control slice as described above, and the synchronization phase parameter is represented by a delay time 42 or an advance time (negative delay) 44. The synchronization phase parameter can be a software-implemented feature in the video processor / control system that allows for the shifting of time zones as shown, i.e., adding several nanoseconds and shifting to the right, or subtracting several nanoseconds and shifting to the left.

[0058] Based on the teachings of this disclosure, users of the disclosed systems and methods, such as filmmakers, artists, or camera operators, can select their preferred camera shutter angle based on desired artistic effects, lighting conditions, lens effects, set design, etc. In one aspect of control, the display / video signal PWM cycle can be reduced, and the pulses slightly widened to reduce the number of timing zones, for example, by increasing the time interval of the zone. In another aspect of control, the desired shutter angle can be adjusted relative to the available frame refresh time. By shifting the timing of the video signal and the camera shutter angle, the desired display timing frame is captured along with any live shots in the foreground. The embodiments disclosed herein thus provide far more control options than prior art systems by facilitating complementary control of both the display PWM signal and the camera timing / shutter angle. Therefore, while the display / video signal control options may be limited by the number of PWM cycles that can be shown per frame interval, when combined with shutter timing / angle control, virtually unlimited artistic effects can be created based on user preferences.

[0059] In a further aspect of this disclosure, a user interface is provided to assist in finding the correct synchronization timing. In one embodiment, as in Figure 5 As shown, the user interface 130 can be used to facilitate user control of synchronization delay time and synchronization on-time parameters based on visual feedback from images generated during setup routines. Furthermore, the user interface 130 and control system can be configured to display solid colors during specific timing slices. Displaying easily recognizable solid colors on the display allows for the combination of the above... Figure 3A and Figure 3B The described visual verification of timing alignment. For example, if red is displayed for the first timing slice, followed by green for the middle timing slice, and then trailing red for the final timing slice, these reference colors can be used to more easily adjust the synchronization timing so that the camera only captures the red + green of the displayed reference image.

[0060] User interface 130 allows the user to precisely select the desired start, end, and on times of the PWM control signal for the LED tiles using a display controller such as controller 116 or 118. User interface 130 includes a graphical representation of the shutter timing 132 of a specific image capture device to be used. The shutter timing of a specific camera can be obtained from the manufacturer's specifications or can be set in the camera, for example, via the camera controller. In some embodiments, the image capture device can communicate directly with the LED display control system to automatically input, for example, the shutter timing of the LED display. Figure 1AThe shutter timing information shown is illustrated. A slider 134, which can be a virtual touchscreen slider, allows the user to set the LED PWM control on-time to correspond to the shutter opening time and fine-tune the relationship to achieve the desired appearance of the image produced by the camera. Each end of the slider can move independently to change the synchronization delay at the beginning of the shutter frame and the dead time at the end. Visual feedback from the display allows the slider position (i.e., the synchronization delay value) to be determined, thereby providing the desired appearance in the captured video stream.

[0061] The test color selection section 136 of the user interface 130 allows for finer tuning of the LED sync delay and shutter open time using color selection boxes 1, 2, and 3. In this illustrative example, three color selection boxes are provided, corresponding to the sync delay time, sync on time, and off / dead time at the end of the frame period. Additional color selection boxes can be provided based on user preferences. Alternatively, any still image can be displayed instead of a static single color. Providing selectable color options for syncing with a specific shutter period is highly useful in finding the off / open / open / closed / closed portion of the camera shutter, especially when the camera manufacturer has not disclosed details of how and when the camera is open. For example, a user can set color 1 to "red" and move the sync delay slider 134 forward and backward until the red color is in the "open" area of ​​the shutter period (i.e., Figure 2 The LED PWM "on" time is set until it is almost invisible in time period B. This allows for precise identification and setting of the start of the LED PWM "on" time, after which color 1 can be reset to black. Next, color 3 can be set to another color, such as "green," and the dead time (PWM "off" period) can be adjusted forward and backward until the captured image is almost invisible in green, after which color 3 can be reset to black. To confirm these settings made by the slider, color 2 can be selected to confirm that the synchronized on-time is aligned with the image capture device to record and output the selected color without artifacts. Once the settings for eliminating artifacts are confirmed, "color 2" is reset to "video," after which the monitor and image capture device can be used normally, where the LED display is synchronized to the shutter time of the image capture device to avoid artifacts in the captured image stream.

[0062] In situations where the timing of a specific camera is unknown, the slider 134 of the user interface 130 ( Figure 5 Together with color selection box 136, it can be used to visually identify the available shutter opening window and set the PWM timing, as described. Alternatively, when a specific camera timing (e.g., shutter opening / closing time and charging / discharging time) is known, the available shutter opening window can be pre-calculated and used as described... Figure 6The alternative user interface shown allows for setting the appropriate PWM cycle length and off time.

[0063] In some embodiments, the user interface 130 may be configured as a graphical user interface (GUI) presented on a user interface display device based on interface instructions executed by a system processor. Such a GUI may present multiple GUI areas, for example, a first GUI area 130a containing control elements as discussed above, and a second GUI area 130b containing a display window 138 presenting a captured image. Figure 11 Examples of captured images are presented in GUI display windows such as window 138.

[0064] In another alternative embodiment, aspects of synchronous control can be embedded in the individual bricks to reduce the bandwidth requirements for brick-to-brick and brick-to-system control communication. Figure 7 A typical LED display tile 150, such as that used to create a video wall, is shown. Advanced components include an LED pixel array 152 and a component housing 154, which includes components such as a tile controller 156 and multiple connectors 158. The tile controller 156 includes functions for processing and storing data to perform actions such as video packet switching, directing video signals to appropriate pixels, and communication between and through the tiles with system control. Each tile controller 156 also controls PWM signals in many tile designs. Tile connectors 158 are typically located on the edge of each tile to facilitate communication with adjacent tiles to relay tile identification information, control signals, and video signals. The connectors can be physical connections or communicate via various wireless protocols, as those skilled in the art will understand.

[0065] In typical applications, the video source is fed into a processor, which divides the feed into brick-sized pieces (or, in IP-based systems, packages the video feed) and sends these pieces or data packets to the bricks. The bricks then transmit this data as a PWM signal, thereby reproducing the video source on the display wall at a brightness set by the PWM signal. Processing the video data and sending it from the controller / processor to the bricks utilizes CPU capacity and communication channel bandwidth. Adding multiple additional time zone controls to the video signal and embedding additional static color signals within specific timing intervals utilizes additional CPU capacity and communication channel bandwidth. In some systems, very high-resolution video wall systems may already be operating at near-CPU capacity and / or communication channel bandwidth, and therefore the synchronization and flicker smoothing techniques disclosed herein cannot be utilized if the time zone and / or static color control signals are embedded along with the video signal. To reduce the demand on the CPU or communication channel, instead of embedding these control signals along with the video signal, the timing of the time zones and the on / off commands and / or the timing of the static color displays (e.g., ...) are... Figure 2During time periods A1 and C2, instructions stored in the tile memory are sent in advance to the tiles (as described above) to enable and disable the PWM of the tile controller and / or display the desired color during the specified timing. By sequencing the control signals in this way and utilizing existing tile memory and processing power, CPU capacity and / or communication channel bandwidth can be freed up to allow for more video to more tiles or higher resolution video, thereby improving controller operation, communication, and overall system performance.

[0066] As a practical example of the above, to provide four subframes for a 60fps system, video could be transmitted at 240fps. However, this high frame rate reduces the overall system processing capacity by a factor of four, requires more sophisticated equipment upstream to send faster data, and may therefore not allow for precise fine-tuning of shutter timing synchronization as described herein. Generating subframe timing zones and static fill colors at the endpoints (i.e., in the brick controller) based on pre-transmitted instructions makes shutter synchronization control possible in this system, which might otherwise be impossible. For example, adding chroma key colors or inverting video fields can be pre-computed and does not require a video payload to transmit such information.

[0067] Figure 8 Further alternative embodiments are shown, in which multiple cameras are used, capturing with different phase shifts, thus exposing at different portions of the video frame time and capturing different video time control slices to allow for a wider variety of video effects. In the example shown, eight (8) time control slices are used to control the video frame time and two cameras are used. Camera 1 and Camera 2 are configured to have the same shutter frequency but with phase shifts. Using as... Figure 8 The control sequence shown allows the “on” time of the LED PWM signal (30) to be independently aligned with each camera if sufficient subframe time control chips are used. As shown therein, time control chips (1) and (2) are aligned with the opening shutter window (B) of camera 1. V1 The corresponding video clips, and the time control clips (6) and (7) are related to the opening shutter window (B) of camera 2. V2 The corresponding active video clips. Based on the principles described above, the time control clips (3), (5) and (8) aligned with the partial charging state of the camera sensor are set to display black to avoid artifacts in the captured video stream, and the time control clip (4) falling within the shutter closing time of the two cameras displays a static color as needed to reduce flicker for live viewers. Figure 8The timing of the two cameras shown is merely illustrative of a multi-camera implementation. Any number of cameras can be used based on the teachings presented herein, typically limited only by the capabilities of the device. Multi-camera implementations offer improvements over existing systems by allowing different background video content to be displayed within frames between specific time sequences, thus permitting different cameras to capture different backgrounds while still focusing on the same foreground action.

[0068] Figure 9 A further alternative system 160 is shown, employing an luminescent three-dimensional volume as a display 112. This alternative system has two image capture devices 110a and 110b, as described above. Figure 8 The timing of these image capture devices is described. In this embodiment, the luminescent three-dimensional volume is determined as follows: Figure 7 The array of luminescent bricks 150 shown is configured as follows. The controller 116 and communication links 120, 122, and 124 can be configured by those skilled in the art based on the teachings of this disclosure and as described elsewhere herein. System 160 allows for the capture of live-action scenes within a virtual environment provided by a three-dimensional volumetric, surround video, with all live-action scenes captured simultaneously by image capture devices 110a and 110b.

[0069] As those skilled in the art will understand, certain control configurations according to this disclosure may result in relatively short capture periods, meaning less light is received by the camera sensor compared to what would otherwise be possible. In such cases, several options can be employed to adjust the brightness of the captured image, such as increasing the PWM on-time (higher duty cycle) to make the display brighter, adjusting the camera aperture to allow more light in, removing any ND (neutral density) filters that the camera may be using, adjusting the shutter angle (within PWM limits), and increasing the camera's sensor sensitivity (ISO). In specific situations with particular equipment, other options for brightness control may be available to achieve any desired image capture quality or artistic effect.

[0070] In some embodiments, control functions such as camera controller 114, display controller 118, or brick controller 156 can be executed as follows: Figure 10 One or more computing devices 200 are shown in the example. In this example, computing device 200 includes one or more processors 202, memory 204, storage device 206, a high-speed interface 208 connected to memory 204 and high-speed expansion port 210, and a low-speed interface 212 connected to low-speed bus 214 and storage device 206. Each of components 202, 204, 206, 208, 210, and 212 uses various buses or as shown in the example. Figure 10Other suitable connections, indicated by arrows, interconnect the components. Processor 202 can process instructions for execution within computing device 200, including instructions stored in memory 204 or on storage device 206 to display graphical information via display 220 through GUI 218 or on an external user interface device coupled to high-speed interface 208. In other embodiments, multiple processors and / or multiple buses may be used in conjunction with multiple memories and various types of memory, depending on the circumstances. Furthermore, multiple computing devices 200 may be connected, each providing a portion of the necessary operation (e.g., as a server library, blade server group, or multiprocessor system).

[0071] The memory 204 stores information within the computing device 200. In one embodiment, the memory 204 is a computer-readable medium. In one embodiment, the memory 204 is one or more volatile memory cells. In another embodiment, the memory 204 is one or more non-volatile memory cells.

[0072] Storage device 206 provides large-capacity storage for computing device 200 and may contain information such as timing control, time slice size and / or static chroma, and timing as described above. In one embodiment, storage device 206 is a computer-readable medium. In various embodiments, storage device 206 may be a floppy disk device, hard disk device, optical disk device, magnetic tape device, flash memory or other similar solid-state storage device, or a device array including devices in a storage area network or other configuration. In one embodiment, a computer program product is tangibly embodied in an information carrier. The computer program product contains instructions that, when executed, perform one or more methods, such as the methods described above. The information carrier is a computer-readable or machine-readable medium, such as memory 204, storage device 206, or memory on processor 202.

[0073] High-speed interface 208 manages bandwidth-intensive operations for computing device 200, while low-speed controller 212 manages lower bandwidth-intensive operations. This assignment of responsibilities is merely exemplary. In one embodiment, high-speed interface 208 is coupled to memory 204, display 220 (e.g., via a graphics processor or accelerator), and high-speed expansion port 210 that can accept various expansion cards (not shown). In another embodiment, low-speed controller 212 is coupled to storage device 206 and low-speed expansion port 214. The low-speed expansion port, which may include various communication ports (e.g., USB, Bluetooth, Ethernet, Wi-Fi), may be coupled to one or more input / input devices (e.g., keyboard, pointing device, scanner) as part of GUI 218 or as a further external user interface, or, for example, via a network adapter to a networked device such as a switch or router.

[0074] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuits, integrated circuits, specially designed ASICs (Application-Specific Integrated Circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor (which may be dedicated or general-purpose), the at least one programmable processor being coupled to receive data and instructions from a storage system, at least one input device, and at least one output device, and to transfer data and instructions to the storage system, at least one input device, and at least one output device.

[0075] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and may be implemented in high-level programming languages ​​and / or object-oriented programming languages, and / or in assembly language / machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0076] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user, and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including sound, speech, or tactile input.

[0077] The systems and technologies described herein can be implemented in computing systems that include: back-end components (e.g., as a data server), middleware components (e.g., an application server), or front-end components (e.g., a client computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such back-end, middleware, or front-end components. Components of the system can be interconnected by wired or wireless digital data communications (e.g., a communication network) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), and the Internet.

[0078] Computing systems can include clients and servers. Clients and servers are typically located far apart and interact via communication networks. The client-server relationship arises from computer programs running on corresponding computers and having a client-server relationship relative to each other.

[0079] The following paragraphs describe additional alternative embodiments of the invention disclosed herein:

[0080] A method for capturing an image of a scene including at least one display driven by a PWM control signal, the method comprising: initiating image capture with an image capture device at an initial time, wherein the PWM control signal is set to off at the initial time; setting the PWM control signal to on after a first partial shutter time of the image capture device; maintaining the PWM control signal on during an open shutter time of the image capture device; setting the PWM control signal to off at or before a second partial shutter time of the image capture device; and repeating each of the setting and maintenance of the PWM control signal on a series of image capture frames.

[0081] The method as described in paragraph

[0057] further includes maintaining the PWM control signal off during the shutter-off period of the image capturing device during each of the captured frames.

[0082] The method as described in paragraph

[0057] further includes setting the PWM control signal to on during all or part of the shutter-off period of the image capturing device during each of the captured frames.

[0083] The method described in any of paragraphs

[0057] ,

[0058] or

[0059] , wherein setting the PWM control signal to on after a partial shutter time period comprises: setting the PWM control signal to on after an initial synchronization delay time following the initial time; identifying whether a leading edge artifact is present or absent in the captured image; and, when a leading edge artifact is present, increasing the synchronization delay time until the leading edge artifact is reduced or eliminated in the captured image.

[0084] The method described in paragraph

[0060] further includes, wherein setting the PWM control signal to be turned on after a portion of the shutter duration, further comprising: reducing the synchronization delay time until the leading edge artifact appears in the captured image when no leading edge artifact is present, and subsequently increasing the synchronization delay time to a value greater than the synchronization delay time at which the artifact appears in the captured image.

[0085] The method described in paragraph

[0060] , wherein setting the PWM control signal to off during or before the second shutter interval comprises: setting the PWM control signal to off after an initial synchronization on-time following the synchronization delay time; identifying whether trailing edge artifacts are present or absent in the captured image; and, when trailing edge artifacts are present, reducing the synchronization on-time until trailing edge artifacts are reduced or eliminated in the captured image.

[0086] The method described in paragraph

[0062] further includes setting the PWM control signal to off during or before the second shutter interval by increasing the synchronization on-time until the trailing edge artifact appears in the captured image when no trailing edge artifact is present, and then reducing the synchronization on-time to a value less than the synchronization on-time when the artifact appears in the captured image.

[0087] The method described in any of paragraphs

[0057] to

[0063] , wherein the setting step and the maintaining step comprise: displaying a first static color on at least one display during an initial synchronization delay time beginning with the initial time; displaying a second static color on at least one display during an initial synchronization on time beginning after the initial synchronization delay time; displaying a third static color on at least one display after the initial synchronization on time; adjusting the initial synchronization delay time and the initial synchronization on time until the first static color and the third static color do not appear in the captured image on the at least one display; selecting the initial synchronization delay time during which the first static color does not appear in the captured image as the synchronization delay time; selecting the initial synchronization on time during which the third static color does not appear in the captured image as the synchronization on time; setting the PWM control signal to on after the synchronization delay time; setting the PWM control signal to off after the synchronization on time; maintaining the PWM control signal on during the synchronization on time; and changing the second static color during the synchronization on time to achieve a desired video feed.

[0088] The method described in any of paragraphs

[0057] to

[0064] further includes: initiating image capture with a second image capture device during a second initial time period, wherein the second initial time period begins after a second partial shutter time of the first image capture device; setting the PWM control signal to on after a first partial shutter time period of the second image capture device; maintaining the PWM control signal on during an open shutter time period of the second image capture device; and setting the PWM control signal to off during or before the second partial shutter time period of the second image capture device.

[0089] The method as described in paragraph

[0065] further includes aligning a first portion of the shutter time of the first image capturing device with a second portion of the shutter time of the second image capturing device.

[0090] The method described in paragraph

[0066] includes the first image capture device and the second image capture device having a first total shutter speed period and a second total shutter speed period, respectively, and the method further includes maintaining a constant phase offset between the first total shutter speed period and the second total shutter speed period.

[0091] The method described in paragraph

[0066] , wherein the first image capturing device is in a closed shutter period during the open shutter period of the second image capturing device.

[0092] The method as described in paragraph

[0068] further includes: aligning the shutter closing period of the first image capture device with the shutter closing period of the second image capture device; and setting the PWM control signal to on during the aligned shutter closing period.

[0093] The method as described in any of paragraphs

[0057] to

[0069] further includes: dividing each image capture frame into a plurality of time slices; setting the PWM control signal to off during a time slice that is wholly or partially aligned with the partial shutter duration; and setting the PWM control signal to on during a time slice that is wholly or partially aligned with the open shutter duration.

[0094] The method described in paragraph

[0070] is wherein the time slices are equal in length and the full number of time slices are present in each image capture frame.

[0095] A method for capturing images of a scene including at least one display driven by a PWM control signal using at least one image capture device, wherein the image capture device captures images in frames, wherein each frame includes at least one partial shutter duration, an open shutter duration, and a closed shutter duration, the method comprising: dividing each frame into a plurality of time slices; setting the PWM control signal to off during a time slice that is wholly or partially aligned with the partial shutter duration; setting the PWM control signal to on during a time slice that is wholly or partially aligned with the open shutter duration; initiating image capture under the PWM control signal setting; and maintaining the PWM control setting over a series of image capture frames.

[0096] The method described in paragraph

[0072] is wherein the time slices are equal in length and the full number of time slices are present in each image capture frame.

[0097] The method as described in paragraphs

[0072] or

[0073] further includes setting the PWM control signal to on during a time slice aligned with the shutter closing period.

[0098] The method described in paragraphs

[0072] or

[0073] , wherein the PWM control signal is set to be on only during a time slice that is fully aligned with the shutter opening period or only during a time slice that is fully aligned with the shutter opening and closing periods.

[0099] A method for capturing an image of a scene including at least one display using at least one image capture device, the method comprising: displaying a first static color on the at least one display during the synchronization delay time starting at an initial time; displaying a second static color on the at least one display during the synchronization on-time starting after the synchronization delay time; displaying a third static color on the at least one display after the synchronization on-time; capturing an image of the at least one display using the image capture device; adjusting the synchronization delay time and the synchronization on-time until the first static color and the third static color no longer appear in the captured image of the at least one display; changing the second static color of a desired video feed; and capturing the scene including the at least one display displaying the desired video feed.

[0100] The method as described in paragraph

[0076] , wherein the at least one display comprises a display driven by a PWM control signal, and wherein the image capturing device captures images in frames, wherein each frame comprises at least one partial shutter duration, an open shutter duration, and a closed shutter duration.

[0101] An image capture system includes: an image capture device configured to capture moving images of a scene, the image capture device including a shutter mechanism for capturing images in a series of frames, each frame including an open shutter period, preceded and followed by partial shutter periods; a system display device including a light-emitting display surface configured to display an image within the scene captured by the image capture device, and a driver configured to drive the light-emitting display surface with a PWM control signal; and a control device that sets the PWM control signal to off during the partial shutter periods of the image capture device and sets the PWM control signal to on during the open shutter periods of the image capture device.

[0102] The system as described in paragraph

[0078] , wherein the control device: divides each image capture frame into multiple time slices; sets the PWM control signal to off during time slices that are fully or partially aligned with the partial shutter duration; and sets the PWM control signal to on during time slices that are fully or partially aligned with the open shutter duration.

[0103] The system as described in paragraphs

[0078] or

[0079] , wherein the control device includes: at least one microprocessor; and a memory operatively communicating with the at least one microprocessor, the memory containing machine-executable instructions that, when executed by the at least one microprocessor, perform the following operations: setting an initial time for initiating image capture; allowing a user to set a synchronization delay time starting from the initial time; allowing the user to set a synchronization on-time starting after the synchronization delay time; turning off the PWM control signal during the synchronization delay time starting from the initial time; turning on the PWM control signal during the synchronization on-time after the synchronization delay time; turning off the PWM control signal after the synchronization on-time; and resetting the initial time at the end of each image capture frame.

[0104] The system as described in paragraph

[0080] includes additional machine-executable instructions that, when executed by the at least one microprocessor, display a graphical user interface (GUI) on a user interface display device, the GUI having at least a first GUI area that allows the user to input a synchronization delay time value and a synchronization connection time value.

[0105] The system as described in paragraphs

[0080] or

[0081] , wherein the memory contains additional machine-executable instructions that, when executed by the at least one microprocessor, perform the following operations: displaying the GUI, the GUI having at least a second GUI area for displaying the image captured by the image capture device; and presenting within the first GUI area the following: (1) a user-selectable option for displaying a video feed or a first still color on the system display device during a synchronization delay time; (2) a user-selectable option for displaying a video feed or a second still color on the system display device during a synchronization on-time starting after the synchronization delay time; and (3) a user-selectable option for displaying a video feed or a third still color on the system display device after the synchronization on-time; thereby, adjusting the synchronization delay time and the synchronization on-time until the first still color and the third still color do not appear in the captured image of the at least one display, determines the synchronization delay time in each image capture frame of the image capture device corresponding to the initial partial shutter duration and the shutter opening duration.

[0106] The system described in any of paragraphs

[0080] to

[0082] , wherein the memory contains additional machine-executable instructions that, when executed by the at least one microprocessor, perform the following operations: dividing each image capture frame into multiple time slices; and setting the PWM control signal to be on or off by time slice.

[0107] The system as described in paragraph

[0083] further includes each frame of the image capture device including a shutter-closing period; and the memory further includes machine-executable instructions that, when executed by the at least one microprocessor, perform the following operations: allowing the user to set the on or off state of the PWM control signal during a time slice falling within the shutter-closing period; and allowing the user to select a still image or video feed for display when the PWM control signal is set to on during the shutter-closing period; and wherein the system display device includes an array of light-emitting tiles, each tile having a tile control configured for a PWM driver for the tile and including at least one tile microprocessor and a tile memory operatively communicating with the at least one tile microprocessor, the tile memory including machine-executable instructions that, when executed by the at least one tile microprocessor, display the still image on the tile when the user selects the still image.

[0108] The system described in any of paragraphs

[0078] to

[0084] further includes a shutter-closing period for each frame of the image capture device; and the control device allows the user to set the PWM signal to be on or off during the shutter-closing period.

[0109] The system described in any of paragraphs

[0078] to

[0085] includes a system display device comprising a three-dimensional (3D) luminescent display body comprising a three-dimensional array of luminescent bricks joined along the edges of the bricks to form a seamless 3D volumetric display wall, and each of the bricks comprising a unique PWM driver for each of the bricks; and the image capturing device comprising at least one camera having a digital image sensor electronically configured with the shutter mechanism.

[0110] The foregoing is a detailed description of illustrative embodiments of the disclosure. It should be noted that, in this specification and the appended claims, unless expressly stated or indicated otherwise, conjunctions used, such as in the phrases “at least one of X, Y, and Z” and “one or more of X, Y, and Z”, should be considered to mean that each item in the conjunction list can exist in any number without including every other item in the list, or can be combined in any number with any or all other items(multiple) in the conjunction list, each of which can also exist in any number. Applying this general rule, the conjunction phrases in the above examples consisting of X, Y, and Z should each cover: one or more of X; one or more of Y; one or more of Z; one or more of X and one or more of Y; one or more of Y and one or more of Z; one or more of X and one or more of Z; and one or more of X, one or more of Y, and one or more of Z.

[0111] Various modifications and additions may be made without departing from the spirit and scope of this disclosure. Features of each embodiment described above may be suitably combined with features of other described embodiments to provide multiple feature combinations in associated new embodiments. Furthermore, while several individual embodiments have been described above, the descriptions herein are merely illustrative of the application of the principles of this disclosure. Additionally, although specific methods herein may be described and / or described as being performed in a particular order, this order is highly variable within the realm of common art in order to implement various aspects of this disclosure. Accordingly, this description is intended to be exemplary only and does not otherwise limit the scope of this disclosure.

[0112] Exemplary embodiments have been disclosed above and illustrated in the accompanying drawings. Those skilled in the art will understand that various changes, omissions, and additions can be made to the specific disclosure herein without departing from the spirit and scope of this disclosure.

Claims

1. A method for image capture control of a scene including at least one display driven by control signals, the method comprising: The initial time is specified as the time when the image capture device initiates image capture, wherein the control signal is set to be off at the initial time; The control signal is set to turn on after the first portion of the shutter time of the image capture device; The control signal is kept on during the shutter opening period of the image capture device; The control signal is set to off during or before the second portion of the shutter duration of the image capture device; and The setting and maintenance of the control signal are repeated for each of a series of image capture frames.

2. The method as described in claim 1, wherein, The control signal is a PWM control signal.

3. The method of claim 1, further comprising, during each of the captured frames, maintaining the control signal off during the shutter-off period of the image capturing device.

4. The method of claim 1, further comprising setting the control signal to on during all or part of the shutter-off period of the image capturing device in each of the captured frames.

5. The method as described in claim 1, 2, 3, or 4, wherein, Setting the control signal to be on after a portion of the shutter duration includes: The control signal is set to turn on after an initial synchronization delay time following the initial time. Identify whether leading-edge artifacts are present or absent in the captured image; and When leading edge artifacts are present, the synchronization delay time is increased until the leading edge artifacts are reduced or eliminated in the captured image.

6. The method of claim 5, wherein, Setting the control signal to on after a portion of the shutter duration further includes: reducing the synchronization delay time until the leading edge artifact appears in the captured image when no leading edge artifact is present, and then increasing the synchronization delay time to a value greater than the synchronization delay time for the artifact to appear in the captured image.

7. The method of claim 5, wherein, Setting the control signal to off during or before the second shutter speed period includes: The control signal is set to off after the initial synchronization turn-on time following the synchronization delay time; Identify whether trailing edge artifacts are present or absent in the captured image; and When trailing edge artifacts are present, the synchronization turn-on time is reduced until the trailing edge artifacts are reduced or eliminated in the captured image.

8. The method of claim 7, wherein, Setting the control signal to off during or before the second shutter speed segment further includes: increasing the sync on time until the trailing edge artifact appears in the captured image when there is no trailing edge artifact, and then reducing the sync on time to a value less than the sync on time when the artifact appears in the captured image.

9. The method according to any one of claims 1, 2, 3 or 4, wherein, The setting step and the maintenance step include: A first static color is displayed on at least one display during an initial synchronization delay period starting at the initial time; A second static color is displayed on at least one display during the initial synchronization on-time, which begins after the initial synchronization delay time. A third static color is displayed on at least one display after the initial synchronization connection time; Adjust the initial synchronization delay time and the initial synchronization on time until the first static color and the third static color no longer appear in the captured image of the at least one display; The initial synchronization delay time is selected as the synchronization delay time when the first static color does not appear in the captured image. The initial synchronization connection time when the third static color does not appear in the captured image is selected as the synchronization connection time. The control signal is set to turn on after the synchronization delay time; The control signal is set to turn off after the synchronization turn-on time; The control signal is maintained on during the synchronization on time; and During the synchronization connection time, the second static color is changed to achieve the desired video feed.

10. The method of any one of claims 1, 2, 3 or 4, further comprising: Image capture is initiated using a second image capture device at a second initial time, wherein the second initial time begins after the second portion of the shutter time of the first image capture device; The control signal is set to turn on after the first portion of the shutter time of the second image capture device; The control signal is kept on during the shutter opening period of the second image capture device; The control signal is set to off during or before the second portion of the shutter time of the second image capturing device.

11. The method of claim 10, further comprising aligning a first portion of the shutter time of the first image capturing device with a second portion of the shutter time of the second image capturing device.

12. The method of claim 11, wherein, The first image capture device and the second image capture device each have a first total shutter speed period and a second total shutter speed period, and the method further includes maintaining a constant phase offset between the first total shutter speed period and the second total shutter speed period.

13. The method of claim 11, wherein, The first image capturing device is in the closed shutter period during the open shutter period of the second image capturing device.

14. The method of claim 13, further comprising: Align the shutter closing time of the first image capture device with the shutter closing time of the second image capture device; as well as The control signal is set to ON during the aligned shutter-closing period.

15. The method of any one of claims 1, 2, 3 or 4, further comprising: Each captured image frame is divided into multiple time slices; The control signal is set to off during a time slice that is fully or partially aligned with the partial shutter speed period; as well as The control signal is set to ON during a time slice that is fully or partially aligned with the shutter opening period.

16. The method of claim 15, wherein, The time slices are equal in length and the full number of time slices exists in each image capture frame.

17. A system for controlling the capture of a moving image, the moving image being displayed on a system display device using an image capture device, wherein, The system display device includes a light-emitting display surface and a driver. The light-emitting display surface is configured to display an image, and the driver is configured to drive the light-emitting display surface via a control signal. The image capture device includes a shutter mechanism for capturing a series of frames of images, each frame including a shutter-open period preceded and followed by partial shutter periods. The system includes a control device. The control signal is set to off during a portion of the shutter speed of the image capture device, and The control signal is set to be on during the shutter opening period of the image capturing device.

18. The system of claim 17, wherein, The control device: Each captured image frame is divided into multiple time slices; The control signal is set to off during a time slice that is fully or partially aligned with the partial shutter speed period; and The control signal is set to ON during a time slice that is fully or partially aligned with the shutter opening period.

19. The system of claim 17 or claim 18, wherein, The control signal is a PWM control signal, and the control device includes: At least one microprocessor; and A memory, which is in operative communication with the at least one microprocessor, contains machine-executable instructions that, when executed by the at least one microprocessor, perform the following operations: Set the initial time for initiating image capture; Allows users to set a synchronization delay time starting from the initial time; The user is allowed to set the synchronization connection time to begin after the synchronization delay time; The PWM control signal is turned off during the synchronization delay time starting from the initial time; The PWM control signal is turned on during the synchronization turn-on time following the synchronization delay time; The PWM control signal is turned off after the synchronization on-time; and The initial time is reset at the end of each image capture frame.

20. The system of claim 19, wherein, The memory contains additional machine-executable instructions that, when executed by the at least one microprocessor, perform the following operations: A graphical user interface (GUI) is displayed on a user interface display device, the GUI having at least a first GUI area that allows the user to input a synchronization delay time value and a synchronization connection time value.

21. The system of claim 20, wherein, The memory contains additional machine-executable instructions that, when executed by the at least one microprocessor, perform the following operations: The GUI is displayed, and the GUI has at least a second GUI area for displaying the image captured by the image capturing device; Presented within the first GUI area: User-selectable option for displaying the video feed or first static color on the system display device during the synchronization delay time; A user-selectable option for displaying a video feed or a second static color on the system display device during the synchronization on-time that begins after the synchronization delay time; as well as A user-selectable option for displaying a video feed or a third static color on the system display device after the synchronization connection time; Accordingly, the synchronization delay time and the synchronization on-time are adjusted until the first static color and the third static color do not appear in the captured image of the at least one display, thereby determining the synchronization delay time in each image capture frame of the image capture device corresponding to the initial partial shutter duration and shutter opening duration.

22. The system of claim 19, wherein, The memory contains additional machine-executable instructions that, when executed by the at least one microprocessor, perform the following operations: Divide each captured image frame into multiple time slices; and The PWM control signal is set to be on or off according to the time slice.

23. The system of claim 22, wherein: Each frame of the image capture device further includes a shutter-closed period; The memory further includes machine-executable instructions that, when executed by the at least one microprocessor, perform the following operations: The user is allowed to set the on or off state of the PWM control signal within a time slice that falls within the shutter-closing period. When the PWM control signal is set to ON during the shutter-off period, the user is allowed to select either a still image or a video feed for display; and The system display device includes an array of light-emitting tiles, each tile having a tile control configured for a PWM driver for the tile and including at least one tile microprocessor and a tile memory operatively communicating with the at least one tile microprocessor, the tile memory containing machine-executable instructions that, when executed by the at least one tile microprocessor, display the static image on the tile when the user selects the static image.

24. The system of claim 17 or claim 18, wherein: Each frame of the image capture device further includes a shutter-closing period; and The control device allows the user to set the control signal to be on or off during the shutter-closing period.

25. The system of claim 17 or claim 18, wherein: The system display device includes a three-dimensional 3D luminescent display body, which comprises a three-dimensional array of luminescent bricks joined along the edges of the bricks to form a seamless 3D volumetric display wall, and each brick includes a unique control signal driver for each brick; and The image capturing device includes at least one camera, the at least one camera having a digital image sensor electronically configured with the shutter mechanism.

26. A computer-readable storage medium storing instructions that, when executed by at least one processor, implement the method as claimed in any one of claims 1 to 16.

Images