Detecting black bars in video content

Abstract

A system for controlling a plurality of lighting units to present a light effect determined based on video content to accompany the presentation of the video content on a display is disclosed. The system includes at least one output interface and at least one processor configured to: position a first analysis region of a predetermined number of analysis regions at an initial position in the video content at which a first color is extracted from the video content, the first analysis region is repositioned relative to the video content to one or more subsequent locations in the video content and one or more subsequent colors are extracted from the video content at the respective one or more subsequent locations, determining the presence of a black bar in the video content based on the first color and / or the one or more subsequent colors, determining an area in the video content based on the first color and / or the one or more subsequent colors, the area comprising the black bar, determining a location of a subset of the predetermined number of analysis regions outside the region including the black bar, extracting one or more colors from one or more of the analysis regions of the subset, determining the light effect based on the extracted one or more colors, and determining the light effect based on the extracted one or more colors. And control the plurality of lighting units via the at least one output interface to present the light effect.

Description

Technical Field

[0001] The present invention relates to a system for controlling a plurality of lighting units to present a light effect determined based on video content to accompany the presentation of the video content on a display.

[0002] The present invention also relates to a method for controlling a plurality of lighting units to present a light effect determined based on video content to accompany the presentation of the video content on a display.

[0003] The present invention also relates to a computer program product enabling a computer system to execute such a method. Background Art

[0004] To create a more immersive entertainment experience, systems capable of controlling lighting units to change light according to video and / or audio content can be used. In other words, the lighting units are controlled to present entertainment light effects. An example of such a system is the Hue system, where an immersive entertainment experience can be created by a Hue (HDMI) Sync Box or a desktop PC (Hue Sync PC desktop application). In these systems, the audio and / or video are analyzed, and the light presented by the group of lighting units changes according to the analyzed content.

[0005] When controlling lighting units based on video content, analysis regions are typically used to ensure that different lighting units present different colors. For example, such that the lighting units on the left side of the TV present colors extracted from the left part of the video content, and the lighting units on the right side of the TV present colors extracted from the right part of the video content. For example, WO 2021 / 239560 discloses mapping different lighting units to different spatial regions (referred to as analysis regions) of the video content. In the method disclosed in WO 2021 / 239560, the size and / or position of the analysis region are determined based on the distance between the light source and a surface (such as a wall) to increase the mixing of the presented colors when the light source is positioned near the surface.

[0006] The Hue sync box also uses analysis regions, and the current Hue sync box extracts thousands of colors from thousands of small analysis regions among these analysis regions. However, if the analysis is to be performed on a device with limited resources or limited resources allocated for this task, the number of analysis regions needs to be reduced, for example, reduced to a few small analysis regions per analysis region. Without taking additional measures, this will typically result in poor light effects, such as flickering. Summary of the Invention

[0007] The inventors have realized that when video content contains black bars (e.g., horizontal or vertical), it is necessary to detect these black bars in order to provide a light effect based on the actual video content rather than the black color of the black bars. However, in the case of limited resources, black bar detection requires allocating appropriate resources for such detection. In addition, the size and / or position of the black bars may vary for each video, or even for each scene of a single video.

[0008] Accordingly, an object is to provide a system that can detect black bars in video content with limited resources to provide an attractive light effect.

[0009] According to a first aspect, this object is achieved by a system for controlling a plurality of lighting units to present a light effect determined based on video content to accompany the presentation of the video content on a display. The system includes at least one output interface and at least one processor, which is configured to:

[0010] - Locate a first analysis region among a predetermined number of analysis regions at an initial position in the video content,

[0011] - Extract a first color from the video content at the initial position,

[0012] - Reposition the first analysis region relative to the video content to one or more subsequent positions in the video content and extract one or more subsequent colors from the video content at the corresponding one or more subsequent positions,

[0013] - Determine the presence of black bars in the video content based on the first color and / or the one or more subsequent colors,

[0014] - Determine a region in the video content that includes the black bars based on the first color and / or the one or more subsequent colors,

[0015] - Determine positions of a subset of the analysis regions among the predetermined number of analysis regions outside the region that includes the black bars,

[0016] - Extract one or more colors from one or more of the analysis regions of the subset,

[0017] - Determine the light effect based on the extracted one or more colors, and

[0018] - Control the plurality of lighting units via the at least one output interface to present the light effect.

[0019] A predetermined amount of analysis regions, such as 16, is typically the maximum amount. This limitation may be imposed due to limited resources. By repositioning the first analysis region and extracting and analyzing colors from one or more subsequent positions of the first analysis region, the position of the black bar is detected, and colors are extracted from other analysis regions that are not located in the black bar region. At least one processor may be configured to iteratively repeat the repositioning until the position of the region including the black bar has been determined. Repositioning the first analysis region is beneficial because it enables black bar detection to be performed (in real time) using limited resources to provide an (attractive) light effect based on the actual video content (rather than based on the black bar).

[0020] The initial position may be a position that is substantially located at the edge of the video content. For example, the edge may be a horizontal upper edge or lower edge, or one of the vertical side edges. Alternatively, the edge may be a vertical edge. This enables the quick determination of whether there is a black bar using limited resources because black bars are typically located at the edges of the video content.

[0021] The at least one processor may be configured to reposition the first analysis region based on the correspondingly extracted colors (i.e., the first color and / or the corresponding subsequent colors). For example, the at least one processor may be configured to reposition the first analysis region towards the edge of the video content if the correspondingly extracted color is not black, and / or be configured to reposition the first analysis region away from the edge of the video content if the correspondingly extracted color is black. The at least one processor may be configured to iteratively repeat this process for each subsequent position until the position of the region including the black bar has been determined.

[0022] The at least one processor may be configured to reposition the first analysis region relative to the video content to one or more subsequent positions in the video content for corresponding subsequent moments in the video content. Thus, the first analysis region may be repositioned in an adaptive manner for different scenes, frames, or shots of the video content.

[0023] The initial position may be a predetermined initial position. For example, the initial position may be a predetermined position at or near the horizontal / vertical edge of the video content. For example, the predetermined position may be a position at predetermined pixel coordinates, a predetermined distance from the edge, a percentage of the horizontal and / or vertical edge of the video content, etc.

[0024] The at least one processor may be configured to determine the initial position based on the metadata of the video content. This is beneficial because by using this information, the position of the black bar can be determined more quickly with fewer resources.

[0025] The at least one processor may be configured to:

[0026] - Infer the existence of a second black bar on the opposite side of the video content based on the existence of the first black bar.

[0027] - Determine a second region in the video content, the second region including the second black bar, and

[0028] - Determine the positions of a subset of the analysis regions of the predetermined number of analysis regions outside the region including the black bar and outside the second region including the second black bar. The at least one processor may be configured to determine the existence of a first black bar in the video content and, based on this, infer that a second black bar also exists on the opposite side (e.g., opposite edge) of the video content. Typically, the video content has two black bars (vertical or horizontal), so detecting a single black bar and inferring the existence of a second black bar is beneficial as it reduces the resources required for black bar detection.

[0029] The at least one processor may also be configured to: (i) determine that no horizontal black bar exists based on the first color and / or the one or more colors, and sequentially reposition the first analysis region to determine whether a vertical black bar exists, and / or (ii) determine that no vertical black bar exists based on the first color and / or the one or more colors, and sequentially reposition the first analysis region to determine whether a horizontal black bar exists. This enables the detection of vertical and horizontal black bars with a limited amount of resources.

[0030] The system may include at least one input interface. The at least one processor may be configured to obtain, via the at least one input interface, for each respective lighting unit of the plurality of lighting units, a position associated with the respective lighting unit, the position being the position of the respective lighting unit relative to the display or the position at which the respective lighting unit creates a light effect relative to the display, and determine the positions of the subset of the analysis regions based on the positions of the plurality of lighting units.

[0031] The at least one processor may be configured to determine a first position of a subset of the analysis regions at a first moment in the video content and a second position of the analysis regions at a second moment in the video content. Thus, a limited number of analysis regions may be distributed adaptively based on, for example, the video content. Certain analysis region positions may be very suitable for the first scene or shot of the video content, while other analysis region positions may be very suitable for the second scene or shot of the video content.

[0032] According to a second aspect, this object is achieved by a method of controlling a plurality of lighting units to present a light effect determined based on video content to accompany the presentation of the video content on a display, the method comprising:

[0033] - Locate a first analysis region among a predetermined number of analysis regions at an initial position in the video content,

[0034] - Extract a first color from the video content at the initial position,

[0035] - Relocate the first analysis region relative to the video content to one or more subsequent positions in the video content, and extract one or more subsequent colors from the video content at the corresponding one or more subsequent positions,

[0036] - Determine the presence of black bars in the video content based on the first color and / or the one or more subsequent colors,

[0037] - Determine a region in the video content that includes the black bars based on the first color and / or the one or more subsequent colors,

[0038] - Determine positions of a subset of the analysis regions of the predetermined number of analysis regions outside the region including the black bars,

[0039] - Extract one or more colors from one or more of the analysis regions of the subset;

[0040] - Determine the light effect based on the extracted one or more colors; and

[0041] - Control the plurality of lighting units to present the light effect.

[0042] According to a third aspect, the object is achieved by a computer program product for a computing device, the computer program product including computer program code to perform the method when the computer program product runs on a processing unit of the computing device.

[0043] As will be appreciated by those skilled in the art, aspects of the present invention may be embodied as a device, a method, or a computer program product. Accordingly, aspects of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware aspects, which may generally be referred to herein as a "circuit", "module", or "system". The functions described in this disclosure may be implemented as algorithms executed by a processor / microprocessor of a computer. In addition, aspects of the present invention may take the form of a computer program product embodied in one or more computer-readable media having computer-readable program code embodied (e.g., stored) thereon.

[0044] Any combination of one or more computer-readable media can be utilized. The computer-readable media can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, by way of example and not limitation, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the computer-readable storage medium can include, but are not limited to, the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this invention, the computer-readable storage medium can be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0045] The computer-readable signal medium can include a propagated data signal having computer-readable program code embodied therein (e.g., in baseband or as part of a carrier wave). Such a propagated signal can take any of a variety of forms, including but not limited to electromagnetic, optical, or any suitable combination thereof. The computer-readable signal medium can be any computer-readable medium that is not a computer-readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

[0046] The program code embodied on the computer-readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wireline, fiber optic cable, RF, etc., or any suitable combination of the foregoing. The computer program code for carrying out operations for aspects of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages such as Java(TM), Smalltalk, C++, etc., and conventional procedural programming languages such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (e.g., through the Internet using an Internet service provider).

[0047] Aspects of the present invention will now be described with reference to the flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus (in particular a microprocessor or a central processing unit (CPU)) to produce a machine such that the instructions, which execute via the processor of the computer, other programmable data processing apparatus, or other device, create means for implementing the functions / actions specified in the flowchart and / or one or more block diagrams.

[0048] These computer program instructions may also be stored in a computer-readable medium that can direct a computer, other programmable data processing apparatus, or other device to function in a particular manner, such that the instructions stored in the computer-readable medium produce an article of manufacture including instructions for implementing the functions / actions specified in the flowchart and / or one or more block diagrams.

[0049] The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other device to produce a computer-implemented process such that the instructions executed on the computer or other programmable apparatus provide a process for implementing the functions / actions specified in the flowchart and / or one or more block diagrams.

[0050] The flowchart and block diagrams in the figures illustrate the architecture, functionality and operation of possible implementations of devices, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which includes one or more executable instructions for implementing the specified (multiple) logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may in fact be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, can be implemented by a system based on dedicated hardware for performing the specified functions or actions, or by a combination of dedicated hardware and computer instructions. Description of the Drawings

[0051] With reference to the accompanying drawings, by way of example, these and other aspects of the invention will be apparent and will be further elucidated, in which:

[0052] Figure 1 is a block diagram of a first example of the system;

[0053] Figure 2 is a block diagram of a second example of the system;

[0054] Figure 3a and 3b illustrates different techniques for black bar detection in video content;

[0055] Figure 4 illustrates an example of switching from horizontal black bar detection to vertical black bar detection; and

[0056] Figure 5 illustrates a method of controlling a plurality of lighting units to present a light effect determined based on video content to accompany the presentation of the video content on a display.

[0057] Corresponding elements in the drawings are labeled with the same reference numerals. Detailed Description

[0058] Figure 1 illustrates a first example of a system for controlling a plurality of lighting units to present a light effect determined based on video content to accompany the presentation of the video content on a display. In Figure 1 the first example, the system is a display device, such as a smart TV.

[0059] In Figure 1 the example, the display device 1 can control the lighting devices 11 and 12 via the bridge 19. For example, the lighting control can be performed by an application running on the display device 1. The lighting device 11 includes light sources / pixels 41 - 43, and the lighting device 12 includes light sources / pixels 44 - 46. For example, each of the light sources 41 - 46 can include one or more LEDs. For example, the bridge 19 can be a Hue bridge. The bridge 19 communicates with the lighting devices 11 and 12 using, for example, Zigbee technology. The display device 1 is connected to the wireless LAN access point 21 via Wi-Fi, for example. The bridge 19, the local media receivers 31 and 32, and the mobile device 35 are also connected to the wireless LAN access point 21 via Wi-Fi or Ethernet, for example.

[0060] Alternatively or additionally, the display device 1 can be capable of communicating directly with the bridge 19 using, for example, Zigbee technology, and / or can be capable of communicating with the bridge 19 via the Internet / cloud (e.g., via the Internet server 29). Alternatively or additionally, the display device 1 can be capable of controlling the lighting devices 11 and 12 directly via Wi-Fi, Bluetooth, or Zigbee or via the Internet / cloud without a bridge.

[0061] The wireless LAN access point 21 is connected to the Internet 25. The media server 27 is also connected to the Internet 25. For example, the media server 27 can be a server for video-on-demand services such as Netflix, Amazon Prime Video, Hulu, Disney+, or Apple TV+. The display device 1 is connected to the local media receivers 31 and 32 via HDMI. The local media receivers 31 and 32 can include one or more streaming or content generation devices such as Apple TV, Microsoft Xbox, and / or Sony PlayStation, and / or one or more cable or satellite TV receivers. In an alternative example, the media receivers 31 and / or 32 are included in the display device.

[0062] The display device 1 includes a receiver 3, a transmitter 4, a processor 5, a memory 7, and a display 23. The processor 5 is configured to obtain, via the receiver 3, a position associated with each respective light source among the light sources 41 - 46. The position is the position of the respective light source relative to the display 23, or the position where the respective light source creates a light effect relative to the display 23. These positions can be automatically obtained, for example, by using RF beacons and / or by analyzing one or more camera images, or can be input by a person, for example, on a mobile device 35 or in a floor plan or building information model (BIM).

[0063] The processor 5 is further configured to determine the positions of a predetermined number of analysis regions based on the positions of the light sources 41 - 46, extract colors from each of the predetermined number of analysis regions, determine a light effect based on the extracted colors, and control the light sources 41 - 46 to present the light effect via the transmitter 4, for example, by sending light commands to the lighting devices 11 and 12. The predetermined amount of analysis regions, for example, 16, is typically the maximum amount. The processor 5 can also be configured to further determine the positions of the predetermined number of analysis regions based on video content.

[0064] In Figure 1 the example of the display device 1 shown, the display device 1 includes one processor 5. In an alternative example, the display device 1 includes multiple processors. The processor 5 of the display device 1 can be a general-purpose processor, such as an ARM-based processor, or a dedicated processor. The processor 5 of the display device 1 can run, for example, a Unix-based operating system. The memory 7 can include one or more memory units. For example, the memory 7 can include solid-state memory.

[0065] For example, the receiver 3 and the transmitter 4 can communicate with the bridge 19 using one or more wired or wireless communication technologies (such as Zigbee), and communicate with the local media receivers 31 and 32 using HDMI. In an alternative example, multiple receivers and / or multiple transmitters are used instead of a single receiver and a single transmitter. In Figure 1 the example shown, a separate receiver and a separate transmitter are used. In an alternative example, the receiver 3 and the transmitter 4 are combined into a transceiver. The display device 1 may include other typical components for a display device, such as a power connector. The present invention can be implemented using a computer program running on one or more processors.

[0066] In Figure 1 the example, the system of the present invention includes a single device. In an alternative example, the system includes multiple devices. In Figure 1 the example, the system of the present invention is a display device. In an alternative example, the system can be another device, such as a mobile device, a laptop, a personal computer, a bridge, an HDMI module, a streaming device, or an Internet server.

[0067] Figure 2 A second example of a system for controlling a plurality of lighting units to present a light effect determined based on video content to accompany the presentation of the video content on a display is shown. In Figure 2 the second example, the system is a personal computer 61.

[0068] The computer 61 includes a receiver 63, a transmitter 64, a processor 65, and a memory 67. The processor 65 is configured to obtain, for each respective light source among the light sources 41 - 46, a position associated with the respective light source via the receiver 63. The position is the position of the respective light source relative to the display 23, or the position where the respective light source creates a light effect relative to the display 23. These positions can be obtained automatically, for example, by using RF beacons and / or by analyzing one or more camera images, or can be obtained by human input, such as using the keyboard of the personal computer 61.

[0069] The processor 65 is further configured to determine the positions of a predetermined number of analysis regions based on the positions of the light sources 41 - 46, extract colors from each of the predetermined number of analysis regions, determine a light effect based on the extracted colors, and control the light sources 41 - 46 to present the light effect via the transmitter 64, for example, by sending light commands to the lighting devices 11 and 12. The predetermined quantity of analysis regions, for example, 16, is typically the maximum quantity. The processor 65 can also be configured to further determine the positions of the predetermined number of analysis regions based on the video content.

[0070] In Figure 2In the example of computer 61 shown, computer 61 includes a processor 65. In an alternative example, computer 61 includes multiple processors. The processor 65 of computer 61 can be, for example, a general-purpose processor from Intel or AMD, or a dedicated processor. The processor 65 of computer 61 can run an operating system such as Windows or Unix-based. The storage device 67 can include one or more memory units. For example, the memory 67 can include one or more hard disks and / or solid-state memories. For example, the memory 67 can be used to store an operating system, applications, and application data.

[0071] For example, the receiver 63 and the transmitter 64 can use one or more wired and / or wireless communication technologies, such as Ethernet and / or Wi-Fi (IEEE 802.11), to communicate with the wireless LAN access point 21. In an alternative example, multiple receivers and / or multiple transmitters are used instead of a single receiver and a single transmitter. In Figure 2 the example shown, a separate receiver and a separate transmitter are used. In an alternative example, the receiver 63 and the transmitter 64 are combined into a transceiver. Computer 61 can include other typical components for a computer, such as a power connector. The present invention can be implemented using a computer program running on one or more processors.

[0072] Figure 3a and 3b show different techniques for detecting black bars in video content. These figures show that at least one processor 5, 65 is configured to position a first analysis region 101 among a predetermined number of analysis regions 101, 102 at an initial position in the video content (e.g., a set of (pixel) coordinates indicated by Figure 3a and 3b the reference numeral 101 in the figures). For example, the predetermined number of analysis regions 101, 102 can be regions including a (predetermined) number of pixels. At least one processor 5, 65 is configured to extract a first color from the video content at the initial position 101. Thereafter, at least one processor 5, 65 (sequentially) relocates the first analysis region 101 relative to the video content to one or more subsequent positions 101’, 101” in the video content and extracts one or more subsequent colors from the video content at the corresponding one or more subsequent positions 101’, 101”. Based on the extracted colors (the first color and / or one or more subsequent colors), at least one processor determines the presence (and position) of black bars in the video content.

[0073] Reference Figure 3a, at least one of the processors 5, 65 may first locate the first analysis region at position 101, extract the color (e.g., black color), and then relocate the first analysis region 101 to the first subsequent position 101', and extract the first subsequent color (e.g., black) at this first subsequent position. Based on this, at least one of the processors 5, 65 may relocate the first analysis region to the second subsequent position 101", and extract the second subsequent color (e.g., non-black, indicating that the black bar does not exist at the second subsequent position 101") at this second subsequent position. Based on this, at least one of the processors 5, 65 may determine the presence of the black bar 300 in the video content, and determine the height and the area including the black bar 300. The relocation may continue until the exact edge of the black bar has been detected. Alternatively, the relocation may continue until an estimate of the position of the black bar (between positions 101' and 101") has been detected.

[0074] Figure 3b Another example is shown where the first analysis region is relocated to converge to the (estimated) position of the edge of the black bar 300. At least one of the processors 5, 65 may first locate the first analysis region at position 101, extract the color (e.g., black color), and then relocate the first analysis region 101 to the first subsequent position 101', and extract the first subsequent color (e.g., non-black, indicating that the black bar does not exist at the second subsequent position 101") at this first subsequent position. Based on this, at least one of the processors 5, 65 may relocate the first analysis region to the second subsequent position 101", and extract the second subsequent color (e.g., black, indicating that the black bar 300 exists at the second subsequent position 101") at this second subsequent position. Based on this, at least one of the processors 5, 65 may determine the presence of the black bar 300 in the video content, and determine the height and the area including the black bar 300. The relocation may continue until the (exact) edge of the black bar has been detected. Alternatively, the relocation may continue until an estimate of the position of the black bar has been detected.

[0075] At least one of the processors 5, 65 may also be configured to infer the presence of a second black bar 302 on the opposite side of the video content based on the presence of the black bar 300. Alternatively, at least one of the processors 5, 65 may use a second analysis region (not shown) to detect the second black bar 302 in a manner similar to the detection of the (first) black bar. At least one of the processors 5, 65 may also be configured to determine a second region in the video content that includes the second black bar 302, and determine a subset 102 of the analysis regions of a predetermined number of analysis regions at positions outside the region including the black bar and outside the second region including the second black bar.

[0076] At least one of processors 5, 65 may be configured to relocate a first analysis region based on a respective extracted color (e.g., a first color and / or respective subsequent colors). For example, the first analysis region may be positioned a predetermined distance from an edge of the video content, and at least one of processors 5, 65 may be configured to relocate the first analysis region towards the edge of the video content if the respective extracted color is not black. Additionally or alternatively, at least one of processors 5, 65 may be configured to relocate the first analysis region away from the edge of the video content if the respective extracted color is black (as exemplified in Figure 3a ). The at least one of processors 5, 65 is configured to iteratively repeat the process for subsequent positions until the position of the region including the black bar 300 has been determined.

[0077] At least one of processors 5, 65 may be configured to relocate the first analysis region relative to the video content to one or more subsequent positions in the video content for respective subsequent moments in the video content. For example, the relocation may be performed once per frame, per group of frames, per predetermined time period (e.g., every 100 ms), etc.

[0078] At least one of processors 5, 65 may be configured to determine positions of a subset of analysis regions 102 (different from the first analysis region) among a predetermined number of analysis regions 101, 102 outside of a region including the black bar 300 (and the second black bar 302). Figure 3a The positioning of the subset of analysis regions 102 is shown, which has been positioned outside of the region including the black bar 300.

[0079] At least one of processors 5, 65 is configured to extract colors from one or more of the subset of analysis regions and determine the light effect based on one or more of the extracted colors. For example, at least one of processors 5, 65 may extract one or more colors from an analysis region by analyzing the color of the video content at the analysis region. The analysis region may be an individual pixel or a region including a predetermined number of pixels (e.g., at least 5x5 pixels or 10x10 pixels). Techniques for extracting colors from video content are known in the art and will not be discussed further in detail herein. Then, at least one of processors 5, 65 controls a plurality of lighting units (41-46, 47-49, 14) via at least one output interface (4, 64) to render the light effect.

[0080] The initial position may be a predetermined initial position. For example, the initial position may be a predetermined position at or near the horizontal / vertical edge of the video content. For example, the predetermined position may be a position that is a predetermined number of pixels, a predetermined distance, a percentage, etc. away from the horizontal and / or vertical edge of the video content. At least one of the processors 5, 65 may be configured to determine the initial position based on the metadata of the video content. The metadata may include information indicating the (approximate) position of one or more black bars. The metadata may indicate, for example: the resolution of the video content, the position indicating the black bar, the type of the video content, etc. At least one of the processors 5, 65 may determine the initial position of the first analysis region based on this.

[0081] The at least one processor may also be configured to: (i) determine, based on the first color and / or one or more colors, that there is no horizontal black bar present and sequentially reposition the first analysis region to determine whether there is a vertical black bar, and / or (ii) determine, based on the first color and / or one or more colors, that there is no vertical black bar present and sequentially reposition the first analysis region to determine whether there is a horizontal black bar present. Figure 4 An example thereof has been shown, which shows that the first analysis region 101 has been moved to the first subsequent position 101'. When no horizontal black bar is detected, the at least one of the processors 5, 65 may reposition the first analysis region to the second subsequent position 101'' and then to the third subsequent position 101''' to determine whether there is a vertical black bar 400 (which may be performed in a similar manner to the detection of the horizontal black bar 300). Then, the at least one of the processors 5, 65 may correspondingly position a subset of the analysis regions 102 such that they are positioned outside the region covered by the (vertical) black bar 400.

[0082] The system may include at least one input interface 3, 63. At least one processor 5, 65 may be configured to obtain, via the at least one input interface, a position associated with each respective illumination unit of the plurality of illumination units, the position being the position of the respective illumination unit relative to the display, or the position at which the respective illumination unit creates a light effect relative to the display, and determine the position of the subset of the analysis regions based on the positions of the plurality of illumination units. For example, if an illumination device 11 having light sources 41, 42, and 43 is positioned to the left of a display 23, and an illumination device 12 having light sources 44, 45, and 46 is positioned to the right of the display 23, then subsets of the analysis region 102 may be positioned to the left and right of the video content (in regions excluding the (multiple) black bars 300, 302, 400). For example, if an illumination device 11 having light sources 41, 42, and 43 is positioned above a display 23, and an illumination device 12 having light sources 44, 45, and 46 is positioned below the display 23, then subsets of the analysis region 102 may be positioned at the top and bottom of the video content (in regions excluding the (multiple) black bars 300, 302, 400).

[0083] Figure 5 A method 500 for controlling a plurality of illumination units to present a light effect determined based on video content to accompany the presentation of the video content on a display is schematically illustrated. The method includes:

[0084] - Positioning 502 a first analysis region of a predetermined number of analysis regions at an initial position in the video content,

[0085] - Extracting 504 a first color from the video content at the initial position,

[0086] - Repositioning 506 the first analysis region relative to the video content to one or more subsequent positions in the video content and extracting 508 one or more subsequent colors from the video content at the respective one or more subsequent positions,

[0087] - Determining 510 the presence of black bars in the video content based on the first color and / or the one or more subsequent colors,

[0088] - Determining 512 a region in the video content that includes the black bars based on the first color and / or the one or more subsequent colors,

[0089] - Determining 514 positions of a subset of the analysis regions of the predetermined number of analysis regions outside the region that includes the black bars,

[0090] - Extract one or more colors from one or more of the analysis regions of the subset;

[0091] - Determine the light effect based on the one or more extracted colors; and

[0092] - Control the plurality of lighting units to present the light effect.

[0093] Steps 506 and 508 may be performed multiple times, for example, until the presence / absence of one or more black bars has been determined, or until the positions of one or more black bars have been (sufficiently) determined.

[0094] When the computer program product runs on a processing unit of a computing device, such as at least one of the processors 5, 65 of systems 1, 61, the method 500 may be executed by the computer program code of the computer program product.

[0095] Various embodiments of the present invention may be implemented as a program product for use with a computer system, where the program(s) of the program product define the functions of the embodiments (including the methods described herein). In one embodiment, the program(s) may be embodied on various non-transitory computer-readable storage media, where as used herein, the phrase "non-transitory computer-readable storage media" includes all computer-readable media, with the sole exception of transitory propagating signals. In another embodiment, the program(s) may be embodied on various transitory computer-readable storage media. Illustrative computer-readable storage media include, but are not limited to: (i) non-writable storage media on which information is permanently stored (e.g., read-only memory devices within a computer, such as CD-ROM disks readable by a CD-ROM drive, ROM chips, or any type of solid-state non-volatile semiconductor memory); and (ii) writable storage media on which changeable information is stored (e.g., flash memory, floppy disks within a floppy disk drive or hard disk drive, or any type of solid-state random-access semiconductor memory). The computer program may run on the processor 302 described herein.

[0096] The terms used herein are for the purpose of describing particular embodiments only and are not intended to limit the present invention. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that when used in this specification, the terms "comprises" and / or "comprising" specify the presence of the stated features, integers, steps, operations, elements and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0097] All structural, material, acts, and equivalents of the means or step plus function elements in the following claims are intended to include any structural, material, or act for performing the function in combination with other claimed elements as specifically claimed. The description of the embodiments of the invention has been presented for purposes of illustration but is not intended to be exhaustive or limiting to the forms disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the invention. The embodiments were chosen and described in order to best explain the principles of the invention and some practical applications, and to enable others of ordinary skill in the art to understand the invention for various embodiments with various modifications suitable for the particular purposes contemplated.

Claims

1. A system (1, 61) for controlling a plurality of lighting units (41 - 46, 47 - 49, 14) to present a light effect determined based on video content to accompany the presentation of the video content on a display (23), the system (1, 61) comprises: at least one output interface (4, 64); and at least one processor (5, 65), configured to: - Locate a first analysis region (101) of a predetermined number of analysis regions (101, 102) at an initial position in the video content, wherein the analysis regions (101, 120) each comprise individual pixels or a predetermined number of pixels, - Extract a first color from the video content at the initial position (101) by analyzing one or more colors of the video content in the first analysis region, - Iteratively relocate the first analysis region (101) relative to the video content to one or more subsequent positions (101’, 101”, 101”’) in the video content, and extract one or more subsequent colors from the video content at the corresponding one or more subsequent positions (101’, 101”, 101”’), until the position of a region including black bars (300, 302, 400) has been determined, wherein the at least one processor (15, 65) is configured to relocate the first analysis region towards the edge of the video content if the first color is not black, and to relocate the first analysis region away from the edge of the video if the first color is black, - Determine the presence of the black bars (300, 302, 400) in the video content based on the first color and the one or more subsequent colors, - Determine the region in the video content that includes the black bars (300, 302, 400) based on the first color and the one or more subsequent colors, - Determine positions of a subset of the analysis regions (102) of the predetermined number of analysis regions (101, 102) outside the region including the black bars (300, 302, 400), - Extract one or more colors from one or more of the analysis regions of the subset (102), - Determine the light effect based on the extracted one or more colors, and - Control the plurality of lighting units (41 - 46, 47 - 49, 14) via the at least one output interface (4, 64) to present the light effect.

2. The system (1, 61) according to claim 1, wherein the initial position is a position located at the edge of the video content.

3. The system (1, 61) according to any one of the preceding claims, wherein the at least one processor (5, 65) is configured to relocate the first analysis region relative to the video content to the one or more subsequent positions in the video content for corresponding subsequent moments in the video content.

4. The system (1, 61) according to any one of the preceding claims, wherein the initial position is a predetermined initial position.

5. The system (1, 61) according to any one of the preceding claims, wherein the at least one processor (5, 65) is configured to determine the initial position based on metadata of the video content, wherein the metadata includes information indicating the positions of one or more black bars.

6. The system (1, 61) according to any one of the preceding claims, wherein the at least one processor (5, 65) is configured to: - Infer the presence of a second black bar on the opposite side of the video content based on the presence of a black bar, - Determine a second region in the video content that includes the second black bar, - Determine positions of a subset of the analysis regions of the predetermined number of analysis regions outside the region including the black bar and outside the second region including the second black bar.

7. The system (1, 61) according to any one of the preceding claims, wherein the at least one processor (5, 65) is further configured to: (i) determine that no horizontal black bar is present based on the first color and / or the one or more colors, and sequentially reposition the first analysis region to determine whether a vertical black bar is present, and / or (ii) determine that no vertical black bar is present based on the first color and / or the one or more colors, and sequentially reposition the first analysis region to determine whether a horizontal black bar is present.

8. The system according to any one of the preceding claims, further comprising at least one input interface (3, 63), wherein the at least one processor is further configured to - via the at least one input interface (3, 63), obtain, for each respective lighting unit of the plurality of lighting units (41-46, 47-49, 14), a position associated with the respective lighting unit, the position being the position of the respective lighting unit relative to the display (23), or the position where the respective lighting unit creates a light effect relative to the display (23), and - Determine the positions of the subset of the analysis regions based on the positions of the plurality of lighting units (41-46, 47-49, 14).

9. A method (500) of controlling a plurality of lighting units to present a light effect determined based on video content to accompany the presentation of the video content on a display, the method comprises: - Position (502) a first analysis region of a predetermined number of analysis regions at an initial position in the video content, wherein the analysis regions (101, 120) each include individual pixels or include a predetermined number of pixels, - Extract (504) a first color from the video content at the initial position by analyzing one or more colors of the video content in the first analysis region, - Iteratively relocate (506) the first analysis region relative to the video content to one or more subsequent positions in the video content and extract (508) one or more subsequent colors from the video content at the respective one or more subsequent positions until the position of the region including the black bars (300, 302, 400) has been determined, wherein if the first color is not black, the first analysis region is relocated towards the edge of the video content, and wherein if the first color is black, the first analysis region is relocated away from the edge of the video content, - Determine (510) the presence of the black bars in the video content based on the first color and the one or more subsequent colors, - Determine (512) the region in the video content that includes the black bars based on the first color and the one or more subsequent colors, - Determine (514) a subset of the analysis regions of the predetermined number of analysis regions at positions outside the region including the black bars, - Extract (516) one or more colors from one or more of the analysis regions of the subset; - Determine (518) the light effect based on the one or more extracted colors; and - Control (520) the plurality of lighting units to present the light effect.

10. A computer program product for a computing device, the computer program product comprising computer program code to perform the method (500) of claim 9 when the computer program product is run on a processing unit of the computing device.

Images