Determining the Image Analysis Area for Entertainment Lighting Based on Distance Metrics

Abstract

While the display displays the image content, a system for controlling a light source based on an analysis of image content (101) in an analysis area (131-141) of the image content is configured to obtain the image content, obtain a distance between the light source and a surface, determine a size and / or location of the analysis area based on the distance, determine characteristics of the image content by analyzing the image content in the analysis area, determine a light effect based on the characteristics, and control the light source to render the light effect.

Description

Technical Field

[0005]

[0001] The present invention relates to a system for controlling a light source based on analysis of image content in an analysis region of the image content while a display displays the image content.

[0002] The present invention further relates to a method for controlling a light source based on analysis of image content in an analysis region of the image content while a display displays the image content.

[0003] The present invention also relates to a computer program product that enables a computer system to execute such a method.

Background Art

[0004] The Philips HueSync application for Window PCs and Apple computers brings the atmosphere of games, music or movies directly into the room where the user is. When the application is active, the selected lights will reproduce the light effects that accompany the content. Recently, a new product called an HDMI (registered trademark) module, an HDMI Sync box, has been added to the Hue entertainment portfolio. This new device is intended for use with streaming and gaming devices that are connected to a TV and do not require a computer.

[0005] A pixelated light strip may be mounted around a television to create an immersive experience, such as the experience produced by the television as described in US 2010 / 0097408 A1. One of the key challenges of a pixelated light strip around a television is that the television is typically placed at different distances in front of a surface by different consumers. The surface may be a wall, or another surface onto which light is projected, such as the back of the cabinet on which the television is placed.

[0006] For example, one consumer might mount their television directly to the wall, while another might place it on a television cabinet or table at a distance of 10-20 cm from the wall. US 2010 / 0097408 A1 discloses changing the angular position of the pixelated light source in the television based on the distance to the wall, but there is still a significant difference in the quality of the entertainment light effect when the television is close to the wall compared to when it is far from the wall.

[0007] Document WO2020 / 089150A1 discloses a system for determining one or more lighting effects based on the analysis of video content. This system applies a selected color extraction method to extract colors from one or more frames of the video content. The lighting effects based on the extracted colors are then rendered to a light source. [Overview of the Initiative] [Problems that the invention aims to solve]

[0008] A first objective of the present invention is to provide a system that can control a light source to render entertainment lighting effects whose quality is not so dependent on the distance between the light source and the surface.

[0009] A second object of the present invention is to provide a method that can be used to control a light source to render entertainment lighting effects whose quality is not so dependent on the distance between the light source and the surface. [Means for solving the problem]

[0010] In a first aspect of the present invention, a system for controlling a light source based on an analysis of the image content in an analysis area of ​​the image content, while a display displays image content, includes at least one input interface, at least one output interface, and at least one processor configured to obtain the image content via the at least one input interface, obtain the distance between the light source and a surface, determine the size and / or location of the analysis area based on the distance, determine the properties of the image content by analyzing the image content in the analysis area, determine a light effect based on the properties, and control the light source via the at least one output interface to render the light effect on the surface. The light effect may include color and / or intensity.

[0011] The characteristics may include, for example, color, for example, pixel value, and / or, for example, intensity, or for example, color, for example, pixel value, and / or, for example, intensity. The characteristics may be, for example, color throughout the application, and the color may be determined by analyzing the image content in the analysis region, i.e., using, for example, a color extraction methodology.

[0012] When a light source is placed relatively far from a surface (e.g., a wall), the blending of light from different light sources in a pixelated lighting device, also called a pixel, occurs automatically through optical mixing. The greater the distance between the light source / pixel and the surface, the more the light emitted by the light source / pixel will blend. At short distances, there is little to no blending between pixels. User perception tests have shown that when blending does not occur, users perceive the entertainment lighting effect as being of lower quality. Therefore, an algorithmic method for blending colors from pixels that depends on the distance between the light source and the surface is used. This distance may be obtained, for example, from a sensor or a user device. In the latter case, the distance is determined from user input. The user device may be, for example, a user input device. The location of the analysis region may be, for example, the centroid of the analysis region or the location of one or more corners. The image content is typically video content. Therefore, the system according to the present invention may include a sensor or a user device. In one embodiment, the light source may include a sensor or a user device (such as a user input device or a user interface device).

[0013] The at least one processor is configured to determine a first analysis region having a first size and a first location when the distance has a first value, and to determine a second analysis region having a second size and a second location when the distance has a second value, wherein the second size is different from the first size and / or the second location is different from the first location, the first analysis region has a greater overlap with adjacent analysis regions than the second analysis region, and the first value may be smaller than the second value. Thus, as the distance between the light source / pixel and the surface decreases, the overlap between adjacent analysis regions increases, and to a greater extent, the light effects determined from the analysis regions are blended, for example, the colors extracted from the analysis regions. The overlap between analysis regions may be increased, for example, by increasing the size of one or more analysis regions.

[0014] In one embodiment, the size according to the present invention may be predetermined or constant, while the location may be determined based on the distance.

[0015] In one embodiment, however alternatively expressed, through the present application, the processor according to the present invention may be configured to determine the image analysis property of the analysis region based on the distance, the image analysis property may include the size and / or location of the analysis region. Thus, the size and / or location may be defined as the image analysis property.

[0016] The light source may be included in a lighting device, the lighting device may include further light sources, the distance may also represent the distance between the further light sources and the surface, and the at least one processor may be configured to determine the size and / or location of a further analysis area of ​​the image content based on the distance, determine further characteristics of the image content by analyzing the image content in the further analysis area, determine further light effects based on the further characteristics, and control the further light sources to render the further light effects. Thus, the distance may, in some examples, be the distance between the lighting device and the surface, which may, in some examples, be considered a device distance. The further light effects may include color and / or intensity. The further characteristics may include, for example, color, for example, pixel value and / or, for example, intensity, or for example, color, for example, pixel value and / or, for example, intensity.

[0017] In many cases, lighting devices are positioned so that the distance between each light source and the surface is the same for all light sources in the lighting device, and assuming that the distance between multiple light sources in a lighting device, for example, all light sources and the surface is the same, therefore works in many situations. Even if the distances between the light sources and the surface are not the same and are slightly different, this assumption will usually result in high-quality entertainment lighting effects.

[0018] The at least one processor may be configured to obtain a device distance between the lighting device and the surface, obtain a further device distance between the lighting device and the surface, and determine the distance by calculating the average of the device distance and the further device distance. This is useful when multiple device distances are available, for example, the distance between two edges of the lighting device and the surface.

[0019] The light source may be included in a lighting device, the lighting device may include further light sources, and the at least one processor may be configured to obtain a further distance between the further light source and the surface, determine the size and / or location of a further analysis area of ​​the image content based on the further distance, determine further characteristics of the image content by analyzing the image content in the further analysis area, determine further light effects based on the further characteristics, and control the further light source to render the further light effects. The further light effects may include color and / or intensity. The further characteristics may include, for example, color, for example, pixel value, and / or, for example, intensity, or for example, color, for example, pixel value, and / or, for example, intensity. Determining different distances for different light sources in the same lighting device is beneficial when there are substantial differences in the distances between these light sources and the surface. This may be, for example, when a lighting device including vertically positioned light sources is placed at an angle to a wall, or when a light strip is mounted horizontally to a curved display or a display placed in a corner.

[0020] The at least one processor may be configured to obtain a device distance between the lighting device and the surface, obtain a further device distance between the lighting device and the surface, determine the distance between the light source and the surface based on the device distance, the further device distance and the position of the light source on the lighting device, and determine the further distance between the further light source and the surface based on the device distance, the further device distance and the position of the further light source on the lighting device. Even if the distance between the light source and the surface cannot be obtained for each light source, it is possible to determine a relatively accurate distance to the surface for each light source based on the obtained (e.g., two) device distances.

[0021] The at least one processor may be configured to estimate the amount of light overlap between the light projected onto the surface by the light source and the light projected onto the surface by the further light source based on the distance, to determine the amount of desired region overlap between the analysis region and the further analysis region based on the estimated amount of light overlap, and to determine the size and / or location of the analysis region and the size and / or location of the further analysis region based on the desired region overlap. This may be beneficial, for example, when the user can change the angle of the light source. If the user cannot change the angle of the light source, it is also possible to use a predetermined mapping between distance and desired region overlap or between distance and size and / or location of the analysis region.

[0022] The at least one processor may be configured to determine the size and / or location of the analysis area based further on the size of the light source, the size of the display, and / or the size of the lighting device including the light source. As a first example, the formula C = p + b / (0.1 * d + 1) may be used to calculate the size of the analysis area in centimeters for a pixel (e.g., an LED) when d is less than a threshold T, where C is the size of the analysis area in centimeters, b is the blending factor, d is the (estimated or measured) numeric distance value in centimeters, and p is the size of the pixel in centimeters. In this example, the smallest analysis area size is used when d is greater than or equal to the threshold T.

[0023] As a second example, a mapping table may be used in which a percentage of overlap is given for multiple ranges of distance to the surface. For example, distances from 0 to 5 cm may be mapped to a 50% overlap, distances from 5 to 10 cm may be mapped to a 25% overlap, and so on. In this case, the size of the analysis area may be determined based on the pixel size and the determined overlap percentage.

[0024] Furthermore, the ratio of the pixel (light source) size to the display size may be used as a parameter. For example, the above formula may be modified so that C is a function of p (pixel size), d (distance to the surface), and r (ratio of display size to pixel size). In the first implementation, the larger the pixel size relative to the display size, the smaller the overlap. For example, if the distance to the wall is 5 cm, on a 55-inch display television (55 inches is the diagonal dimension), the overlap between adjacent analysis areas may be 40% for 6.25 cm pixels and 20% for 12.5 cm pixels, while on a 75-inch display television, the overlap between adjacent analysis areas may be 50% for 6.25 cm pixels and 25% for 12.5 cm pixels.

[0025] The at least one processor may be configured to determine the size and / or location of the analysis area based further on the distance between the light source and the display. For example, if a larger analysis area is used when the distance between the light source and the display is greater, the entertainment lighting effect may appear better.

[0026] The light source may include a plurality of light elements that cannot render different light effects. In other words, when these light elements render a light effect, these light sources render the same light effect. Pixelated lighting devices often include a plurality of such light sources. In a pixelated lighting device, a light source is also referred to as a pixel or a segment. The light element may be, for example, an LED.

[0027] In a second aspect of the present invention, a method of controlling a light source based on analysis of image content in an analysis region of the image content while a display displays the image content includes obtaining the image content, obtaining a distance between the light source and a surface, determining a size and / or location of the analysis region based on the distance, determining characteristics of the image content by analyzing the image content in the analysis region, determining a light effect based on the characteristics, and controlling the light source to render the light effect on the surface. The method may be executed by software operating on a programmable device. The light effect may include color and / or intensity. The software may be provided as a computer program product. The characteristics may include, for example, color, such as a pixel value, and / or, for example, intensity, or may be, for example, color, such as a pixel value, and / or, for example, intensity.

[0028] Furthermore, a computer program for practicing the methods described herein, as well as a non-transitory computer-readable storage medium storing the computer program, are provided. The computer program may be downloaded, for example, by an existing device, or uploaded to an existing device, or may be stored during manufacture of these systems.

[0029] A non-transitory computer-readable storage medium stores at least one software code portion that, when executed or processed by a computer, causes a display to display image content while performing executable operations for controlling a light source based on an analysis of the image content in an analysis region of the image content.

[0030] The executable operations include obtaining the image content, obtaining a distance between the light source and a surface, determining a size and / or location of the analysis region based on the distance, determining characteristics of the image content by analyzing the image content in the analysis region, determining a light effect based on the characteristics, and controlling the light source to render the light effect on the surface. The light effect may include color and / or intensity. The characteristics may include, for example, color, such as pixel values, and / or, for example, intensity, or may be, for example, color, such as pixel values, and / or, for example, intensity.

[0031] As will be understood by those skilled in the art, aspects of the present invention may be embodied as a device, method, or computer program product. Accordingly, aspects of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware aspects, all of which may be referred to herein as a "circuit," "module," or "system." The functions described in this disclosure may be implemented as algorithms executed by a computer's processor / microprocessor. Additionally, aspects of the present invention may take the form of a computer program product embodied in one or more computer-readable media having, for example, computer-readable program code embodied thereon.

[0032] Any combination of one or more computer-readable media may be used. The computer-readable media may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any preferred combination of the above. More specific examples of computer-readable storage media include, but not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any preferred combination of the above. In the context of the present invention, the computer-readable storage medium may be any tangible medium capable of containing or storing a program for use by or in connection with an instruction execution system, apparatus, or device.

[0033] Examples of computer-readable signaling media include propagating data signals having computer-readable program code embodied within them, for example, within the baseband or as part of a carrier wave. Such propagating signals may take any of various forms, including, but not limited to, electromagnetic, optical, or any preferred combination thereof. The computer-readable signaling medium may not be a computer-readable storage medium, but any computer-readable medium capable of communicating, propagating, or transmitting programs for use by or in connection with instruction execution systems, apparatus, or devices.

[0034] The program code embodied on a computer-readable medium may be transmitted using any suitable medium, including, but not limited to, wireless, wired, optical fiber, cable, RF, or any preferred combination thereof. The computer program code for performing the operations according to the embodiments of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java®, Smalltalk®, and C++, and conventional procedural programming languages ​​such as the C programming language or similar programming languages. This program code may be executed as a standalone software package entirely on the user's computer, partially on the user's computer, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or this connection may be made to an external computer (for example, via the Internet using an Internet service provider).

[0035] Aspects of the present invention will be described below with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It will be understood that each block in a flowchart and / or block diagram, and combinations of blocks within a flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions may be provided to a processor, particularly a microprocessor or central processing unit (CPU), of a general-purpose computer, a dedicated computer, or other programmable data processing device, in order to create a machine, thereby creating means for instructions executed via the processor of a computer, other programmable data processing device, or other device to perform the functions / actions specified within the blocks of the flowchart and / or block diagram.

[0036] These computer program instructions may also be stored in a computer-readable medium capable of instructing a computer, other programmable data processing device, or other device to function in a particular manner, thereby creating a product in which the instructions stored in the computer-readable medium include instructions that perform functions / actions specified within blocks of a flowchart and / or block diagram.

[0037] Computer program instructions may also be loaded onto a computer, other programmable data processing device, or other device to create a computer execution process, causing a series of operational steps to be executed on that computer, other programmable data processing device, or other device, thereby providing a process for instructions executed on a computer or other programmable device to perform a function / action specified within a block of a flowchart and / or block diagram.

[0038] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for performing a specified logical function. It should also be noted that in some alternative implementations, the functions described within a block may be performed in an order different from that shown in the diagram. For example, two blocks shown consecutively may actually be executed substantially simultaneously, or they may be executed in reverse order depending on the functionality they are involved in. It should also be noted that each block in a block diagram and / or flowchart, and any combination of blocks in such block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system or a combination of dedicated hardware and computer instructions that performs a specified function or action. [Brief explanation of the drawing]

[0039] These and other aspects of the present invention are evident from the following drawings and will be further illustrated by reference to those drawings, for example. [Figure 1] This is a block diagram of the first embodiment of the system. [Figure 2] This is a block diagram of a second embodiment of the system. [Figure 3] An example of a lighting device at a first distance from the wall is shown. [Figure 4] An example of a lighting device with a second distance from the wall is shown. [Figure 5] An example of a lighting device that is a third distance from the wall is shown. [Figure 6] Figure 1 shows an example of a residential floor plan in which the system may be used. [Figure 7] This is a flowchart of the first embodiment of the method. [Figure 8] An example of a video frame is shown. [Figure 9] An example of an analysis region used to analyze the video frames in Figure 8 is shown. [Figure 10] An example of an analysis region used to analyze the video frames in Figure 8 is shown. [Figure 11] An example of an analysis region used to analyze the video frames in Figure 8 is shown. [Figure 12] An example of an analysis region used to analyze the video frames in Figure 8 is shown. [Figure 13] An example of an analysis region used to analyze the video frames in Figure 8 is shown. [Figure 14] This is a flowchart of the second embodiment of the method. [Figure 15] This is a flowchart of the third embodiment of the method. [Figure 16] This is a flowchart of the fourth embodiment of the method. [Figure 17] This is a block diagram of an exemplary data processing system for carrying out the method of the present invention. [Modes for carrying out the invention]

[0040] Corresponding elements in the drawing are indicated by the same reference number.

[0041] Figure 1 shows a first embodiment of a system for controlling a light source based on an analysis of the image content in an analysis area of ​​the image content, where a display shows the image content. The HDMI module 1 may be, for example, a Hue Play HDMI Sync Box. In the example of Figure 1, the image content is rendered on display 23. Alternatively, the image content may be rendered on multiple displays, for example, a video wall.

[0042] In the example shown in Figure 1, the HDMI module 1 can control lighting devices 13-15 via a bridge 19. The bridge 19 may be, for example, a Hue bridge. The bridge 19 communicates with the lighting devices 13-15 using, for example, Zigbee® technology. The HDMI module 1 is connected to a wireless LAN access point 21 via, for example, Wi-Fi®. The bridge 19 is also connected to the wireless LAN access point 21 via, for example, Wi-Fi or Ethernet®.

[0043] Alternatively or additionally, HDMI module 1 may communicate directly with bridge 19 using, for example, Zigbee technology, and / or communicate with bridge 19 via the internet / cloud. Alternatively or additionally, HDMI module 1 may control lighting devices 13-15 without using a bridge, for example, directly via Wi-Fi, Bluetooth®, or Zigbee, or via the internet / cloud.

[0044] The wireless LAN access point 21 is connected to the internet 25. The media server 27 is also connected to the internet 25. The media server 27 may be a server for a video-on-demand service such as Netflix, Amazon Prime Video, Hulu, Disney+, or Apple TV+. The HDMI module 1 is connected to the local media receivers 31, 32 and the display 23 via HDMI. The local media receivers 31 and 32 may include one or more streaming or content-generating devices, such as Apple TV, Microsoft Xbox One and / or Sony PlayStation 4, and / or one or more cable or satellite television receivers.

[0045] In the example in Figure 1, lighting devices 13 and 14 are arrays of vertically arranged light sources, such as Philips Hue Signe, and lighting device 15 is an array of horizontally arranged light sources, such as a horizontally placed light strip. Lighting device 13 includes four light sources (pixels) 41-44 and one distance sensor 67, lighting device 14 includes four light sources (pixels) 46-49 and no distance sensor, and lighting device 15 includes five light sources (pixels) 61-65 and two distance sensors 68 and 69. Distance sensors 67-69 may include, for example, one or more infrared distance sensors and / or one or more ultrasonic distance sensors.

[0046] Lighting devices 13-15 are also called pixelated lighting devices. In practice, pixelated lighting devices contain four or more pixels. In the example in Figure 1, each light source (pixel) contains two light elements, such as LEDs, which cannot render the same light effect. Light sources 41-44, 46-49, and 61-65 are also called individually addressable segments of light elements.

[0047] The HDMI module 1 includes a receiver 3, a transmitter 4, a processor 5, and memory 7. The processor 5 is configured to obtain video content from, for example, a media receiver 31 or 32 via the receiver 3, obtain the distance between the light source and the surface of each lighting device from, for example, one or more sensors 67-69 or a user device 29, and for each light source, determine the size and / or location of the analysis area associated with the light source based on the distance. The user device 29 may be, for example, a mobile phone or a tablet.

[0048] The processor 5 is further configured to determine the characteristics of the video content by analyzing the video content in the analysis area, to determine the color and / or intensity for the light effect based on the characteristics, and to control the light sources 41-44, 46-49, and 61-65 via the transmitter 4 to render the light effect. The characteristics may include, for example, color, for example, pixel value, and / or, for example, intensity, or for example, color, for example, pixel value, and / or, for example, intensity.

[0049] In the example in Figure 1, the distance between the light source and the surface is known for each of the lighting devices 13-15, and therefore the analysis area can be determined based on these distances. For light sources where the distance between the light source and the surface is unknown, an analysis area with distance-independent size and location may be used.

[0050] In the example in Figure 1, two device distances are obtained from lighting device 15 and one device distance is obtained from lighting device 13. A device distance is the distance between the lighting device and the surface onto which light is projected, such as a wall. A second device distance may be obtained for lighting device 13 from user device 29. Alternatively, a single device distance obtained from lighting device 13 may be considered to represent the distance between all light sources of lighting device 13 and the surface, or the second device distance may be determined to be zero if the top of lighting device 13 is intended to be inclined with respect to the surface. One or more device distances for lighting device 14 are obtained from user device 29.

[0051] If only a single device distance is available for a lighting device, the distance between the surface and each light source of the lighting device is assumed to be this single device distance. If multiple device distances are available for a lighting device, the distance between the surface and each light source of the lighting device may be the average of the device distances, or the distance between the light source and the surface may be determined based on the device distance and the position of the light source on the lighting device.

[0052] As an example of the former, if distance sensor 68 measures a distance of 30 cm and distance sensor 69 measures a distance of 50 cm, the distance between the surface and each light source of the lighting device 15 may be considered to be 40 cm. As an example of the latter, if distance sensor 68 measures a distance of 30 cm and distance sensor 69 measures a distance of 50 cm, the distances between light sources 61-65 and the surface may be considered to be 30, 35, 40, 45, and 50 cm, respectively.

[0053] Assuming that the distance between each light source and the surface of the lighting device is the same, it is not necessary to determine the distance for each light source after determining the (average) device distance. The processor 5 may directly determine the size and / or location of the analysis area based on the (average) device distance.

[0054] In the embodiment of the HDMI module 1 shown in Figure 1, the HDMI module 1 includes one processor 5. In an alternative embodiment, the HDMI module 1 includes multiple processors. The processor 5 of the HDMI module 1 may be, for example, an ARM-based general-purpose processor or an application-specific processor. The processor 5 of the HDMI module 1 may run, for example, a Unix-based operating system. The memory 7 may include one or more memory units. The memory 7 may include, for example, solid-state memory.

[0055] The receiver 3 and transmitter 4 may use one or more wired or wireless communication technologies, such as Zigbee to communicate with the bridge 19 and HDMI to communicate with the display 23 and local media receivers 31, 32. In an alternative embodiment, multiple receivers and / or multiple transmitters are used instead of a single receiver and a single transmitter. In the embodiment shown in Figure 1, separate receivers and separate transmitters are used. In an alternative embodiment, the receiver 3 and transmitter 4 are combined into a transceiver. The HDMI module 1 may include other components typical of network devices, such as a power connector. The present invention may be implemented using a computer program running on one or more processors.

[0056] In the embodiment shown in Figure 1, the system of the present invention is an HDMI module. In alternative embodiments, the system may be another device, such as a mobile device, laptop, personal computer, bridge, media rendering device, streaming device, or internet server. In the embodiment shown in Figure 1, the system of the present invention includes a single device. In alternative embodiments, the system includes multiple devices.

[0057] Video content analysis may be performed in real time, i.e., immediately before the light source is controlled and the video content is displayed. Alternatively, video content analysis may be performed earlier, for example, by using automated light scripting. Automated light scripting may be performed, for example, by the internet server mentioned above. In automated light scripting, video content analysis is typically performed in the cloud before the user views / streams it (for example, it may be performed near real time with a 5-minute buffer). This may be used to ensure perfect synchronization between the content and the light effects.

[0058] A processing system operating in the cloud may use a user profile, indicating the distance to the surface, to generate a personalized script. Alternatively, the system may pre-generate sets of scripts for several common distances (e.g., 0-5cm, 5-10cm, and 11cm or more), and when the user starts streaming a movie, the system may select the script that best matches the user's setup. The latter would conserve cloud resources when popular movies are being streamed.

[0059] Figure 2 relates to a second embodiment of a system for controlling a light source based on an analysis of the image content in an analysis area of ​​the image content, where a display shows image content, and a mobile device 51. The mobile device 51 may be, for example, a smartphone or a tablet. Lighting devices 13-15 can be controlled by the mobile device 51 via a bridge 19. The mobile device 51 is connected to a wireless LAN access point 21, for example, via Wi-Fi.

[0060] The mobile device 51 includes a receiver 53, a transmitter 54, a processor 55, a memory 57, and a display 59. Image content is preferably displayed on an external display 23, but may be displayed on the display 59 of the mobile device 51. The processor 55 is configured to obtain video content via the receiver 53, for example, from a media server 27, and to obtain the distance between the light source and the surface of each lighting device, for example, from one or more sensors 67-69 (see Figure 1) or from the input interface of the mobile device 51 itself (e.g., a touchscreen display or microphone).

[0061] The processor 55 is further configured to determine, for each light source, the size and / or location of an analysis area associated with the light source based on distance, determine the characteristics of the video content by analyzing the video content in the analysis area, determine the color and / or intensity for a light effect based on the characteristics, and control the light sources 41-44, 46-49, and 61-65 via the transmitter 54 to render the light effect. Alternatively, “determine a color and / or intensity for a light effect” may be expressed as “determine a light effect,” and the light effect may include color and / or intensity.

[0062] In the embodiment of the mobile device 51 shown in Figure 2, the mobile device 51 includes one processor 55. In an alternative embodiment, the mobile device 51 includes multiple processors. The processor 55 of the mobile device 51 may be a general-purpose processor from, for example, ARM or Qualcomm, or an application-specific processor. The processor 55 of the mobile device 51 may run, for example, an Android or iOS operating system. The display 59 may be, for example, a touchscreen display. The display 59 may comprise, for example, an LCD or OLED display panel. The memory 57 may include one or more memory units. The memory 57 may include, for example, solid-state memory.

[0063] The receiver 53 and transmitter 54 may use one or more wireless communication technologies, such as Wi-Fi (IEEE 802.11), to communicate with a wireless LAN access point 21, for example. In an alternative embodiment, multiple receivers and / or multiple transmitters are used instead of a single receiver and a single transmitter. In the embodiment shown in Figure 2, separate receivers and separate transmitters are used. In an alternative embodiment, the receiver 53 and transmitter 54 are combined into a transceiver. The mobile device 51 may further include a camera (not shown). This camera may include, for example, a CMOS or CCD sensor. The mobile device 51 may include other components typical of a mobile device, such as a battery and a power connector. The present invention may be implemented using a computer program running on one or more processors.

[0064] In the embodiment shown in Figure 2, the lighting devices 13-15 are controlled via a bridge 19. In an alternative embodiment, one or more of the lighting devices 13-15 are controlled directly, for example, via Bluetooth®, without using a bridge. In the embodiments shown in Figures 1 and 2, the system of the present invention includes only local devices. In an alternative embodiment, the system of the present invention includes one or more internet / cloud servers.

[0065] Figure 3 shows an example of the lighting device 15 from Figures 1 and 2, where the distance from the wall 81 is a first distance 71. The lighting device 15 is mounted on the back of the display 23 in Figure 1. Figure 4 shows an example of the lighting device 15 where the distance from the wall 81 is a second distance 72. Figure 5 shows an example of the lighting device 15 where the distance from the wall 81 is a third distance 73. The first distance 71 is shorter than the second distance 72. The second distance 72 is shorter than the third distance 73.

[0066] Figure 6 shows an example of a space in which the system of Figure 1 is used. The floor 91 of the house includes an entrance hall 93, a kitchen 94, and a living room 95. Lighting devices 13-15 are installed in the living room 65. The vertically positioned lighting devices 13 and 14 are placed to the left and right of a display 23, which may be, for example, a television. The horizontally positioned lighting device 15 is mounted on the back of the display 23.

[0067] The wireless LAN access point 21 is installed in the entrance hall 93. The HDMI module 1 is installed in the living room 65 next to the display 23. The bridge 19 is installed in the living room 65 near the wireless LAN access point 21. Person 99 is watching television. The lighting device 15 is at a second distance 72 from the wall, as shown in Figure 4. The lighting device 13 is at a distance 76 from the display 23. The lighting device 14 is at a distance 77 from the display 23.

[0068] Figure 7 shows a first embodiment of a method for controlling a light source based on an analysis of the image content in an analysis area of ​​the image content, while the display shows the image content. Step 201 includes obtaining a video frame.

[0069] Step 202 includes obtaining lighting device information relating to the lighting devices. For example, the lighting device information may include, for each lighting device, the width of a horizontally positioned lighting device or the height of a vertically positioned lighting device and the size of the light source / pixel.

[0070] Lighting device information about a particular lighting device may be obtained, for example, when a user adds this lighting device to their lighting system. For example, when a new lighting device is added, an HDMI module, mobile device, or bridge may download and store lighting device information from the lighting device. If this information is stored in the bridge, the HDMI module or mobile device may later retrieve this information from the bridge.

[0071] In a simpler embodiment, step 202 may be omitted. In a more advanced embodiment, the lighting device information may include further information about the lighting device, such as pixel density (e.g., the number of light pixels per meter), pixel spacing (e.g., the distance between the centers of two individual pixels), and / or the optical properties of the pixels (e.g., the beam width and beam angle of the pixelated light source).

[0072] Step 203 includes obtaining the distance between each light source of the lighting device, i.e., each pixel, and a surface, such as a wall. Typically, one or more device distances are obtained from a sensor or a user device. The distances between each light source of the lighting device and the surface may be the same. Alternatively, different distances for each light source may be determined based on the obtained one or more device distances.

[0073] The distance between a pixelated light source and the area to be illuminated, i.e., the surface, can be determined in various ways. In a simple embodiment, the user provides input regarding the distance, for example, via a smartphone UI. This can be a numerical or approximate input, for example, by selecting an icon indicating how the television is installed (wall-mounted or freestanding). In a more advanced embodiment, the distance between the light source and the area to be illuminated, i.e., the surface, is determined automatically via an external sensor, for example, by using one or more sensors (e.g., time-of-flight sensors) incorporated into or attached to the lighting device, or by analyzing a (depth) image from the television setup captured by a smartphone camera.

[0074] Step 205 includes determining, for each light source, the size and / or location of the analysis area associated with the light source based on the distance. If the distance has a first value, a first analysis area having a first size and a first location is determined in step 205 for the first light source; if the distance has a second value, a second analysis area having a second size and a second location is determined in step 205 for this first light source.

[0075] The first analysis region has greater overlap with adjacent analysis regions than the second analysis region, and the first value is smaller than the second value. The size of the second is different from the size of the first, and / or the location of the second is different from the location of the first. While it may be possible to create greater overlap by changing the location of at least one adjacent analysis region, it may be easier to achieve this by using a larger analysis region (as well).

[0076] In the embodiment shown in Figure 7, the equation C = p + b / (0.1 * d + 1) is used to calculate the size of the analysis area in centimeters for a pixel when d is less than a certain threshold T, where C is the size of the analysis area in centimeters, b is the blending coefficient, d is the (estimated or measured) distance value in centimeters, and p is the size of the pixel in centimeters. If d is greater than or equal to the threshold T, the smallest analysis area size is used.

[0077] A pixel may contain multiple light elements, such as multiple LEDs. A 6-centimeter pixel may contain, for example, six LED packages, one of which are arranged in each centimeter of a lighting device (e.g., a light strip). All six LEDs are controlled to a single color. The six LEDs may include, for example, three RGB LEDs and three white LEDs.

[0078] For example, in the case of horizontally arranged pixels with a blending factor of 6, a pixel width of 6 cm, and a distance of 50 cm from the wall, the width of the color analysis area may be calculated as follows: 6 + 6 / (0.1 * 50 + 1) = 7 cm If the TV is placed against the wall (0cm), the width of the color analysis area will be 12cm. This means it will take 50% of the two adjacent pixels. If it is placed 10cm from the wall, the width of the color analysis area will be 9cm (taking 25% of the two adjacent pixels). In this example, the threshold T is greater than 50cm. If the threshold T is less than 50cm and the distance to the wall is 50cm, the width of the color analysis area will be determined to be 6cm.

[0079] For vertically arranged pixels, the height of the color analysis area may be determined as described above with respect to the width of the color analysis area for horizontally arranged pixels.

[0080] The pixel color analysis area size may be determined by multiplying the width (in centimeters) of the color analysis area size by the amount of horizontal pixels in the video content divided by the width (in centimeters) of the lighting device positioned horizontally, or by multiplying the height (in centimeters) of the color analysis area size by the amount of vertical pixels in the video content divided by the height (in centimeters) of the lighting device positioned vertically.

[0081] For horizontally arranged pixels, the height of the color analysis area may be determined independently of the distance to the wall, for example, by being a fixed value, or by being determined based on the width of the color analysis area. For vertically arranged pixels, the width of the color analysis area may be determined independently of the distance to the wall, for example, by being a fixed value, or by being determined based on the height of the color analysis area.

[0082] The horizontally and vertically arranged pixels may be part of a light strip attached to a display device, or they may be part of a lighting device located a short distance away from the display device, such as a floor-standing light fixture like a Hue Signe placed near a wall and contributing to the entertainment experience.

[0083] In a modified version of this embodiment, the ratio of the pixel (light source) size to the display size is further used as a parameter. For example, the above equation may be modified so that C is a function of p (pixel size), d (distance to the surface), and r (ratio of display size to pixel size). In the first implementation, the larger the pixel size relative to the display size, the smaller the overlap. For example, if the distance to the wall is 5 cm, on a 55-inch display television (55 inches is the diagonal dimension), the overlap between adjacent analysis areas may be 40% for 6.25 cm pixels and 20% for 12.5 cm pixels, while on a 75-inch display television, the overlap between adjacent analysis areas may be 50% for 6.25 cm pixels and 25% for 12.5 cm pixels. Information indicating the display size may be obtained, for example, in step 202, or in a separate step performed before, after, or (partially) in parallel with step 202.

[0084] In a simpler embodiment, the size of the color analysis area may be determined by a simple binary determination. When the TV (with or incorporating a light source) is mounted on a stand, the color analysis area maps one-to-one to the pixels. Thus, if a pixel is 6 cm wide, the color analysis area has a corresponding width. When the TV is mounted against a wall, the color analysis area is expanded to 50% of the color analysis area of ​​two adjacent pixels.

[0085] In a slightly less straightforward embodiment, a mapping table may be used in which overlap rates are given for multiple ranges of distance to the surface. For example, distances from 0 to 5 cm may be mapped to a 50% overlap rate, distances from 5 to 10 cm may be mapped to a 25% overlap rate, and so on. In this case, the size of the analysis area may be determined based on the pixel size and the determined overlap rate.

[0086] In more advanced embodiments, more complex functions are used. Furthermore, the blend coefficient b may be variable and may be selected by the user or the system, for example, based on the type of content.

[0087] Step 206 includes checking whether there are any further lighting devices that should be controlled based on the analysis of video content for which the analysis area has not yet been determined. If so, steps 203 and 205 are repeated for these further lighting devices. Otherwise, step 207 is performed. Step 201 may be performed, for example, in parallel with at least one or more of steps 202-206, before step 202 is performed, or between steps 206 and 207.

[0088] Step 207 involves determining the characteristics of the video content by analyzing the video content, typically the video frames of the video content, within an analysis region associated with one of the light sources. In the embodiment of Figure 7, colors are extracted from the video content in the analysis region. Various color extraction methods can be used, such as taking the average color or the triprimary colors in this region. The color extraction method may also be modified depending on the overall and absolute size (amount) of the analysis region. For example, for smaller, non-overlapping regions, taking the average may work best (a stable but non-saturated method), while for larger, overlapping regions, the trimean method may work best (resulting in less stable but more saturated colors).

[0089] If desired, the size and / or location of the color analysis area determined in step 203 may be adjusted in step 207 based, for example, on color and / or luminance contrast and / or the number of edges in the content. For example, if a video frame contains high-contrast elements that align with light pixels, the analysis area may be reduced even if the distance to the wall is short. Similarly, if the content is already very smooth, overlapping areas would not be beneficial and would only result in desaturation of the light effect. This analysis can be performed pixel by pixel, which would allow overlap to be reduced in some pixels and increased in others. However, this would typically need to be done per video frame and would only be possible if the system could analyze the content fast enough.

[0090] Step 209 includes determining the light effect to be rendered with this light source based on its characteristics. If a color is extracted in step 207, this color may be used as the color for the light effect to be rendered with the light source.

[0091] Step 210 includes checking whether there are any further light sources associated with the analysis area whose light effects have not yet been determined. If so, steps 207 and 209 are repeated for these further light sources. Otherwise, step 211 is performed. Step 211 includes controlling the light sources to render the light effects determined in step 209 by sending light control commands specifying one of the light effects to the corresponding light sources, either directly to the light sources or to the lighting device containing the light sources.

[0092] Step 212 involves checking whether the end of the video content has been reached. If not, steps 207-211 are repeated for the next portion of the video content, for example, the next frame.

[0093] Figure 8 shows an example of a video frame 101 of video content, rendered, for example, on display 23 in Figure 1. Figure 9 shows an example of analysis regions 111-121 that can be used to extract characteristics from the video frame 101. In this example, multiple areas of the screen are mapped to different lighting devices, and each analysis region is analyzed separately, for example, the average color is extracted from each analysis region. For example, analysis regions 111-114 may be mapped to pixels 41-44 of lighting device 13 in Figure 1, analysis regions 114-118 may be mapped to pixels 61-65 of lighting device 15 in Figure 1, and analysis regions 118-121 may be mapped to pixels 46-49 of lighting device 14 in Figure 1.

[0094] Figure 10 shows further examples of analysis regions that can be used to extract characteristics from video frame 101. In this example, analysis regions 131-134 may be mapped to pixels 41-44 of lighting device 13 in Figure 1, analysis regions 134-138 may be mapped to pixels 61-65 of lighting device 15 in Figure 1, and analysis regions 138-141 may be mapped to pixels 46-49 of lighting device 14 in Figure 1.

[0095] Analysis regions 131-141 in Figure 10 are larger than analysis regions 111-121 in Figure 9. As a result, there is overlap between adjacent analysis regions in Figure 10, whereas there is no such overlap between adjacent analysis regions in Figure 9. Such overlap is beneficial when the distance between the pixels of a pixelated lighting device and the surface, such as a wall, is relatively small.

[0096] In the examples in Figures 9 and 10, each analysis region has the same size. However, as shown in Figure 11, it is also possible to use analysis regions of different sizes. Of the analysis regions 151-154 on the left side of the video frame, analysis region 151 is the largest and analysis region 154 is the smallest.

[0097] Furthermore, Figure 11 shows that it is possible to increase the size of an analysis region without increasing the overlap between adjacent analysis regions. For example, if the pixelating lighting device is sufficiently far from the surface and overlap between adjacent analysis regions is not required, the size of the analysis region may still be increased to focus the lighting effect more towards the ambiance rather than focusing only on colors close to the side of the screen where the light source is located. Thus, without changing the overlap, the size of the analysis region can be changed to occupy a larger portion of the video frame without affecting adjacent analysis regions.

[0098] In the example in Figure 11, the pixels / light sources of the lighting device have different distances from the surface. For example, the pixelated light device may be tilted relative to the wall, i.e., it may have a certain angle to the wall, it may be mounted on the back of a curved display, or it may be placed behind the display at the corner of two walls. In the example in Figure 11, the light source associated with analysis region 151 is the furthest from the surface, and the light source associated with analysis region 154 is the closest to the surface. The farther the light source is from the wall, the greater the effect it has (relative to the wall). Therefore, in some cases, even if the distance to the wall is large, it may be beneficial to analyze a larger portion of the video frame so that the large effect on the wall reflects a larger portion of the video frame rather than only a small portion of the side video frame.

[0099] In the example in Figure 10, each pair of adjacent analysis regions has the same overlap. However, as shown in Figure 12, it is also possible to use different overlaps for different pairs of adjacent analysis regions. Of the analysis regions 161-164 on the left side of the video frame, adjacent analysis regions 163 and 164 have the largest overlap, while adjacent analysis regions 161 and 162 have the smallest overlap. This can be beneficial, for example, when the pixels / light sources of a lighting device are at different distances from the surface.

[0100] By individually determining the distance to the surface for each light source, it becomes possible to use different overlaps for different pairs of adjacent analysis regions. In the example in Figure 12, the light source associated with analysis region 161 is furthest from the surface, and the light source associated with analysis region 164 is closest to the surface. In the example in Figure 12, all analysis regions are the same size but are stacked differently vertically, with relatively small overlap areas between adjacent pixels far from the wall and relatively large overlap areas between adjacent pixels close to the wall. Thus, only the position, not the size, of the analysis region depends on the distance to the surface.

[0101] Furthermore, Figure 12 shows that it is possible to change the overlap between two adjacent analysis areas without changing the total overlap between all analysis areas. If the angle of the pixelated lighting device relative to the wall is increased, and therefore the distance between light sources related to analysis area 164 is increased, the overlap between adjacent analysis areas 163 and 164 may be decreased, and the overlap between adjacent analysis areas 161 and 162 may be increased, thereby keeping the total overlap the same.

[0102] In the example in Figure 12, there is still a small overlap between adjacent analysis areas 161 and 162. However, for analysis area 161, which is related to the light source furthest from the surface, it is possible for there to be no overlap at all with the adjacent analysis area 162. Furthermore, the overlap between adjacent areas 163 and 164 may be even greater than shown in Figure 12.

[0103] In the examples shown in Figures 9 to 12, all analysis regions have a rectangular shape. It is also possible to use one or more different shapes for analysis regions 171 to 174, as shown in Figure 13.

[0104] Figure 14 shows a portion of a second embodiment of a method for controlling a light source based on an analysis of the image content in an analysis area of ​​the image content, while the display shows the image content. In this second embodiment, step 203 in Figure 7 is carried out by substeps 231-237. Following step 203, steps 205-212 are performed as shown in Figure 7.

[0105] Step 231 includes obtaining one or more device distances between the lighting device and the surface. Typically, these one or more device distances are obtained from sensors or from user devices, such as a user input device. If one end of a vertically positioned lighting device needs to be inclined with respect to the wall, and a device distance is obtained for the other end, a second device distance of zero may be automatically obtained.

[0106] Next, step 232 includes checking whether one device distance was obtained in step 231, or whether multiple device distances were obtained in step 231. If it is determined in step 232 that a single device distance was obtained in step 231, then step 233 is performed. Step 233 includes determining further distances between the light source and the surface based on the single device distance determined in step 232. Typically, the further distances between the light source and the surface are equal to the single device distance. Step 205 is performed after step 233.

[0107] If, in step 232, it is determined that multiple device distances have been obtained in step 231, then step 235 is performed. Step 235 includes calculating the average of the multiple device distances determined in step 231. If two device distances, for example, the device distances at both ends of a lighting device, are determined in step 231, a single average is calculated in step 235. If three or more device distances are determined in step 231, multiple averages may be calculated in step 235. Next, step 237 includes determining the distance between an additional light source and a surface based on the (multiple) averages calculated in step 235. If a single average is calculated in step 235, the distance between an additional light source and a surface is typically equal to this single average. Step 205 is performed after step 237.

[0108] Figure 15 shows a portion of a third embodiment of a method for controlling a light source based on an analysis of the image content in an analysis area of ​​the image content, while the display shows the image content. Compared to the second embodiment in Figure 14, steps 235 and 237 are replaced by steps 251 and 253, steps 255 and 257 are added before step 205, and step 205 is performed by step 259.

[0109] Step 251 includes determining the position of light sources on the lighting device. Step 253 includes determining the distance between each light source and the surface based on the position of the light sources determined in step 251 and at least two of the device distances determined in step 231. The device distances are expressed with respect to a reference point on the lighting device, for example, the edge of the lighting device. If the position of a light source is between two reference points, the distance between this light source and the surface is determined based on two device distances corresponding to these reference points and the distance between the position of the light source and these reference points.

[0110] Step 255 includes estimating the amount of light overlap between light projected onto the surface by adjacent light sources based on the distance determined in step 203. The obtained illumination device information (see step 202 in Figure 7) may be used to estimate the amount of overlap. For example, beam width, beam angle, and distance to the wall may be used to more accurately calculate the amount of light overlap between illumination areas of different pixels projected onto the wall. Beam width and beam angle typically affect optical mixing. For example, using a wider beam and / or a sharper beam angle (due to a longer distance to the surface) results in more optical mixing.

[0111] Step 257 includes determining a desired amount of region overlap between adjacent analysis regions of adjacent light sources based on the estimated amount of light overlap. The desired amount of region overlap may be lower if more optical mixing occurs. Step 259 includes determining the size and / or location of the analysis regions based on the desired amount of region overlap. Step 206 is performed after step 259.

[0112] A fourth embodiment of a method for controlling a light source based on an analysis of the image content in an analysis area of ​​the image content, while the display shows the image content, is shown in Figure 16. In this fourth embodiment, additional steps 271, 273 and / or 275 are optionally performed before step 205, and step 205 is performed by step 277.

[0113] Step 271 includes determining the size of the (multiple) light sources of the lighting device. Step 273 includes determining the size of the lighting device. Step 275 includes determining the distance between the (multiple) light sources and the display. Step 277 includes determining the size and / or location of the analysis area based on the distance determined in step 203, and optionally based on the size of the (multiple) light sources of the lighting device determined in step 271, the size of the lighting device determined in step 273, and / or the distance between the (multiple) light sources and the display. Step 206 is performed after step 277.

[0114] The embodiments in Figures 7 and 14-16 differ from one another in multiple ways, i.e., multiple steps are added or replaced. Modifications of these embodiments add or replace only a subset of these steps, and / or omit one or more steps. For example, one or more of steps 271, 273, and 275 may be added to the embodiments of Figure 14 and / or Figure 15, and steps 255, 257, and 259 of Figure 15 may be added to the embodiment of Figure 14, and / or omitted from the embodiment of Figure 15.

[0115] Figure 17 shows a block diagram illustrating an exemplary data processing system capable of performing the methods described with reference to Figures 7 and 14-16.

[0116] As shown in Figure 17, the data processing system 300 may include at least one processor 302 coupled to the memory element 304 via a system bus 306. Therefore, the data processing system may store program code in the memory element 304. Furthermore, the processor 302 may execute program code accessed from the memory element 304 via the system bus 306. In one embodiment, the data processing system may be implemented as a computer suitable for storing and / or executing program code. However, it should be understood that the data processing system 300 may be implemented in the form of any system including a processor and memory capable of performing the functions described herein.

[0117] The memory element 304 may include one or more physical memory devices, such as local memory 308 and one or more mass storage devices 310. Local memory may refer to random-access memory or other non-persistent memory devices commonly used during the actual execution of program code. Mass storage devices may be implemented as hard drives or other persistent data storage devices. The processing system 300 may also include one or more cache memories (not shown) that provide temporary storage for at least some of the program code to reduce the number of times the program code must be retrieved from the mass storage device 310 during execution. Furthermore, the processing system 300 may use memory elements of another processing system, for example, if the processing system 300 is part of a cloud computing platform.

[0118] Input / output (I / O) devices, indicated as input device 312 and output device 314, can optionally be coupled to the data processing system. Examples of input devices, but not limited to, include keyboards, pointing devices such as mice, and microphones (e.g., for voice and / or speech recognition). Examples of output devices, but not limited to, include monitors or displays and speakers. The input and / or output devices may be coupled to the data processing system directly or via an intermediary I / O controller.

[0119] In one embodiment, the input and output devices may be implemented as a combined input / output device (shown in Figure 17 by dashed lines surrounding input device 312 and output device 314). An example of such a combined device is a touch-sensitive display, sometimes referred to as a “touchscreen display” or simply a “touchscreen.” In such embodiments, input to the device may be provided by the movement of a physical entity, such as a stylus or a user’s finger, on or near the touchscreen display.

[0120] The network adapter 316 may also be coupled to the data processing system, enabling the data processing system to be coupled to other systems, computer systems, remote network devices, and / or remote storage devices via an intervening private or public network. The network adapter may include a data receiver for receiving data transmitted to the data processing system 300 by the aforementioned systems, devices, and / or networks, and a data transmitter for transmitting data from the data processing system 300 to the aforementioned systems, devices, and / or networks. Modems, cable modems, and Ethernet® cards are examples of various types of network adapters that may be used with the data processing system 300.

[0121] As shown in Figure 17, the memory element 304 may store the application 318. In various embodiments, the application 318 may be stored in local memory 308, one or more mass storage devices 310, or separately from those local memory and mass storage devices. It should be understood that the data processing system 300 may also run an operating system (not shown in Figure 17) that facilitates the execution of the application 318. The application 318 is implemented in the form of executable program code and can be executed by the data processing system 300, for example, by a processor 302. In response to the execution of the application, the data processing system 300 may be configured to perform one or more operation or method steps described herein.

[0122] Various embodiments of the present invention may be implemented as a program product for use with a computer system, the program of this program product defining the functionality of the embodiments (including the methods described herein). In one embodiment, the program may be contained on various non-temporary computer-readable storage media, and as used herein, the expression “non-temporary computer-readable storage media” includes all computer-readable media, with the sole exception being temporary propagating signals. In another embodiment, the program may be contained on various temporary computer-readable storage media. Exemplary computer-readable storage media include, but are not limited to, (i) non-writable storage media on which information is stored permanently (e.g., read-only memory devices inside a computer, such as CD-ROM disks, ROM chips, or any type of non-volatile solid-state semiconductor memory readable by a CD-ROM drive), and (ii) writable storage media on which modifiable information is stored (e.g., flash memory, floppy disks inside a diskette drive or hard disk drive, or any type of random-access solid-state semiconductor memory). The computer program may be executed on the processor 302 described herein.

[0123] The terminology used herein is intended solely to describe specific embodiments and is not intended to limit the invention. Where used herein, the singular forms “a,” “an,” and “the” are intended to include the plural form unless the context explicitly indicates otherwise. Where used herein, the terms “contains” and / or “contains” specify the presence of a described feature, integer, step, operation, element, and / or component, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0124] The corresponding structures, materials, actions, and equivalents of all means-plus-function or step-plus-function elements in the following claims are intended to include any structures, materials, or actions for performing a function in combination with other claimed elements specifically claimed. The description of embodiments of the present invention has been presented for illustrative purposes only and is not intended to be exhaustive or to limit implementations of the disclosed forms. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the invention. The embodiments have been selected and described to best illustrate the principles and some practical applications of the present invention and to enable those other skilled in the art to understand the invention with respect to various embodiments having various modifications suitable for specific applications conceivable.

Claims

1. A system for controlling a light source based on an analysis of the image content in an analysis area of ​​the image content, wherein the display shows image content, and the system is A light source, wherein the light source is configured to project light onto a surface, and the surface is a wall, At least one input interface, At least one output interface, The image content is obtained via at least one input interface. Obtain the distance between the light source and the surface from the sensor or user device. Based on the distance, the size and / or location of the analysis area is determined. The characteristics of the image content are determined by analyzing the image content in the aforementioned analysis area. The photo effect is determined based on the aforementioned characteristics, and The light source is controlled via the at least one output interface to render the light effect on the surface. A processor configured as follows: A system that includes this.

2. The system according to claim 1, wherein the at least one processor is configured to determine a first analysis region having a first size and a first location when the distance has a first value, and to determine a second analysis region having a second size and a second location when the distance has a second value, wherein the second size is different from the first size and / or the second location is different from the first location, the first analysis region has greater overlap with adjacent analysis regions than the second analysis region, and the first value is smaller than the second value.

3. The system according to claim 2, wherein the first size is larger than the second size.

4. The system according to any one of claims 1 to 3, wherein the sensor includes one or more infrared distance sensors and / or one or more ultrasonic distance sensors.

5. The light source is included in a lighting device, the lighting device includes a further light source, the distance also represents the distance between the further light source and the surface, and the at least one processor is Based on the distance, determine the size and / or location of the further analysis area of ​​the image content. Further characteristics of the image content are determined by analyzing the image content in the aforementioned further analysis area. Further optical effects are determined based on the aforementioned further characteristics, and Control the additional light source to render the additional light effect on the aforementioned surface. The system according to any one of claims 1 to 4, configured as described above.

6. The aforementioned at least one processor is To obtain the device distance between the illumination device and the surface, To obtain a further device distance between the illumination device and the surface, and The distance is determined by calculating the average of the device distance and the further device distance. The system according to claim 5, configured as described above.

7. The light source is included in a lighting device, the lighting device includes a further light source, and the at least one processor is To obtain a further distance between the further light source and the surface, Based on the aforementioned further distance, determine the further size and / or location of the further analysis area of ​​the image content. Further characteristics of the image content are determined by analyzing the image content in the aforementioned further analysis area. Further optical effects are determined based on the aforementioned further characteristics, and Control the additional light source to render the additional light effect on the aforementioned surface. The system according to any one of claims 1 to 4, configured as described above.

8. The system according to any one of claims 1 to 7, wherein the determined light effect includes color and / or intensity, and / or any further determined light effect includes color and / or intensity.

9. The aforementioned at least one processor is Based on the aforementioned distance, the amount of light overlap between the light projected onto the surface by the aforementioned light source and the light projected onto the surface by a further light source is estimated. Based on the estimated amount of light overlap, a desired amount of regional overlap between the analysis region and a further analysis region is determined, and Based on the desired amount of region overlap, the size and / or location of the analysis region and the size and / or location of the further analysis region are determined. The system according to any one of claims 5 to 8, configured as described above.

10. The system according to any one of claims 1 to 9, wherein the at least one processor is configured to determine the size and / or location of the analysis area based further on the size of the light source, the size of the display, and / or the size of the lighting device including the light source.

11. The system according to any one of claims 1 to 10, wherein the at least one processor is configured to determine the size and / or location of the analysis area based further on the distance between the light source and the display.

12. The system according to any one of claims 1 to 11, wherein the light source includes a plurality of light elements, and the plurality of light elements are incapable of rendering different light effects.

13. The system according to claim 4, wherein the sensor is a time-of-flight sensor.

14. A method for controlling a light source based on an analysis of the image content in an analysis area of ​​the image content, wherein the display shows image content, and the method is: To obtain the aforementioned image content, To obtain the distance between the light source and the surface, Determining the size and / or location of the analysis area based on the aforementioned distance, The characteristics of the image content are determined by analyzing the image content in the aforementioned analysis area. Determining the photo effect based on the aforementioned characteristics, Controlling the light source to render the light effect on the surface, Methods that include...

15. A computer program or set of computer programs including at least one software code portion, or a computer-readable storage medium storing the computer program or set of computer programs, wherein the software code portion is configured to perform the method described in claim 14 when executed on a system including at least one processor as described in any one of claims 1 to 13.

Images