Image analysis region for determining entertainment lighting based on distance metrics.

Abstract

A system for controlling a light source based on the analysis of image content (101) in an analysis region (131-141) of the image content when displaying image content on a display, the system is configured to obtain the image content, obtain the distance between the light source and a surface, determine the size and / or position of the analysis region based on the distance, determine the characteristics of the image content by analyzing the image content in the analysis region, determine the light effect based on the characteristics, and control the light source to present the light effect.

Description

Technical Field

[0001] The present invention relates to a system for controlling a light source based on the analysis of image content in an analysis area of ​​the image content when displaying image content on a display.

[0002] The present invention further relates to a method for controlling a light source based on the analysis of image content in an analysis area of ​​the image content when displaying image content on a monitor.

[0003] The present invention also relates to a computer program product that enables a computer system to execute this method. Background Technology

[0004] The Philips HueSync app for Windows PCs and Apple computers brings the ambiance of games, music, or movies directly into the user's room. When the app is active, selected lights will play lighting effects to accompany the content. A new product called the HDMI Sync box, an HDMI module, has recently been added to the Hue entertainment portfolio. This new device does not require a computer and is designed for use with streaming and gaming devices connected to a TV.

[0005] Pixelated light strips can be installed around a TV to create an immersive experience, such as the experience described in US 2010 / 0097408A1. One of the key issues with pixelated strips around a TV is that the TV is often placed at different distances in front of the surface by different consumers. This surface can be a wall or other surface on which light is projected, such as the back of a cabinet where the TV is already placed.

[0006] For example, some consumers may mount their TVs directly to the wall, while others may place them on a TV cabinet or table 10-20 cm away from the wall. Although US 2010 / 0097408 A1 discloses changing the angular position of the pixelated light source in a TV based on its distance from the wall, there remains a significant difference in the quality of the entertainment light effect when the TV is close to the wall versus when it is further away.

[0007] Document WO2020 / 089150A1 discloses a system for determining one or more lighting effects based on video content analysis. Thereby, a selected color extraction method is applied to extract colors from one or more frames of the video content. Lighting effects based on the extracted colors are then applied to a light source. Summary of the Invention

[0008] The first objective of this invention is to provide a system capable of controlling a light source to produce an entertainment light effect, the quality of which depends less on the distance between the light source and the surface.

[0009] A second objective of this invention is to provide a method for controlling a light source to produce an entertainment light effect, the quality of which depends less on the distance between the light source and the surface.

[0010] The invention is set forth in the appended set of independent and dependent claims.

[0011] In a first aspect of the invention, a system for controlling a light source based on analysis of image content in an analysis region of the image content while displaying image content on a display includes at least one input interface, at least one output interface, and at least one processor configured to obtain the image content via the at least one input interface, obtain a distance between the light source and a surface, determine the size and / or position of the analysis region based on the distance, determine characteristics of the image content by analyzing the image content in the analysis region, determine a lighting effect based on the characteristics, and control the light source via the at least one output interface to render the lighting effect onto the surface. The lighting effect may include color and / or intensity.

[0012] The characteristics may include, for example, color (e.g., pixel value) and / or intensity. Throughout this application, the characteristics may be, for example, color, which can be determined by analyzing the image content in the analysis region, i.e., using, for example, a color extraction method.

[0013] When a light source is placed relatively far from a surface (e.g., a wall), the mixing of light from different light sources in a pixelated lighting device (also called a pixel) occurs automatically through optical mixing. The greater the distance between the light source / pixel and the surface, the more the light emitted by the light source / pixel will mix. At short distances, there will be almost no mixing between pixels. User perception tests have shown that users find the quality of entertainment lighting effects to be lower when no mixing occurs. For this reason, an algorithmic approach is used to mix colors from pixels, which depends on the distance between the light source and the surface. This distance can be obtained, for example, from a sensor or a user device. In the latter case, the distance is determined by user input. For example, the user device can be a user input device. For example, the location of the analysis area can be the centroid of the analysis area or the location of one or more corners. The image content is typically video content. Therefore, a system according to the invention may include a sensor or a user device. In various aspects, the light source may include a sensor or a user device (e.g., a user input device or a user interface device).

[0014] The at least one processor can be configured to determine a first analysis region having a first size and a first position when the distance has a first value, and to determine a second analysis region having a second size and a second position when the distance has a second value, the second size being different from the first size and / or the second position being different from the first position, the first analysis region having a greater overlap with adjacent analysis regions than the second analysis region, and the first value being smaller than the second value. Therefore, as the distance between the light source / pixel and the surface decreases, the overlap between adjacent analysis regions increases to a greater extent, resulting in a higher degree of blending of the light effects determined from the analysis regions (e.g., blending of colors extracted from the analysis regions). For example, the overlap between analysis regions can be increased by increasing the size of one or more analysis regions.

[0015] In various aspects, the dimensions according to the invention can be predefined or constant, while the position can be determined based on the distance. In various aspects, the position according to the invention can be predefined or constant, while the dimensions can be determined based on the distance.

[0016] Throughout this application, but alternatively, the processor according to the invention can be configured to determine image analysis attributes of the analysis region based on the distance, wherein the image analysis attributes may include the size and / or location of the analysis region. Therefore, size and / or location can be defined as image analysis attributes.

[0017] The light source may be included in a lighting device, which may include additional light sources. The distance may also represent the distance between the additional light source and the surface. The at least one processor may be configured to: determine the size and / or position of an additional analysis region of the image content based on the distance; determine additional characteristics of the image content by analyzing the image content in the additional analysis region; determine additional lighting effects based on the additional characteristics; and control the additional light source to present the additional lighting effects. Therefore, in this example, the distance may be the distance between the lighting device and the surface; in this example, this may be considered a device distance. The additional lighting effect may include color and / or intensity. The additional characteristics may include, for example, color (e.g., pixel value) and / or, for example, intensity.

[0018] Typically, lighting fixtures are positioned such that the distance between each light source and the surface is the same for all light sources of the fixture, and therefore the assumption that the distances between multiple (e.g., all) light sources and the surface are the same applies in many cases. Even if the distances between the light sources and the surface are not identical, but slightly different, this assumption usually still results in high-quality entertainment lighting effects.

[0019] The at least one processor can be configured to obtain a device distance between the lighting device and the surface, obtain an additional device distance between the lighting device and the surface, and determine the distance by calculating the average of the device distance and the additional device distance. This is advantageous if multiple device distances are obtained (e.g., the distance between the two edges of the lighting device and the surface).

[0020] The light source may be included in a lighting device, which may include additional light sources, and the at least one processor may be configured to: obtain an additional distance between the additional light source and the surface; determine the size and / or position of an additional analysis region of the image content based on the additional distance; determine additional characteristics of the image content by analyzing the image content in the additional analysis region; determine additional lighting effects based on the additional characteristics; and control the additional light source to present the additional lighting effects. The additional lighting effects may include color and / or intensity. The additional characteristics may include, for example, color (e.g., pixel value) and / or, for example, intensity. It is advantageous to determine different distances for different light sources of the same lighting device if there is a significant difference in the distance between the light source and the surface. This may be the case, for example, when a lighting device including a vertically arranged light source is placed against a wall, or when a light strip is horizontally attached to a curved display or a display placed in a corner.

[0021] The at least one processor can be configured to: obtain a device distance between the lighting device and the surface; obtain an additional device distance between the lighting device and the surface; determine the distance between the light source and the surface based on the device distance, the additional device distance, and the position of the light source on the lighting device; and determine the additional distance between the additional light source and the surface based on the device distance, the additional device distance, and the position of the additional light source on the lighting device. If the distance between the light source and the surface is not obtained for each light source, it is still possible to determine a relatively accurate distance to the surface for each light source based on the already obtained (e.g., two) device distances.

[0022] The at least one processor may be configured to: estimate the amount of light overlap between light projected onto the surface by the light source and light projected onto the surface by the additional light source based on the distance; determine a desired area overlap between the analysis area and the additional analysis area based on the estimated light overlap; and determine the size and / or position of the analysis area and the size and / or position of the additional analysis area based on the desired area overlap. For example, this may be advantageous if the user can change the angle of the light source. If the user cannot change the angle of the light source, it may also be possible to use a predetermined mapping between distance and desired area overlap, or between distance and analysis area size and / or position.

[0023] The at least one processor can be configured to further determine the size and / or location of the analysis region based on the size of the light source, the size of the display, and / or the size of the lighting device including the light source. As a first example, when d is less than a certain threshold T, the formula C=p+b / (0.1*d+1) can be used to calculate the size of the analysis region in centimeters (e.g., LED pixels), where C is the size of the analysis region in centimeters, b is a blending factor, d is an estimated or measured numerical distance value in centimeters, and p is the pixel size in centimeters. In this example, when d is greater than or equal to the threshold T, the minimum analysis region size is used.

[0024] As a second example, a mapping table can be used, where the percentage of overlap is given for multiple distance ranges to the surface. For example, distances between 0 and 5 cm can be mapped to a 50% overlap percentage, distances between 5 and 10 cm can be mapped to a 25% overlap percentage, and so on. The size of the analysis region can then be determined based on the pixel size and the determined overlap percentage.

[0025] The ratio between the pixel (light source) size and the display size can be further used as a parameter. For example, the above formula can be modified so that C is a function of p (pixel size), d (distance to the surface), and r (ratio between the display size and the pixel size). In the first embodiment, the larger the pixel size relative to the display size, the smaller the overlap percentage. For example, with a distance of 5 cm to the wall, for a TV with a 55-inch display (55 inches is a diagonal size), the overlap between adjacent analysis areas might be 40% for a 6.25 cm pixel and 20% for a 12.5 cm pixel, and for a TV with a 75-inch display, the overlap between adjacent analysis areas might be 50% for a 6.25 cm pixel and 25% for a 12.5 cm pixel. The at least one processor can be configured to further determine the size and / or position of the analysis area based on the distance between the light source and the display. For example, when the distance between the light source and the display is large, the entertainment lighting effect may appear better if a larger analysis area is used.

[0026] The light source may include multiple light elements that cannot produce different lighting effects. In other words, if these light elements produce a lighting effect, then they produce the same lighting effect. Pixelated lighting devices typically include multiple such light sources. In pixelated lighting devices, the light source is also referred to as a pixel or segment. For example, the light element can be an LED.

[0027] In a second aspect of the invention, a method for controlling a light source based on analysis of image content in an analysis region of the image content while a display shows image content includes: obtaining the image content; obtaining a distance between the light source and a surface; determining the size and / or position of the analysis region based on the distance; determining characteristics of the image content by analyzing the image content in the analysis region; determining a lighting effect based on the characteristics; and controlling the light source to project the lighting effect onto the surface. The method can be executed by software running on a programmable device. The lighting effect may include color and / or intensity. The software may be provided as a computer program product. The characteristics may include, for example, color (e.g., pixel values) and / or, for example, intensity.

[0028] In addition, a computer program for implementing the methods described herein is provided, as well as a non-transitory computer-readable storage medium for storing the computer program. The computer program may be downloaded or uploaded to an existing device, for example, or stored during the manufacture of these systems.

[0029] A non-transitory computer-readable storage medium stores at least one portion of software code that, when executed or processed by a computer, is configured to perform executable operations for controlling a light source based on analysis of image content in an analysis area of ​​the image content when the image content is displayed on a display.

[0030] The executable operations include: obtaining the image content, obtaining the distance between the light source and the surface, determining the size and / or position of the analysis region based on the distance, determining characteristics of the image content by analyzing the image content in the analysis region, determining a lighting effect based on the characteristics, and controlling the light source to project the lighting effect onto the surface. The lighting effect may include color and / or intensity. The characteristics may include, for example, color (e.g., pixel value) and / or, for example, intensity.

[0031] As those skilled in the art will appreciate, aspects of the present invention can be embodied as devices, methods, or computer program products. Therefore, aspects of the present invention can take the form of entirely hardware embodiments, entirely software embodiments (including firmware, resident software, microcode, etc.), or embodiments combining software and hardware aspects, all of which are generally referred to herein as “circuit,” “module,” or “system.” The functionality described in this disclosure can be implemented as algorithms executed by a computer’s processor / microprocessor. Furthermore, aspects of the present invention can take the form of computer program products embodied in one or more computer-readable media having computer-readable program code embodied thereon (e.g., stored thereon).

[0032] Any combination of one or more computer-readable media can be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any suitable combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the context of this invention, a computer-readable storage medium can be any tangible medium that can contain or store a program used by or in conjunction with an instruction execution system, apparatus, or device.

[0033] Computer-readable signal media may include propagated data signals having computer-readable program code embodied therein (e.g., in baseband or as part of a carrier wave). Such propagated signals may take any of a variety of forms, including, but not limited to, electromagnetic, optical, or any suitable combination thereof. A computer-readable signal medium may be any computer-readable medium that is not a computer-readable storage medium and may convey, propagate, or transmit a program used by or in conjunction with an instruction execution system, apparatus, or device.

[0034] Program code embodied on a computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, fiber optic, cable, RF, or any suitable combination thereof. Computer program code used to carry out the operations of various aspects of the invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​(such as Java™, Smalltalk, or C++) and conventional procedural programming languages ​​(such as the "C" programming language or similar programming languages). The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)) or can be connected to an external computer (e.g., via the Internet provided by an Internet service provider).

[0035] Various aspects of the invention are described below with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, particularly a microprocessor or central processing unit (CPU), to produce a machine such that the instructions, executable via the processor of the computer, other programmable data processing apparatus, or other device, create means for implementing the functions / actions specified in the flowchart illustrations and / or one or more block diagram blocks.

[0036] These computer program instructions may also be stored in a computer-readable medium that can instruct a computer, other programmable data processing apparatus, or other device to operate in a particular manner, such that the instructions stored in the computer-readable medium produce an article of writing, which includes instructions that implement functions / actions specified in flowcharts and / or one or more block diagrams.

[0037] Computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other device to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide for implementing the functions / actions specified in the flowchart and / or one or more block diagram boxes.

[0038] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, comprising one or more executable instructions for implementing a specified logical function(s). It should also be noted that in some alternative implementations, the functions described in the blocks may not appear in the order shown in the figures. For example, two blocks shown consecutively may actually be executed substantially simultaneously, or sometimes these blocks may be executed in reverse order, depending on the functions involved. It will also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a system based on dedicated hardware, or a combination of dedicated hardware and computer instructions, that performs the specified function or action. Attached Figure Description

[0039] Referring to the accompanying drawings, these and other aspects of the invention will be clear and further illustrated by way of example, in which:

[0040] Figure 1 This is a block diagram of the first embodiment of the system;

[0041] Figure 2 This is a block diagram of the second embodiment of the system;

[0042] Figure 3 An example of a lighting device with a first distance from the wall is shown;

[0043] Figure 4 An example of a lighting device with a second distance to a wall is shown;

[0044] Figure 5 An example of a lighting device with a third distance to the wall is shown;

[0045] Figure 6 It shows where it can be used Figure 1 An example of a floor plan of a residential system;

[0046] Figure 7 This is a flowchart of the first embodiment of the method;

[0047] Figure 8An example of a video frame is shown;

[0048] Figure 9-13 The analysis is shown Figure 8 An example of the analysis area of ​​a video frame;

[0049] Figure 14 This is a flowchart of the second embodiment of the method;

[0050] Figure 15 This is a flowchart of the third embodiment of the method;

[0051] Figure 16 This is a flowchart of the fourth embodiment of the method; and

[0052] Figure 17 This is a block diagram of an exemplary data processing system for performing the methods of the present invention.

[0053] Corresponding elements in the accompanying drawings are represented by the same reference numerals. Detailed Implementation

[0054] Figure 1 A first embodiment of a system for controlling a light source based on the analysis of image content in an analysis area when an image is displayed on a monitor is shown: an HDMI module 1. For example, the HDMI module 1 could be a Hue PlayHDMI Sync Box. Figure 1 In this example, the image content is displayed on monitor 23. Alternatively, the image content can be displayed on multiple monitors (e.g., a video wall).

[0055] exist Figure 1 In this example, HDMI module 1 can control lighting devices 13-15 via bridge 19. For example, bridge 19 could be a Hue bridge. Bridge 19 communicates with lighting devices 13-15, for example, using Zigbee technology. HDMI module 1 is connected to wireless LAN access point 21, for example, via Wi-Fi. Bridge 19 is also connected to wireless LAN access point 21, for example, via Wi-Fi or Ethernet.

[0056] Alternatively or additionally, HDMI module 1 may be able to communicate directly with bridge 19, for example, using Zigbee technology, and / or may be able to communicate with bridge 19 via the Internet / cloud. Alternatively or additionally, HDMI module 1 may be able to control lighting devices 13-15 without a bridge (e.g., directly via Wi-Fi, Bluetooth, or Zigbee, or directly via the Internet / cloud).

[0057] Wireless LAN access point 21 is connected to the Internet 25. Media server 27 is also connected to the Internet 25. For example, media server 27 could be a server for a video-on-demand service such as Netflix, Amazon Prime Video, Hulu, Disney+, or Apple TV+. HDMI module 1 is connected via HDMI to display 23 and local media receivers 31 and 32. Local media receivers 31 and 32 could include one or more streaming or content generation devices, such as Apple TV, Microsoft Xbox One, and / or Sony PlayStation 4, and / or one or more cable or satellite TV receivers.

[0058] exist Figure 1 In the examples, lighting devices 13 and 14 are vertically arranged light source arrays, such as the Philips HueSigne, and lighting device 15 is a horizontally arranged light source array, such as a horizontally placed light strip. Lighting device 13 includes four light sources (pixels) 41-44 and one distance sensor 67; lighting device 14 includes four light sources (pixels) 46-49 and no distance sensor; and lighting device 15 includes five light sources (pixels) 61-65 and two distance sensors 68 and 69. For example, distance sensors 67-69 may include one or more infrared distance sensors and / or one or more ultrasonic distance sensors.

[0059] Lighting fixtures 13-15 are also known as pixelated lighting fixtures. In fact, pixelated lighting fixtures consist of more than four or five pixels. Figure 1 In the example, each light source (pixel) includes two light elements (e.g., LEDs), which cannot produce different lighting effects. Light sources 41-44, 46-49, and 61-65 are also referred to as individually addressable segments of light elements.

[0060] The HDMI module 1 includes a receiver 3, a transmitter 4, a processor 5, and a memory 7. The processor 5 is configured to: acquire video content via the receiver 3 (e.g., from media receivers 31 or 32), such as the distance between the light source and the surface of each lighting device from one or more of sensors 67-69 or from user equipment 29; and determine the size and / or location of an analysis area associated with each light source based on that distance. For example, user equipment 29 may be a mobile phone or a tablet computer.

[0061] The processor 5 is also configured to: determine the characteristics of the video content by analyzing the video content in the analysis area, determine the color and / or intensity of the light effect based on the characteristics, and control the light sources 41-44, 46-49 and 61-65 via the transmitter 4 to present the light effect. The characteristics may include, for example, color (e.g., pixel value) and / or, for example, intensity.

[0062] exist Figure 1 In the example, for each of the lighting devices 13-15, the distance between the light source and the surface is known, and therefore the analysis area can be determined based on these distances. Analysis areas with dimensions and locations independent of distance can be used for light sources where the distance between the light source and the surface is unknown.

[0063] exist Figure 1 In the example, two device distances are obtained from lighting device 15, and one device distance is obtained from lighting device 13. Device distance is the distance between a lighting device and a surface (e.g., a wall) on which light is projected. A second device distance for lighting device 13 can be obtained from user device 29. Alternatively, the single device distance obtained from lighting device 13 can be considered to represent the distance between all light sources and surfaces of lighting device 13, or, if the top of lighting device 13 is intended to rest against a surface, the second device distance can be determined to be zero. One or more device distances for lighting device 14 are obtained from user device 29.

[0064] If only a single device distance is obtained for a lighting device, the distance between the surface and each light source of the lighting device is assumed to be that single device distance. If multiple device distances are obtained for a lighting device, the distance between the surface and each light source of the lighting device can be the average of the device distances, or the distance between the light source and the surface can be determined based on the device distances and the position of the light source on the lighting device.

[0065] As an example of the former, if distance sensor 68 measures a distance of 30 cm and distance sensor 69 measures a distance of 50 cm, then the distance between the surface and each light source of the lighting device 15 can be considered as 40 cm. As an example of the latter, if distance sensor 68 measures a distance of 30 cm and distance sensor 69 measures a distance of 50 cm, then the distances between light sources 61-65 and the surface can be considered as 30, 35, 40, 45, and 50 cm, respectively.

[0066] If it is assumed that the distance between each light source and the surface of the lighting device is the same, then after determining the (average) device distance, it is not necessary to determine the distance of each light source. The processor 5 can then directly determine the size and / or location of the analysis area based on the (average) device distance.

[0067] exist Figure 1 In the embodiment of the HDMI module 1 shown, the HDMI module 1 includes a processor 5. In alternative embodiments, the HDMI module 1 includes multiple processors. The processor 5 of the HDMI module 1 may be a general-purpose processor (e.g., ARM-based) or a dedicated processor. For example, the processor 5 of the HDMI module 1 may run a Unix-based operating system. The memory 7 may include one or more memory cells. For example, the memory 7 may include solid-state memory.

[0068] For example, receiver 3 and transmitter 4 can communicate with bridge 19 using one or more wired or wireless communication technologies (e.g., Zigbee), and can communicate with display 23 and local media receivers 31 and 32 using one or more wired or wireless communication technologies (e.g., HDMI). In alternative embodiments, multiple receivers and / or multiple transmitters are used instead of a single receiver and a single transmitter. Figure 1 In the illustrated embodiment, a separate receiver and a separate transmitter are used. In an alternative embodiment, receiver 3 and transmitter 4 are combined into a transceiver. HDMI module 1 may include other components typically used in network devices, such as a power connector. This invention can be implemented using a computer program running on one or more processors.

[0069] exist Figure 1 In one embodiment, the system of the present invention is an HDMI module. In an alternative embodiment, the system may be another device, such as a mobile device, laptop, personal computer, bridge, media presentation device, streaming device, or internet server. Figure 1 In one embodiment, the system of the present invention includes a single device. In an alternative embodiment, the system includes multiple devices.

[0070] Video content analysis can be performed in real time, that is, before the light source is controlled and the video content is displayed. Alternatively, video content analysis can be performed earlier, for example, by using an automated lighting script. For instance, an automated lighting script can be executed by the aforementioned internet server. In an automated lighting script, video content analysis is completed before the user views / streams it (typically in the cloud) (it can also be done near real-time (e.g., with a 5-minute buffer)). This can be used to ensure perfect synchronization between the content and the lighting effects.

[0071] A processing system running in the cloud can use user profiles to generate personalized scripts, where the user profile indicates the distance to the surface. Alternatively, the system can pre-generate script sets for several common distances (e.g., 0-5 cm, 5-10 cm, and greater than 10 cm), and when a user begins streaming a movie, the system can then select the script closest to the user's settings. The latter will save cloud resources when streaming popular movies.

[0072] Figure 2 A second embodiment of a system for controlling a light source based on the analysis of image content in an analysis area when image content is displayed on a display is shown: a mobile device 51. For example, the mobile device 51 may be a smartphone or a tablet. Lighting devices 13-15 may be controlled by the mobile device 51 via a bridge 19. The mobile device 51 is connected to a wireless LAN access point 21, for example, via Wi-Fi.

[0073] Mobile device 51 includes a receiver 53, a transmitter 54, a processor 55, a memory 57, and a display 59. Image content is preferably displayed on an external display 23, but may also be displayed on the display 59 of mobile device 51. Processor 55 is configured to obtain video content via receiver 53, for example, from media server 27, and, for example, from one or more of sensors 67-69 (see...). Figure 1 Alternatively, the distance between the light source and the surface of each lighting device can be obtained from the input interface of the mobile device 51 itself (such as a touch screen display or microphone).

[0074] The processor 55 is also configured to determine the size and / or position of an analysis region associated with each light source based on distance, determine the characteristics of the video content by analyzing the video content in the analysis region, determine the color and / or intensity of the light effect based on the characteristics, and control the light sources 41-44, 46-49, and 61-65 to present the light effect via the transmitter 54. Alternatively, the phrase "determining the color and / or intensity of the light effect" can be expressed as "determining the light effect," where the light effect may include color and / or intensity.

[0075] exist Figure 2In the embodiment of the mobile device 51 shown, the mobile device 51 includes a processor 55. In alternative embodiments, the mobile device 51 includes multiple processors. The processor 55 of the mobile device 51 may be a general-purpose processor (e.g., from ARM or Qualcomm) or a dedicated processor. For example, the processor 55 of the mobile device 51 may run the Android or iOS operating system. For example, the display 59 may be a touchscreen display. For example, the display 59 may include an LCD or OLED display panel. The memory 57 may include one or more memory cells. For example, the memory 57 may include solid-state memory.

[0076] For example, receiver 53 and transmitter 54 can use one or more wireless communication technologies, such as Wi-Fi (IEEE 802.11), to communicate with wireless LAN access point 21. In alternative embodiments, multiple receivers and / or multiple transmitters are used instead of a single receiver and a single transmitter. Figure 2 In the illustrated embodiment, a separate receiver and a separate transmitter are used. In an alternative embodiment, receiver 53 and transmitter 54 are combined into a transceiver. Mobile device 51 may also include a camera (not shown). For example, the camera may include a CMOS or CCD sensor. Mobile device 51 may include other components typically used in mobile devices, such as a battery and power connector. The invention can be implemented using a computer program running on one or more processors.

[0077] exist Figure 2 In one embodiment, lighting devices 13-15 are controlled via bridge 19. In an alternative embodiment, one or more of lighting devices 13-15 are controlled without a bridge (e.g., directly via Bluetooth). Figure 1 and Figure 2 In one embodiment, the system of the present invention includes only a local device. In an alternative embodiment, the system of the present invention includes one or more Internet / cloud servers.

[0078] Figure 3 It shows a first distance 71 to the wall 81. Figure 1 and Figure 2 Example of lighting device 15. Lighting device 15 has been attached to Figure 1 The back of the monitor 23. Figure 4 An example of a lighting device 15 with a second distance 72 to the wall 81 is shown. Figure 5 An example of a lighting device 15 with a third distance 73 to a wall 81 is shown. The first distance 71 is shorter than the second distance 72. The second distance 72 is shorter than the third distance 73.

[0079] Figure 6 Describing the use of Figure 1 An example of the space in the system. Floor 91 of the residence includes a corridor 93, a kitchen 94, and a living room 95. Lighting fixtures 13-15 have been installed in the living room 95. Vertically arranged lighting fixtures 13 and 14 have been placed on the left and right sides of a display 23, which may be, for example, a TV. Horizontally arranged lighting fixture 15 has been attached to the back of the display 23.

[0080] Wireless LAN access point 21 has been installed in hallway 93. HDMI module 1 has been installed next to monitor 23 in living room 95. Bridge 19 has been installed in living room 95 near wireless LAN access point 21. A person 99 is watching TV. Figure 4 As shown, lighting device 15 has a second distance 72 to the wall. Lighting device 13 has a distance 76 to the display 23. Lighting device 14 has a distance 77 to the display 23.

[0081] Figure 7 The diagram illustrates a first embodiment of a method for controlling a light source based on the analysis of image content in an analysis area of ​​the image content when the display shows image content. Step 201 includes obtaining video content.

[0082] Step 202 includes obtaining lighting device information related to the lighting device. For example, for each lighting device, the lighting device information may include the width of a horizontally arranged lighting device or the height of a vertically arranged lighting device, as well as the size of the light source / pixel.

[0083] For example, when a user adds a lighting device to their lighting system, lighting device information associated with that device becomes available. For instance, when a new lighting device is added, an HDMI module, mobile device, or bridge can download and store the lighting device information. While this information is stored on the bridge, the HDMI module or mobile device may be able to retrieve it from the bridge later.

[0084] In a simpler embodiment, step 202 may be omitted. In a more advanced embodiment, the lighting device information may include additional information about the lighting device, such as pixel density (e.g., the number of light pixels per meter), pixel pitch (e.g., the distance between the centers of two individual pixels), and / or may indicate the optical properties of the pixels (e.g., the beam width and beam angle of the pixelated light source).

[0085] Step 203 involves obtaining the distance between each light source (i.e., each pixel) of the lighting device and a surface (e.g., a wall). Typically, one or more device distances are obtained from a sensor or from a user device. The distance between each light source of the lighting device and the surface can be the same. Alternatively, a different distance for each light source can be determined based on the obtained one or more device distances.

[0086] The distance between the pixelated light source and the area to be illuminated (i.e., the surface) can be determined in various ways. In a simple embodiment, the user provides the distance input, for example, via a smartphone UI. This can be a numerical or approximate input, such as through a selection icon for how to install the TV (wall-mounted / freestanding). In a more advanced embodiment, the distance between the light source and the area to be illuminated (i.e., the surface) is determined automatically via external sensors, such as by using one or more sensors embedded in or attached to the lighting device (e.g., a time-of-flight sensor), or by analyzing (depth) images from the TV settings, such as those captured by a smartphone camera.

[0087] Step 205 includes determining the size and / or location of an analysis region associated with each light source based on distance. When the distance has a first value, a first analysis region with a first size and a first location is determined for the first light source in step 205; and when the distance has a second value, a second analysis region with a second size and a second location is determined for the first light source in step 205.

[0088] The first analysis region has greater overlap with its adjacent analysis regions than the second analysis region, and the first value is smaller than the second value. The second size differs from the first size and / or the second position differs from the first position. Although it may be possible to create greater overlap by changing the position of only at least one adjacent analysis region, it is easier to achieve this by using a larger analysis region as well.

[0089] exist Figure 7 In this embodiment, when d is less than a certain threshold T, the formula C=p+b / (0.1*d+1) is used to calculate the size of the analysis region of the pixel (in centimeters), where C is the size of the analysis region (in centimeters), b is the mixing factor, d is the (estimated or measured) numerical distance value (in centimeters), and p is the size of the pixel (in centimeters). When d is greater than or equal to the threshold T, the minimum analysis region size is used.

[0090] A pixel can include multiple light elements, such as multiple LEDs. A 6-centimeter pixel can include, for example, six LED packages, one per centimeter in a lighting device (e.g., a light strip). All six LEDs are controlled by a single color. For example, the six LEDs can include three RGB LEDs and three white LEDs.

[0091] For example, for a horizontally arranged pixel with a blending factor of 6, a pixel width of 6 cm, and a distance of 50 cm from the wall, the width of the color analysis region can be calculated as follows: 6 + 6 / (0.1 * 50 + 1) = 7 cm. If the TV is placed against the wall (0 cm), the width of the color analysis region will be 12 cm. This means it will occupy 50% of two adjacent pixels. When placed 10 cm from the wall, the color analysis region will be 9 cm wide (occupying 25% of two adjacent pixels). In this example, the threshold T is greater than 50 cm. If the threshold T is less than 50 cm and the distance to the wall is 50 cm, the width of the color analysis region will be determined to be 6 cm.

[0092] For vertically arranged pixels, the height of the color analysis region can be determined in a manner that relates to the width of the color analysis region for horizontally arranged pixels, as described above.

[0093] The color analysis area size in pixels can be determined by multiplying the width of the color analysis area size in centimeters by the horizontal number of pixels in the video content and dividing by the width of the horizontally arranged lighting fixture in centimeters; or by multiplying the height of the color analysis area size in centimeters by the vertical number of pixels in the video content and dividing by the height of the vertically arranged lighting fixture in centimeters.

[0094] For horizontally arranged pixels, the height of the color analysis region can be determined independently of the distance to the wall (e.g., it can be a fixed value), or it can be determined based on, for example, the width of the color analysis region. For vertically arranged pixels, the width of the color analysis region can be determined independently of the distance to the wall (e.g., it can be a fixed value), or it can be determined based on, for example, the height of the color analysis region.

[0095] Horizontally and vertically arranged pixels can be part of a light strip attached to a display device; or they can be part of a lighting device located slightly away from the display device, such as a floor lamp (like Hue Signe) placed near a wall to enhance the entertainment experience.

[0096] In a variation of this embodiment, the ratio between the pixel (light source) size and the display size is further used as a parameter. For example, the above formula can be modified so that C is a function of p (pixel size), d (distance to the surface), and r (ratio between the display size and the pixel size). In the first embodiment, the larger the pixel size relative to the display size, the smaller the overlap percentage. For example, with a distance of 5 cm from the wall, for a TV with a 55-inch display (55 inches is a diagonal size), the overlap between adjacent analysis areas might be 40% for a 6.25 cm pixel and 20% for a 12.5 cm pixel, and for a TV with a 75-inch display, the overlap between adjacent analysis areas might be 50% for a 6.25 cm pixel and 25% for a 12.5 cm pixel. For example, information indicating the display size can be obtained in step 202 or in a separate step performed before, after, or (partially) in parallel with step 202.

[0097] In a simpler embodiment, determining the size of the color analysis area can be a simple binary decision. If the TV (with an attached or embedded light source) is mounted on a bracket, the color analysis area is mapped one-to-one to the pixel. Therefore, if the pixel is 6 cm wide, the color analysis area has a corresponding width. If the TV is mounted against a wall, the color analysis area expands and occupies 50% of the color analysis area of ​​two adjacent pixels.

[0098] In a slightly less simplistic embodiment, a mapping table can be used, where a percentage of overlap is given for multiple distance ranges to the surface. For example, distances between 0 and 5 cm could be mapped to a 50% overlap percentage, distances between 5 and 10 cm could be mapped to a 25% overlap percentage, and so on. The size of the analysis region can then be determined based on the pixel size and the determined overlap percentage.

[0099] In more advanced embodiments, more complex functions are used. Furthermore, the blending factor b can be variable and selected by the user or the system (e.g., based on the content type).

[0100] Step 206 involves checking, based on the analysis of video content in areas where no analysis regions have yet been identified, whether there are additional lighting devices to be controlled. If so, steps 203 and 205 are repeated for that additional lighting device. If not, step 207 is executed. For example, step 201 may be executed in parallel with at least a portion of one or more steps 202-206, either before step 202 or between steps 206 and 207.

[0101] Step 207 involves determining the characteristics of the video content by analyzing the video content (typically video frames of the video content) in an analysis region associated with one of the light sources. Figure 7 In this embodiment, color is extracted from the video content within the analysis region. Various color extraction methods can be used, such as taking the average color or trimean color of the region. The color extraction method can also vary depending on the total and absolute dimensions of the analysis region. For example, averaging may work best for smaller, non-overlapping regions (a stable but desaturating method), while for larger, overlapping regions, trimean averaging may work best (resulting in less stable but more saturated colors).

[0102] If desired, the size and / or position of the color analysis area determined in step 203 can be adjusted in step 207, for example, based on color and / or brightness contrast and / or the number of edges in the content. For example, if a video frame contains high-contrast elements aligned with light pixels, the analysis area can be reduced even at short distances to walls. Similarly, if the content is already very smooth, overlapping areas will offer no benefit and will only result in desaturation of the lighting effects. This analysis can be performed on a per-pixel basis, thus allowing for reduced overlap for some pixels and increased overlap for others. However, this will generally only be possible if the system can analyze the content fast enough, as it will require analysis on every single video frame.

[0103] Step 209 includes determining the light effect to be presented on the light source based on this characteristic. If a color was extracted in step 207, that color can be used as the color of the light effect to be presented on the light source.

[0104] Step 210 includes checking if there is another light source already associated with the analysis area but for which a lighting effect has not yet been determined. If so, steps 207 and 209 are repeated for that additional light source. If not, step 211 is performed. Step 211 includes controlling the light source to present the lighting effect determined in step 209 by transmitting a light control command specifying one of the lighting effects directly to the corresponding light source or to a lighting device including the light source.

[0105] Step 212 involves checking if the end of the video content has been reached. If not, steps 207-211 are repeated for the next part of the video content (e.g., the next frame).

[0106] Figure 8 It shows that, for example Figure 1 An example of video frame 101 of the video content presented on display 23. Figure 9An example of analysis regions 111-121 that can be used to extract features from video frame 101 is shown. In this example, multiple regions of the screen are mapped to different lighting devices, and each analysis region is analyzed individually, for example, the average color is extracted from each analysis region. For example, analysis regions 111 to 114 can be mapped to... Figure 1 The pixels 41-44 of the lighting device 13, and the analysis area 114 to 118 can be mapped to Figure 1 The lighting device 15 has pixels 61-65, and the analysis area 118 to 121 can be mapped to... Figure 1 The lighting equipment has 14 pixels, 46-49.

[0107] Figure 10 Another example of an analysis region that can be used to extract features from video frame 101 is shown. In this example, analysis regions 131 to 134 can be mapped to... Figure 1 The pixels 41-44 of the lighting device 13, and the analysis area 134 to 138 can be mapped to Figure 1 The lighting device 15 has pixels 61-65, and the analysis area 138 to 141 can be mapped to... Figure 1 The lighting equipment has 14 pixels, 46-49.

[0108] Figure 10 The analysis region 131-141 is greater than Figure 9 The analysis area is 111-121, and the result is that in Figure 10 There is overlap between adjacent analysis regions, while Figure 9 There is no such overlap between adjacent analysis regions. This overlap is beneficial if the distance between the pixels of the pixelated lighting device and the surface (e.g., a wall) is relatively small.

[0109] exist Figure 9 and 10 In the example, each analysis region has the same size. However, analysis regions of different sizes can also be used, such as... Figure 11 As shown. Among the analysis regions 151-154 on the left side of the video frame, analysis region 151 is the largest, and analysis region 154 is the smallest.

[0110] Figure 11It also demonstrates that it is possible to increase the size of the analysis region without increasing the overlap between adjacent analysis regions. For example, if the pixelated lighting device is far enough from the surface that overlap between adjacent analysis regions is not required, the size of the analysis region can still be increased to focus the lighting effect more toward the environment, rather than just focusing on the color on the side of the screen closest to the light source. Therefore, without changing the overlap, the size of the analysis region can be changed so that it occupies a larger portion of the video frame without affecting adjacent analysis regions.

[0111] exist Figure 11 In the examples, the pixels / light sources of the lighting device have different distances from the surface. For example, the pixelated light device can be leaned against a wall (i.e., at an angle relative to the wall), attached to the back of a curved display, or placed behind a display in the corner between two walls. Figure 11 In the example, the light source associated with analysis region 151 is farthest from the surface, while the light source associated with analysis region 154 is closest to the surface. The farther the light source is from the wall, the greater its effect (on the wall). Therefore, it can sometimes be beneficial to analyze a larger portion of the video frame, even if the distance to the wall is large, so that a large effect on the wall reflects a larger portion of the video frame, rather than just a small portion of the video frame on the side.

[0112] exist Figure 10 In the example, each pair of adjacent analysis regions has the same overlap. However, it is also possible to use different overlaps for different pairs of adjacent analysis regions, such as... Figure 12 As shown. In the analysis regions 161-164 on the left side of the video frame, adjacent analysis regions 163 and 164 have the greatest overlap, while adjacent analysis regions 161 and 162 have the least overlap. For example, this could be beneficial if the pixels / light sources of the lighting device have different distances to the surface.

[0113] By individually determining the distance to the surface for each light source, it is possible to use different overlaps for different pairs of adjacent analysis regions. Figure 12 In the example, the light source associated with analysis region 161 is farthest from the surface, while the light source associated with analysis region 164 is closest to the surface. Figure 12 In the example, all analysis regions have the same size, but they are stacked differently on top of each other; the overlap between adjacent pixels farther from the wall is relatively small, and the overlap between adjacent pixels closer to the wall is relatively large. Therefore, only the location (and not the size) of the analysis region depends on its distance from the surface.

[0114] Figure 12It was also demonstrated that it is possible to change the overlap between two adjacent analysis regions without changing the total overlap between all analysis regions. When the angle of the pixelated lighting device relative to the wall increases and the distance between the light sources associated with analysis region 164 increases as a result, the overlap between adjacent analysis regions 163 and 164 may decrease, and the overlap between adjacent analysis regions 161 and 162 may increase, thus keeping the total overlap the same.

[0115] exist Figure 12 In the example, there is still a small overlap between adjacent analysis regions 161 and 162. However, analysis region 161, associated with the light source farthest from the surface, may not overlap with adjacent analysis region 162 at all. Furthermore, the overlap between adjacent regions 163 and 164 may be even greater than that between adjacent analysis regions 161 and 162. Figure 12 The one shown is larger.

[0116] exist Figures 9 to 12 In the example, all analysis regions have a rectangular shape. It's also possible to use one different shape or multiple different shapes, such as... Figure 13 The analysis area is shown in Figures 171-174.

[0117] Figure 14 This illustration shows a portion of a second embodiment of a method for controlling a light source based on the analysis of image content in an analysis area of ​​the image content when the display shows image content. In this second embodiment, Figure 7 Step 203 is implemented by sub-steps 231-241. After step 203, steps 205-212 are executed, as follows: Figure 7 As shown.

[0118] Step 231 includes obtaining one or more device distances between the lighting fixture and the surface. Typically, these device distances are obtained from sensors or from user equipment (e.g., user input devices). If one end of the vertically arranged lighting fixture needs to lean against a wall, and the device distance for the other end has been obtained, then a second device distance of zero can be automatically obtained.

[0119] Next, step 232 includes checking whether a single device distance was obtained in step 231 or whether multiple device distances were obtained in step 231. If it is determined in step 232 that a single device distance was obtained in step 231, then step 233 is executed. Step 233 includes determining the distance between an additional light source and the surface based on the single device distance determined in step 232. Typically, the distance between the additional light source and the surface is equal to the single device distance. Step 205 is executed after step 233.

[0120] If it is determined in step 232 that multiple device distances were obtained in step 231, then step 235 is performed next. Step 235 includes calculating the average of the multiple device distances determined in step 231. If, for example, two device distances at opposite ends of a lighting device were determined in step 231, then a single average is calculated in step 235. If more than two device distances were determined in step 231, then multiple averages can be calculated in step 235. Next, step 237 includes determining the distance between an additional light source and a surface based on the average(s) calculated in step 235. If a single average was calculated in step 235, the distance between the additional light source and the surface is generally equal to that single average. Step 205 is performed after step 237.

[0121] Figure 15 This illustrates a portion of a third embodiment of a method for controlling a light source based on the analysis of image content in an analysis area of ​​the image content when the display shows image content. Figure 14 Compared to the second embodiment, steps 235 and 237 have been replaced by steps 251 and 253, steps 255 and 257 have been added before step 205, and step 205 is implemented by step 259.

[0122] Step 251 includes determining the position of a light source on the lighting equipment. Step 253 includes determining the distance between each light source and the surface based on at least two of the equipment distances determined in step 231 and the light source positions determined in step 251. The equipment distances are indicated relative to reference points on the lighting equipment (e.g., the edges of the lighting equipment). If the position of the light source is between two reference points, the distance between the light source and the surface is determined based on two equipment distances corresponding to these reference points and the distance between the position of the light source and these reference points.

[0123] Step 255 includes estimating the amount of light overlap between light projected onto the surface by adjacent light sources based on the distance determined in step 203. See also Figure 7 In step 202, the obtained lighting equipment information can be used to estimate the amount of overlap. For example, beam width, beam angle, and distance to the wall can be used to more accurately calculate the amount of light overlap between lighting areas, such as those projected onto a wall by different pixels. Beam width and beam angle typically affect optical blending. For example, more optical blending occurs when a wider beam is used and / or when the beam angle is sharper (due to a longer distance to the surface).

[0124] Step 257 involves determining the desired region overlap between adjacent analysis regions of adjacent light sources based on the estimated optical overlap. The desired region overlap may be lower when more optical mixing occurs. Step 259 involves determining the size and / or location of the analysis region based on the desired region overlap. Step 206 is performed after step 259.

[0125] Figure 16 A fourth embodiment of a method for controlling a light source based on analysis of image content in an analysis area of ​​the image content when displaying image content on a monitor is shown. In this fourth embodiment, additional steps 271, 273, and / or 275 may be performed prior to step 205, and step 205 is implemented by step 277.

[0126] Step 271 includes determining the size of the (multiple) light sources of the lighting device. Step 273 includes determining the size of the lighting device. Step 275 includes determining the distance between the (multiple) light sources and the display. Step 277 includes determining the size and / or location of the analysis area based on the distance determined in step 203 and optionally based on the size of the (multiple) light sources of the lighting device determined in step 271, the size of the lighting device determined in step 273, and / or the distance between the (multiple) light sources and the display. Step 206 is performed after step 277.

[0127] Figure 7 and Figures 14 to 16 The embodiments differ from each other in several ways, namely, multiple steps have been added or substituted. In variations of these embodiments, only a subset of these steps are added or substituted and / or one or more steps are omitted. For example, one or more of steps 271, 273, and 275 may be added to... Figure 14 and / or Figure 15 The embodiments, and Figure 15 Steps 255, 257, and 259 can be added Figure 14 Implementation examples and / or from Figure 15 The example is omitted.

[0128] Figure 17 The description indicates that the procedure can be performed as shown in the reference. Figure 7 as well as Figures 14 to 16 A block diagram of an exemplary data processing system for the described method.

[0129] like Figure 17As shown, the data processing system 300 may include at least one processor 302 coupled to a memory element 304 via a system bus 306. Thus, the data processing system can store program code within the memory element 304. Furthermore, the processor 302 can execute program code accessed from the memory element 304 via the system bus 306. In one aspect, the data processing system may be implemented as a computer suitable for storing and / or executing program code. However, it should be understood that the data processing system 300 may be implemented in the form of any system including a processor and memory capable of performing the functions described herein.

[0130] Memory element 304 may include one or more physical memory devices, such as, for example, local memory 308 and one or more mass storage devices 310. Local memory may refer to random access memory or (multiple) other non-persistent memory devices generally used during the actual execution of program code. Mass storage devices may be implemented as hard disk drives or other persistent data storage devices. Processing system 300 may also include one or more cache memories (not shown) that provide temporary storage for at least some program code to reduce the number of times program code must be retrieved from mass storage device 310 during execution. For example, if processing system 300 is part of a cloud computing platform, processing system 300 may also be able to use memory elements of another processing system.

[0131] The input / output (I / O) devices, depicted as input device 312 and output device 314, may optionally be coupled to the data processing system. Examples of input devices may include, but are not limited to, a keyboard, a pointing device such as a mouse, a microphone (e.g., for sound and / or speech recognition), etc. Examples of output devices may include, but are not limited to, a monitor or display, a speaker, etc. The input and / or output devices may be coupled to the data processing system directly or through an intermediate I / O controller.

[0132] In embodiments, the input and output devices may be implemented as a combined input / output device (in... Figure 17 (Dashed lines are used to illustrate input device 312 and output device 314). An example of such a combined device is a touch-sensitive display, sometimes also called a "touchscreen display" or simply a "touchscreen". In such embodiments, input to the device can be provided by the movement of a physical object, such as, for example, a stylus or a user's finger, on or near the touchscreen display.

[0133] Network adapter 316 can also be coupled to the data processing system to enable it to couple to other systems, computer systems, remote network devices, and / or remote storage devices via an intermediate private or public network. The network adapter may include a data receiver for receiving data transmitted to the data processing system 300 from the systems, devices, and / or networks, and a data transmitter for transmitting data from the data processing system 300 to the systems, devices, and / or networks. Modems, cable modems, and Ethernet cards are examples of different types of network adapters that can be used with the data processing system 300.

[0134] like Figure 17 As illustrated, memory element 304 can store application program 318. In various embodiments, application program 318 may be stored in local memory 308, one or more mass storage devices 310, or separately from local memory and mass storage devices. It should be understood that data processing system 300 may further execute an operating system that facilitates the execution of application program 318. Figure 17 (Not shown in the image). The application program 318, implemented in the form of executable program code, can be executed by the data processing system 300 (e.g., by the processor 302). In response to executing the application program, the data processing system 300 can be configured to perform one or more operational or method steps described herein.

[0135] Various embodiments of the present invention can be implemented as a program product for use with a computer system, wherein the program(s) of the program product define the functionality of the embodiments (including the methods described herein). In one embodiment, the program(s) may be contained on a variety of non-transitory computer-readable storage media, wherein, as used herein, the expression “non-transitory computer-readable storage media” includes all computer-readable media, with the sole exception of transient propagation signals. In another embodiment, the program(s) may be contained on a variety of transient computer-readable storage media. Illustrative computer-readable storage media include, but are not limited to: (i) non-writable storage media on which information is permanently stored (e.g., read-only memory devices within a computer, such as CD-ROM discs readable by a CD-ROM drive, ROM chips, or any type of solid-state non-volatile semiconductor memory); and (ii) writable storage media on which changeable information is stored (e.g., flash memory, floppy disks within a floppy disk drive or hard disk drive, or any type of solid-state random access semiconductor memory). The computer program may run on the processor 302 described herein.

[0136] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that, when used in this specification, the term “comprising” specifies the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0137] All the means or steps plus functional elements in the following claims are intended to include any structure, material, action, and equivalent for performing a function in combination with other claimed elements as specifically claimed. Descriptions of embodiments of the invention have been shown for illustrative purposes, but are not intended to be exhaustive or limited to the implementations in the disclosed forms.

Claims

1. A system (1, 51) for controlling a light source (61) based on analysis of image content (101) in an analysis region (114, 134) of the image content (101) when displaying image content (101) on a display (23), the system (1, 51) comprising: The light source (61) is configured to project light onto a surface, wherein the surface is a wall; At least one input interface (3, 53); At least one output interface (4, 54); and At least one processor (5, 55) is configured as follows: -The image content (101) is obtained via the at least one input interface (3, 53). - Obtain the distance (71-73) between the light source (61) and the surface (81) from the sensor (67-69) or from the user equipment (29). - Determine the size and / or location of the analysis region (114, 134) based on the distance (71-73). - The characteristics of the image content are determined by analyzing the image content (101) in the analysis region (114, 134). - Determine the light effect based on the aforementioned characteristics, and - Control the light source (61) via the at least one output interface (4, 54) to present the light effect onto the surface.

2. The system (1, 51) according to claim 1, wherein, The at least one processor (5, 55) is configured to determine a first analysis region (134) having a first size and a first position when the distance has a first value, and to determine a second analysis region (114) having a second size and a second position when the distance has a second value, the second size being different from the first size and / or the second position being different from the first position, the first analysis region having greater overlap with adjacent analysis regions than the second analysis region, and the first value being smaller than the second value.

3. The system (1, 51) according to claim 2, wherein, The first dimension is larger than the second dimension.

4. The system (1, 51) according to claim 1, wherein, The sensor includes one or more infrared distance sensors and / or one or more ultrasonic distance sensors.

5. The system (1, 51) according to any one of claims 1 to 4, wherein, The light source (61) is included in the lighting device (15), which includes additional light sources (62-65), and the distance (71-73) also represents the distance between the additional light sources (62-65) and the surface (81), and the at least one processor (5, 55) is configured to: -Based on the distance (71-73), determine additional dimensions and / or additional locations of additional analysis regions (115-118, 135-138) of the image content (101). - Further characteristics of the image content (101) are determined by analyzing the image content (101) in the additional analysis regions (115-118, 135-138). -Based on the aforementioned additional characteristics, determine additional lighting effects, and - Control the additional light source (62-65) to present the additional light effect on the surface.

6. The system (1, 51) according to claim 5, wherein, The at least one processor (5, 55) is configured to: - Obtain the device distance (71-73) between the lighting device (15) and the surface (81), -To obtain additional device distance between the lighting device (15) and the surface (81), and The distance is determined by calculating the average of the distance to the device and the distance to the other device.

7. The system (1, 51) according to any one of claims 1 to 4, wherein, The light source (61) is included in the lighting device (15), which includes additional light sources (62-65), and the at least one processor (5, 55) is configured to: -To obtain an additional distance between the additional light source (62-65) and the surface (81), -Based on the additional distance, determine additional dimensions and / or additional locations of additional analysis regions (115-118, 135-138) of the image content (101). - Further characteristics of the image content (101) are determined by analyzing the image content (101) in the additional analysis regions (115-118, 135-138). -Based on the aforementioned additional characteristics, determine additional lighting effects, and - Control the additional light source (62-65) to present the additional light effect on the surface.

8. The system according to claim 5, wherein, The determined lighting effects include color and / or intensity, and / or wherein the determined additional lighting effects include color and / or intensity.

9. The system (1, 51) according to claim 5, wherein, The at least one processor (5, 55) is configured to: -Based on the distance, estimate the amount of light overlap between the light projected onto the surface (81) by the light source and the light projected onto the surface (81) by the other light source. - Based on the estimated optical overlap, determine the desired region overlap between the analysis region and the other analysis region, and - Based on the desired region overlap, determine the size and / or location of the analysis region and the size and / or location of the additional analysis region.

10. The system (1, 51) according to any one of claims 1-4, wherein, The at least one processor (5, 55) is configured to further determine the size and / or location of the analysis area based on the size of the light source (61), the size of the display, and / or the size of the lighting device including the light source (61).

11. The system (1, 51) according to any one of claims 1-4, wherein, The at least one processor (5, 55) is configured to further determine the size and / or location of the analysis area based on the distance between the light source (61) and the display (23).

12. The system (1, 51) according to any one of claims 1-4, wherein, The light source (61) includes multiple light elements, which cannot present different light effects.

13. The system (1, 51) according to claim 4, wherein, The sensor in question is a time-of-flight sensor.

14. A method for controlling a light source based on analysis of image content in an analysis area of ​​the image content when displaying image content on a monitor, the method comprising: - Obtain the image content described in (201); - Obtain the distance between the light source and the surface described in (203); - Determine the size and / or location of the analysis area based on the distance; -The characteristics of the image content are determined by analyzing the image content in the analysis area; - Determine the light effect (209) based on the aforementioned characteristics; as well as - Control (211) the light source to present the light effect on the surface.

15. A computer program product storing at least one software code portion, said software code portion being configured to perform the method of claim 14 when run on a system comprising at least one processor according to any one of the preceding claims 1-13.

Images