System and method for controlling lighting effect

Abstract

A control system for controlling a lighting effect associated with a plurality of lighting devices connected to the control system, the control system comprising: a data receiver configured to receive a media data comprising a time variable series of images, each image of the series of images comprising a plurality of pixels, each pixel having a color data; a processor configured to: select a subset of pixels from the plurality of pixels of a respective image of the series of images; generate a control signal based on the color data of the selected subset of pixels of the respective image; and a transmitter configured to send the control signal to the plurality of lighting devices to control the lighting effect associated with the plurality of lighting devices.

Description

SYSTEM AND METHOD FOR CONTROLLING LIGHTING EFFECTTECHNICAL FIELD

[0001] This disclosure relates to a control system and a method for controlling a lighting effect associated with a plurality of lighting devices connected to the control system. This disclosure also relates to a smart lighting system including the control system and the plurality of lighting devices.BACKGROUND

[0002] It greatly enhances user experience when light effects can be rendered based on the content that is played on a monitor, e.g., a movie, video games, etc. A dynamic lighting system may be used to control the color and brightness of the lighting device to match the contents that is played on the monitor. The light effects matching the contents can bring an atmosphere of a video game or movie to the user. For example, the light effects matching the contents played on the monitor can let gamers enjoy the flashes of weapon fire or sit in the glow of the force fields as if they were real. The light effects extend the visuals beyond the screen and creates an immersive ambient lighting experience.

[0003] Therefore, there exists a need for providing an improved control system and method for controlling a lighting effect associated with a plurality of lighting devices connected to the control system.SUMMARY

[0004] According to a first aspect of the present disclosure, a control system for controlling a lighting effect associated with a plurality of lighting devices connected to the control system is provided. The control system may comprise: a data receiver configured to receive a media data comprising a time variable series of images, each image of the series of images comprising a plurality of pixels, each pixel having a color data; a processor configured to: select a subset of pixels from the plurality of pixels of a respective image of the series of images; generate a control signal based on the color data of the selected subset of pixels of the respective image; and a transmitter configured to send the control signal to the plurality of lighting devices to control the lighting effect associated with the plurality of lighting devices.

[0005] According to a second aspect of the present disclosure, A smart lighting system is provided. The smart lighting system may comprise the control system as described herein and the plurality of lighting devices connected to the control system.

[0006] According to a third aspect of the present disclosure, a method of controlling a lighting effect associated with a plurality of lighting devices connected to a control system by the control system, the control system comprising a data receiver, a processor and a transmitter is provided. The method may comprise: receiving, by the data receiver, a media data comprising a time variable scries of images, each image of the scries of images comprising a plurality of pixels, each pixel having a color data; selecting, by the processor, a subset of pixels from the plurality of pixels of a respective image of the series of images; generating, by the processor, a control signal based on the color data of the selected subset of pixels of the respective image; and sending, by the transmitter, the control signal to the plurality of lighting devices to control the lighting effect associated with the plurality of lighting devices.BRIEF DESCRIPTION OF THE DRAWINGS

[0007] In the drawings, like reference characters generally refer to like parts throughout the different views. The drawings arc not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the disclosure. Tn the following description, various embodiments of the disclosure are described with reference to the following draw ings, in which:

[0008] FIG. 1 shows a control system for controlling a lighting effect associated with a plurality of lighting devices connected to the control system according to some embodiments of the present disclosure;

[0009] FIG. 2 shows a control system for controlling a lighting effect associated with a plurality of lighting devices connected to the control system according to some embodiments of the present disclosure;

[0010] FIG. 3 shows a control system for controlling a lighting effect associated with a plurality of lighting devices connected to the control system according to some embodiments of the present disclosure;

[0011] FIG. 4 shows a control system for controlling a lighting effect associated with a plurality of lighting devices connected to the control system according to some embodiments of the present disclosure;

[0012] FIG. 5 shows an example of a time variable series of images;

[0013] FIG. 6A shows an example of an image;

[0014] FIG. 6B shows an example of selecting a subset of pixels from the plurality of pixels of an image of FIG. 6A via an interlace scan by fractionating the plurality of pixels of the image by an interlace factor of 2;

[0015] FIG. 6C shows an example of selecting a subset of pixels from the plurality of pixels of an image of FIG. 6A via an interlace scan by fractionating the plurality of pixels of the image by an interlace factor of 2;

[0016] FIG. 7 shows an example of selecting a subset of pixels from a plurality of pixels of an image according to some embodiments of the present disclosure;

[0017] FIG. 8 shows a flow chart of a process of classifying each block in an image as one of a missing block and a passing block according to some embodiments of the present disclosure;

[0018] FIG. 9A shows an example of an image in a time variable series of images included in a media data according to some embodiments of the present disclosure;

[0019] FIG. 9B shows an example of a lighting effect associated with a plurality of lighting devices (e.g., a light wall) controlled by the control signal generated based on the color data of the selected subset of pixels of the image of FIG. 9A;

[0020] FIG. 10A shows an example of dividing a selected subset of pixels into two groups, including a first group in a left part of an image and a second group in a right part of the image;

[0021] FIG. 10B shows an example of a lighting effect associated with the left light bar controlled by a control signal generated based on the color data of the first group of selected subset of pixels;

[0022] FIG. 10C shows an example of a lighting effect associated with the right light bar controlled by a control signal generated based on the color data of the second group of selected subset of pixels;

[0023] FIG. 11A shows an example of a color palette that corresponds to the scene data of campfire (or referred to as campfire color palette);

[0024] FIG. 1 IB shows an example of the color codes consisted in the campfire color palette;

[0025] FIGs.l2A to 12D show an example of determining a block motion when each image of at least three sequential images contains a respective target block having a same or similar color value according to an embodiment of the present disclosure;

[0026] FIGs. 12E to 12F respectively show an example of predicting a position of a target block having a same or similar color value in a subsequent image following the at least three sequential images, according to an embodiment of the present disclosure;

[0027] FIG. 12G shows a position of the respective target block in the at least three sequential images and a predicted position of the respective target block in a subsequent image following the at least three sequential images in a demonstrating image;

[0028] FIGs.l 3A to 13C show an example of determining an object motion when each image of at least three sequential images contains respective target blocks having a same or similar color value according to an embodiment of the present disclosure;

[0029] FIGs.l3D to 13E respectively show an example of predicting a position of a target object in a subsequent image following the at least three sequential images according to an embodiment of the present disclosure;

[0030] FIG. 14 shows a smart lighting system according to some embodiments of the present disclosure;

[0031] FIG. 15 is a flow chart depicting a method of controlling a lighting effect associated with a plurality of lighting devices connected to a control system according to some embodiments of the present disclosure.DETAILED DESCRIPTION

[0032] The following detailed description refers to the accompanying drawings that show, by way of illustration, specific details, and embodiments in which the disclosure may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the disclosure. Other embodiments may be utilized, and structural, logical, optical and electrical changes may be made without departing from the scope of the disclosure. The various embodiments arc not necessarily mutually exclusive, as some embodiments can be combined with one or more other embodiments to form new embodiments.

[0033] Embodiments described in the context of one of the methods or devices are analogously valid for the other methods or devices. Similarly, embodiments described in the context of a method are analogously valid for a device, and vice versa.

[0034] Features that are described in the context of an embodiment may correspondingly be applicable to the same or similar features in the other embodiments. Features that arc described in the context of an embodiment may correspondingly be applicable to the other embodiments, even if not explicitly described in these other embodiments. Furthermore, additions and / or combinations and / or alternatives as described for a feature in the context of an embodiment may correspondingly be applicable to the same or similar feature in the other embodiments.

[0035] It should be understood that the terms “on”, “over”, “top”, “bottom”, “down”, “side”, “back”, “left”, “right”, “front”, “back”, “lateral”, “side”, “up”, “down”, “vertical”, “horizontal” etc., when used in the following description are used for convenience and to aid understanding of relative positions or directions, and not intended to limit the orientation of any device, or structure or any part of any device or structure. In addition, the singular terms "a", "an", and "the" include plural references unless context clearly indicates otherwise. Similarly, the “or” is intended to include “and” unless the context clearly indicates otherwise.

[0036] It will be further understood that the terms “comprise” (and any form of comprise, such as “comprises” and “comprising”), “have” (and any form of have, such as “has” and “having”), “include” (and any form of include, such as “includes” and “including”), and “contain” (and any form of contain, such as “contains” and “containing”) are open-ended linking verbs. As a result, a method or device that “comprises,” “has,” “includes” or “contains” one or more steps or elements possesses those one or more steps or elements, but is not limited to possessing only those one or more steps or elements. Likewise, a step of a method or an element of a device that “comprises,” “has,” “includes” or “contains” one or more features possesses those one or more features, but is not limited to possessing only those one or more features. Furthermore, a device or structure that is configured in a certain way is configured in at least that way, but may also be configured in ways that are not listed.

[0037] Approximating language, as used herein throughout the specification and claims, may be applied to modify any quantitative representation that could permissibly vary without resulting in a change in the basic function to which it is related. Accordingly, a value modified by a term or terms, such as “about,” “substantially”, is not limited to the precise value specified but within tolerances that are acceptable for the operation of the embodiment for an application for which it is intended. In some instances, the approximating language may correspond to the precision of an instrument for measuring the value.

[0038] The term “exemplary” may be used herein to mean “serving as an example, instance, or illustration”. Any aspect or design described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects or designs.

[0039] The terms “at least one” and “one or more” may be understood to include a numerical quantity greater than or equal to one (e.g., one, two, three, four, [...], etc.). The term “a plurality” may be understood to include a numerical quantity greater than or equal to two (e.g., two, three, four, five, [...], etc.). The phrase “at least one of” with regard to a group of elements may be used herein to mean at least one element from the group consisting of the elements. For example, the phrase “at least one of’ with regard to a group of elements may be used hereinto mean a selection of: one of the listed elements, a plurality of one of the listed elements, a plurality of individual listed elements, or a plurality of a multiple of listed elements.

[0040] The words “plural” and “multiple” in the description and the claims expressly refer to a quantity greater than one. Accordingly, any phrases explicitly invoking the aforementioned words (e.g., “a plurality of (objects)”, “multiple (objects)”) referring to a quantity of objects expressly refer to more than one of the said objects. The terms “group (of)”, “set (of)”, “collection (of)”, “series (of)”, “sequence (of)”, “grouping (of)”, etc., and the like in the description and in the claims, if any, refer to a quantity equal to or greater than one, i.c. one or more.

[0041] The term “first”, “second”, “third” detailed herein are used to distinguish one element from another similar element and may not necessarily denote order or relative importance, unless otherwise stated.

[0042] As used herein, the phrase of the form of “at least one of A or B” may include A or B or both A and B. Correspondingly, the phrase of the form of “at least one of A or B or C”, or including further listed items, may include any and all combinations of one or more of the associated listed items.

[0043] The following examples pertain to various aspects of the present disclosure.

[0044] Example 1 is a control system for controlling a lighting effect associated with a plurality of lighting devices connected to the control system, the control system comprising: a data receiver configured to receive a media data comprising a time variable series of images, each image of the series of images comprising a plurality of pixels, each pixel having a color data; a processor configured to: select a subset of pixels from the plurality of pixels of a respective image of the series of images;generate a control signal based on the color data of the selected subset of pixels of the respective image; and a transmitter configured to send the control signal to the plurality of lighting devices to control the lighting effect associated with the plurality of lighting devices.

[0045] In Example 2, the subject matter of Example 1 may optionally include that the control signal is used to control a color effect of each lighting device of the plurality of lighting devices so as to generate the lighting effect.

[0046] In Example 3, the subject matter of Example 1 may optionally include that the subset of pixels from the plurality of pixels of a respective image of the series of images is selected via an interlace scan by fractionating the plurality of pixels of a respective image of the series of images by an interlace factor.

[0047] In Example 4, the subject matter of Example 1 may optionally include that the processor is configured to: divide each image into m columns and n rows such that each image has m*n blocks, wherein m and n are both an integer greater than or equal to 2; classify each block of the m*n blocks as one of a missing block and a passing block, wherein a block of the m*n blocks is classified as the missing block when no pixel in that block is within a predetermined color range; wherein a block of the m*n blocks is classified as the passing block when at least one pixel in that block is within the predetermined color range.

[0048] In Example 5, the subject matter of Example 4 may optionally include that the processor is configured to: classify each column of the m columns as one of a selected column, an unselected column and an interpolated column, wherein the column is classified as the selected column when there is no missing block in the column; wherein the column is classified as the unselected column when a number of missing blocks in the column equals to or is greater than a preset threshold; wherein the column is classified as the interpolated column when a number of missing blocks in the column is greater than 0 and less than the preset threshold, wherein the selected subset of pixels consists of: (i)pixcls of the passing blocks of the selected column; (ii)pixels of the passing blocks of the interpolated column; (iii) pixels of a respective missing block of the at least one missing blocks in the interpolated column, when each of adjacent blocks that are adjacent to the respective missing block has an average intensity of any of the red, green and blue color of all pixels in that adjacent block that is less than or equals to an interpolating threshold; (iv)pixels of an interpolating block for a respective missing block of the at least one missing blocks in the interpolated column, wherein the interpolating block is an adjacent block that is adjacent to the respective missing block, when the adjacent block that is adjacent to the respective missing block has an average intensity of any of the red, green and blue color of all pixels in the adjacent block that is greater than the interpolating threshold.

[0049] In Example 6, the subject matter of Example 1 may optionally include that the processor is further configured to: predict a scene data for each image based on the color data of the selected subset of pixels of the image by a scene prediction model; and obtain the control signal for the plurality of lighting devices based on the predicted scene data.

[0050] In Example 7, the subject matter of Example 6 may optionally include that the scene prediction model is trained based on a plurality of training scene data and a color data of a selected subset of pixels of a plurality of training images, each training scene data corresponds to a training image of the plurality of training images.

[0051] In Example 8, the subject matter of Example 1 may optionally include that the processor is further configured to: divide each image of at least three sequential images of the scries ofimages into m columns and n rows such that each image has trfn blocks, each block having a color value representing an average intensity of the red, green and blue color of all pixels in the block; determine a block motion when each image of the at least three sequential images contains a respective target block having a same or similar color value or determine an object motion when each image of the at least three sequential images contains respective target blocks having same or similar color values.

[0052] In Example 9, the subject matter of Example 8 may optionally include that the color value is the same or similar when a difference between the color value of the respective target block in each image of the at least three sequential images is less than or equals to a similarity threshold.

[0053] In Example 10, the subject matter of Example 8 may optionally include that the processor is further configured to: predict a position of a target block having the same or similar color value in a subsequent image following the at least three sequential images based on positions of the respective target blocks in the at least three sequential images.

[0054] In Example 11, the subject matter of Example 10 may optionally include that the processor is further configured to: generate a predicted control signal based on a color data of a predicted subset of pixels of the subsequent image following the at least three sequential images to control the lighting effect associated with each lighting device of the plurality of lighting devices, wherein the color data of the predicted subset of pixels of the subsequent image following the at least three sequential images is obtained based on the predicted position of the target block in the subsequent image, and the position of the target block and the color data of the selected subset of pixels of the at least three sequential images.

[0055] In Example 12, the subject matter of Example 1 may optionally include a data converter configured to convert a format of the media data such that the converted format of the media data is compatible with the processor of the control system.

[0056] In Example 13, the subject matter of Example 1 may optionally include that the processor is configured to reduce a frame rate of the media data to a predetermined frame rate value.

[0057] In Example 14, the subject matter of Example 1 may optionally include that the media data is transmitted to a display device to display the series of images over time.

[0058] In Example 15, the subject matter of Example 14 may optionally include that a refresh rate of the display device is synchronized with a frame rate of the media data.

[0059] In Example 16, the subject matter of Example 1 may optionally include that the transmitter is in wireless communication with the plurality of lighting devices.

[0060] Example 17 is a smart lighting system, the smart lighting system comprising the control system of Example 1; and the plurality of lighting devices connected to the control system.

[0061] In Example 18, the subject matter of Example 17 may optionally include that the plurality of lighting devices each comprises at least a red, a green and a blue light emitting device.

[0062] Example 19 is a method of controlling a lighting effect associated with a plurality of lighting devices connected to a control system by the control system, the control system comprising a data receiver, a processor and a transmitter, the method comprising: receiving, by the data receiver, a media data comprising a time variable series of images, each image of the series of images comprising a plurality of pixels, each pixel having a color data; selecting, by the processor, a subset of pixels from the plurality of pixels of a respective image of the series of images; generating, by the processor, a control signal based on the color data of the selected subset of pixels of the respective image; and sending, by the transmitter, the control signal to the plurality of lighting devices to control the lighting effect associated with the plurality of lighting devices.

[0063] In Example 20, the subject matter of Example 19 may optionally include controlling a color effect of each lighting device of the plurality of lighting devices using the control signal so as to generate the lighting effect.

[0064] The present disclosure provides a control system for controlling a lighting effect associated with a plurality of lighting devices connected to the control system. FIG. 1 and FIG. 2 respectively shows a control system 100 for controlling a lighting effect associated with a plurality of lighting devices 190 connected to the control system 100 according to some embodiments of the present disclosure. The control system 100 may be configured to control the plurality of lighting devices 190 to render dynamic light effects corresponding to a video content being displayed on a display device 10, e.g., a TV.

[0065] The control system 100 may include a data receiver 110, a processor 120 and a transmitter 130.

[0066] The data receiver 110 may be configured to receive a media data. The media data may include a video content. The media data may include a time variable scries of images 90 (sec FIG. 5). For example, the series of images 90 (or referred to as time variable series of images 90) may include a first to N-th image, where N is an integer that is greater than or equal to 2.

[0067] The media data may be output to (e.g., via an output interface of the control system 100) a display device 10 to display the series of images over time. The output media data (outputl)may be displayed on the display device 10 (c.g., a TV monitor, a mobile phone display, a projector, a pc / laptop display, etc).

[0068] Each image of the first to N-th images may include a plurality of pixels, each pixel having a color data. The color data may include a red color data (or referred to as a value for the red channel) representing an intensity of the red color, a green color data (or referred to as a value for the green channel) representing an intensity of the green color, and a blue color data (or referred to as a value for the blue channel) representing an intensity of the blue color. Each of the red, green and blue color data (or referred to as RGB values) may include n (c.g., 8) bits data such that the color data may represent a combination of 23n(e.g., 224) different colors. Each of the red, green and blue color data may range from 0 to 255.

[0069] The color data of each pixel may also include a brightness of the pixel. The brightness may refer to the overall intensity or luminance of a pixel of the image, irrespective of its color. The brightness may be calculated using a weighted average of the RGB values. The brightness may provide information about how light or dark an image appears overall, without considering the specific colors the image presents. The RGB values of a pixel may describe the color composition of individual pixels in an image, and the brightness of a pixel may provide a measure of the overall lightness or darkness of the pixel.

[0070] The processor 120 may be configured to select a subset of pixels from the plurality of pixels of a respective image of the series of images (e.g., the first to N-th images). The processor 120 may be configured to generate a control signal based on the color data of the selected subset of pixels of the respective image. The processor 120 may include a field programmable gated array (FPGA), an application specific integrated chip (ASIC), a programmable circuit board (PCB), or other suitable integrated chip (IC) device.

[0071] The transmitter 130 may be configured to send the control signal (output2) to the plurality of lighting devices 190 to control the lighting effect associated with the plurality of lighting devices 190 (including 190a, 190b, ...., collectively 190). The control signal may be generated based on the respective image and be used to control a color effect of each lighting device of the plurality of lighting devices 190 so as to generate the lighting effect. The color effect of each lighting device may include a brightness of a color of the lighting device, RGB values of the color including a color shade for the red, green, blue color, a duration of the color, etc. The control signals generated by the time variable series of images may be used to control the plurality of lighting devices 190 to render dynamic light effects corresponding to the media data (e.g., a video content) being displayed on a display device 10, e.g., a TV.

[0072] The transmitter 130 may send the control signal to the plurality of lighting devices 190 using wired or wireless communication technologies such as Wi-Fi, Bluetooth, etc. Tire control system 100 may include a router 180 configured to receive the control signal from the transmitter 130 and relay the control signal to the plurality of lighting devices 190.

[0073] FIG. 3 shows a control system 200 for controlling a lighting effect associated with a plurality of lighting devices (e.g., the plurality of lighting devices 190) connected to the control system 200 according to some embodiments of the present disclosure. FIG. 4 shows a control system 300 for controlling a lighting effect associated with a plurality of lighting devices connected to the control system 300 according to some embodiments of the present disclosure. Referring to FIG. 3 and FIG. 4, the control system 200 / 300 may, similar to the control system 100 of FIG. 1 and FIG. 2, include a data receiver 210 / 310, a processor 220 / 320 and a transmitter 230 / 330.

[0074] The components of the control system 200 / 300 of FIG. 3 / F1G.4 that are like the corresponding components of the control system 100 of FIG. 1 and FIG. 2 are similarly numbered. The description of these components made with respect to the control system 200 / 300 of FIG. 3 / FIG.4 may be applicable with respect to the control system 100 of FIG. 1 and FIG. 2, and vice versa.

[0075] Referring to FIG. 3, the control system 200 may include a data receiver 210. The data receiver 210 may be configured to receive a media data including a time variable series of images 90 as shown in FIG. 5, each image of the series of images 90 including a plurality of pixels, each pixel having a color data. The data receiver 210 may include a data splitter 250 (e.g., a High-Definition Multimedia Interface (HDMI) 2.1 splitter) configured to split an input signal (e.g., a HDMI 2.1 input signal) into a first output signal (e.g., a HDMI 2.1 ouput signal) and a second output signal (e..g, a HDMI 2.0 output signal). The first output signal may be referred to as output3-l and the second output signal may be referred to as output3-2. Each of the first and second output signals may include the media data.

[0076] The control system 200 may include a processor 220. The control system 200 may include a data converter configured to convert a format of the media data such that the converted format of the media data is compatible with the processor 220 of the control system 200. The data converter may include a HDMI 2.0 to low- voltage differential signaling (LVDS) converter 254 configured to convert a HDMI 2.0 output signal to a LVDS signal. The processor 220 may include a field programmable gated array (FPGA) 256. The FPGA 256 may be configured to convert the LVDS signal to a Mobile Industry Processor Interface (MIPI) signal. The FPGA 256 may be configured to reduce a frame rate of the media data to a predeterminedframe rate value. The FPGA 256 may be configured to down-scale the media data. The FPGA 256 may be configured to synchronize a refresh rate of the display device 10 with the frame rate of the media data. The FPGA 256 may be configured to adjust clock domain to ensure correct LVDS signal based on frame-rate change information from the HDMT 2.0 to LVDS converter 254. The FPGA 256 may be configured to conduct a process of color picking. The process of color picking may refer to a process of selecting a subset of pixels from the plurality of pixels of a respective image of the series of images as described herein.

[0077] The processor 220 may include an artificial intelligence (Al) chip 258 configured to process the MIPI signal and generate a control signal based on the MIPI signal. Tire Al chip 258 may be configured to conduct a process of LED data mapping. The process of LED data mapping may refer to a process of generating a control signal based on the color data of the selected subset of pixels of the respective image of the series of images as described herein. The processor 220 may further include a microcontroller unit 27O.The microcontroller unit 270 may be configured to detect whether the frame rate of the media data (e.g., the LVDS signal) is changed, and send to the FPGA 256 an information signal that the frame rate of the media data (e.g., the LVDS signal) is changed such that the FPGA 256 may synchronize a refresh rate of the display device 10 with the frame rate of the media data (e.g., the LVDS signal).

[0078] The transmitter 230 may include a Wi-Fi 7 / Bluetooth low energy (BLE) module 260 configured to receive the control signal from the processor 220 and send the control signal to the plurality of lighting devices.

[0079] Referring to FIG. 4, the control system 300 may include a data receiver 310. The data receiver 310 may be configured to receive a media data including a time variable scries of images 90 as shown in FIG. 5, each image of the series of images 90 including a plurality of pixels, each pixel having a color data. The data receiver 310 may include a data splitter 350 (e.g., a HDMI 2.1 splitter) configured to split an input signal (e.g., a HDMI 2.1 input signal) into a first output signal (e.g., a HDMI 2.1 ouput signal) and a second output signal (e..g, a HDMI 2.1 output signal). The first output signal may be referred to as output4- 1 and the second output signal may be referred to as output4-2. Each of the first and second output signals may include the media data.

[0080] The control system 300 may include a processor 320. The processor 320 may include a data converter configured to convert a format of the media data such that the converted format of the media data is compatible with the processor 320 of the control system 300. The data converter may include a HDMI2.1 to Display Port 2.1 Converter 354 configured to convert aHDMI 2.1 signal to a display port 2.1 signal. The data converter may be configured to synchronize a refresh rate of the display device 10 with the frame rate of the media data.

[0081] The processor 320 may include an Al chip 358 configured to process the display port 2.1 signal and generate a control signal based on the display port 2.1 signal. The AT chip 358 may be configured to conduct a process of color picking. The process of color picking may refer to a process of selecting a subset of pixels from the plurality of pixels of a respective image of the series of images as described herein. The Al chip 358 may be configured to conduct a process of LED data mapping. The process of LED data mapping may refer to a process of generating a control signal based on the color data of the selected subset of pixels of the respective image of the series of images as described herein. The processor 320 may further include a microcontroller unit 370. The microcontroller unit 370 may be configured to detect whether the frame rate of the media data (e.g., the HDMI 2.1 signal) is changed, and send to the data converter an information signal that the frame rate of the media data (e.g., the HDMI 2.1 signal) is changed such that the data converter may synchronize a refresh rate of the display device 10 with the frame rate of the media data (e.g., the HDMI 2.1 signal).

[0082] The transmitter 330 may include a Wi-Fi 7 / BLE module 360 configured to receive the control signal from the processor 320 and send the control signal to the plurality of lighting devices.

[0083] The Wi-Fi 7 / BLE module 260 / 360 may be further configured to function as a microcontroller unit.

[0084] The transmitter 230 / 330 may include an analog microphone configured to act as an audio meter and pick up surrounding sound. The analog microphone may be configured to generate a sound command signal based on the surrounding sound (e.g., the audio amplitude of the surrounding sound) to control the lighting effect associated with the plurality of lighting devices. The transmitter 230 / 330 may include an on / off power button configured to power on or power off the transmitter 230 / 330. The transmitter 230 / 330 may include a tri-color lightemitting diode (LED) indicator 262 / 362 configured to indicate whether the control signal or the sound command signal is sent to the plurality of lighting devices to control the lighting effect associated with the plurality of lighting devices. When the control signal generated based on the media data received from the data receiver 210 / 310 is sent to the plurality of lighting devices, the tri-color LED indicator 262 / 362 may indicate a first color (e.g., a yellow color). When the sound command signal generated based on the surrounding sound picked up by the analog microphone is sent to the plurality of lighting devices, the tri-color LED indicator 262 / 362 may indicate a second color (e.g., an orange color) different from the first color. Whenno sound command signal or control signal is sent to the plurality of lighting devices, the transmitter 230 / 330 may enter a standby mode, and the tri-color LED indicator 262 / 362 may indicate a third color (e.g., a red color) which is different from the first and second color. The plurality of lighting devices may include a front facing panel addressable Red Green Blue (ARGB) 264 / 364 configured to present chroma effects. The transmitter 230 / 330 may be configured to send the control signal to the front facing panel ARGB 264 / 364 to control the chroma effects. The front facing panel ARGB 264 / 364 may include a LED strip designed along a front case of the control system 200 / 300.

[0085] The microcontroller unit 270 / 370 may be configured to act as a system headquarter to synchronize and coordinate each component of the control system 200 / 300 for correct functions. For example, the microcontroller unit 270 may be configured to read (i) High dynamic range (HDR) / Standard Dynamic Range (SDR) mode, color space format etc. from the data splitter 250 (e.g., the HDM1 2.1 splitter); (ii) frame rate information from the data converter (e.g., the HDMI 2.0 to LVDS converter 254); (iii) color-picking status from the FPGA 256; (iv) neural processing unit (NPU) processing status and LED data mapping status from the Al Chip 258; and (v) WIFI Transmit (TX) / Receive (RX) status from the transmitter 230 (e.g., the Wi-Fi 7 / BLE module 260). For example, the microcontroller unit 370 may be configured to read (i) HDR / SDR mode, color space format etc. from the data splitter 350 (e.g., the HDMI 2.1 splitter); (ii) frame rate information from the data converter (e.g., the HDMI 2.1 to Display Port 2.1 Converter 354); (iii) color-picking status, neural processing unit (NPU) processing status and LED data mapping status from the Al Chip 358; and (iv) WIFI Transmit (TX) / Rcccivc (RX) status from the transmitter 330 (e.g., the Wi-Fi 7 / BLE module 360). The color-picking status may indicate a status of the process of selecting a subset of pixels from the plurality of pixels of a respective image of the series of images. The LED data mapping status may indicate a status of the process of generating a control signal based on the color data of the selected subset of pixels of the respective image of the series of images.

[0086] The control system 100 / 200 / 300 may include a memory 140 / 240 / 340. The memory 140 / 240 / 340 may be configured to store the control signal.

[0087] In accordance with some embodiments of the present disclosure, the data splitter 250 / 350 may be compliant with HDMI 2.1 , HDMI 2.0b, HDMI 1 .4b input signal and output signal. The data splitter 250 / 350 may be compliant with High-bandwidth Digital Content Protection (HDCP) 2.2 / 2.3 and HDCP 1.4. The data rate of the input signal and the output signal may be up to 48Gbps (Fixed Rate Link (FRL) 12Gbps / 4 Lane). The data splitter 250 / 350 may be configured to counteract distortions of input signal due to various factors such as noise,attenuation, dispersion, and interference. Parameters such as filter coefficients, equalization depth, and adaptation algorithms may be adjusted to optimize performance for specific communication channels or signal conditions. The data splitter 250 / 350 may be configured to support High Dynamic Range (HDR) and Dynamic / Static Metadata. The data splitter 250 / 350 may be configured to support Variable Refresh Rate (VRR), Free-Sync, G-Sync. The data splitter 250 / 350 may be configured to support Auto Low Latency Mode (ALLM). The data splitter 250 / 350 may be configured to support Programmable Voltage Swing, Slew-Rate and Pre-emphasis. The data splitter 250 / 350 may be configured to support AC-coupling on Transition minimized differential signaling (TMDS) input / output. The data splitter 250 / 350 may be configured to support Color Space Converter in TMDS mode. The data splitter 250 / 350 may be configured to support High dynamic range (HDR). The data splitter 250 / 350 may be configured to support High dynamic range (HDR) to Standard Dynamic Range (SDR) conversion. The data splitter 250 / 350 may be configured to support display stream compression (DSC) encoded stream pass-through from input (e.g., HDMI / DP input).

[0088] An input interface of the data converter may be compliant with HDMI 2.0b, HDMI 1 ,4b signals. The input interface of the data converter may be compliant with HDCP 2.2 / 2.3 and HDCP 1.4. The data rate of a signal input to the input interface of the data converter may be up to 18Gbps. The data rate of a signal output from an output interface of the data converter may be up to 1.5Gbps per lane. The data converter may be configured to counteract distortions of input signal due to various factors such as noise, attenuation, dispersion, and interference. The data converter may be configured to support Variable Refresh Rate (VRR), Free-Sync, G- Sync. The data converter may include a color space converter. The data converter may be configured to support Programmable Voltage Swing, Slew-Rate and Pre-emphasis.

[0089] The Al chip 258 / 358 may include a neural processing unit (NPU).The NPU may have a capability that is in trillion operations per second (TOPS) unit. The NPU may be configured to detect images such as gaming scenes, objects, faces, etc, e.g., by transacting mega data.

[0090] The Wi-Fi 7 / BLE module 260 / 360 may support a maximum data rate of 46 Gbps.

[0091] The control system 200 / 300 may incur a lighting delay which is less than 2ms. The lighting delay may refer to a lag from a first time when the control system 200 / 300 receives the input signal (e.g., a HDMI 2.1 input signal) to a second time when control system 200 / 300 sends the control signal to the plurality of lighting devices.

[0092] According to various non-limiting embodiments, the processor 120 / 220 / 320 may be configured to downscale the color data of the media data. For example, for a media data including a video of 3840 x 2160 (4K) at 120Hz video and 24 bits red green blue (RGB) colordepth, the color data is around 2388.8 Mbit per second. The color data of the media data may be downscaled to avoid potential signal lag. FIG. 5 shows an example of a time variable series of images 90. Referring to FIG. 5, each image 90 (or referred to as image frame) may be divided into m columns and n rows such that each image 90 has m*n blocks 70, wherein m and n are both an integer that is greater than or equal to 2. The m and n may be set based on the lightemitting diode (LED) latency requirements. The color data of raw pixels in each block of an image may be averaged to generate 8 bit red color data, 8 bit green color data, 8 bit blue color data for the block. Frame rate may be downscaled from a high frame rate (e.g., 60Hz, 120Hz, 240Hz) to a low frame rate (e.g., 5 ~30 Hz). The color data after downscaling may equal to n*m*R*p, where m is the number of columns of blocks 70 in an image, n is the number of rows of blocks 70 in an image, R is the frame rate (i.e., the frame number per second), p is bit number for the red, green and blue color data (or referred to as RGB values) for each block. The color data after downscaling becomes 61 ,44Kbits per second (i.e., 7.68K bytes per second). The original color data is around 2388.8 Mbit color data per second for a media data including a video of 3840 x 2160 (4K) at 120Hz video and 24 bits RGB color depth. The down-scale ratio is about 388800:1.

[0093] According to various non-limiting embodiments, a subset of pixels may be selected from the plurality of pixels of a respective image of the series of images 90 via an interlace scan by fractionating the plurality of pixels of a respective image of the series of images 90 by an interlace factor.

[0094] An interlaced scan may refer to dividing an image into alternating lines or fields. The interlace factor may refer to a ratio or factor by which an image is interlaced. When the interlace factor is 2, an interlaced scan may divide an image into two fields including an odd field and an even field, where the odd field may contain all the odd-numbered lines (i.e., line 1, 3, 5, etc.), while the even field may contain all the even-numbered lines (i.e., line 2, 4, 6, etc.).

[0095] FIG. 6A shows an example of an image 90. FIG. 6B and FIG. 6C respectively shows an example of selecting a subset of pixels from the plurality of pixels of an image 90 of FIG. 6A via an interlace scan by fractionating (e.g., dividing, splitting) the plurality of pixels of the image 90 by an interlace factor of 2. In FIG. 6B, pixels of the odd field containing all the odd- numbered lines 82 (i.e., line 1 , 3, 5, etc.) may be selected as the subset of pixels. In FIG. 6C, pixels of the even field containing all the even-numbered lines 84 (i.e., line 2, 4, 6, etc.) may be selected as the subset of pixels.

[0096] According to various non-limiting embodiments, the processor 120 / 220 / 320 may be configured to divide each image 90 into m columns and n rows such that each image 90 hasm*n blocks, wherein m and n arc both an integer that is greater than or equal to 2. Each block of the m*n blocks may be classified as one of a missing block and a passing block. A block of the m*n blocks may be classified as the missing block when no pixel in that block is within a predetermined color range. A block of the m*n blocks may be classified as the passing block when at least one pixel in that block is within the predetermined color range. An adjacent block that is adjacent to a respective missing block in a column may be used as an interpolating block for the respective missing block, when the adjacent block that is adjacent to the respective missing block has an average intensity of any of the red, green and blue color of all pixels in the adjacent block that is greater than the interpolating threshold (e.g.,0). The adjacent block may refer to a block that is in the same row with the respective missing block and in a column that is next to a column of the respective missing block. Each missing block may have two adjacent blocks arranged in a preceding column and a succeeding column to the column of the missing block.

[0097] The processor 120 / 220 / 320 may be configured to classify each column of the m columns as one of a selected column, an unsclcctcd column and an interpolated column, wherein the column may be classfied as the selected column when there is no missing block in that column, or classified as the unselected column when a number of missing blocks in that column equals to or is greater than a preset threshold, or may be classified as the interpolated column when a number of missing blocks in that column is greater than 0 and less than the preset threshold. The selected subset of pixels that may be used to generate the control signal for controlling the lighting effect associated with the plurality of lighting devices 190 may consist of the following:(i) pixels of the passing blocks of the selected column;(ii) pixels of the passing blocks of the interpolated column;(iii) pixels of a respective missing block of the at least one missing blocks in the interpolated column, when each of adjacent blocks that are adjacent to the respective missing block has an average intensity of any of the red, green and blue color of all pixels in that adjacent block that is less than or equals to an interpolating threshold (c.g., 0);(iv) pixels of an interpolating block for a respective missing block of the at least one missing blocks in the interpolated column, wherein the interpolating block is an adjacent block that is adjacent to the respective missing block, when the adjacent block that is adjacent to the respective missing block has an averageintensity of any of the red, green and blue color of all pixels in the adjacent block that is greater than the interpolating threshold (e.g.,0).

[0098] FIG. 7 shows an example of selecting a subset of pixels from a plurality of pixels of an image 90 according to some embodiments of the present disclosure. FIG. 8 shows a flow chart of a process 820 of classifying each block in an image 90 as one of a missing block and a passing block according to some embodiments of the present disclosure.

[0099] With reference to FIG. 7, the image 90 may be divided into m columns and n rows such that each image 90 has m*n blocks, wherein m and n arc both an integer that is greater than or equal to 2. Each block may include a plurality of pixels. Each of the red, green and blue color data (or referred to as RGB values) of any pixel in each block may include n (e.g., 8) bits data. The red color data may include n bits data represented by R(n-1 ) to R0; the green color data may include n bits data represented by G(n-l) to GO; the blue color data may include n bits data represented by B(n- 1) to B0. With reference to FIG. 8, the process 820 may include a step 822 of setting the parameters a, b and c. The parameters a, b and c may be set as an integer that is greater than or equals to 0, and is less than or equals to (n-1). The setting of parameters a, b and c may be referred to as color detection threshold setting. The process 820 may include a step 824 of checking whether at least one pixel in the block is within a predetermined color range for each block in the image 90 by checking whether any of R(n-1 ) to Ra, G(n-1 ) to Gb or B (n-1) to Be of any pixel in the block is 1 for each block in the image 90, wherein n is the number of bits of the red, green and blue color data of any pixel, and the parameters a, b, c are set in step 822. The block may be classified as the passing block when any of R(n-l) to Ra, G(n-l) to Ga, or B(n-l) to Ba of any pixel in the block is 1 (step 826). The block may be classified as the missing block when no pixel in the block has any of R(n-l) to Ra, G(n-l) to Ga or B(n-l) to Ba that is 1 (step 828). For example, when the parameters a, b and c are set as 4, and the number of bits of the red, green and blue color data of any pixel is 8, checking whether at least one pixel in the block is within a predetermined color range may be completed by checking whether any of R7 to R4, G7 to G4 or B7 to B4 of any pixel in the block is 1 (i.e., checking whether an intensity of any of a red, green and blue color of any pixel ranges from 16 to 255). The block may be classified as the passing block when any of R7 to R4, G7 to G4 or B7 to B4 of any pixel in the block is 1 (i.e., at least one pixel in that block having an intensity of any of a red, green and blue color ranging from 16 to 255). The block may be classified as the missing block when no pixel in the block has any of R7 to R4, G7 to G4 or B7 to B4 that is 1 (i.e., no pixel in that block having an intensity of any of a red, green and blue color ranging from 16 to 255).

[0100] Referring back to FIG. 7, each block 70 may be elassfied as one of a missing block 71 and a passing block 72 using the process 820 described above. In FIG. 7, the block classified as the missing block 71 is not labelled with a white dot and the block classified as the passing block 72 is labelled with a white dot. Each column of the m columns may be classified as one of a selected column, an unselected column and an interpolated column.

[0101] For example, column Cb may be classified as the selected column as there is no missing block in the column (i.e., all the blocks in the column Cb are passing blocks). The pixels of the passing blocks of the column Cb may be consisted in the selected subset of pixels used to generate the control signal.

[0102] Column Cb+1 and column Cb+2 may be classified as the interpolated column as a number of missing blocks in each of the column Cb+1 and column Cb+2 is greater than 0 and less than a preset threshold (e.g., 9). An adjacent block that is adjacent to a respective missing block in these columns may be used as an interpolating block for the respective missing block when the adjacent block that is adjacent to the respective missing block has an average intensity of any of the red, green and blue color of all pixels in the adjacent block that is greater than the interpolating threshold (e.g., Oj.The pixels of the interpolating blocks may be consisted in the selected subset of pixels used to generate the control signal. For example, an adjacent block (the block in the seventh row and in the column Cb) that is adjacent to a respective missing block (the block in the seventh row and in the column Cb+1) may be used as an interpolating block for the respective missing block in the column Cb+1. An adjacent block (the block in the sixth, eighth, ninth, tenth, eleventh row and in the column Cb+1, respectively) that is adjacent to a respective missing block (the blocks in the sixth, eighth, ninth, tenth, eleventh row and in the column Cb+2, respectively) may be used as an interpolating block for the respective missing block in the column Cb+2. Pixels of an interpolated column (e.g., column Cb+2) may include pixels of interpolating blocks of adjacent columns (e.g., Column Cb+1 ). The pixels of a respective missing block of the at least one missing block in these columns may be consisted in the selected subset of pixels used to generate the control signal, when each of adjacent blocks that are adjacent to the respective missing block has an average intensity of any of the red, green and blue color of all pixels in that adjacent block that is less than or equals to an interpolating threshold (e.g., 0). For example, the pixels of a respective missing block of the at least one missing block in the column Cb+2 (the block in the seventh row and in the column Cb+2) may be consisted in the selected subset of pixels used to generate the control signal, as each of adjacent blocks that are adjacent to the respective missing block has anaverage intensity of any of the red, green and blue color of all pixels in that adjacent block that is less than or equals to an interpolating threshold.

[0103] Column Cb+3 and column Cb+4 may be classified as the unselected column as a number of missing blocks in the column equals to or is greater than the preset threshold (e.g., 9). No pixels in the blocks of these columns may be consisted in the selected subset of pixels used to generate the control signal.

[0104] Column Ca, column Ca+1 , column Ca+2 and column Ca+3 may be classified as the interpolated column as a number of missing blocks in each of the these columns is greater than 0 and less than a preset threshold (e.g., 9). An adjacent block that is adjacent to a respective missing block in these columns may be used as an interpolating block for the respective missing block when the adjacent block that is adjacent to the respective missing block has an average intensity of any of the red, green and blue color of all pixels in the adjacent block that is greater than the interpolating threshold (e.g., 0). The pixels of the interpolating blocks may be consisted in the selected subset of pixels used to generate the control signal. For example, an adjacent block (the blocks in the first and second row and in the column Cb, respectively) each having an average intensity of any of the red, green and blue color of all pixels in the adjacent block that is greater than the interpolating threshold and adjacent to a respective missing block in the column Ca (the blocks in the first and second row and in the column Ca, respectively) may be used as an interpolating block for the respective missing block in the column Ca.. An adjacent block (the block in the third, fourth, fifth, sixth, seventh and ninth row and in the column Ca+2, respectively) having an average intensity of any of the red, green and blue color of all pixels in the adjacent block that is greater than the interpolating threshold and adjacent to a respective missing block in the column Ca+3 (the block in the third, fourth, fifth, sixth, seventh and ninth row and in the column Ca+3, respectively) may be used as an interpolating block for the respective missing block in the column Ca+3. The pixels of the interpolating blocks may be consisted in the selected subset of pixels used to generate the control signal. The pixels of a respective missing block of the at least one missing block in these columns may be consisted in the selected subset of pixels used to generate the control signal, when each of adjacent blocks that arc adjacent to the respective missing block has an average intensity of any of the red, green and blue color of all pixels in that adjacent block that is less than or equals to an interpolating threshold (e.g., 0). For example, the pixels of a respective missing block of the at least one missing block in columns Ca+1, Ca+2 and Ca+3 (the blocks in the first and second row and in the column Ca+1, Ca+2 and Ca+3, respectively) may be consisted in the selected subset of pixels used to generate the control signal, as each of adjacent blocks that arcadjacent to the respective missing block has an average intensity of any of the red, green and blue color of all pixels in that adjacent block that is less than or equals to an interpolating threshold.

[0105] Column Ca+4 may be classified as the unselected column as a number of missing blocks in the column Ca+4 equals to or is greater than the preset threshold (e.g., 9). No pixels in the blocks of the column Ca+4 may be consisted in the selected subset of pixels used to generate the control signal.

[0106] FIG. 9A shows an example of an image 90 in a time variable scries of images included in a media data according to some embodiments of the present disclosure. FIG. 9B shows an example of a lighting effect associated with a plurality of lighting devices (e.g., a light wall) controlled by the control signal generated based on the color data of the selected subset of pixels of the image 90 of FIG. 9A.

[0107] The selected subset of pixels in an image may be divided into a plurality of groups, each group of the selected subset of pixels may be used to generate a control signal to control a group of lighting devices of a plurality of groups of lighting devices. For example, the selected subset of pixels may be divided into 2 groups, including a first group in a left part of the image 90 and a second group in a right part of the image 90. The first group of the selected subset of pixels may be used to generate a control signal to control a first group of lighting devices (e.g., a light bar disposed in the left of a display device). The second group of the selected subset of pixels may be used to generate a control signal to control a second group of lighting devices (e.g., a light bar disposed in the right of the display device).

[0108] FIGs. 10A to 10C shows an example of controlling a lighting effect associated with two groups of lighting device (i.e., left light bar and right light bar) using a control signal generated by the color data of two groups of selected subset of pixels respectively. FIG. 10A shows an example of dividing a selected subset of pixels of an image 90 into two groups, including a first group in a left part of the image 90 (labelled as Division 1) and a second group in a right part of the image 90 (labelled as Division 2). FIG. 10B shows an example of a lighting effect associated with the left light bar controlled by a control signal generated based on the color data of the first group of selected subset of pixels. FIG. 10C shows an example of a lighting effect associated with the right light bar controlled by a control signal generated based on the color data of the second group of selected subset of pixels.

[0109] According to various non-limiting embodiments, the processor 120 / 220 / 320 may be configured to predict a scene data for an image of the time variable series of images based on the color data of selected subset of pixels of the image by a scene prediction model. Thecontrol signal used to control the lighting effect associated with the plurality of lighting devices may be generated / obtained based on the predicted scene data.

[0110] The scene data may be classified into various categories, such as fire, balloon, firework, sky, lamp, flower and plant, etc. Examples of the scene data may include campfire, fireweed, firefox, soul fire, and fire and flames in the category of fire, hot air balloon ride, balloon boy and savannah balloons in the category of balloon, sky blue, deep sky color, tropical sky and sunset sky in the category of sky, lamp, lava lamp, neon lamp, salt lamp and floor lamp in the category of lamp, cactus flower, pink flower, sunflower, pink flower and may flower in the category of flower, seaside plant, holly plant, cannibal plants, coffee plant and neon plant in the category of plant.

[0111] Each scene data may correspond to a color palette consisting of a plurality of different color codes (e.g., five different color codes). A color code may be represented in Hex and / or RGB value ranging from 0 to 255.

[0112] Hexadecimal (Hex) and RGB (Red, Green, Blue) are two different ways to represent colors. Hex code represents colors using a six-digit combination of numbers and letters, prefixed with a hash (#) symbol. Each pair of digits represents the intensity of the red, green, and blue color channels, respectively. For example, #FF0000 represents the color red with full intensity in the red channel and no intensity in the green and blue channels. RGB values represent colors using three numbers that range from 0 to 255, indicating the intensity of the red, green and blue color channels. For example, RGB (255, 0, 0) represents the color red with full intensity in the red channel and no intensity in the green and blue channels.

[0113] FIG. 11 A shows an example of a color palette that corresponds to the scene data of campfire (or referred to as campfire color palette). FIG. 1 IB shows an example of the color codes consisted in the campfire color palette.

[0114] The scene prediction model may be trained based on a plurality of training scene data and a color data of selected subset of pixels of the plurality of training images, each training image of the plurality of training images corresponds to a training scene data. Each training image may be labelled to correspond to a training scene data. In the selected subset of pixels of the training image, a number of continuous horizontal and vertical pixels that have a color data that are within a predetermined tolerance (e.g., + / -1 % tolerance) of the color codes consisted in the color palette that corresponds to a training scene data may be determined. The training image may be labelled to correspond to the training scene data when the number of continuous horizontal and vertical pixels exceeds a preset threshold.

[0115] The trained scene prediction model may be stored in the memory 140 / 240 / 340. The scene prediction model may be trained by the Al chip 258 / 358.

[0116] According to various non-limiting embodiments, the processor 120 / 220 / 320 may be configured to divide each image of at least three sequential images of the series of images into m columns and n rows such that each image has m*n blocks. Each block in the m*n blocks in the image may have a color value representing an average intensity of the red, green and blue color of all pixels in the block. A block motion may be determined when each image of the at least three sequential images contains a respective target block having a same or similar color value.

[0117] The processor 120 / 220 / 320 may be configured to predict a position of a target block having the same or similar color value in a subsequent image following the at least three sequential images based on positions of the respective target blocks in the at least three sequential images. The color value of a respective target block in the at least sequential images and a subsequent image following the at least three sequential images may be regarded as the same or similar when a difference between the color values of the target blocks in the at least sequential images and the subsequent image is less than or equals to a similarity threshold.

[0118] FIGs.12A to 12D show an example of determining a block motion when each image of at least three sequential images (four images 93, 94, 95, 96) contains a respective target block (block 73, 74, 75, 76, respectively) having a same or similar color value according to an embodiment of the present disclosure. FIG. 12A shows that a first image (image 93) of the at least three sequential images contains a target block (block 73), FIG. 12B shows that a second image (image 94) of the at least three sequential images contains a target block (block 74), FIG. 12C shows that a third image (image 95) of the at least three sequential images contains a target block (block 75), FIG. 12D shows that a fourth image (image 96) of the at least three sequential images contains a target block (block 76), and the target blocks (blocks 73, 74, 75, 76) have a same or similar color value. The target block (blocks 73, 74, 75, 76) are labelled by “A” in FIGs 12A to 12D, respectively. The color value of the respective target block in each of the at least three sequential images may be regarded as the same or similar when a difference between the color value of the respective target block (blocks 73, 74, 75, 76) in each of the at least three sequential images (images 93, 94, 95, 96) is less than or equals to a similarity threshold. As each of the at least three sequential images (images 93, 94, 95, 96) contains a respective target block (blocks 73, 74, 75, 76 respectively) having a same or similar color value, a block motion may be determined. As shown in FIG. 12G, the respective target block (e.g., block 73) may include a plurlaity of pixels. The color value of the respective target block (c.g., block 73) mayrepresent an average intensity of the red, green and blue color of all pixels in the respective target block (e.g., block 73).

[0119] FIGs. 12E to 12F respectively show an example of predicting a position of a target block having a same or similar color value in a subsequent image following the at least three sequential images, according to an embodiment of the present disclosure. With reference to FIGs. 12E and 12F, a position of a target block (e.g., blocks 77, 78) having a same or similar color value in a subsequent image (e.g., images 97, 98 respectively) following the at least three sequential images (images 93, 94, 95, 96) may be predicted based on positions of the respective target blocks (blocks 73, 74, 75, 76) in the at least three sequential images (images 93, 94, 95, 96). The target block (blocks 77, 78) are labelled by A’ in FIGs 12E to 12F, respectively.

[0120] FIG. 12G shows a position of the respective target block (blocks 73, 74, 75, 76) in the at least three sequential images (images 93, 94, 95, 96) and a predicted position of the respective target block (e.g., blocks 77, 78) in a subsequent image (e.g., images 97, 98 respectively) following the at least three sequential images (images 93, 94, 95, 96) in a demonstrating image 99.

[0121] With reference to FIG. 12G, the determined block motion (i.e., changing of a position of the respective target block (blocks 73, 74, 75, 76)) is indicated by the arrows in solid line (i.e., arrows 41 , 42, 43). The determined block motion (i.e., changing of aposition of the respective target block (blocks 73, 74, 75, 76)) may be predicted to continue as indicated by the arrows in dashed line (i.e., arrows 44, 45) such that a position of the respective target block (e.g., blocks 77, 78) in a subsequent image (e.g., images 97, 98 respectively) may be predicted based on positions of the respective target blocks (blocks 73, 74, 75, 76) in the at least three sequential images (images 93, 94, 95, 96).

[0122] The processor 120 / 220 / 320 may be configured to generate a predicted control signal based on a color data of a predicted subset of pixels of the subsequent image following the at least three sequential images to control the lighting effect associated with each lighting device of the plurality of lighting devices. The color data of the predicted subset of pixels of the subsequent image following the at least three sequential images may be obtained based on the predicted position of the target block in the subsequent image, and the position of the target block and the color data of the selected subset of pixels of the at least three sequential images.

[0123] According to various non-limiting embodiments, the processor 120 / 220 / 320 may be configured to divide each image of at least three sequential images of the series of images into m columns and n rows such that each image has m*n blocks. Each block in the m*n blocks in the image may have a color value representing an average intensity of the red, green andblue color of all pixels in the block. An object motion may be determined when each image of the at least three sequential images contains respective target blocks having same or similar color values. An object motion may be determined when each image of the at least three sequential images contains a respective target object and the corresponding target blocks in the respective target object have a same or similar color value.

[0124] FIGs.l3A to 13C show an example of determining an object motion when each image of at least three sequential images (images 53, 54, 55) contains respective target blocks having a same or similar color value according to an embodiment of the present disclosure. With reference to FIGs. 13A to 13C, each of the at least three sequential images (images 53, 54, 55) contains a respective target object (objects 63, 64, 65, respectively). The target object (objects 63, 64, 65) are labelled by “A” in FTGs 13A to 13C, respectively. Each target object has a plurality of blocks. For example, the target object 63 has a plurality of blocks (blocks 73a, 73b, 73c), the target object 64 has a plurality of blocks (blocks 74a, 74b, 74c), the target object 65 has a plurality of blocks (blocks 75a, 75b, 75c). The corresponding target blocks in the respective target object (objects 63, 64, 65 respectively) have a same or similar color value. In other words, corresponding target blocks 73a, 74a, 75a have a same or similar color value, corresponding target blocks 73b, 74b, 75b have a same or similar color value, corresponding target blocks 73c, 74c, 75c have a same or similar color value. Accordingly, an object motion may be determined when each image of the at least three sequential images (images 53, 54, 55) contains a respective target object (objects 63, 64, 65) and the corresponding target blocks in the respective target object (objects 63, 64, 65 respectively) have a same or similar- color value.

[0125] The determined object motion (i.c., changing of a position of the respective target objects (objects 63, 64, 65) is indicated by the arrows in solid line (i.e., arrows 46, 47) in FIG. 13A and FIG. 13B.

[0126] The processor 120 / 220 / 320 may be configured to predict positions of target blocks having the same or similar color values in a subsequent image following the at least three sequential images based on positions of the respective target blocks in the at least three sequential images. The color value of two target blocks may be regarded as the same or similar when a difference between the color value of the two target blocks is less than or equals to a similarity threshold. The processor 120 / 220 / 320 may be configured to predict a position of a target object in a subsequent image following the at least three sequential images based on positions of the respective target object in the at least three sequential images, the corresponding target blocks in the respective target object in the subsequent image and the at least three sequential images having a same or similar color value.

[0127] FIGs.l3D to 13E respectively show an example of predicting a position of a target object in a subsequent image following the at least three sequential images according to an embodiment of the present disclosure. With reference to FIGs. 13D to 13E, a position of a target object (e.g., objects 66, 67) in a subsequent image (e.g., images 56, 57 respectively) after the at least three sequential images (images 53, 54, 55) may be predicted based on positions of the respective target object (objects 63, 64, 65) in the at least three sequential images (images 53, 54, 55). The target objects (objects 66, 67) are labelled by A’ in FIGs 13D to 13E, respectively. The target object (e.g., objects 66, 67) in a subsequent image (e.g., images 56, 57 respectively) may include a plurality of target blocks. The corresponding target blocks in the respective target object (objects 63, 64, 65, 66, 67) in the at least three sequential images (images 53, 54, 55) and the subsequent images (images 56, 57) may have a same or similar color value.

[0128] The determined object motion (i.e., changing of a position of the respective target object (objects 63, 64, 65)) may be predicted to continue as indicated by the arrows in dashed line (i.e., arrows 48, 49) such that a position of the respective target object (e.g., objects 66, 67) in a subsequent image (e.g., images 56, 57 respectively) may be predicted based on positions of the respective target objects (objects 63, 64, 65) in the at least three sequential images (images 53, 54, 55).

[0129] The processor 120 / 220 / 320 may be configured to generate a predicted control signal based on a color data of a predicted subset of pixels of the subsequent image following the at least three sequential images to control the lighting effect associated with each lighting device of the plurality of lighting devices. The color data of the predicted subset of pixels of the subsequent image following the at least three sequential images may be obtained based on the predicted position of the target object in the subsequent image, and the position of the target object and the color data of the selected subset of pixels of the at least three sequential images.

[0130] The present disclosure provides a smart lighting system. FIG. 14 shows a smart lighting system 600 according to some embodiments of the present disclosure. The smart lighting system 600 may include a control system 100 / 200 / 300 as described above and a plurality of lighting devices 190 (including 190a, 190b,..., collectively 190) connected to the control system 100 / 200 / 300.

[0131] The plurality of lighting devices 190 may be housed in the smart lighting system 600. The lighting devices 190 may be configured to emit light. In some embodiments, each lighting device 190 may be configured to selectively emit light in various brightness and colours, for example, a color of red, green, blue, or any combinations thereof based on thecontrol signal. Each lighting device 190 may include at least a red, a green and a blue light emitting device. Each lighting device 190 may include an RGB LED (red-green-blue light emitting diode). In some embodiments, the light emitted by the lighting devices 190 may be matched with the media data being displayed on the display device 10. Accordingly, the smart lighting system 600 may provide the user with refined and enhanced user experiences by producing a variety of lighting colours and brightness over time matched with the media data being displayed on the display device 10.

[0132] In some embodiments, the lighting device 190 may be mounted on the display device 10, e.g., mounted at side parts of the display device 10. In some embodiments, the lighting devices 190 may be mounted on a computer peripheral device 30 such as a keyboard, a mouse, a touchpad, etc.

[0133] The present disclosure provides a method 900 of controlling a lighting effect associated with a plurality of lighting devices 190 connected to a control system 100 / 200 / 300 described above by the control system 100 / 200 / 300, the control system 100 / 200 / 300 including a data receiver 110 / 210 / 310, a processor 120 / 220 / 320 and a transmitter 130 / 230 / 330. FIG. 15 is a flow chart depicting a method 900 of controlling a lighting effect associated with a plurality of lighting devices 190 connected to a control system 100 / 200 / 300 according to some embodiments of the present disclosure. The method 900 may include: i) a step 910 of receiving, by the data receiver, a media data comprising a time variable series of images, each image of the series of images comprising a plurality of pixels, each pixel having a color data; ii) a step 920 of selecting, by the processor, a subset of pixels from the plurality of pixels of a respective image of the series of images; iii) a step 930 of generating, by the processor, a control signal based on the color data of the selected subset of pixels of the respective image; and iv) a step 940 of sending, by the transmitter, the control signal to the plurality of lighting devices to control the lighting effect associated with the plurality of lighting devices.

[0134] The method 900 may include a step of controlling a color effect of each lighting device of the plurality of lighting devices using the control signal so as to generate the lighting effect.

[0135] While the disclosure has been particularly shown and described with reference to specific embodiments, it should be understood by those skilled in the ail that various changes in form and detail may be made therein without departing from the spirit and scope of thedisclosure as defined by the appended claims. The scope of the disclosure is thus indicated by the appended claims and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced.

Claims

CLAIMS1.A control system for controlling a lighting effect associated with a plurality of lighting devices connected to the control system, the control system comprising: a data receiver configured to receive a media data comprising a time variable series of images, each image of the series of images comprising a plurality of pixels, each pixel having a color data; a processor configured to: select a subset of pixels from the plurality of pixels of a respective image of the series of images; generate a control signal based on the color data of the selected subset of pixels of the respective image; and a transmitter configured to send the control signal to the plurality of lighting devices to control the lighting effect associated with the plurality of lighting devices.

2. The control system of claim 1, wherein the control signal is used to control a color effect of each lighting device of the plurality of lighting devices so as to generate the lighting effect.

3. The control system of claim 1, wherein the subset of pixels from the plurality of pixels of a respective image of the series of images is selected via an interlace scan by fractionating the plurality of pixels of a respective image of the series of images by an interlace factor.

4. The control system of claim 1, wherein the processor is configured to: - divide each image into m columns and n rows such that each image has m*n blocks, wherein m and n are both an integer greater than or equal to 2; classify each block of the urn blocks as one of a missing block and a passing block, wherein a block of the m*n blocks is classified as the missing block when no pixel in that block is within a predetermined color range; wherein a block of the m*n blocks is classified as the passing block when at least one pixel in that block is within the predetermined color range.5.The control system of claim 4, wherein the processor is configured to: - classify each column of the m columns as one of a selected column, an unsclcctcd column and an interpolated column,wherein the column is classified as the selected column when there is no missing block in the column; wherein the column is classified as the unselected column when a number of missing blocks in the column equals to or is greater than a preset threshold; wherein the column is classified as the interpolated column when a number of missing blocks in the column is greater than 0 and less than the preset threshold, wherein the selected subset of pixels consists of: -(i)pixcls of the passing blocks of the selected column;(ii)pixels of the passing blocks of the interpolated column;(iii) pixels of a respective missing block of the at least one missing blocks in the interpolated column, when each of adjacent blocks that are adjacent to the respective missing block has an average intensity of any of the red, green and blue color of all pixels in that adjacent block that is less than or equals to an interpolating threshold;(iv)pixcls of an interpolating block for a respective missing block of the at least one missing blocks in the interpolated column, wherein the interpolating block is an adjacent block that is adjacent to the respective missing block, when the adjacent block that is adjacent to the respective missing block has an average intensity of any of the red, green and blue color of all pixels in the adjacent block that is greater than the interpolating threshold.

6. The control system of claim 1, wherein the processor is further configured to: - predict a scene data for each image based on the color data of the selected subset of pixels of the image by a scene prediction model; and obtain the control signal for the plurality of lighting devices based on the predicted scene data.

7. The control system of claim 6, wherein the scene prediction model is trained based on a plurality of training scene data and a color data of a selected subset of pixels of aplurality of training images, each training scene data corresponds to a training image of the plurality of training images.

8. The control system of claim 1, wherein the processor is further configured to: - divide each image of at least three sequential images of the series of images into m columns and n rows such that each image has m*n blocks, each block having a color value representing an average intensity of the red, green and blue color of all pixels in the block; determine a block motion when each image of the at least three sequential images contains a respective target block having a same or similar color value or determine an object motion when each image of the at least three sequential images contains respective target blocks having same or similar color values.

9. The control system of claim 8, wherein the color value is the same or similar when a difference between the color value of the respective target block in each image of the at least three sequential images is less than or equals to a similarity threshold.

10. The control system of claim 8, wherein the processor is further configured to: - predict a position of a target block having the same or similar color value in a subsequent image following the at least three sequential images based on positions of the respective target blocks in the at least three sequential images.1 l.The control system of claim 10, wherein the processor is further configured to:- gcncratc a predicted control signal based on a color data of a predicted subset of pixels of the subsequent image following the at least three sequential images to control the lighting effect associated with each lighting device of the plurality of lighting devices, wherein the color data of the predicted subset of pixels of the subsequent image following the at least three sequential images is obtained based on the predicted position of the target block in the subsequent image, and the position of the target block and the color data of the selected subset of pixels of the at least three sequential images.

12. The control system of claim 1, further comprising: a data converter configured to convert a format of the media data such that the converted format of the media data is compatible with the processor of the control system.13.Thc control system of claim 1, wherein the processor is configured to reduce a frame rate of the media data to a predetermined frame rate value.

14. The control system of claim 1 , wherein the media data is transmitted to a display device to display the series of images over time.

15. The control system of claim 14, wherein a refresh rate of the display device is synchronized with a frame rate of the media data.

16. The control system of claim 1, wherein the transmitter is in wireless communication with the plurality of lighting devices.17.A smart lighting system comprising: the control system of claim 1; and the plurality of lighting devices connected to the control system.

18. The smart lighting system of claim 17, wherein the plurality of lighting devices each comprises at least a red, a green and a blue light emitting device.19.A method of controlling a lighting effect associated with a plurality of lighting devices connected to a control system by the control system, the control system comprising a data receiver, a processor and a transmitter, the method comprising: receiving, by the data receiver, a media data comprising a time variable series of images, each image of the series of images comprising a plurality of pixels, each pixel having a color data; selecting, by the processor, a subset of pixels from the plurality of pixels of a respective image of the series of images; generating, by the processor, a control signal based on the color data of the selected subset of pixels of the respective image; and sending, by the transmitter, the control signal to the plurality of lighting devices to control the lighting effect associated with the plurality of lighting devices.2O.Thc method of claim 19, further comprising controlling a color effect of each lighting device of the plurality of lighting devices using the control signal so as to generate the lighting effect.

Images