Determining the luminance distribution

Abstract

A method and computer device are provided for determining a spatial luminance distribution over at least one image of a scene illuminated by at least one illuminant (121). The image includes RGB channels. A linear combination of the RGB channels is applied to determine the luminance distribution. The linear combination has a set of coefficients including a respective coefficient for each of the RGB channels. The coefficients are determined by identifying a spectral power distribution (SPD) of the at least one illuminant and performing a search to determine values ​​for the set of coefficients that minimize a spectral mismatch between a) the identified SPD weighted by the linear combination and b) the identified SPD weighted by a luminosity function.

Description

[Technical field]

[0001] The present disclosure relates to determining luminance distribution. [Background technology]

[0002] The overall lighting present in an environment can be heavily influenced by a controllable lighting system that includes one or more illuminants. There may also be other light sources in the environment, such as natural light coming from a window. In general, the overall lighting can include controllable (part of the lighting system) and non-controllable (not part of the lighting system) components. Summary of the Invention [Problem to be solved by the invention]

[0003] To achieve high quality lighting, the controller of the lighting system should consider the lighting quality aspects in the environment, such as lighting distribution, directionality, dynamics, glare, melanopic equivalent daylight illuminance, and spectral power density in the visual and non-visual parts of the spectrum (from UV to IR). To do so, a useful first step is to measure the luminance distribution in the environment, because the luminance distribution contains many important information about these lighting quality aspects and can provide sufficient information to achieve high quality lighting. Therefore, a luminance distribution measuring device is a useful device for a lighting quality control system.

[0004] The luminance distribution measuring device may also have other applications, such as a stand-alone measuring device, for example to ensure that the lighting in a workspace plane meets certain standards.

[0005] CN 108509887 A relates to the technical field of display, in particular to a method and device for obtaining ambient illumination information, and displays electronic equipment. The disclosed method includes the steps of: extracting a color image of a reference object that is actually taken from an image including the reference object; generating an observation point matrix by sampling according to the color image of the reference object; solving the light source linear function coefficient according to a set of linear equations formed by the light source linear function coefficient, the transfer matrix and the observation point matrix; and obtaining ambient illumination information according to the light source linear function coefficient. Through this method, the illumination information of the surrounding environment can be calculated accurately and efficiently, which is a prerequisite for realistically rendering a virtual object in a real environment under real illumination conditions. [Means for solving the problem]

[0006] According to a first aspect disclosed herein, there is provided a method for determining a spatial luminance distribution over at least one image of a scene illuminated by at least one illuminant, the image including RGB channels, the method comprising: forming a combination of the RGB channels to determine the luminance distribution, the combination having a set of coefficients including a respective coefficient for each of the RGB channels, the coefficients being determined by identifying a spectral power distribution (SPD) of the at least one illuminant, and performing a search to determine values ​​of the set of coefficients that minimize a spectral mismatch between a) the identified SPD weighted by the combination and b) the identified SPD weighted by a luminosity function.

[0007] In one example, the at least one image includes a set of low dynamic range (LDR) images, and the method includes constructing a high dynamic range (HDR) image from the set of LDR images, and the combination is applied to the constructed HDR image.

[0008] In one example, identifying the spectral power distribution includes determining a gamut of at least one image and comparing the determined gamut to a set of predetermined gamuts for known spectral power distributions.

[0009] In one example, the gamut is a red-blue gamut.

[0010] In one example, identifying the spectral power distribution includes receiving a predetermined indication of the spectral power distribution.

[0011] In one example, the combination is a linear combination.

[0012] In one example, the method includes determining at least one Light Performance Indicator (LPI) from the luminance distribution, the LPI being a combined metric derived from the area of ​​the luminance distribution.

[0013] In one example, at least one of the LPIs is a luminance LPI that indicates luminance values ​​in an area of ​​the luminance distribution.

[0014] In one example, at least one of the LPIs is a contrast LPI, which indicates the difference in luminance over an area of ​​the luminance distribution.

[0015] In one example, the method includes identifying an orientation of a user, and at least one of the LPIs is determined for an area of ​​the luminance distribution that corresponds to an area located in front of the user.

[0016] In one example, at least one of the LPIs is a glare LPI indicating an amount of glare experienced by a user, and the method includes determining a glare value from the luminance distribution by determining a background luminance and a luminance of a glare source, estimating a displacement of the glare source from a line of sight of the user, estimating a solid angle subtended by the glare source from the user, and determining an amount of glare based on the background luminance, the luminance of the glare source, the estimated displacement of the glare source from a line of sight of the user, and the estimated solid angle subtended by the glare source from the user.

[0017] In one example, at least one of the LPIs is a non-visual LPI that indicates an expected non-visual effect on a user.

[0018] In one example, the method includes identifying a number of task areas within a luminance distribution and determining an LPI for each respective task area.

[0019] In one example, the method includes controlling at least one illuminant based on the determined at least one LPI.

[0020] There is also provided a computing device comprising computer executable code embodied in a computer readable storage medium and configured to, when executed by one or more processors, perform the method of the first aspect or any example thereof.

[0021] Also provided is a system that includes a computing device and a camera.

[0022] Also described is a camera-based sensor device for use in a controllable lighting system, the camera-based sensor including a communications interface, a camera for capturing images of a scene, each image including an array of pixels, the processor configured to determine at least one Light Performance Indicator (LPI) from images captured by the camera, the LPI being a composite metric derived from a plurality of pixels of the array of pixels of the image, and to transmit via the communications interface the determined at least one LPI to the controllable lighting system for use by the controllable lighting system making control decisions based on the LPI, wherein the processor does not transmit any images of the scene.

[0023] In one example, at least one of the LPIs is a brightness LPI that indicates brightness values ​​at multiple pixels in an image captured by a camera.

[0024] In one example, at least one of the LPIs is a contrast LPI that indicates the difference in brightness among multiple pixels in an image captured by a camera.

[0025] In one example, at least one of the LPIs is a colour LPI that indicates colour values ​​at a number of pixels in an image captured by a camera.

[0026] In one example, the color may be a color temperature.

[0027] In one example, at least one of the LPIs indicates a combination of brightness or luminance and color or color temperature. In a particular example, the LPI may indicate the position of the combination relative to a Kruithof curve.

[0028] In one example, at least one of the LPIs is a color contrast LPI that indicates color differences among multiple pixels in an image captured by a camera.

[0029] In one example, at least one of the LPIs is a non-visual LPI that indicates expected non-visual effects to a user present in the scene, such as melanopic radiance, s-cone-opic radiance, m-cone-opic radiance, l-cone-opic radiance, rhodopic radiance, etc.

[0030] In one example, the processor is configured to determine a location and orientation of a user present within the scene, and at least one of the LPIs is a glare LPI indicating an amount of glare experienced by the user.

[0031] In one example, the processor is configured to determine a luminance distribution from an array of pixels, and at least one of the LPIs is a luminance LPI indicative of luminance values ​​at a plurality of pixels in an image captured by the camera.

[0032] In one example, a scene includes multiple task areas and an LPI is determined for each respective task area.

[0033] In one example, the processor is configured to determine an LPI for each of a number of users present in the scene.

[0034] In one example, the processor is configured to determine the same type of LPI for each of a number of hypothesized user locations within the scene, and to generate an average LPI from the multiple LPIs.

[0035] In one example, the processor is configured to determine a current activity performed by a user present in the scene, and the at least one LPI is dependent on the determined current activity.

[0036] Also described is a method of controlling a controllable lighting system, the method comprising: capturing an image of a scene using a camera, the image including an array of pixels; determining at least one Light Performance Indicator (LPI) from the image captured by the camera, the LPI being a composite metric derived from a plurality of pixels of the array of pixels of the image; and transmitting the determined at least one LPI to the controllable lighting system for use by the controllable lighting system making control decisions based on the LPI, where the image from the camera is not transmitted.

[0037] In one example, the method includes receiving at least one LPI at a controller of a controllable lighting system, comparing the received at least one LPI with corresponding user preferences to determine settings for devices in the controllable lighting system, and controlling the devices in accordance with the determined settings.

[0038] In one example, the method includes determining user satisfaction with settings of devices in a controllable lighting system and modifying corresponding user preferences accordingly. [Brief description of the drawings]

[0039] To assist in understanding the present disclosure and to show how embodiments may be practiced, reference is made to the accompanying drawings, by way of example. [Figure 1] 1 illustrates a schematic diagram of an exemplary lighting system for illuminating an environment. [Diagram 2] 2 shows a schematic diagram of a camera unit of the illumination system in more detail; [Diagram 3] FIG. 2 illustrates a high level functionality of a lighting system according to examples described herein. [Figure 4] 4 illustrates generally a flow chart showing an exemplary method performed by a processor of the camera unit. [Diagram 5] 13 illustrates generally a flow chart showing another exemplary method performed by a processor of the camera unit. [Figure 6] 3 shows an exemplary luminance distribution of an environment determined from an image captured by a camera. [Figure 7] Indicates user preference data. [Figure 8] Indicates user preference data. [Figure 9] 1 illustrates an exemplary luminosity function. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0040] Currently, people spend more than 90% of their time indoors, making the indoor environment paramount for people's health and well-being. Healthy building design is therefore becoming an increasingly important issue for building owners, regulators and tenants. The trend of healthy building design is suggested to grow rapidly. Given the beneficial effects of daylight, artificial lighting schemes that reflect its natural rhythm and provide the light nutrition required to support health and well-being are key to creating context-aware healthy indoor environments. Light sensing and monitoring is therefore crucial.

[0041] A controllable lighting system allows lighting in an environment to be controlled in response to input from various sensors. It is recognized herein that camera-based sensors that capture images of an environment can lead to privacy or security concerns. This is particularly problematic because camera-based sensors can offer many advantages over other types of sensors, such as infrared motion detectors, due to the fact that they provide spatial information.

[0042] This disclosure describes devices and methods that enable the use of camera-based sensors while maintaining user privacy and data security. To achieve this, one or more "Light Performance Indicators" (LPIs) are derived in the camera unit (camera-based sensor device). The LPIs are derived from one or more images taken by the camera in the camera unit and contain the information necessary for the controller of the lighting system to make control decisions. Each LPI is a combined metric derived from multiple arrays of arrays of pixels of the images. In other words, the LPIs only contain a limited number of identifiers and do not contain trackable pictures of people or their activities. In this case, it is these LPIs, not the images, that are transmitted from the camera unit to the controller. That is, the images themselves do not leave the camera unit, providing enhanced privacy and security. In other words, two steps are performed. Step 1: Convert the measured light distribution into LPI, which is calculated inside the camera unit. Step 2: Use an optimization / cost function to calculate how different light settings can improve the LPI value. This can be done outside the camera unit.

[0043] Thus, the camera unit has an interface that does not exchange images, but exchanges LPI. In other words, the communication interface of the camera unit defines a privacy boundary where images are never exchanged. In one example, the camera unit is provided in the form of an integrated camera-based sensor device, where the camera and communication interface are integrated in the same housing (along with a processor and memory, described below). In these examples, the communication interface is configured to communicate LPI (and not images) to an external system, such as a lighting system controllable from the integrated camera-based sensor device.

[0044] Some examples of LPI may utilize information about the luminance distribution in an environment. Known devices that measure the luminance distribution in an environment are expensive, specialized devices. Even when using such devices, the individual steps to determine the luminance distribution from the raw measurements need to be done manually, which requires expert level skills. Both of these factors limit the uptake of luminance distribution measurement devices.

[0045] The present disclosure also describes devices and methods that allow for the determination of the luminance distribution from one or more images captured by a camera. This allows, in embodiments, practical accuracy to be maintained at low cost and requiring only standard components. Furthermore, the luminance distribution measurement can be fully automated. This allows for easy integration into lighting control systems.

[0046] 1 illustrates generally an exemplary lighting system 100 for illuminating an environment 110. The environment 110 may be, for example, a walled room. A user 111 is shown within the environment 110.

[0047] The lighting system 100 includes a controller 120, one or more illuminants 121, and a camera unit 200. The controller 120 is operatively coupled to each of the illuminants 121 and the camera unit 200 by respective wired or wireless connections. The controller 120 may also be connected to a network 123 as shown in FIG. 1. An example of a network is the Internet. The controller 120 may also be connected to a memory 124. The memory 124 is a computer storage device. The memory 124 may be directly coupled to the controller 120 as shown in FIG. 1 (i.e., a local memory) or may be a remote memory accessible via the network 123. For example, the memory 124 may be a server accessible by the controller 120 via the Internet.

[0048] Illuminants 121 are light sources (also called lighting fixtures) for generating light. Controller 120 is configured to send control commands to illuminants 121 to control the lighting in environment 110. As shown in FIG. 1, illuminants 121 are disposed in environment 110. That is, each illuminant 121 is configured to illuminate at least a portion of environment 110 by emitting visible light into environment 110. In this example, four illuminants 121 are shown that are ceiling mounted illuminants. However, it should be understood that more or fewer illuminants may be present in environment 110. It should also be understood that different types of illuminants may be present. Examples of other types of illuminants include floor lamps, desk lamps, spotlights, etc. Not all illuminants 121 need be of the same type.

[0049] The environment 110 may include one or more light sources that are not themselves part of the lighting system 100. An example of such a light source is a natural light source, e.g., a window 112, as shown in FIG. 1. The control device 120 may control the effect of these types of light sources using other types of controllable devices. One example is controllable shades or blinds for the window 112. The controller 120 may be configured to control the shades or blinds to cover or uncover the window to change the amount of natural light entering through the window 112.

[0050] The controller 120 is described herein as performing functions related to both identifying and correcting lighting issues in the environment 110. However, it should be understood that these steps may be performed by separate devices in the lighting system 100. In particular, the controller 120 may in some examples only identify a lighting issue (e.g., lighting is too bright) and pass responsibility for correcting it to a separate control device. One reason for doing so is that correcting a lighting issue may be desirable but may not be feasible due to power consumption constraints, etc. In this example, the controller 120 may be configured to determine a desired change to a lighting setting (e.g., increase brightness) without needing to be aware of non-lighting-based requirements (e.g., limited power consumption) that may prevent it from being enacted.

[0051] Note that "brightness" may be thought of simply as the magnitude of one or more RGB values ​​in an image. However, a better measure of "brightness" experienced by the user 111 is luminance. Below, a method is described for determining luminance values ​​(luminance distribution) from an image. Thus, instead of (more naive) luminance values, (more sophisticated) luminance values ​​may be used.

[0052] Environment 110 may include one or more objects 113. Figure 1 shows a chair placed within environment 110. The chair is an example of an object 113. Different objects respond differently to the light emitted by illuminant 121 by absorbing and reflecting different wavelengths to different degrees.

[0053] FIG. 2 shows diagrammatically the camera unit 200 of the illumination system 100 in more detail.

[0054] The camera unit 200 includes a camera 201, a processor 202, a communication interface 203, and an internal memory 204. The processor 202 is operatively coupled to each of the camera 201, the communication interface 203, and the internal memory 204.

[0055] The camera 201 is configured to capture an image of a scene in the environment 110. The term "scene" refers to the portion of the environment 110 that is captured in the image, i.e., the portion of the environment 110 within the field of view of the camera 201. It should be noted that the camera unit 200 may be located inside or outside the environment 110 itself. In either case, the camera 201 may be a wide-angle camera. The advantage of a wide-angle camera is that the resulting image represents a large area (larger scene) of the environment 110. The scene captured by the camera 201 may be substantially all of the environment 110. For example, the camera 201 may be a wide-angle camera mounted on a ceiling that has a 360-degree view of the environment 110. In this specification, the terms "scene" and "environment" are used interchangeably.

[0056] Camera 201 captures RGB images. An RGB image is represented in RGB color space by individual values ​​for each of the red R, green G, and blue B channels. That is, an image captured by camera 201 includes (e.g., floating point) RGB values ​​for each pixel, as known in the art. Each channel includes an array of scalar (grayscale) pixel values. For example, the red channel includes a grayscale image representing the response of the red sensor of camera 201 at each point in the image.

[0057] As described in more detail below, the processor 202 of the camera unit 200 is configured to receive and convert images from the camera 201 into one or more Light Performance Indicators (LPIs). The LPIs are transmitted to the controller 120 of the lighting system 100 instead of the images themselves. The LPIs contain information that is used by the controller 120 in making control decisions. In other words, the processor 202 is configured to "strip down" the information (the images) into a format that is still useful to the controller 120, but that does not have any privacy concerns associated with the images.

[0058] Each LPI is a measure of the human experience of the lighting conditions present in the scene, e.g., brightness, glare, contrast, color, etc. The LPI may be determined using a function that models the human experience of the lighting conditions, e.g., a function on the values ​​obtained from each of the RGB channels. The function may, for example, take a combination of RGB values ​​for each pixel, each parameterized by a respective parameter (e.g., each weighted by a respective coefficient). The processor 202 may perform a training or search process to adjust the parameters or coefficients to identify values ​​that best model the human experience, e.g., values ​​that minimize the spectral mismatch between the response of the camera system and the human eye.

[0059] Each LPI is essentially a measure or metric related to how humans experience lighting. Various examples are described herein, but it should be understood that this is not an exhaustive list. In particular, many of the examples are given in quantitative models. However, the inventors envision that quantitative models will be proposed and validated for many more examples in the future. In many examples, performance is expressed numerically, possibly as the probability that a human can tolerate a light level, the probability that a room user will intervene in the light settings, subjective interpretations are quantified, a mean satisfaction score, the productivity of a worker in solving a task for a particular light setting, reading speed in words per second, the degree to which people with medical adaptations and disabilities can still safely perform a task, the expected rate at which people will experience fatigue, etc. It may be possible to determine some of these factors in a well-conditioned laboratory setting, but this requires expert-level skill and careful control of the variables involved (e.g., setting the light to create specific test conditions). The techniques described herein address the specific challenge of determining or predicting LPI in a light distribution measured purely by a camera, which may involve other inputs such as user-specific inputs (position, orientation, etc.). An appropriate model allows many LPIs to be calculated numerically by an automated system.

[0060] FIG. 3 illustrates high-level functioning of a lighting system 100 according to examples described herein.

[0061] 3 shows a camera unit 200 including a camera 201, an image 210 captured by the camera 201, a luminance distribution 215, an exemplary LPI 230, and a memory 204. The memory 204 is shown to store user data 214 and environmental data 224. Examples of user data 214 include user position, user orientation, user gaze, etc. Examples of environmental data include task area, wall area, etc. Exemplary LPIs include overall brightness 230, glare 231, task area lighting 232, wall lighting 233, etc.

[0062] 3 also shows memory 124, controller 120, and environment 110. Memory 124 is shown to store user data 125, lighting system data 126, and user preference data 127. Examples of user data 125 include user activity. Examples of lighting system data 126 include illuminant positions and illuminant orientations. Environment 110 is shown to include illuminants 121, as well as other controllable devices 122. Examples of other controllable devices 122 include shades and blinds covering windows 112. Also shown is user input 114. Examples of user input include explicit and implicit user input. These are described in more detail below. Examples of ways in which a user may provide input to controller 120 include via a switch (e.g., mounted on a wall), via a smartphone or smartwatch, etc.

[0063] As mentioned above, the processor 202 is configured to determine one or more LPIs from images captured by at least the camera 201. Note that only a few exemplary LPIs are shown in Figure 3. A larger list of exemplary LPIs is provided later below.

[0064] In some examples, the processor 202 may consider additional factors in determining one or more LPIs. Figure 3 illustrates two broad categories of such additional factors: environmental data and user data.

[0065] Environmental data refers to information related to the environment 110. Examples of environmental data include the location and optionally orientation of an area of ​​interest within the environment 110. For example, a "task area" may be an area of ​​interest. A task area is an area where the user 111 or other users typically perform a task, e.g., a desk area. The lighting requirements of a task area typically differ from the lighting requirements of other areas in the environment 110. For example, the user 111 may want his desk (task area) to be illuminated more brightly than other parts of the environment 110.

[0066] User data refers to information related to a user, such as a user's physical attributes, etc. Examples of user data include user position data, user orientation data, user gaze direction, etc.

[0067] Other data may also be considered by the processor 202 in determining one or more LPIs. Examples include occupancy of the environment 110, facial recognition, eye tracking, etc.

[0068] The environmental data, user data, and other data may be a) predetermined and stored in a memory, such as memory 204, b) determined by external sensors and received by processor 202, c) determined by processor 202 from one or more images captured by camera 201, or d) a combination of one or more of these. Techniques for determining a user's position and / or orientation using one or more sensor devices, among others, are known in the art, for example using data captured by a user's computing device (such as a smartphone).

[0069] In cases where the processor 202 is unable to determine the position and / or orientation of the user 111, the processor 202 may still determine one or more LPIs for many "hypothetical" user positions and determine an average LPI for these "hypothetical" positions. In other words, the memory 204 may store an indication of one or more predefined user positions within the environment 110. The processor 202 may be configured to obtain one of these predefined user positions from the memory 204 and use the obtained position as described herein. In other examples, the processor 202 may be configured to obtain multiple predefined user positions from the memory 204. In such cases, the processor 202 may use each obtained position to determine one or more LPIs and average the resulting multiple LPIs to determine a single output LPI to provide to the controller 120.

[0070] The predefined user positions may be stored in memory 204 during the commissioning process. For example, the commissioner may determine a user position to be used in those cases where processor 202 is unable to determine a current (actual, real) user position. The predefined user positions may correspond to locations within environment 110 that are more likely to be occupied by a user. One example of such a location is a desk. Thus, in one example, the predefined user positions correspond to the location of a desk within environment 110.

[0071] The memory 204 may also be configured to have one or more predefined user orientations, similar to those described above in connection with predefined user positions, in which case the processor 202 may similarly use one or more predefined user orientations if it is unable to determine the current (actual, real) user orientation.

[0072] Once the processor 202 has determined one or more LPIs, it transmits the LPIs to the controller 120. The controller 120 can adjust the illumination provided by one or more illuminants 121 accordingly to improve the value of the LPIs. For example, the controller 120 can determine from the LPIs that a task area in which the user 111 is working is not sufficiently illuminated. In this case, the controller 120 can control one or more illuminants 121 that emit light to that task area to increase light output.

[0073] To do so, controller 120 may compare the received LPI with corresponding user preferences, which may be stored in memory 124. If the LPI indicates that the current value of one or more parameters (e.g., the overall or average brightness in environment 110) does not equal the user's preferences for that parameter, controller 120 controls illuminant 121 to adjust the brightness accordingly.

[0074] A user preference may be associated with a corresponding tolerance. A user tolerance for a given preference value is an indication of the likelihood that the user 111 will accept a given setting for that value. This is described in more detail below.

[0075] Some LPIs may be more specific. For example, an LPI may indicate that a particular task area (e.g., identified by a task area identification number) is underlit. To remedy this, controller 120 may access a database (e.g., memory 124) that stores lighting system data, where "lighting system data" refers to information related to the position, and optionally the orientation, of illuminants 121 within environment 110. Thus, in this example, controller 120 may access memory 124 to determine which illuminants 121 are positioned to illuminate the underlit task area. Controller 120 can then control those illuminants to increase the brightness in that task area.

[0076] Whether the controller 120 decides to make a change to the lighting in the environment 110 may further depend on the activity currently being performed by the user 111. For example, if the user 111 is asleep, the controller 120 may decide not to increase the brightness in the environment 110 even if the received LPI indicates that the brightness is "too low". In some examples, the current activity of the user 111 may be determined based on a predefined schedule, e.g., stored in the memory 124. In other examples, the current activity of the user 111 may be estimated based on inputs from one or more devices in the environment 110. Examples of such devices include the smartphone of the user 111 and a smartwatch worn by the user 111. Data from the smartphone or smartwatch may be accessible by the controller 120 (e.g., via the network 123 or directly, e.g., via Bluetooth or WiFi). Data from the smartwatch, smartphone, or other device may be used to determine the current activity of the user 111. For example, heart rate data from a smartwatch can indicate that user 111 is exercising or stressed, and application data from a smartphone can indicate that user 111 is watching a video or reading a message or other content.

[0077] In additional or alternative examples, the current activity of user 111 may be determined based on calendar or agenda data. The calendar or agenda entries may indicate whether user 111 is, for example, in a meeting. The environment 110 may be a room that can be booked by a user, and data related to the booking (e.g., start and end times, number of attendees, etc.) may be managed by a room booking system. In such a case, data from the room booking system may be used to estimate the number of people in the environment 110. Data from the room booking system may also be used to determine user activity, for example, when indicating whether a presentation, conversation, discussion, etc. is currently in progress.

[0078] In further additional or alternative examples, the current activity of the user 111 may be determined based on audio captured within the environment 110, for example using a microphone. In some particular examples, the current activity of the user 111 may be related to the user's mood or excitement. Techniques for determining mood or excitement level from audio are known in the art.

[0079] User preferences may vary for different activities or mood / excitement levels.

[0080] Controller 120 may be configured to update user preferences in response to explicit or implicit input received from user 111. This is discussed in more detail below.

[0081] Some LPIs may take into account the subjective experience of the user 111. The subjective experience of the user 111 depends not only on the spectrum of illumination provided by the illuminant 121 but also on the response of the human eye to those wavelengths. Therefore, how the user 111 experiences the illumination in the environment 110 is best described by a luminance value. Luminance is a photometric measure of brightness in that it takes into account the sensitivity of the human eye. Therefore, a luminance distribution, which indicates the brightness perceived (by the user 111) at different locations in the environment 110, is valuable information for the lighting system 100.

[0082] Therefore, as part of determining one or more LPIs, the processor 202 of the camera unit 200 may determine a luminance distribution from one or more images captured by the camera 201 .

[0083] A method for determining a luminance distribution from one or more images captured by the camera 201 is first described. Various exemplary LPIs are then given. Where an LPI is described as requiring a luminance distribution, the luminance distribution may be determined from one or more images captured by the camera 201 (as described shortly below) or may be determined by an additional sensor and provided to the processor 202, for example, via the communication interface 203.

[0084] The RGB values ​​of the image captured by the camera 201 may be transformed into a different color space having luminance as one of the components and typically two chrominance components as the other components. That is, the luminance value may be determined as a combination of the RGB values. One example of a particularly advantageous color space is the CIE XYZ color space, since it was developed to have a color matching function V(λ) that resembles the luminous sensitivity curve of the human eye for photopic vision. The transformation from RGB to the XYZ (or other) color space may be performed using a transformation matrix that indicates the dependence on the white point applied by the camera 201 and the primaries of the selected color space. Thus, the luminance Y may be determined as a linear combination of the RGB values, as shown in Equation 1. TIFF0007614117000001.tif1269 formula 1 where r, g and b are weighting coefficients for the R, G and B values, respectively, extracted from the transformation matrix.

[0085] The transformation (and therefore the weighting coefficients) used to properly map between RGB space and XYZ space or other spaces depends on the illumination (spectral power distribution (SPD) of the illumination provided by one or more illuminants 121) when the RGB image was captured. Prior art systems assume that the illumination is of a standard illuminant with a known SPD. For example, for the sRGB color space, this is the standard illuminant D65. Because of these assumptions, prior art systems use fixed weighting coefficients r, g, and b for the transformation.

[0086] This disclosure recognizes that prior art systems suffer from poor accuracy in converting to luminance values. This is due to several factors. First, in the real world, an environment may be illuminated by many different types of illuminants with different SPDs. Second, the responsivity of the camera 201 may not perfectly match the standard sRGB spectral responsivity.

[0087] The present disclosure adapts weighting factors for determining the luminance distribution from a given image depending on the SPD of the illuminants present in the environment. In this way, more accurate luminance values ​​can be determined by optimizing the weighting factors used to take into account the SPD. The method described herein also takes into account deviations in the response of the camera 201 from the standard sRGB spectral responsivity.

[0088] In this case, the task is to determine the values ​​of r, g and b that determine the most accurate luminance value Y from the RGB values ​​of a given pixel of the image. This is designed to improve the performance of the spectral match and luminance distribution measurement. In the following, following the explanation of the theory, an exemplary implementation is described in more detail with reference to FIG. 4.

[0089] The relative spectral responsivity s of the camera 201 rel (λ) is importantly defined as a linear combination of the individual responses of the red R(λ), green G(λ) and blue B(λ) channels, using the same conversion coefficients as above. TIFF0007614117000002.tif15127 formula 2 where k r、g、b , is the calibration factor. The calibration factor is determined by the integral of the luminosity function V relative to the camera response s as shown in Equation 3. rel is chosen to be equal to the integral of TIFF0007614117000003.tif1951 formula 3

[0090] The choice of luminosity function V depends on the particular implementation, which is described in more detail below in the context of non-visual effects. For purposes of explanation, a typical luminosity function models the average spectral sensitivity of human visual perception of brightness. One such luminosity function is the CIE photopic luminosity function V(λ). Different luminosity functions may also be used, as described in more detail below.

[0091] In either case, the total power detected by the camera 201 should be equal to the total power that would have been detected by the human eye from the image. Therefore, to meaningfully compare the two, the response of the camera 201 is first scaled so that the total power detected by the camera 201 is equal to the total power that would have been detected by the human eye, per Equation 4. TIFF0007614117000004.tif2298 formula 4 Here, s * is the scaled response of the camera, and Φ is the SPD. The SPD may be determined in various ways, as described in more detail below.

[0092] Then, the scaled response s of the camera 201 is * can be directly compared to the human eye. The absolute value of the difference between the camera 201 and the human eye is a measure of the spectral mismatch, as shown in Equation 5. TIFF0007614117000005.tif1974 formula 5

[0093] Then, the general spectral mismatch f1 ` The function f1 is minimized, i.e., as shown in Equation 6.` The weighting coefficients r, g, b are determined to find a set of values ​​of r, g, and b that minimizes TIFF0007614117000006.tif36104 formula 6

[0094] The coefficients r, g, b obtained by the above method may then be used to determine the luminance value of each pixel of the image according to the above equation 1. Thus, the image itself can be converted into an array of luminance values ​​representing the luminance distribution in the environment.

[0095] In an alternative example, the general spectral mismatch can be defined as the root mean square of the absolute difference between the luminosity function weighted by the SPD and the camera's response weighted by the SPD, as shown in Equation 7: TIFF0007614117000007.tif29139 formula 7

[0096] However, once the spectral mismatch is defined, the same process of minimizing the spectral mismatch under the constraints of Equation 6 is applied. The resulting coefficient values ​​(which minimize the spectral mismatch) can then be used to determine the luminance values, as in Equation 1.

[0097] It should be noted that in any of the above examples, one or more integrals may be calculated discretely, for example in increments of 1 nm, and that the upper and lower integral limits refer to the visible spectrum in human vision, and thus the given range of 380-780 nm is only an example.

[0098] FIG. 4 is a flow chart illustrating an exemplary method performed by processor 202.

[0099] In S401 , the processor 202 receives an RGB image from the camera 201 .

[0100] At S402, the processor 202 identifies the spectral power distribution (SPD) of the illumination in the environment 101.

[0101] The spectral power distribution (SPD) represents the radiant power at each wavelength. The SPD of the illumination provided by illuminant 121 is needed to determine the coefficients, as described above.

[0102] As mentioned above, the SPD of the illumination in the environment 101 affects the color gamut imaged by the camera 201. The gamut is related to the surface colors of the scene to some extent. For example, a scene imaged under incandescent lighting provides a scene where red values ​​are expected to be higher, and therefore the gamut is expected to lie near higher R values. While every light source is expected to have its own gamut, light sources with similar SPD are expected to have very similar gamuts. This can be exploited to distinguish between different light sources. Therefore, based on the gamut of the scene, the SPD of the illuminant 121 can be estimated.

[0103] The memory 204 may store a set of predefined gamuts, each associated with a predefined SPD. The association between a gamut and an SPD may be determined during a commissioning process by capturing images of an environment under lighting with known SPD and determining the associated gamut from the captured images.

[0104] Thus, the processor 202 may identify the SPD by determining a gamut from the RGB image and accessing the memory 204 to identify a predefined gamut that most closely matches the determined gamut. The processor 202 may then use the SPD from the memory 204 that is associated with the identified predefined gamut.

[0105] Note that if the environment is illuminated by light sources (e.g., illuminant 121, natural light, etc.) with multiple different SPDs, the gamut is affected by all SPDs present. In other words, the SPDs of the light sources in the environment 110 are added together to produce an overall SPD. It is the overall SPD that is estimated from the extracted gamut. Thus, the estimated predetermined SPD (associated with the predetermined gamut that is most similar to the extracted gamut) will be the predetermined SPD that most closely matches the overall SPD.

[0106] For example, if environment 110 is illuminated by a combination of 50% natural light and 50% LED lighting provided by illuminant 121, the overall SPD will be the combination of the SPDs of the natural light and the LED lighting. If none of the predefined gamuts are associated with this exact type of lighting, processor 202 will still determine the closest match.

[0107] The gamut (stored in memory 204 and determined from the RGB image) may be a red-blue gamut (a gamut based on the red and blue channels imaged by camera 201), which is particularly representative of the colors in the image and therefore works better than other gamuts (blue-green, green-red) for estimating SPD.

[0108] The predefined gamuts are based on a set of SPDs likely to be present in real-life scenarios, such as LED, fluorescent, and daylight. For each SPD, a theoretical gamut is determined using the camera's spectral responsivity. To estimate the illuminant of the scene, the imaged gamut of the scene is compared to all predefined gamuts. Based on the correlation between the imaged gamut and the predefined gamut, the processor 202 determines which SPD is most likely, or in other words, which SPD has the highest probability. Which gamut from the memory 204 most closely matches the identified gamut may be determined using Bayesian estimation.

[0109] In another example, the SPD indication can be provided to the processor 202 by one of the illuminants 121 via the communication interface 203. The SPD indication can be defined during a commissioning process, such as during installation of the lighting system 100, or as a step in the manufacturing process of the illuminant 121. In either case, the illuminant 121 includes a memory that stores an indication of the SPD of that illuminant that can be provided to the processor 202. Alternatively, the SPD indication can be stored in another memory, such as the memory 124.

[0110] In yet another example, the SPD can be measured directly using an illuminance spectrophotometer.

[0111] In S403, the processor 202 determines the values ​​of a set of coefficients r, g, b for converting the RGB values ​​of the image into luminance values.

[0112] Based on the most likely SPD, the processor 202 determines weighting factors for R, G, and B to most accurately calculate the luminance value within the distribution, as described above. The weighting factors are determined such that the general spectral mismatch of the R, G, and B combinations is minimized, as shown in Equations 4 and 5 above. This means that the R, G, and B values ​​of a given pixel are combined such that the luminance function weighted by the SPD is most closely matched (see Equation 2). This optimization, shown in Equation 6 or Equation 7, results in three weighting factors for R, G, and B that are used to most accurately calculate the corresponding luminance value.

[0113] At S404, the processor 202 uses the coefficient values ​​from step S403 to determine a luminance distribution, which involves determining the luminance value of each pixel of the RGB image, or at least a portion thereof, by taking a linear combination of the RGB values ​​of that pixel with the determined coefficient value, as shown in Equation 8. TIFF0007614117000008.tif15103 formula 8

[0114] This is done for each individual pixel. The result is an image of the luminance channel, i.e. a luminance distribution, that contains the luminance values ​​of each pixel in an area. The area over which the luminance distribution is formed may include some or all of the area of ​​the original input image (or images).

[0115] This process can also be transposed to other sensitivities in the visible part of the spectrum, such as α-opics, which are discussed in more detail below.

[0116] Although the processor 202 can determine the luminance distribution from any RGB image using the methods described above, to obtain a more accurate luminance distribution, it is preferable that the dynamic range of each pixel in the RGB image is as high as possible. One way to achieve this is to use a High Dynamic Range (HDR) image.

[0117] The HDR image is constructed from multiple Low Dynamic Range (LDR) images captured using different exposures, for example, using sequential exposure bracketing. The LDR images are merged into a single HDR image. Thus, the camera 201 is configured in some examples to capture multiple LDR images of the environment 101. The processor 202 then combines these LDR images into an HDR image. The processor 202 can then use the HDR images in determining the luminance distribution as described herein. That is, a linear combination is applied to the constructed HDR images.

[0118] The construction of the HDR image can be performed in parallel with the determination of the coefficients for the linear combination. This is described below with reference to Figure 5, which shows a flow chart illustrating an example method performed by the processor 202. In this example, the memory 204 stores a set of predefined color gamuts, each associated with a respective SPD.

[0119] In S510, the processor 202 receives a set of LDR images from the camera 201. Each LDR image is an RGB image.

[0120] In S511, the processor 202 extracts a color gamut from one or more of the received RGB images. For example, the processor 202 may extract a color gamut from a first LDR image of the LDR images. Alternatively, the processor 202 may extract a color gamut from each of the LDR images and determine an average gamut.

[0121] At S512, the processor 202 identifies an SPD to use in determining the coefficients. To do so, the processor 202 accesses the memory 204 to determine a predefined color gamut that most closely matches the gamut extracted at S511. This may be done using Bayesian estimation. The SPD is assumed to be the one associated with the gamut in the memory 204 that most closely matches the color gamut identified from the captured image. In other words, the SPD can be estimated by the processor 202 from the image captured by the camera 201. This is done using the insight that the SPD of the illumination provided by the illuminant 121 affects the colors captured by the camera 201.

[0122] In S513, the processor 202 determines the coefficients using the luminosity function and the SPD, as described above.

[0123] In S520, the processor 202 constructs an HDR image from the received LDR image. As shown in FIG. 5 and described above, this is performed in parallel with the gamut extraction in S511, the SPD identification in S512, and the coefficient determination in S513. Due to this, the parallel tasks may be performed by separate controller modules. That is, it should be understood that the processor 202 may be implemented as a first control module and a second control module. The first control module is configured to perform at least steps S511 to S513, and the second control module is configured to perform at least step S520. The control modules may be implemented by different processors. If privacy is not a concern, the control modules may be implemented elsewhere in the lighting system 100. For example, the second control module may be implemented locally in the camera unit 200, and the first control module may be implemented elsewhere in the lighting system 100, such as in a server.

[0124] In S530, the processor 202 determines a luminance distribution from the constructed HDR image using the determined coefficients.

[0125] Prior art sensors applied to lighting control systems can typically only provide one piece of information: the illuminance at a particular point in space, i.e., a scalar value representing the illuminance in the field of view of the sensor. A luminance distribution, on the other hand, provides luminance values ​​(i.e., a luminance distribution) for a complete set of points (each point in some or all of the captured images) covering an area or volume of space. Figure 6 shows an exemplary luminance distribution 600 of the environment 110 determined from images captured by the camera 201 as described above.

[0126] This information allows for a wide range of valuable insights related to the user's 111 perception within the environment 110, compared to a single scalar value as provided by prior art sensors. To do so, according to embodiments disclosed herein, the processor 202 is configured to derive one or more LPIs from the images captured by the camera 201. The LPIs relate to brightness, task area lighting levels, non-visual effects, dynamics, etc. Each LPI is a composite metric derived from multiple pixels from an array of pixels in the images captured by the camera 201. As discussed above, this may or may not involve first converting each pixel into a luminance value.

[0127] Various examples of LPI are now described.

[0128] A task area illumination level is an example of an LPI. The processor 202 may determine one or more task areas from an image captured by the camera 201. For example, the task area may be a desk surface in the environment 110. The processor 202 may then determine from the luminance distribution the luminance values ​​present within the task areas. In one example, the processor 202 may take an average luminance value over a portion of the luminance distribution that corresponds to the identified task area. For example, the processor 202 may identify an average illuminance value within each task area and provide the identified illuminance to the controller 120 along with an indication of the respective task area (e.g., task area 1 luminance=value 1, task area 2 luminance=value 2, etc.).

[0129] People tend to have preferences for a certain lighting level over their task area, e.g., 300 lux or 500 lux. Thus, the task area lighting level LPI may be used by controller 120 to determine whether a task area is under- or over-lit (e.g., by comparing the determined task area lighting level with a target lighting level for that task area). Controller 120 may then control one or more corresponding illuminants 121 to increase or decrease the lighting level for that task area accordingly.

[0130] 7 and 8 show examples of user preference data as described above in relation to FIG. 3. Note that both Illuminance and Luminance values ​​are shown. If the reflectance of surfaces in the environment 110 is known (assuming the surfaces are Lambertian reflectors), the Illuminance can be extracted from the Luminance. The controller 120 may access the user preference data, for example from the memory 124.

[0131] Figure 7 relates to a single user, where the probability that a given illumination in the environment 110 is considered insufficient 701, satisfactory 702 or excessive 703 by the user is shown.

[0132] Controller 120 may compare the received LPI to the user preference data to determine user satisfaction. Controller 120 may also determine user satisfaction for multiple users, thereby determining an average or overall user satisfaction.

[0133] 8 relates to the preferences of several (three in this example) users 801, 802, 803. The preference data for each user is represented as a curve with a maximum at the user's preferred illuminance and a width representing the user's tolerance range. For example, user 801 prefers a lower illuminance than user 803, but they both have a similar tolerance range. User 802 prefers an illuminance value between users 801 and 803, but is more tolerant of deviations from this preferred value than the other two users 801, 803.

[0134] In the case of multiple users, controller 120 may determine user satisfaction based on the task area of ​​that particular user, i.e., controller 120 may compare the current intensity value of the particular task area (as indicated in the received LPI) with the preference data of the user (e.g., the user working at that desk) associated with that task.

[0135] User preference data for values ​​other than brightness may be expressed and taken into account by controller 120 in a similar manner as described above. For example, user preference data relating to each user's comfort with different levels of contrast may be stored in memory 124.

[0136] Glare is another example of LPI. The processor 202 may be configured to identify (potential) glare sources from the luminance distribution. The processor 202 may then quantify the amount of glare experienced by the user 111. Glare is a function of the luminance and solid angle of the glare source as viewed by the user 111, the background luminance, and the orientation of the user 111 relative to the glare source. As an example, one useful definition of glare is the Unified Glare Rating. The Unified Glare Rating (UGR) is a measure of glare in a given environment proposed by Sorensen in 1987 and adopted by the International Commission on Illumination (CIE). It is defined as Equation 9: TIFF0007614117000009.tif1672 formula 9 where log is the logarithm base 10 and L b is the background luminance, and L n are the luminances of each light source numbered n, and ω n is the solid angle of the light source as seen by the observer, and p n is the Guth position index, which depends on the distance from the line of sight of the user 111. Note that URG is given only as an example and other metrics of glare may be used.

[0137] The processor 202 may determine the background luminance and the luminance of the glare source from the luminance distribution itself.

[0138] The processor 202 may use face recognition and / or eye tracking techniques known in the art to estimate the offset of the glare source from the goose's position indicator or the user's 111 line of sight, and the solid angle for the glare source. Among other things, such techniques are known and used in the field of signage and interactive shop windows (shop owners want to know which products are attracting attention from potential customers). For the purposes described herein, these techniques can be used to determine in which direction a user (e.g., user 111) is looking (viewing angle). The glare LPI can then be determined using this viewing angle. Other LPIs that require information about the viewing angle of the user 111 may use these techniques as well.

[0139] Glare is generally undesirable. Thus, the controller 120 may control one or more devices in the lighting system 100 to reduce glare. For example, the controller 120 may do so in response to receiving an LPI indicating that the amount of glare experienced by the user 111 exceeds a threshold amount of glare. For example, the controller 120 may determine that the amount of glare emanating from the user 111's computer screen exceeds a threshold amount. In response to this determination, the controller 120 may control one or more devices of the lighting system 100 to correct the excessive glare. This may be accomplished, for example, by reducing the brightness setting of one or more illuminants 121 that are causing the glare.

[0140] The controller 120 may have the additional functionality of intervening in the lighting control to reduce glare by dimming or turning off one or more illuminants 121 that are / are causing glare. If the glare is due to a non-controllable light source (e.g., natural light from a window), the controller 120 may control different devices to reduce the presence of this non-controllable light source. As an example, the controller 120 may deploy blinds or shades over a window through which natural light enters the environment 100.

[0141] The glare value may be determined for each task area using the techniques described above, i.e., the processor 202 may determine a glare value for each task area in the environment 110.

[0142] Uniformity (also called contrast) is another example of an LPI. "Uniformity" refers to the change in brightness in a distribution, i.e., the variation in brightness in an image. Brightness may be determined based on RGB values ​​of the image. For example, the processor 202 may generate a contrast LPI that indicates the difference or variation in brightness of areas of the image captured by the camera 201. A similar "luminance contrast" LPI may be generated from the luminance distribution.

[0143] In another example, the processor 202 determines the uniformity of luminance within the environment, since the techniques disclosed herein allow a luminance distribution to be determined (as opposed to a single luminance value).

[0144] In other words, the processor 202 may be configured to analyze changes in brightness or luminance in the image. This allows the processor 202 to generate an LPI indicative of areas of high contrast. The processor 202 may then transmit this LPI to the controller 120. The controller 120 may then identify whether the amount of non-uniformity is within an acceptable range. For example, user preference data (e.g., from the memory 124) may indicate an acceptable contrast range for one or more users. In general, contrast that is too strong is distracting, while contrast that is too weak is dull. The user preference data may be compared to the received contrast value to determine whether the contrast is acceptable, too high, or too low. The controller 120 may be configured to control one or more devices in the lighting system 100 to make the contrast experienced by one or more users more acceptable (i.e., increase the contrast if it is too low, or decrease the contrast if it is too high).

[0145] Uniformity of chromaticity is another example of an LPI. Large color variations within the environment 110 are generally undesirable. For example, sunlight entering through the window 112 may have a different color than the artificial light from the illuminant 121. It is generally desirable to control the illuminant 121 to match the color of the sunlight to create a uniform lighting atmosphere within the environment 110. Thus, the processor 202 may generate a color contrast LPI that indicates the difference or variation in color of areas of the image captured by the camera 201.

[0146] Colour uniformity may be determined using similar techniques as described above in relation to luminance uniformity. Non-linear operations on individual colour channels are preferred to quantify the colour difference. The LPI preferably includes the absolute value or square of these colour distances. For example, to calculate the colour distance, as a first step a (non-linear) conversion from RGB to XY chromaticity locations in the colour triangle is required. The colour distance can be obtained from the distance between the chromaticity locations of two differently illuminated areas.

[0147] Controller 120 may then control one or more devices in lighting system 100 to improve color uniformity. For example, this may include controlling illuminant 121 to change its color output to more closely match that of sunlight.

[0148] Other examples of LPIs concern non-visual effects. It is well known that lighting (and light in general) can affect the human body in non-visual ways. Examples include acute effects such as alertness, melatonin suppression, pupillary reflex, brain activity, heart rate, circadian effects such as sleep-wake regulation, and therapeutic effects such as depression and mood.

[0149] It should be noted that the method for determining the luminance distribution described above involved the use of a luminosity function V(λ), which describes the way human vision responds to different wavelengths of light. However, the same technique can be used to model non-visual effects of lighting in the environment 110.

[0150] To do so, the luminosity function is simply replaced by a function that represents the desired non-visual response of the human eye. Such a function is called the α-opic action spectrum, and represents the response of different cell types in the human eye associated with their respective physiological effects. Examples of suitable functions include melanopic radiance, s-cone-opic radiance, m-cone-opic radiance, l-cone-opic radiance, rhodopic radiance, etc. Figure 9 shows several different examples of these functions. Each function represents the relative strength of the effect associated with a particular type of cell in the human eye to different wavelengths. For example, a physiological effect caused by a first type of cell having a response represented by function 901 is more responsive to shorter wavelengths than a physiological effect caused by a second type of cell having a response represented by function 902.

[0151] By using the α-opic to represent a particular physiological effect, the strength of that effect induced in the user 111 by the current lighting in the environment 110 can be estimated. Thus, one or more LPIs may be non-visual LPIs indicating an estimated strength of the particular effect.

[0152] The estimated intensity of the effect may be compared to a user preference for that effect to determine whether the current intensity is acceptable. The user preference may be time-dependent. For example, the user preference may be for lower melanopic effects in the evening than in the morning. The controller 120 may then control one or more devices in the lighting system 100 accordingly to adjust the effect. For example, the controller 120 may control the illuminant 121 to output less blue light in the evening to reduce melanopic effects.

[0153] In other examples, the expected non-visual effects for the user can be simply estimated using color values ​​from the image, for example blue areas of the image may be assumed to produce melanopic effects.

[0154] White is another example of an LPI. Colors can be expressed as "color temperature", which is the temperature of an ideal blackbody radiator that emits light of a color equivalent to the color in the image. To calculate the color temperature from the image captured by the camera 201, the processor 202 converts the RGB values ​​to XYZ coordinates. These are converted to a standardized u,v color space, and a non-linear mapping conforming to the CIE 1960 UCS gives the color temperature.

[0155] In a colour-tunable system (i.e., a lighting system 100 in which the colour of the light output by the illuminant 121 is controllable), the LPI based on colour and colour difference includes such calculations. Among other things, the LPI can indicate whether a combination of colour temperature and luminance satisfies the Kruithof criterion. The (non-linear) Kruithof curve describes the region of illuminance levels and colour temperatures that are often considered comfortable or pleasing to an observer. In particular, the combination of low levels of cool light or high intensity of warm light is perceived as unpleasant.

[0156] Information such as the Kruithof curve describing acceptable combinations of illuminance and color temperature may be stored in memory 124. Controller 120 may then compare the received LPI, which indicates color temperature and illuminance (which may be separate LPIs), with the acceptable combinations to determine whether the current combination is acceptable.

[0157] If the controller 120 determines that the current combination is not acceptable, it can initiate actions to lower or increase the color temperature and / or illuminance as appropriate to reach an acceptable value.

[0158] Further examples of LPI come from chronobiology. Human sleep is primarily regulated by two processes: the circadian pacemaker and the homeostatic sleep drive. Many mathematical models of the circadian pacemaker are known, such as the "Kronauer model". Exposure to light affects the human body clock in a way that can be described by many nonlinear equations that depend on the moment of light exposure to the human body clock. This may be simplified to a function of time of day if light exposure is predictable (e.g., natural light is the main influence). Light exposure has a weighted impact on the human body clock, and in particular, on sleep. This is typically referred to in the context of the "light dose response curve". Thus, another example of LPI is the impact of lighting on the user's 111 body clock.

[0159] Another example of an LPI is a medical LPI for the impact of lighting on the efficacy of a drug. In studies on the efficacy of medication, it is known that the amount of light exposure affects the efficacy of a particular drug dose. Thus, the processor 202 may be configured to determine the expected impact of the current lighting on the efficacy of one or more drugs and report these as LPIs. In a further example, the processor 202 may determine how much the current lighting affects the efficacy of the drug. The processor 202 may then indicate in the LPI a change corresponding to the dosage that counteracts the illumination-induced change in efficacy. Alternatively, the controller 120 may perform this step upon receiving the medical LPI from the camera unit 200.

[0160] One or more LPIs may be determined by controller 202 at a predetermined time interval, e.g., once per second, once per ten seconds, etc. To do so, camera 201 is configured to capture and provide images to controller 202 at predetermined time intervals. Controller 202 can then be configured to determine, for each image (or set of images) received from camera 201, a corresponding luminance distribution. Controller 220 may determine any of the above-mentioned LPIs from the dynamically determined luminance distributions.

[0161] Even when configured to capture multiple LDR images for use in constructing an HDR image, the camera 201 may be configured to capture images at a predetermined time interval as well. In such cases, the predetermined time interval may be longer than the above, for example, once per minute, once per five minutes, etc.

[0162] As mentioned above and shown in Figure 3, the controller 120 may update the user preference data in response to input from the user 111. The input may be explicit or implicit.

[0163] One example of explicit user input is the user 111 manually controlling one or more devices in the lighting system 100 to change one or more lighting conditions. For example, if the user 111 controls the illuminant 121 to increase its brightness (e.g., using a wall switch or using a personal computing device connected to the controller 120 via the network 123), the controller 120 may determine that the user 111 prefers brighter lighting. The controller 120 may then update the user preference data in the memory 124 accordingly.

[0164] Another example of explicit input is user 111 explicitly indicating his or her satisfaction with the current lighting conditions in environment 110. For example, user 111 may provide an indication of his or her satisfaction to controller 120 over network 123 using a personal computing device, such as a smartphone.

[0165] An example of an implicit input is when user 111 does not react negatively to a change in a lighting setting, e.g., if controller 120 increases the brightness in environment 110 and user 111 does not manually intervene to decrease the brightness, controller 120 may determine that the new brightness setting is acceptable to user 111. That is, controller 120 may update the user's brightness user preference data in memory 124.

[0166] Various potential use cases are described below.

[0167] A first exemplary use case is a Daylight Glare Probability (DGP) sunshading controller.

[0168] Automatic sun shading is increasingly being implemented in buildings because it can improve not only comfort but also energy performance. However, automatic shading devices have a bad reputation because of the discomfort they cause due to "false on" and "false off". Typically, these sun shading systems are controlled by photocells placed on the roof of the building. First, the photocells are not placed in the most relevant locations. Second, the photocells lose spatial information by averaging the light falling on the sensor.

[0169] Glare is the most important reason why one wants to apply sunshades. Therefore, the embodiment uses a DGP measurement device to control the sunshade. The DGP is based on luminance distribution measurements, performed in the relevant room, using low-cost components as described herein. The DGP can be extracted using appropriate software known in the art. If the glare exceeds a certain threshold, the sunshade is activated.

[0170] As a result, the sunshade is activated only when there is actual glare that is expected to annoy the room occupants, and therefore, it is expected that there will be less unnecessary movement of the shading device that causes discomfort.

[0171] A second example use case is as part of a lighting quality controller.

[0172] Lighting control systems tend to optimize energy performance or a single lighting quality aspect (e.g., illuminance on a desktop). However, lighting quality has many aspects that are all related. Optimizing a single lighting quality aspect does not necessarily provide high quality lighting. The main problem is that not all aspects are easily measured and often require different measurement devices.

[0173] It is recognized herein that the luminance distribution measuring device can simultaneously extract information about multiple relevant aspects of lighting quality. Therefore, it is an excellent sensor for a control system that provides high quality lighting. When the sensor is placed in a room, most likely the ceiling, a closed loop control system can be developed that can optimize lighting to provide high quality lighting.

[0174] Instead of optimizing only illuminance or energy performance, lights can be optimized for a combination of light quantity, glare, distribution, directionality, and dynamics. This allows control systems to set lighting conditions more precisely without being counterproductive.

[0175] A third example use case is a desktop light sensor.

[0176] The light sensor approximates the illuminance on the desktop for controlling lighting by measuring the luminance of a small area. The luminance measurement correlates with the illuminance using the reflection of the desktop. However, since the measurement area is very small, it is not guaranteed that this area represents the entire desktop. This area may be covered by, for example, a paper that has a completely different reflection than the desktop, which would then cause a large error in the illuminance approximation. As a result, this may lead to a change in the actual illuminance on the desktop, since the control system makes the wrong selection based on the wrong input.

[0177] The embodiment addresses this problem by using a luminance distribution measuring device. This means that the opening angle is significantly increased and therefore the luminance of the entire desktop can be measured. Additionally, since the luminance distribution utilizes an image, desktop obstructions can be recognized by image processing. By ignoring occluded areas, the illuminance is measured only for the relevant area.

[0178] Based on this methodology, illuminance is measured over an entire relevant area, free of obstructions, instead of a small area that is not necessarily relevant, and lighting can therefore be more accurately provided by the control system.

[0179] A fourth exemplary use case is wall brightness control.

[0180] It has been shown in the literature that the luminance and uniformity of non-horizontal surfaces (e.g., walls) within the field of view affect the visual and psychological evaluation of an office space. Furthermore, the preferred light level on the desktop is lower for non-uniform wall luminance. Therefore, including wall luminance as a target in a lighting control system would improve comfort and energy performance. However, walls have a large surface area and therefore are not easily measured using current methods. As a result, they cannot be included in the control system.

[0181] In the embodiments disclosed herein, the luminance distribution of the wall can be continuously measured using the camera unit 200. If the camera unit 200 is placed accordingly, most likely in a vertical orientation, the wall luminance can be measured simultaneously with the desktop luminance (illuminance). This provides continuous data relevant to the control system (closed loop) so that energy can be limited by changing the wall luminance so that lower illuminance on the desktop is allowed.

[0182] All relevant content is measured at once using the camera unit 200. Based on the measurement, the illuminated environment is set such that wall luminance allows the desktop illumination to be lowered so that energy can be saved.

[0183] A fifth exemplary use case relates to screen visibility.

[0184] More and more surgeries are being performed with robots controlled by a doctor. The doctor controls the robot based on live camera or X-ray information. It is very important that the doctor can see this information as well as possible. Although the images shown can be enhanced to improve visibility, it may be more effective to improve the illuminated environment so that the doctor is provided with optimal conditions to view the images.

[0185] To enhance the visibility of an image on a screen, the luminance contrast should be optimized. Additionally, veiling reflection should be prevented. Both are aspects that can be measured by the camera unit 200. By developing a closed-loop control system with the measurement data and images / videos of the camera unit 200 as inputs, the conditions can be optimized to enhance the visibility of the screen.

[0186] Image enhancement has reached its limits, and even with large investments only a small increase in visibility can be achieved. Developing a closed-loop system including the camera unit 200 can be more effective. Furthermore, the camera unit 200 can also be used to optimize lighting for different tasks in the operating room.

[0187] A sixth exemplary use case is occupancy-based dimming.

[0188] Dimming lighting fixtures based on occupancy is energy efficient, however, in an open office, occupancy-based dimming can be irritating by automatically switching or dimming lighting fixtures when people leave or arrive.

[0189] It is recognized herein that energy can be saved while maintaining high comfort by dimming background zones while maintaining sufficient illumination of the work area. Using the camera unit 200, the brightness of different areas can be measured simultaneously. Thus, when an occupant leaves, the light can be partially dimmed using recommendations regarding the available background, ambient and task areas.

[0190] The advantage is that all relevant areas can be measured, each occupant having their own area, but still with only one measuring device. Furthermore, occupancy sensing can be included in the measurements of the camera unit 200.

[0191] A seventh exemplary use case is found in directional lighting for Non-Image-Forming (NIF) effects.

[0192] It is recognized herein that light coming from certain angles has a more stimulating effect on humans. At certain moments during the day, stimulation is needed and at other moments it is not needed, and the stimulation is very time-dependent, just as daylight is also time-dependent. Therefore, optimizing the lighting and shading / reflecting devices to achieve the required stimulation is a complex problem.

[0193] With the camera unit 200, the non-image forming (NIF) effect can be approximated. Similar to the luminance picture, a radiance map can be determined for the entire space. Based on this, the camera unit 200 can be implemented in a closed-loop control system that optimizes the light so that the (stimulating) top half of the image sensor (using a fisheye lens) has a certain ratio to the bottom half.

[0194] Also, with the device the luminance distribution can be measured simultaneously with the α-opics, allowing the stimulation effect to be optimized without adversely affecting visual comfort.

[0195] An eighth exemplary use case is Non-Image-Forming (NIF) vs. Image-Forming (IF) optimization.

[0196] Non-image and image forming requirements change throughout the day. However, a ratio can be developed that captures the relationship between the NIF and IF requirements for a given time of day. Based on this ratio, lighting can be optimized so that the lighting is stimulating but not at the wrong moment while maintaining high quality lighting.

[0197] In an embodiment, non-image forming (NIF) effects can be approximated using the camera unit 200. Similar to a luminance picture, a radiance map can be determined for the entire space. Based on this, the camera unit 200 can be implemented in a closed-loop control system that optimizes the light according to the NIF IF ratio.

[0198] Also, with the device the luminance distribution can be measured simultaneously with the α-opics, allowing the stimulation effect to be optimized without adversely affecting visual comfort.

[0199] A ninth example use case is two-sided lighting quality optimization.

[0200] For lighting quality, generally only visual aspects are considered. However, it is possible to distinguish high quality lighting for non-visual aspects as well. The problem is that visually high quality lighting does not necessarily provide non-visual quality. Therefore, it is difficult to optimize both.

[0201] In particular, non-visual aspects are highly time-dependent, meaning that non-visual lighting quality is not, or is less relevant for every moment during the day. Visual lighting is less time-dependent. According to embodiments disclosed herein, it can be determined which type of lighting quality is most relevant based on time, and then this type is optimized. Both types of lighting quality can be determined using the camera unit 200.

[0202] With the camera unit 200, both qualities can be measured simultaneously, so only one device is needed to measure a room or an area of ​​a room. Furthermore, as the device is placed in the room, a closed loop setup can be developed to improve accuracy. Using this technique, lighting can be controlled in many ways that were not possible before.

[0203] Other variations to the disclosed embodiments can be understood by those skilled in the art upon study of the drawings, the disclosure, and the appended claims, and can be implemented in practicing the claimed invention. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain means are recited in mutually different dependent claims does not indicate that a combination of these means cannot be used to advantage. The computer program may be stored / distributed in a suitable medium, such as an optical storage medium or a solid-state medium, provided together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems. Any reference signs in the claims should not be interpreted as limiting the scope.

Claims

1. 1. A method for determining a spatial luminance distribution over at least one image of a scene illuminated by at least one illuminant, said image comprising RGB values ​​for each pixel, the method comprising: forming a combination of the RGB values ​​to determine the luminance distribution, the combination having a set of coefficients including a respective coefficient for each of the RGB values; Including, The coefficients are: identifying a spectral power distribution (SPD) of the at least one illuminant; and determining values ​​of a set of coefficients that minimize a spectral mismatch between a) the identified SPDs weighted by the combination and b) the identified SPDs weighted by a luminosity function; The method is determined by.

2. 2. The method of claim 1, wherein the at least one image comprises a set of low dynamic range (LDR) images, the method comprising constructing a high dynamic range (HDR) image from the set of LDR images, and the combination is applied to the constructed HDR image.

3. 3. The method of claim 1 or 2, wherein identifying the spectral power distribution comprises determining a gamut of the at least one image and comparing the determined gamut with a set of predetermined gamuts of known spectral power distributions.

4. The method of claim 3, wherein the gamut is a red-blue color gamut.

5. The method of claim 1 or 2, wherein identifying the spectral power distribution comprises receiving a predetermined indication of the spectral power distribution.

6. The method of claim 1 , wherein the combination is a linear combination.

7. 7. The method of claim 1, further comprising determining at least one Light Performance Indicator (LPI) from the luminance distribution, the LPI being a composite metric derived from an area of ​​the luminance distribution.

8. The method of claim 7 , wherein at least one of the LPIs is a luminance LPI indicative of luminance values ​​in an area of ​​the luminance distribution.

9. The method according to claim 7 or 8, wherein at least one of the LPIs is a contrast LPI, which indicates differences in luminance over areas of the luminance distribution.

10. 10. The method of claim 7, further comprising identifying an orientation of a user, and wherein at least one of the LPIs is determined for an area of ​​the luminance distribution that corresponds to an area located in front of the user.

11. At least one of the LPIs is a glare LPI indicating an amount of glare experienced by a user, the method comprising: determining a background luminance and a luminance of a glare source from said luminance distribution; estimating a displacement of the glare source from the user's line of sight; estimating a solid angle from the user to the glare source; and determining an amount of glare based on the background luminance, the luminance of the glare source, the estimated displacement of the glare source from a line of sight of the user, and the estimated solid angle of the glare source from the user; 11. The method of claim 7, further comprising determining the glare value by:

12. 12. The method of claim 7, wherein at least one of the LPIs is a non-visual LPI indicative of an expected non-visual effect on a user.

13. 13. A method according to any one of claims 7 to 12, comprising identifying a plurality of task areas within the luminance distribution and determining an LPI for each respective task area.

14. A computing device comprising computer executable code embodied in a computer readable storage medium and configured to perform the method of any one of claims 1 to 13 when executed by one or more processors.

15. A system comprising the computing device of claim 14 and a camera.

Images