A controller for controlling a plurality of lighting units and a method thereof

Abstract

A method of controlling a plurality of lighting units located in a space according to a light scene comprising a plurality of light settings is disclosed. The method comprises: obtaining an original mapping of the light settings onto the plurality of lighting units, controlling the plurality of lighting units according to the respective light settings as defined by the original mapping, - receiving a signal indicative of that an augmented reality device has been activated, - obtaining location information indicative of a location of the augmented reality device in the space relative to the plurality of lighting units, - adjusting the original mapping of the light settings onto the plurality of lighting units by remapping the plurality of light settings onto the plurality of lighting units, wherein each light setting is mapped onto a lighting unit based on the light setting's respective color, saturation, intensity and / or temporal aspects and based on the lighting unit's respective location relative to the location of the augmented reality device, and - controlling the plurality of lighting units according to the remapped light settings when the augmented reality device is activated.

Description

[0001] A controller for controlling a plurality of lighting units and a method thereof

[0002] FIELD OF THE INVENTION

[0003] The invention relates to method of controlling a plurality of lighting units located in a space according to a light scene. The invention further relates to a computer program product for executing the method. The invention further relates to a controller for controlling a plurality of lighting units located in a space according to a light scene.

[0004] BACKGROUND OF THE INVENTION

[0005] Home environments typically contain multiple controllable lighting units for creation of atmosphere, accent or task lighting. These controllable lighting units may be controlled via a user interface of a control device, such as a smartphone, via a wireless network. A user may select a light scene via the user interface of the control device, whereupon the lighting units are controlled according to light settings defined by the light scene. Alternatively, the light scene may be activated automatically (e.g. based on a scheduled routine, based on a sensor that has been triggered, etc.) or the lighting units may be controlled according to light settings that are based on media content (e.g. an image, video, music, etc.).

[0006] The light settings of a light scene are to be mapped onto the lighting units according to a (predefined / generated / random) mapping. This mapping may be defined by a user, for example via a user interface that enables the user to select certain light settings for certain lighting units. Alternatively, the mapping may be performed automatically and may, for example, be random or be based on the light rendering properties or types of the lighting units. The light scene and the associated mapping may then be stored, such that it can be recalled at a later moment in time.

[0007] US 2020245435A1 discloses a method of controlling a plurality of lighting devices, comprising: obtaining a 360 degree panoramic image, rendering the 360 degree panoramic image at least partially on an image rendering device, obtaining positions of the plurality of lighting devices relative to the image rendering device, mapping the 360 degree panoramic image onto the plurality of lighting devices based on the positions of the plurality of lighting devices, such that each lighting device is associated with a part of the 360 degree panoramic image, determining, for each lighting device, a light setting based on an image characteristic of the part of the 360 degree image, and controlling each lighting device according to the respective light setting.

[0008] SUMMARY OF THE INVENTION

[0009] A user who is present in the space wherein a lighting system is located may operate an augmented reality device. The inventors have realized that the light output of lighting devices affects the view of a user operating the augmented reality device. Certain light settings may have a negative impact on the quality of the view of a user through the augmented reality device. For instance, a user may have created a light scene in his or her living room (or the light scene may have been generated automatically), and the light scene may have been created such that lighting units in the field of view of the augmented reality device are, for example, (heavily) saturated, too bright or (heavily) dimmed, which may negatively affect the quality of the view of the augmented reality device. Consequently, the view may for example be colored, overexposed or underexposed. It is therefore an object of the present invention to provide a method and a controller for mapping light settings to lamps to improve the quality of the view of an augmented reality device.

[0010] According to a first aspect, the object is achieved by a method of controlling a plurality of lighting units located in a space according to a light scene comprising a plurality of light settings, the method comprising: obtaining an original mapping of the light settings onto the plurality of lighting units, controlling the plurality of lighting units according to the respective light settings as defined by the original mapping, receiving a signal indicative of that an augmented reality device has been activated, obtaining location information indicative of a location of the augmented reality device in the space relative to the plurality of lighting units, adjusting the original mapping of the light settings onto the plurality of lighting units by remapping the plurality of light settings onto the plurality of lighting units, wherein each light setting is mapped onto a lighting unit based on the light setting’s respective color, saturation, intensity and / or temporal aspects and based on the lighting unit’s respective location relative to the location of the augmented reality device, and controlling the plurality of lighting units according to the remapped light settings when the augmented reality device is activated. The light settings are defined by the light scene, which light scene is applied to the plurality of lighting units according to the respective light settings as defined by the original mapping before the augmented reality device has been activated. The light scene may be a predefined light scene, which may for instance be activated based on a user input, based on a sensor input, based on a schedule lighting control routine, etc. The mapping of the light settings onto the plurality of lighting units is adjusted based the locations of the lighting units relative to the augmented reality device and based on the color, saturation, intensity and / or temporal aspects of the light settings. The mapping may be based on image quality requirements of images captured by a camera of the augmented reality device, based on the content to be rendered as an overlay on a display of the augmented reality device, etc. Each light setting may be associated with an influence value indicating how the light setting influences (the quality of) the view of the augmented reality device. The light settings may be mapped onto the lighting units based on the respective influence value such that when the lighting units are controlled according to the light settings, the quality the view of the augmented reality device is improved or optimized. If, for example, a user would select a light scene comprising the plurality of light settings (e.g. three light settings for three lighting units), the light settings would be mapped onto the lighting units based on the locations of the lamps relative to the augmented reality device, and based on the influence of the respective light spectra of the respective light settings on the view of the augmented reality device. Advantageously, the quality of the view of an augmented reality device is improved. Additionally, if another user (e.g. not operating an augmented reality device) is in the same space, the other user will still be able to experience the light scene, and a change of the lighting ambience in the space is kept to a minimum.

[0011] The method may comprise: determining the location of the augmented reality device relative to the plurality of lighting units by determining locations of the plurality of lighting units relative to a field of view of the augmented reality device based on the location information, and the adjustment of the original mapping may be determined based on respective locations of the plurality of lighting units relative to the field of view of the augmented reality device. For instance, the location information may comprise the location (and, optionally, the orientation) of the augmented reality device relative to the space, and the field of view of the augmented reality device may be determined based on the location (and, optionally, the orientation) of the augmented reality device relative to the space.

[0012] The method may further comprise: determining distances between the augmented reality device and the plurality of lighting units based on the location information, and the adjustment of the original mapping may be further based on the distances between the augmented reality device and the plurality of lighting units. The mapping may, for example, be performed such that one or more light settings of which the light spectrum positively affects the quality of the view of the augmented reality device are mapped onto one or more first lighting units of which the light effect is in closer proximity to the augmented reality device compared to one or more second lighting units. The light emission of the lighting units in closer proximity of the augmented reality device may have a larger influence on the view of the augmented reality device. Taking the distance between the augmented reality device and the lighting units (and / or their light effects) into account is beneficial, because it further improves the quality of the view of the augmented reality device.

[0013] The location of the augmented reality device may be predefined or user- defined, or the location of the augmented reality device may be obtained by detecting a current location of the augmented reality device. The predefined location (and, optionally, the orientation of the augmented reality device) may, for example, have been defined by a user. The user may have provided information about a predefined / frequently used location (and / or orientation) of the augmented reality device on a map of the space. In another example, the predefined / frequently used location may be derived from the map of the space (e.g. form a building information model, a user-created map, etc.).

[0014] Additionally or alternatively, the location (and optionally the orientation) of the augmented reality device may be obtained by detecting a current location (and optionally the orientation) of the augmented reality device. The system may also monitor the location and / or orientation of the augmented reality device over time, and determine a typical or frequent location and / or orientation of the augmented reality device based on the monitoring. The location and / or the orientation may, for example, be detected by an (indoor) positioning system, for instance based on signal characteristics of signals transmitted between one or more of the lighting units and the augmented reality device. Additionally, the method may further comprise: repeatedly detecting the current location and / or orientation of the augmented reality device, and remapping the plurality of light settings onto the plurality of lighting units if a difference between a new location and / or orientation of the augmented reality device and a previous location and / or orientation of the augmented reality device exceeds a threshold. Hence, the mapping may only be adjusted if the augmented reality device is moved towards a substantially different location or is orientated in a substantially different direction. This is beneficial, because the (re)mapping does not need to be performed constantly.

[0015] The method may further comprise: determining a current mode of operation of the augmented reality device, and controlling the plurality of lighting units according to the original mapping or according to the adjusted mapping in dependence on the current mode of operation of the augmented reality device. The current mode of operation may be an application running on the augmented reality device, a setting of the augmented reality device, an AR mode or a VR mode of the augmented reality device, etc. For some modes, the adjusted mapping may not be required, and the lighting units may then be controlled according to the original mapping.

[0016] The augmented reality device may be a video see-through augmented reality device, and the method may comprise: determining if a see-through mode of the augmented reality device is switched on or off, and controlling the plurality of lighting units according to the original mapping if the see-through mode is switched off, and controlling the plurality of lighting units according to the adjusted mapping if the see-through mode is switched on. Video see-through augmented reality devices typically have a see-through (AR) mode and a non-see-through (VR) mode. If the non-see-through mode has been activated, adjustment of the mapping may not be required.

[0017] The method may further comprise: determining a current mode of operation of the augmented reality device, and determining the adjustment of the original mapping further based on the current mode of operation of the augmented reality device. Different modes of operation may have different lighting requirements. Hence, it is beneficial to determine the adjustment of the mapping based thereon. The mode of operation may be indicative of an application running on the augmented reality device, and the method may comprise: determining the adjustment of the original mapping further based on the application running on the augmented reality device.

[0018] The method may further comprise: obtaining light rendering properties of the plurality of lighting units, and determining the adjustment of the original mapping further based on the light rendering properties of the plurality of lighting units. For instance, a colored light setting may be mapped onto a lighting unit configured to emit colored light and a white light setting may be mapped onto a lighting unit configured to emit white light.

[0019] The step of obtaining the location information indicative of the location of the augmented reality device relative to the plurality of lighting units may comprise: obtaining one or more images captured by a camera of the augmented reality device, analyzing the one or more images to extract the locations of the plurality of lighting units relative to the camera. One or more images may thus be used to determine the locations of the lighting units relative to the camera (and therewith relative to the augmented reality device). The lighting units may be recognized in the field of view based on their light output (e.g. based on their color, intensity, based on a modulation of the light emission, etc.), or a dark-room calibration or white-room calibration may be executed, wherein one or more lighting units are sequentially switched on / off which is used to determine their locations relative to the camera of the augmented reality device, and therewith relative to the augmented reality device.

[0020] Additionally or alternatively, the step of obtaining the location information indicative of the location of the augmented reality device relative to the plurality of lighting units may comprise: obtaining first location information indicative of the locations of the plurality of lighting units relative to the space, obtaining second location information indicative of the location of the augmented reality device relative to the space, and determining the location of the augmented reality device in the space relative to the plurality of lighting units based on the first and second location information. The first and second location information may, for example, be received from an (indoor) positioning system.

[0021] The method further may further comprise: receiving an activation signal indicative of that the augmented reality device has been activated by a user, and performing the step of adjusting the original mapping when the augmented reality device has been activated by the user. This is beneficial, because lighting atmosphere remains unchanged because it is kept to the original mapping when the user is not operating the augmented reality device.

[0022] According to a second aspect, the object is achieved by a computer program product for a computing device, the computer program product comprising computer program code to perform any of the above-mentioned methods when the computer program product is run on a processing unit of the computing device.

[0023] According to a third aspect, the object is achieved by a controller for controlling a plurality of lighting units located in a space according to a light scene comprising a plurality of light settings, the controller comprising: an input interface configured to receive a signal indicative of that an augmented reality device has been activated, a processor configured to: obtain an original mapping of the light settings onto the plurality of lighting units, control the plurality of lighting units according to the respective light settings as defined by the original mapping, obtain location information indicative of a location of the augmented reality device in the space relative to the plurality of lighting units, adjust the original mapping of the light settings onto the plurality of lighting units by remapping the plurality of light settings onto the plurality of lighting units, wherein each light setting is mapped onto a lighting unit based on the light setting’s respective color, saturation, intensity and / or temporal aspects and based on the lighting unit’s respective location relative to the location of the augmented reality device, and control the plurality of lighting units according to the remapped light settings when the augmented reality device is activated.

[0024] According to a fourth aspect, the object is achieved by a lighting system comprising: the controller, a plurality of lighting units and a communication unit configured to communicate lighting control commands to the plurality of lighting units.

[0025] It should be understood that the computer program product, the controller and the lighting system may have similar and / or identical embodiments and advantages as the above-mentioned methods.

[0026] BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The above, as well as additional objects, features and advantages of the disclosed systems, devices and methods will be better understood through the following illustrative and non-limiting detailed description of embodiments of devices and methods, with reference to the appended drawings, in which:

[0028] Fig. 1 shows schematically an example of a lighting system comprising a controller for controlling a plurality of lighting units;

[0029] Figs. 2 shows schematically an example of a lighting system comprising a plurality of lighting units in a space that are controlled based on their location relative to an augmented reality device;

[0030] Fig. 3 shows schematically an example of a field of view of a camera of an augmented reality device; and

[0031] Fig. 4 shows schematically a method of controlling a plurality of lighting units located in a space according to a light scene. All the figures are schematic, not necessarily to scale, and generally only show parts which are necessary in order to elucidate the invention, wherein other parts may be omitted or merely suggested.

[0032] DETAILED DESCRIPTION

[0033] Fig. 1 shows schematically an example of a lighting system 100 comprising a controller 102 for controlling a plurality of lighting units 112, 114 in a space 130. The controller 102 comprises a processor 106 (e.g. a microcontroller, circuitry, a microchip, etc.) configured to obtain location information indicative of a location of an augmented reality device 120 in the space 130 relative to the plurality of lighting units 112, 114, map the plurality of light settings onto the plurality of lighting units 112, 114, wherein each light setting is mapped onto a respective lighting unit based on the light setting’s respective color, saturation, intensity and / or temporal aspects and based on the lighting unit’s respective location relative to the location of the augmented reality device 120, and control the plurality of lighting units 112, 114 according to the mapped light settings.

[0034] The processor 106 is configured to obtain an original mapping of the light settings onto the plurality of lighting units (according to which the lighting units are controlled before the augmented reality device has been activated), and adjust the original mapping of the light settings onto the plurality of lighting units 112, 114 by remapping the plurality of light settings onto the plurality of lighting units 112, 114, wherein each light setting is mapped onto a lighting unit based on the light setting’s respective color, saturation, intensity and / or temporal aspects and based on the lighting unit’s respective location relative to the location of the augmented reality device 120.

[0035] The lighting units 112, 114 comprise one or more (LED) light sources. The lighting units 112, 114 may be light bulbs, light strips, TLEDs, light tiles, etc. The lighting units 112, 114 may comprise one or more individually (LED) light sources. The lighting units 112, 114 may be individually controllable light sources of a luminaire (e.g. an LED strip, a ceiling panel, a chandelier, etc.). The lighting units 112, 114 may comprise a control unit, such as a microcontroller (not shown), for controlling the light output generated by the one or more light sources based on received lighting control commands (which may be based on the generated light settings / light scene, which may be received from the controller 102). A lighting control command may comprise lighting control instructions for controlling the light output, such as the color, intensity, saturation, beam size, beam shape, etc. of the one or more light sources. The controller 102 may be comprised in any type of lighting control device. The controller 102 may, for example, be comprised in a mobile device (e.g. a smartphone, a wearable device, a (tablet) pc, etc.), in the augmented reality device 120, in a central lighting controller (e.g. a bridge, a router, a central home controller, a smart voice assistant, etc.), a remote server connected to the lighting units 112, 114 via a network / the internet, etc. The controller 102 may be configured to control the lighting units 112, 114. The controller 102 may comprise a communication unit 104 configured to communicate lighting control commands via any wired or wireless communication protocol (e.g. Ethernet, DALI, Bluetooth, Wi-Fi, Li-Fi, Thread, ZigBee, etc.) to the lighting units 112, 114, either directly or indirectly.

[0036] The augmented reality device 120 may be any type of augmented reality device (a head-worn augmented reality device, a portable augmented reality device such as a mobile phone, etc.). The augmented reality device 120 may comprise an augmented reality (AR) display. The AR display of the augmented reality device 120 may be configured to augment a view of the real environment 130 with virtual content. A processing unit coupled to or comprised in the AR display may, for example, render a virtual overlay on the view of the real environment with virtual content. The AR display may, for example, be a screen of a mobile device configured to render images captured by a camera located on the opposite side of the mobile device (as shown in Fig. 2). Alternatively, the display may be a see-through display (e.g. one or more eyeglasses) and the overlay may be projected onto the see-through display. Alternatively, the AR display may be a contact lens. Such AR displays are known in the art and will therefore not be discussed in detail.

[0037] The augmented reality device 120 may be activated by a user via a user interface, such as a button, a touch screen, a voice interface, etc. The use interface may provide the signal indicative of that the augmented reality device 120 has been activated. In examples wherein the augmented reality device 120 is a head-worn device, the augmented reality device 120 may comprise a sensor configured to detect when a user is wearing the augmented reality device 120. The sensor may provide the signal indicative of that the augmented reality device 120 has been activated.

[0038] The processor 106 is configured to control the plurality of lighting units 112, 114 according to the light scene. The light scene is defined as a plurality of predefined light settings for the plurality of lighting units 112, 114. The light settings may be mapped onto the plurality of lighting units 112, 114 according to an original (predefined) mapping and the lighting units are controlled according to the original mapping before the augmented reality device 120 has been activated. A user may, for example, select a light scene (which may be indicative of the plurality of light settings for the plurality of lighting units 112, 114) via a user interface (e.g. by providing a voice command to activate the light scene, by selecting the light scene via a touch-sensitive display, by activating the light scene by operating a light switch, etc.). Alternatively, the light scene may be activated when a sensor (e.g. a presence sensor, a light sensor, etc.) has been triggered. Alternatively, the light scene may be activated based on a lighting control routine or a scheduled light scene, which may be activated based on the time of day. The processor 106 may be further configured to receive an input indicative of an activation of the light scene, and map the plurality of light settings onto the plurality of lighting units 112, 114 (based on the light settings’ colors, saturation and / or intensity and based on the lighting units’ respective locations relative to the location (and orientation) of the camera 120) when the light scene is activated according to the original mapping.

[0039] The plurality of light settings of the light scene may be based on colors of one or more images, and the input may be indicative of a selection of an image. The image may, for example, be selected by a user via a user interface of a mobile device such as a smartphone. The colors may be extracted from the one or more images or be associated with the one or more images. The colors may be extracted from the image by analyzing color values of pixels or groups of pixels of the image. The extracted colors may, for example, be dominant colors from the image. Techniques for extracting colors from images are known in the art and will therefore not be discussed in detail.

[0040] The processor 106 is configured to obtain location information indicative of a location of the augmented realty device 120 relative to the plurality of lighting units 112, 114. The controller 102 may comprise an input interface for obtaining the location information. The input interface may be the communication unit 104, which may be configured to obtain the location information indicative of the location of the augmented realty device 120 relative to the plurality of lighting units 112, 114. Additionally or alternatively, the input interface 104 may be an input to the processor 106, and the processor 106 may obtain the location information from a memory 140, which may be comprised in the controller 102. Alternatively, the location information may be obtained (e.g. by the processor 106) by extracting the location information from one or more images captured by a camera of the augmented realty device 120. The processor 106 may, for example, be configured to obtain one or more images captured by a camera of the augmented realty device 120. The one or more images may be analyzed to extract the locations of the plurality of lighting units 112, 114 relative to the camera (and therewith relative to the augmented realty device 120). One or more images may thus be used to determine the locations of the lighting units relative to the camera 120. The lighting units may be recognized in the field of view based on their light output (e.g. based on their color, intensity, based on a modulation of the light emission, etc.), or a dark-room / white-room calibration may be executed, wherein one or more lighting units are sequentially switched on / off to determine their locations within the field of view of the camera based on said switching. Distances between the camera and the lighting units 112, 114 may be further determined based on the analysis of the image. Additionally, the camera may be a depth camera configured to provide depth information to the processor 106 to determine the distances between the camera of the augmented realty device 120 and the lighting units 112, 114. Alternatively, the augmented realty device 120 may comprise one or more depth sensors, and the processor 106 may be configured to determine the distances based on data from the depth sensors and images captured by the camera. Techniques for determining the location of a lighting unit relative to a field of view of a camera are known in the art and will therefore not be discussed in further detail. Fig. 3 illustrates an example wherein the processor 106 may receive one or more images form the camera 320 and analyze these to detect the locations of the lighting units 312-318 in the environment relative to the camera 320.

[0041] Additionally or alternatively, the processor 106 may be configured to obtain first location information indicative of the locations of the plurality of lighting units 112, 114 relative to the space 130 and to obtain second location information indicative of the location of the augmented realty device 120 relative to the space 130, and determine the location of the augmented realty device 120 in the space 120 relative to the plurality of lighting units 112, 114 based on the first and second location information. The first and second location information may, for example, be received from an (indoor) positioning system (such as an RF-based indoor positioning system or a visible light communication (VLC) based positioning system), it may be based on the signal strength of signals transmitted between one or more lighting units and the augmented realty device 120, etc. The first and second location information may be indicative of coordinates of the lighting units 112, 114 and the augmented realty device 120 relative to the space 130. Additionally, the location information may be indicative of the orientation of the augmented realty device 120 relative to the space 130. The orientation may for example be based on data from an orientation sensor comprised in the augmented realty device 120, based on a predetermined orientation of the augmented realty device 120 (e.g. defined by a user via a user interface), etc. Such techniques of obtaining location and / or orientation information are known in the art and will therefore not be discussed in further detail.

[0042] The processor 106 may be further configured to obtain information indicative of the field of view 240 of the augmented realty device 120, 220. The processor 106 may be configured to obtain information of the field of view (the angle of view) of the augmented realty device 120, 220 based on properties of the augmented reality device 120, 220 (e.g. the display size) and / or based on properties of a camera of the augmented reality device 120 (e.g. the field of view / focal length of the camera), 220. The processor 106 may be further configured to determine the locations of the plurality of lighting units 112, 114 with respect to the field of view 240 of the augmented reality device 120, 220. The processor 106 may be further configured to determine the mapping further based on the locations of the plurality of lighting units 112, 114 with respect to the field of view 240.

[0043] The processor 106 is further configured to adjust the original mapping of the plurality of light settings onto the plurality of lighting units 112, 114 (to create an adjusted mapping), wherein each light setting is mapped onto a respective lighting unit based on the light setting’s respective color, saturation, intensity and / or temporal aspects and based on the lighting unit’s respective location relative to the location (and, optionally, orientation) of the augmented reality device 120, 220. The processor 106 is configured to control the plurality of lighting units 112, 114 according to the adjusted mapping when / at the moment that the augmented reality device 120 is activated. Fig. 2 illustrates an example of a mapping of a light scene 250 onto a plurality of lighting units 212, 214, 216, 218. The processor 106 may receive light scene information of the light scene 250 comprising a plurality of light settings cl, c2, c3, c4 that are to be mapped onto a plurality of lighting units 212, 214, 216, 218. The light settings may have been mapped onto the lighting units 212, 214, 216, 218 according to an original mapping (e.g. cl to lighting unit 218, c2 to lighting unit 216, c3 to lighting unit 214 and c4 to lighting unit 212). This original mapping may not be optimized for the augmented reality device 220, and the processor 106 may therefore determine an adjusted mapping t, for example, improve the view of the augmented reality device 120. The processor 106 may further receive location information indicative of the locations of the plurality of lighting units 212, 214, 216, 218 relative to a location and / or an orientation of the augmented reality device 220 in the space 230, for instance from an (indoor) positioning system. The processor 106 may further receive the color, saturation, intensity and / or temporal aspects of the light settings. In this example, the light settings may be a white light setting (cl), a desaturated blue light setting (c2), a blue light setting (c3) and a purple light setting (c4). The processor 106 may determine the mapping of the plurality of light settings cl, c2, c3, c4 onto the plurality of lighting units 212, 214, 216, 218 based on their location and color, saturation, intensity and / or temporal aspects, resulting in that light setting cl may be mapped to a lighting unit in front of the augmented reality device 220 (i.e. on lighting unit 212), that light setting c2 may be mapped to a lighting unit in the peripheral view of the augmented reality device 220 (i.e. on lighting unit 216) and that that light settings c3 and c4 may be mapped to lighting units outside the field of view of the augmented reality device 220 (i.e. on lighting units 214, 218).

[0044] In the example of Fig. 2, the light scene comprises 4 light settings cl-c4, and the lighting system 100 comprises 4 lighting units 212-218. It should be understood that this is merely an example. In another example, the light scene may comprise a number of light settings (e.g. 5 light settings) higher than the number of lighting devices (e.g. 3) in the lighting system. The original mapping may define a first subset of light settings (e.g. a first subset of 3 light settings) and the processor 106 may control the lighting devices according to the first subset of light settings. When adjusting the original mapping, the processor 106 may select a second subset (different from the first subset) and map these onto the lighting devices unit based on the light setting’s respective color, saturation, intensity and / or temporal aspects and based on the lighting unit’s respective location relative to the location of the augmented reality device 120. If another user (e.g. not operating an augmented reality device) is in the same space, the other user will still be able to experience the light scene, and a change of the lighting ambience in the space is kept to a minimum.

[0045] Fig. 3 shows another example of a mapping of a light scene 350 onto a plurality of lighting units 312, 314, 316, 318. The processor 106 may receive light scene information of the light scene 350 comprising a plurality of light settings cl, c2, c3, c4 that are to be mapped onto a plurality of lighting units 312, 314, 316, 318. The light settings may have been mapped onto the lighting units 312, 314, 316, 318 according to an original mapping (e.g. cl to lighting unit 318, c2 to lighting unit 316, c3 to lighting unit 314 and c4 to lighting unit 312). The processor 106 may further receive location information indicative of the locations of the plurality of lighting units 312, 314, 316, 318 relative to a location and / or an orientation of the augmented reality device 320 in the space 330. In this example, the augmented reality device 320 may comprise a camera configured to capture an image of the space 330, and the locations of the lighting unis 312, 314, 316, 318 relative to the camera may be determined by analyzing the image, for instance by the processor 106. The processor 106 may further receive the color, saturation, intensity and / or temporal aspects of the light settings. In this example, the light settings may be a white light setting (cl), a desaturated red light setting (c2), a red light setting (c3) and a blue light setting (c4). The processor 106 may determine the mapping of the plurality of light settings cl, c2, c3, c4 onto the plurality of lighting units 312, 314, 316, 318 based on their location and color, saturation, intensity and / or temporal aspects, resulting in that light setting cl may be mapped to a lighting unit in front of the camera (i.e. on lighting unit 312), that light setting c2 may be mapped to a lighting unit in the less central in the field of view of the camera (i.e. on lighting unit 318) and that that light settings c3 and c4 may be mapped to lighting units in the periphery of the field of view of the camera (i.e. on lighting units 314, 316).

[0046] In another example, the processor 106 may be configured to map light settings onto respective lighting units based on the light settings’ temporal aspects and based on the lighting units’ respective locations relative to the location (and, optionally, orientation) of the augmented reality device 120. The temporal aspects may be defined as the changes of the light setting over time. In other words, the light settings may be dynamic light settings that change over time. The processor 106 may be configured to determine the mapping based on a dynamicity level of the light settings, wherein the dynamicity level is indicative of a number of changes (e.g. of the color and / or intensity) of a respective light setting over time and / or a level of contrast of the changes (e.g. of the color and / or intensity) of a respective light setting over time. For example, a first light setting with a first (low) dynamicity level (e.g. a first (low) number of changes and / or a second (low) contrast of changes of the light setting) may be mapped onto a lighting device located in the center of the field of view of the augmented reality device 120, and a second light setting with a second (higher) dynamicity level (e.g. a second (low) number of changes and / or a second (higher) contrast of changes of the light setting) may be mapped onto a lighting device located in the periphery of the field of view or outside the field of view of the augmented reality device 120.

[0047] The processor 106 may be further configured to determine distances between the augmented reality device 120 and the plurality of lighting units 112, 114 based on the location information. The processor 106 may be further configured to determine the adjusted mapping further based on the distances between the augmented reality device 120 and the plurality of lighting units 112, 114. The distances may be determined based on the locations of the lighting units 112, 114 relative to the augmented reality device 120 and / or the space 130. If, for example, a first lighting unit is in closer proximity to augmented reality device 120 compared to a second lighting unit, a first light setting (e.g. a first light setting with a first (low) saturation and / or a first (low) brightness) may be mapped onto the first lighting unit 112 based the lighting unit’s distance to augmented reality device 120, and a second light setting (e.g. a second light setting with a higher saturation and / or brightness than the first light setting) may be mapped onto the second lighting unit 114 based on the second lighting unit’s distance to the camera.

[0048] The processor 106 is further configured to control the plurality of lighting units 112, 114 according to the remapped light settings light settings (as shown in Figs. 2 and 3). The controller 102 may comprise the communication unit 104 configured to communicate lighting control commands indicative of the light settings to the plurality of lighting units 112, 114. The processor 106 may be configured to control the control the plurality of lighting units 112, 114 by switching to the remapped light settings light settings instantly, or the processor 106 may be configured to control the plurality of lighting units 112, 114 by transitioning from the original mapping to the remapped light settings over a period of time (e.g. to provide a smooth transition).

[0049] The processor 106 may be configured to determine the adjusted mapping based on image quality requirements of the augmented reality device 120. Each light setting may be associated with an image influence value indicating how the light setting influences (the quality of) images (e.g. virtual overlay images) rendered on a display of the augmented reality device 120. The image influence values may be indicative of the influence respective light spectra of the respective light settings have on the images rendered on the augmented reality device 120. These associations may be stored in a look-up table (e.g. in memory 140 or in a remote memory), which may be accessible by the processor 106. The light settings may be mapped onto the lighting units based on the respective image influence value such that when the lighting units are controlled according to the light settings, the quality of the images rendered on the augmented reality device 120 are improved. If, for example, a user would select a light scene comprising the plurality of light settings (e.g. four light settings for four lighting units, as illustrated in Figs. 2 and 3), the light settings would be remapped onto the lighting units based on the locations of the lighting units relative to the augmented reality device 120and based on the image influence values.

[0050] The processor 106 may be further configured to determine a current mode of operation of the augmented reality device 120, and control the plurality of lighting units 112, 114 according to the original mapping or according to the adjusted mapping in dependence on the mode of operation of the augmented reality device 120. The processor 106 may be configured to receive an input signal indicative of the current mode of operation of the augmented reality device 120 (e.g. from the augmented reality device 120, from a central (home) control system, form a software application, etc.). Different modes of operation may require different illumination, and the mapping of the light scene may therefore be determined based on the current mode of operation. For instance, if a certain application is running on the augmented reality device 120 (e.g. a messaging application), the plurality of lighting units may be controlled according to the original mapping, because no dedicated illumination of the space if required, while if, for example, another application is running on the augmented reality device 120 (e.g. an AR videogame), the plurality of lighting units may be controlled according to the adjusted mapping, because the other application requires optimization of the illumination in the space (e.g. to create contrast between the AR content and the environment, or to create a certain atmosphere that matches with the AR content). In another example, the augmented reality device 120 may be a video see-through augmented reality device the first mode of operation may be a see-through mode, and the second mode of operation may be a non-see-through mode. For the non-see-through mode, an adjusted mapping may not be required, and the processor 106 may control the plurality of lighting units 112, 114 according to the original mapping.

[0051] The processor 106 may be further configured to determine the mapping of the light settings onto the plurality of lighting units 112, 114 based on the current mode of operation of the augmented reality device 120. The processor 106 may be configured to determine the current mode of operation of the augmented reality device 120, for instance based on an input signal indicative of the current mode of operation. The input signal may for example be received from the augmented reality device 120, from a central (home) control system, form a software application, etc. For instance, if the augmented reality device 120 is set to a first mode wherein the user operating the augmented reality device 120 is playing an augmented reality game, the quality requirements of the renderings on the display of the augmented reality device 120 may be higher compared to a second mode wherein the user is reading a text message from a friend. The processor 106 may then to determine the adjusted mapping for the first mode different from the second mode.

[0052] The processor 106 may be configured to obtaining light rendering properties of the plurality of lighting units 112, 114, and determine the adjustment of the original mapping further based on the light rendering properties of the plurality of lighting units. For instance, a colored light setting may be mapped onto a lighting unit configured to emit colored light and a white light setting may be mapped onto a lighting unit configured to emit white light. The processor 106 may be further configured to obtain one or more images captured by a camera of the augmented reality device 120, and to analyze the one or more images to extract one or more image quality parameters of the one or more images based on the analyzed one or more images. The processor 106 may be further configured to adjust the mapping of the plurality of light settings onto the plurality of lighting units based on the one or more image quality parameters of the one or more images. The processor 106 may be configured to iteratively repeat these steps until a target image quality of the one or more images is achieved. If, for example, the one or more images are (partially) overexposed due to light emitted by to a lighting unit onto which a bright light setting has been mapped, the processor 106 may adjust the mapping such that a different (e.g. a less bright) light setting is mapped onto that lighting unit. If, for example, the one or more images are (partially) underexposed, the processor 106 may adjust the mapping such that a different (e.g. a brighter) light setting is mapped onto a lighting unit in the field of view of the camera 120. If, for example, the one or more images are colored due to a colored light setting mapped onto a lighting unit, the processor 106 may adjust the mapping such that a different (e.g. desaturated or different colored) light setting is mapped onto that lighting unit.

[0053] Fig. 4 shows schematically a method 400 of controlling a plurality of lighting units located in a space according to a light scene comprising a plurality of light settings. The method comprises: obtaining 402 an original mapping of the light settings onto the plurality of lighting units, controlling 404 the plurality of lighting units according to the respective light settings as defined by the original mapping, receiving 406 a signal indicative of that an augmented reality device has been activated, obtaining 408 location information indicative of a location of the augmented reality device in the space relative to the plurality of lighting units, adjusting 410 the original mapping of the light settings onto the plurality of lighting units by remapping the plurality of light settings onto the plurality of lighting units, wherein each light setting is mapped onto a lighting unit based on the light setting’s respective color, saturation, intensity and / or temporal aspects and based on the lighting unit’s respective location relative to the location of the augmented reality device, and controlling 412 the plurality of lighting units according to the remapped light settings when the augmented reality device is activated. The method 400 may be executed by computer program code of a computer program product when the computer program product is run on a processing unit of a computing device, such as the processor 106 of the controller 102.

[0054] It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims.

[0055] In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. Use of the verb "comprise" and its conjugations does not exclude the presence of elements or steps other than those stated in a claim. The article "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer or processing unit. In the device claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

[0056] Aspects of the invention may be implemented in a computer program product, which may be a collection of computer program instructions stored on a computer readable storage device which may be executed by a computer. The instructions of the present invention may be in any interpretable or executable code mechanism, including but not limited to scripts, interpretable programs, dynamic link libraries (DLLs) or Java classes. The instructions can be provided as complete executable programs, partial executable programs, as modifications to existing programs (e.g. updates) or extensions for existing programs (e.g. plugins). Moreover, parts of the processing of the present invention may be distributed over multiple computers or processors or even the ‘cloud’.

[0057] Storage media suitable for storing computer program instructions include all forms of nonvolatile memory, including but not limited to EPROM, EEPROM and flash memory devices, magnetic disks such as the internal and external hard disk drives, removable disks and CD-ROM disks. The computer program product may be distributed on such a storage medium, or may be offered for download through HTTP, FTP, email or through a server connected to a network such as the Internet.

Claims

CLAIMS:

1. A method (400) of controlling a plurality of lighting units located in a space according to a light scene comprising a plurality of light settings, the method comprising: obtaining (402) an original mapping of the light settings onto the plurality of lighting units, controlling (404) the plurality of lighting units according to the respective light settings as defined by the original mapping, receiving (406) a signal indicative of that an augmented reality device has been activated, obtaining (408) location information indicative of a location of the augmented reality device in the space relative to the plurality of lighting units, adjusting (410) the original mapping of the light settings onto the plurality of lighting units by remapping the plurality of light settings onto the plurality of lighting units, wherein each light setting is mapped onto a lighting unit based on the light setting’s respective color, saturation, intensity and / or temporal aspects and based on the lighting unit’s respective location relative to the location of the augmented reality device, and controlling (412) the plurality of lighting units according to the remapped light settings when the augmented reality device is activated.

2. The method (400) of any preceding claim, wherein the method comprises: determining the location of the augmented reality device relative to the plurality of lighting units by determining locations of the plurality of lighting units relative to a field of view of the augmented reality device based on the location information, wherein the adjustment of the original mapping is determined based on respective locations of the plurality of lighting units relative to the field of view of the augmented reality device.

3. The method (400) of any preceding claim, wherein the method further comprises: determining distances between the augmented reality device and the plurality of lighting units based on the location information, and wherein adjustment of the originalmapping is further based on the distances between the augmented reality device and the plurality of lighting units.

4. The method (400) of any preceding claim, wherein the location of the augmented reality device is predefined or user-defined, or wherein the location of the augmented reality device is obtained by detecting a current location of the augmented reality device.

5. The method (400) of any preceding claim, wherein the augmented reality device is a video see-through augmented reality device, and wherein the method comprises: determining if a see-through mode of the augmented reality device is switched on or off, and controlling the plurality of lighting units according to the original mapping if the see-through mode is switched off, and controlling the plurality of lighting units according to the adjusted mapping if the see-through mode is switched on.

6. The method (400) of any preceding claim, wherein the method further comprises: determining a current mode of operation of the augmented reality device, and determining the adjustment of the original mapping further based on the current mode of operation of the augmented reality device.

7. The method (400) of claim 6, wherein the mode of operation is indicative of an application running on the augmented reality device, and wherein the method comprises: determining the adjustment of the original mapping further based on the application running on the augmented reality device.

8. The method (400) of any preceding claim, further comprising: obtaining light rendering properties of the plurality of lighting units, determining the adjustment of the original mapping further based on the light rendering properties of the plurality of lighting units.

9. The method (400) of any preceding claim, wherein the step of obtaining the location information indicative of the location of the augmented reality device relative to the plurality of lighting units comprises: obtaining one or more images captured by a camera of the augmented reality device, analyzing the one or more images to extract the locations of the plurality of lighting units relative to the camera.

10. The method (400) of any preceding claim, wherein the step of obtaining the location information indicative of the location of the augmented reality device relative to the plurality of lighting units comprises: obtaining first location information indicative of the locations of the plurality of lighting units relative to the space, obtaining second location information indicative of the location of the augmented reality device relative to the space, and determining the location of the augmented reality device in the space relative to the plurality of lighting units based on the first and second location information.

11. The method (400) of any preceding claim, wherein the method further comprises: receiving an activation signal indicative of that the augmented reality device has been activated by a user, and performing the step of adjusting the original mapping when the augmented reality device has been activated by the user.

12. A computer program product for a computing device, the computer program product comprising computer program code to perform the method (400) of any of the preceding claims when the computer program product is run on a processing unit of the computing device.

13. A controller (102) for controlling a plurality of lighting units located in a space (130) according to a light scene comprising a plurality of light settings, the controller (102) comprising: an input interface (104) configured to receive a signal indicative of that an augmented reality device (120) has been activated, a processor (106) configured to:obtain an original mapping of the light settings onto the plurality of lighting units (112, 114), control the plurality of lighting units (112, 114) according to the respective light settings as defined by the original mapping, obtain location information indicative of a location of the augmented reality device (120) in the space (130) relative to the plurality of lighting units (112, 114), adjust the original mapping of the light settings onto the plurality of lighting units (112, 114) by remapping the plurality of light settings onto the plurality of lighting units (112, 114), wherein each light setting is mapped onto a lighting unit based on the light setting’s respective color, saturation, intensity and / or temporal aspects and based on the lighting unit’s respective location relative to the location of the augmented reality device (120), and control the plurality of lighting units (112, 114) according to the remapped light settings when the augmented reality device (120) is activated.

14. A lighting system (100) comprising: the controller (102) of claim 13, a plurality of lighting units (112, 114), and a communication unit configured to communicate lighting control commands to the plurality of lighting units.

Images