Method for controlling lighting and lighting system for implementing said method

Abstract

A lighting control method is provided, comprising the steps of: - capturing image data of an area with a camera module at different times, - comparing information obtained from image data captured with the camera module at different times, and - activating lighting elements depending on the results of this comparison, whereby features are extracted from the image data captured with the camera module and the comparison is based exclusively on such features; as well as a lighting system with at least one camera module and at least one first processing unit for processing image data captured with the camera module, and with at least one lighting element that can be activated and / or deactivated depending on the results of the processing of image data captured with the camera module, wherein the lighting system is designed and configured.to carry out such a procedure.

Description

[0001] Automated lighting control tailored to the actual use of a room is an essential component of resource-efficient energy management, especially in work environments such as (open-plan) offices, workshops, and commercially used halls. This requires the provision of detectors or sensors that monitor the room for occupancy and, based on the monitoring results, provide only the necessary lighting.For example, a significant contribution to energy savings can be made if, in the case of an open-plan office where there is only one employee working overtime, the entire office is no longer illuminated but rather the desk is specifically illuminated, or if, during a night-time inspection tour of a machine operator through a large production hall with automated production, not the entire hall is illuminated but only the area around the machine operator's current position.

[0002] A well-known and proven approach for implementing such lighting controls for many years is the use of image information recorded with camera modules and evaluated for the presence of a person by means of image comparisons.

[0003] A key problem is that there can be a multitude of changes in an image of a spatial area to be monitored that are not related to the presence of a person. For example, a change in lighting conditions, caused, say, by a cloud obscuring the sun, will change the brightness values ​​of many pixels in the image without a person actually being present. If one attempts to detect a presence by comparing it with a static reference image taken under defined conditions without any presence, not only can an employee sitting at their desk activate the office lights, but also their briefcase left at their desk. If one attempts to detect a presence by detecting movement in the spatial area to be monitored, the movement of a houseplant in the airflow of the air conditioning or ventilation system can lead to false detection.

[0004] Of course, in each of these scenarios, it is easy to devise corrections to counteract false detections. Ambient light effects will normally affect large parts of the image in the same or similar way; the position of the forgotten briefcase will always be identical, and the movement of the houseplant will be limited to a narrow spatial area; one can also try to analyze the shape of a supposedly present presence and compare it with an image of a person. The problem with this, however, is that the already considerable computational effort required for such an evaluation continues to rise, especially since resolution requirements may also increase, and a frame rate of several frames per second is regularly desired for the camera module, which in turn increases the amount of image data to be analyzed.In addition, high evaluation speeds are required to ensure that the switching process to be triggered is perceived by the user as sufficiently fast and to avoid the user initially being left in the dark due to evaluation times that are too long.

[0005] The object of the invention is to provide an improved method for controlling lighting, as well as a lighting system for implementing the method. This object is achieved by a method having the features of patent claim 1 and a lighting system having the features of patent claim 11.

[0006] The method for controlling lighting according to the invention comprises at least the steps of recording image data of an area with a camera module at different times, comparing information obtained from image data recorded with the camera module at different times, and activating lighting elements depending on the results of this comparison.

[0007] Essential to the invention is that features are extracted from the image data captured by the camera module and that the comparison is carried out at least partly, preferably exclusively, on the basis of such features. Unlike previously, the comparison that leads to the decision regarding the switching of the lighting elements is no longer entirely pixel-based, but at least partially feature-based.

[0008] The feature space represents an abstraction of the problem essential for lighting control: whether a person is present in the surveillance area, so that the lighting must be activated. In the abstracted feature space, this problem can be solved more abstractly and often more simply and / or unambiguously due to the decoupling of a multitude of additional information initially contained in the image that is irrelevant to this question.

[0009] An abstract feature space also allows for a broader interpretation of the "movement" itself; based on the features, it is often easier to classify movements belonging to "objects" (namely people) that are important for lighting control.

[0010] The extraction of features from the image data captured by the camera module can be achieved in a variety of ways. For example, algorithms such as SIFT, SURF, or ORB can be used; however, it can also be achieved using deep learning methods, machine learning methods, or other algorithms.

[0011] In a preferred variant of the method, a feature space is formed using the extracted features, which is preferably adapted dynamically so that the addition or disappearance of features can be immediately taken into account.

[0012] The features extracted from current image data of an image captured with the camera module are preferably assigned a vector in the feature space. In a preferred development of this variant, a background vector in the feature space is furthermore formed from features extracted from the respective current image data of an image captured with the camera module across a plurality of images captured by the camera module. Other aspects can also be taken into account when creating the background vector, for example, how stable the position of a feature remains, in order to distinguish between an inanimate object that is not moving and an animate object that remains virtually stationary, such as an employee working at a desk.

[0013] These feature vectors then make it possible, in particular, for the comparison of information obtained from image data recorded with the camera module at different times to include a difference formation or a more abstract derivation of the difference between the vector in the feature space formed by the features extracted from current image data of an image recorded with the camera module and the background vector, which is an operation that can normally be implemented with a very manageable computational effort.

[0014] In particular, the background vector can be updated in the feature space using the vector formed by the features extracted from current image data of an image captured with the camera module, for example, through weighted addition or other weighted adjustment. In this way, newly added inanimate objects, such as a package placed on a desk by a courier, can be integrated into the background instead of constantly feigning a presence.

[0015] In a particularly advantageous variant of the method, features are extracted from the image data recorded with the camera module using a first computing unit, and the comparison is made on the basis of these features using a second computing unit that is spatially separate from the first computing unit. Logically, the extracted features must then be transmitted via a communication interface from the first computing unit to the second computing unit, which is preferably done using wireless communication or a bus system. As already mentioned, however, as a consequence of the feature-based comparison - especially if only the features of the areas in which a change was detected are transmitted - the amount of data to be transmitted is manageable, even if the camera module records several images per second.

[0016] It is precisely the manageability of the data volume and the possibility of creating a much more cost-effective execution of feature-based comparisons in terms of computational effort compared to pixel-based comparisons that then also enable the second computing unit to be in signal communication with several first computing units, each of which extracts features from image data of different camera modules, so that the second computing unit carries out the feature-based comparisons for several or all camera units.

[0017] Accordingly, when using this variant of the method, detectors can be designed which essentially comprise a camera module, a first processing unit with memory which only has to be able to handle a reduced range of tasks, e.g. the extraction of features from the images taken by the camera module and, if necessary, also a first comparison, as well as a communication module for forwarding the extracted features to a central control module which is responsible for several or all detectors and which, with more computing power, can carry out a comparison if one has not yet been carried out and / or a more complex classification, which also has a corresponding communication module in addition to the second processing unit, and can therefore be implemented cost-effectively.

[0018] In such a configuration, it is also possible to consider the information obtained from multiple camera modules during the feature-based comparison of image data captured by the respective camera module at different times when activating a lighting element. This can be used, for example, to proactively activate a lighting unit in the area of ​​a second detector based on a feature-based detection of a presence in the area of ​​a first detector.

[0019] It can be advantageous if the central processing unit can process, or does process, features from multiple devices simultaneously. The central processing unit can also consider a common feature vector / space from all devices. This also makes it possible to concentrate computing operations on different device units accordingly.

[0020] The lighting system according to the invention has at least one camera module and at least one first computing unit for processing image data recorded with the camera module, as well as at least one lighting element that can be activated and / or deactivated depending on the results of the processing of image data recorded with the camera module. A computing unit is understood to be a CPU (central processing unit), an NPU (neural processing unit), or a TPU (tensor processing unit) along with the respective associated working memory and program memory. The lighting system is characterized in that the lighting system is designed and configured to carry out a method according to one of claims 1 to 10.Accordingly, the first computing unit is programmed in particular to extract features from image data recorded with the camera module, and either the first computing unit or another computing unit in the lighting system carries out a comparison of features extracted from such image data, the results of which then lead to the activation or deactivation of the lighting element, in particular by emitting corresponding control signals.

[0021] The lighting system preferably comprises multiple detectors. These detectors each have a camera module, a first processing unit designed and configured to extract features from image data captured by the camera module, and a communications module for transmitting extracted features to a central control unit. Accordingly, the memory size and computing power requirements of these detectors are not too high, and they can be manufactured relatively inexpensively.

[0022] Preferably, the central control unit then comprises a communications module for receiving extracted features from at least one detector, as well as a second computing unit designed and configured to compare features received from the at least one detector. This second computing unit can be much more powerful than the first computing units.

[0023] The invention is described below with reference to Figure 1 explained in more detail, which shows a flow diagram of a cycle of a method according to the invention.

[0024] In step S1, image acquisition takes place, i.e., the recording of image data using a camera module. This produces a current image of the room being monitored by the camera module.

[0025] In step S2, feature extraction from this current image takes place, which, as already mentioned above, can be realized using a variety of different methods such as SIFT, SURF or ORB or even with deep learning methods, but can also be based on other machine learning methods or apply other feature extraction algorithms.

[0026] The image information is reduced to the characteristics or features considered essential.

[0027] The extracted features of the current image, including position information regarding their position in the monitored space, are then stored in step S3 and further processed in feature space. This leaves the image plane, and further processing and evaluation of this image takes place only in a feature space, while the next set of camera image data can already be acquired at the image plane and subjected to feature extraction.

[0028] During further processing, the features stored in step S3 are compared with background features in step S4. In the application case of interest in this application, lighting control, it is typically changes in the relevant features of a scene that are intended to trigger control signals. While static features are identified and extracted, they are not intended to lead to a control signal. This can be achieved by comparing them with a suitably formed background vector in the feature space.

[0029] If the comparison in step S4 reveals significant changes in the recorded live features, the conditions that are appropriately defined for sending control signals to the lighting, the position of these changes is defined in step S5. In step S6, based on this position information, it is determined in which zones of the lighting system a reaction should occur before the corresponding control signal is sent in step S7.

[0030] Longer-term changes in the scene that are not associated with a presence to be detected can be taken into account by adapting the background features. To do this, in step S8, the current live feature vector can be downscaled using an adaptation factor and added to the current background feature vector, which is then saved as a new background feature vector in step S9. If a new feature added to the live feature vector proves to be static, so that it repeatedly appears in subsequent live feature vectors, its contributions will accumulate and it will gain full weight in the background feature vector over the course of several evaluation cycles. If, on the other hand, it is dynamic, it will appear changed in subsequent live feature vectors and statistically average out of the background feature vector. List of reference symbols

[0031] S1-S9Procedure steps

Claims

1. A method for controlling lighting comprising the steps of - recording image data of an area with a camera module at different times, - comparing information obtained from image data recorded with the camera module at different times, and - activating lighting elements depending on the results of this comparison, characterized in that Features are extracted from the image data recorded with the camera module and that the comparison is carried out at least also, preferably exclusively, on the basis of such features.

2. Method according to claim 1, characterized in that a feature space is formed using the extracted features.

3. Method according to claim 2, characterized in that the feature space is dynamically adapted.

4. Method according to claim 2 or 3, characterized in thatthe features extracted from current image data of an image captured with the camera module are assigned a vector in the feature space.

5. Method according to claim 4, characterized in that a background vector is formed in the feature space from features that are extracted from the respective current image data of an image taken with the camera module across a plurality of images taken by the camera module.

6. Method according to claim 5, characterized in that Comparing information obtained from image data recorded with the camera module at different times comprises forming a difference between the vector in the feature space formed by the features extracted from current image data of an image recorded with the camera module and the background vector.

7. Method according to claim 5 or 6, characterized in thatthe background vector is updated using the vector in the feature space formed by the features extracted from current image data of an image taken with the camera module.

8. Method according to one of the preceding claims, characterized in that the extraction of features from the image data recorded with the camera module is carried out by means of a first computing unit and that the comparison on the basis of these features is carried out by means of a second computing unit which is spatially separated from the first computing unit.

9. Method according to claim 8, characterized in that the second computing unit is in signal communication with a plurality of first computing units, each of which extracts features from image data of different camera modules, so that the second computing unit carries out the feature-based comparisons for several or all camera units.

10. Method according to claim 9, characterized in thatWhen activating a lighting element, the information obtained from several camera modules during feature-based comparison of image data recorded with the respective camera module at different times is taken into account.

11. Lighting system with at least one camera module and at least one first processing unit for processing image data recorded with the camera module, as well as with at least one lighting element that can be activated and / or deactivated depending on the results of the processing of image data recorded with the camera module, characterized in that the lighting system is designed and arranged to carry out a method according to one of claims 1 to 10.

12. Lighting system according to claim 11, characterized in thatthe lighting system comprises a plurality of detectors, each having a camera module, a first computing unit designed and configured to extract features from image data recorded with the camera module, and a communication module for transmitting extracted features to a central control unit.

13. Lighting system according to claim 12, characterized in that the central control unit has a communication module for receiving extracted features from at least one detector and a second computing unit designed and configured to compare received features.

Images