Method for controlling infrared illumination source

Abstract

To provide a method and a video capturing system for controlling an infrared illumination source illuminating an area captured by a camera.SOLUTION: A method comprises the steps of: modulating emission from an infrared illumination source between a first emission intensity and a second emission intensity at a first frequency during a first prescribed period; upon capturing images using an image sensor of a camera during a first predetermined time period, determining a radiation level indicator for each one of a plurality of the images captured by the image sensor; evaluating a sequence of determined radiation level indicators to determine whether a frequency resulting from the modulated emission at the first frequency is detected or not in the sequence of radiation level indicators; and If the evaluation results in the frequency resulting from the modulated emission at the first frequency being determined as not detected in the sequence of radiation level indicators, stopping the infrared illumination source from illuminating the area captured by the camera.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to the control of infrared light-emitting diodes, and more particularly to the control of infrared light-emitting diodes used to illuminate an area captured by a camera.

Background Art

[0002] Cameras for capturing video, i.e., video, are often used for observing and / or monitoring areas of interest. These areas of interest can, over time, be exposed to change lighting conditions over 24 hours, especially when the scene being captured is daylight during part of the 24-hour span and low light to no light during another part of the 24-hour span, i.e., at night. To enable a camera to capture videos related to both daytime and nighttime, many cameras today are provided with at least two different capture modes, i.e., a day mode and a night mode.

[0003] The image sensor of a camera is sensitive to light in the infrared spectrum, at least light in the near-infrared spectrum, and an infrared cut filter positioned in the optical path between the lens and the image sensor stops infrared light from reaching the image sensor in order to reproduce the colors of the captured image visible to the human eye. However, at low illuminance levels, such as at night, since all light detectable by the image sensor contributes to the image quality, the infrared cut filter is removed from the optical path to enable infrared light to be detected by the image sensor. By utilizing this light in the low-frequency spectrum, the usefulness of the camera is extended to low-illuminance environments as well. Low-illuminance images from a camera can be enhanced by illuminating the captured scene using infrared light-emitting diodes. Illuminating the scene using such infrared light-emitting diodes improves the ability of the camera to deliver high-quality images using infrared light and / or near-infrared light. The infrared light and / or near-infrared light from such illumination is not visible to the human eye, but an image sensor without an infrared cut filter detects it.

[0004] However, the light-emitting diode should be active only when needed, such as when the area captured by a camera is experiencing low illumination, and the infrared cut-off filter should be removed from the optical path in low-illumination situations and reinserted into the optical path when the lighting conditions improve. The activation and deactivation of the light-emitting diode and the removal and reinsertion of the infrared cut-off filter depend on whether there is sufficient light in the scene.

[0005] European Patent Application Publication No. 3886539 (Axis AB) of September 29, 2021 describes a method for evaluating the ambient light level during video acquisition in a video camera to control the positioning of an infrared cut-off filter for removing or not removing infrared light. The method includes capturing a stream of images when the scene is illuminated by an IR illuminator having a first output level and then capturing a stream of images when the scene is illuminated by an IR illuminator having a reduced output level. These two streams of images are then evaluated for a measure representing the ambient light level. The method provides an excellent way to determine when to switch between night mode and day mode in a camera, but there are situations and embodiments where it can be difficult to appropriately evaluate these streams, for example, due to the level of noise components in the detected signal or because the reduced level is not recognized as being sufficiently reduced to trigger detection.

[0006] From the above, it is understood that there is room for a solution that further facilitates the determination of switching additional illumination using an IR illuminator and / or removing the infrared cut-off filter from the optical path when it is no longer needed. SUMMARY OF THE INVENTION

[0007] The present invention is defined by the appended independent claims. Additional features and advantages of the concepts disclosed herein are described in the following description, become apparent in part from the description, or may be learned by the practice of the described technology. The features and advantages of the concepts may be realized and obtained by means of the instrumentalities and combinations particularly pointed out in the appended claims. These and other features of the described technology will become more fully apparent from the following description and appended claims, or may be learned by the practice of the disclosed concepts described herein.

[0008] In a first aspect, a method for controlling at least one infrared illumination source, an IR illumination source, for illuminating an area captured by a camera is to modulate the light emission from the IR illumination source between a first emission intensity and a second emission intensity at a first frequency during a first predetermined period, capture an image using the image sensor of the camera during the first predetermined period, determine a radiation level indicator for each of a plurality of images captured by the image sensor, evaluate a sequence of the determined radiation level indicators to determine whether a frequency resulting from the light emission modulated at the first frequency is detected in the sequence of the radiation level indicators, and if it is determined that the evaluation result at the frequency resulting from the light emission modulated at the first frequency is not detected in the sequence of the radiation level indicators, the IR illumination source stops illuminating the area captured by the camera. One advantage of this method is that it is a simple, robust and effective way to control the IR illumination source that illuminates the scene captured by the camera. Another advantage is that the method enables the IR illumination source to be turned off when illumination is no longer required, thereby enabling energy savings.

[0009] In some embodiments, the evaluation of the sequence of the radiation level indicators includes applying a transformation to the sequence of the radiation level indicators to transform the sequence of the radiation level indicators into the frequency domain. The use of frequency transformation facilitates the evaluation of the received radiation and is simple to implement and operate.

[0010] In some further embodiments, the detection or non-detection determination of the frequency resulting from the modulated emission at the first frequency in the sequence of radiation level indicators includes determining non-detection of the frequency resulting from the modulated emission at the first frequency if the converted sequence of the radiation level indicators yields a value of the frequency resulting from the modulated emission at a first frequency that is below a threshold.

[0011] In some embodiments, the second emission intensity is 50 to 95 percent of the first emission intensity of the IR illumination source, and in some embodiments, the first predetermined period during which the IR illumination source is modulated is from 1 second to 10 minutes.

[0012] In some further embodiments, the determination of the radiation level indicator of the captured image includes determining the upper quartile luminance value of the region of the captured image. One advantage of this feature is that it can facilitate the detection of fluctuations in the introduced illumination when the focus of the detection shifts towards fluctuations at higher detected radiation intensities.

[0013] In some embodiments, the method also includes generating a signal to instruct the camera to insert an IR filter into the optical path leading to the image sensor of the camera if it is determined that the evaluation result at the frequency resulting from the modulated emission at the first frequency is not detected in the sequence of radiation level indicators.

[0014] In some additional embodiments, the method includes operating the IR illumination source in a normal mode during a delay period if it is determined that the evaluation result at the frequency resulting from the modulated emission at the first frequency is detected in the sequence of radiation level indicators, where the normal mode is such that the IR illumination source is operated as it was operated before the first period during which the IR illumination source was modulated, before the test process is restarted again. One advantage of this procedure is that the method can detect the switch from bad environmental lighting conditions to acceptable environmental lighting conditions without continuously modulating the emission from the IR illumination source.

[0015] In some further embodiments, the control method includes at least a two - step process where the process of stopping the IR illumination source from illuminating the area captured by the camera is related to the final step, and the initial steps include modulating the light emission from the IR illumination source between a first initial emission intensity and a second initial emission intensity at a second frequency during a second predetermined period, capturing an image using the camera during the second predetermined period, determining a radiation level indicator for each of the plurality of captured images, creating a sequence of initial radiation level indicators including the radiation level indicators determined from the plurality of captured images, and evaluating the sequence of initial radiation level indicators to determine whether a frequency resulting from the light emission modulated at the second frequency is detected in the sequence of initial radiation level indicators. If it is determined that the evaluation result at the frequency resulting from the light emission modulated at the second frequency is not detected in the sequence of radiation level indicators, the next step in the at least two - step process is initiated. One advantage of this approach is that the modulated light emission from the IR illumination source cannot be interfering and / or obstructive to the actual scene or a user looking at the scene captured by the camera. One reason for this is that under lighting conditions where the variations can be very significant, such as in darker ambient lighting conditions, the variations can be maintained at a low amplitude or level of variation.

[0016] In some embodiments, the first frequency and the second frequency are the same frequency. In some further embodiments, the control method is a two - step process, and the next step of the at least two - step process is the final step.

[0017] Some embodiments can include evaluating an image captured from a camera for movement within a region captured by the camera and initiating modulation of light emission from an IR illumination source when the movement evaluation indicates a movement value lower than a movement threshold. One advantage is that the risk of moving objects within the scene that can affect the correct detection of variations within the scene is reduced. Another advantage can be that the risk of IR modulation that can affect movement detection is reduced.

[0018] In some embodiments, evaluating a sequence of radiation level indicators determined to determine whether a frequency resulting from modulated light emission at a first frequency is detected in the sequence of radiation level indicators further includes determining whether the first frequency, or a frequency resulting from aliasing of the first frequency and the frame rate of the captured image, is detected in the sequence of radiation level indicators.

[0019] According to a second aspect, the method is executed in a video capture system configured for this purpose. The advantages described in relation to the method are applicable to corresponding features of the video capture system.

[0020] According to a third aspect, a computer-readable medium including instructions for causing a computer to execute the method when executed by the computer. Thus, the method can be implemented as computer-executable code that can be made available in various ways. The advantages described in relation to the method are applicable to corresponding features of the computer-readable medium.

[0021] The further scope of application of the present invention will become apparent from the detailed description given hereinafter. However, various changes and modifications within the scope of the present invention will be apparent to those skilled in the art from this detailed description, so it should be understood that the detailed description and specific examples, while indicating preferred embodiments of the present invention, are given by way of illustration only. Accordingly, it should be understood that the present invention is not limited to the specific components of the described apparatus or the steps of the described method, and that such apparatus and methods may vary. It should also be understood that the terms used herein are for the purpose of describing particular embodiments only and are not intended to be limiting. As used herein and in the appended claims, the articles "a", "an", "the", and "said" are intended to mean that there is one or more elements unless the context clearly indicates otherwise. Thus, for example, reference to "a sensor" or "the sensor" may include several sensors and the like. Further, "comprising" does not exclude other elements or steps.

[0022] To best explain how the above-described embodiments are implemented and to define other advantages and features of the present disclosure, a more specific description is provided below and shown in the accompanying drawings. It should be understood that these drawings show only exemplary embodiments of the present invention and should not be considered as limiting the scope, and the embodiments will be described and explained with additional specificity and detail using the accompanying drawings.

Brief Description of the Drawings

[0023]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

DETAILED DESCRIPTION OF THE INVENTION

[0024] Furthermore, in the drawings, like reference numerals indicate like or corresponding parts throughout several views.

[0025] Here, specific embodiments will be described more fully with reference to the accompanying drawings. It will be apparent to those skilled in the art that various modifications and variations can be made without departing from the concept of the present invention. Other embodiments will be apparent to those skilled in the art from consideration of the specification and the embodiments disclosed herein. The embodiments herein are provided by way of example so that the disclosure is thorough and complete, and to fully convey the scope of the concept of the present invention, and the claims are to be construed as including all equivalents of the concept of the present invention that are apparent to those skilled in the art to which the concept of the present invention pertains. Unless otherwise stated, different embodiments can be combined with each other.

[0026] The present invention relates to a method, an apparatus, and a system for controlling at least one infrared illumination source (hereinafter also referred to as an IR illumination source). The IR illumination source may be an infrared light emitting diode (hereinafter referred to as an IR LED), or a device including a plurality of IR LEDs arranged to emit IR light or near-IR light to an area of interest for capturing an image. The IR illumination source may be, or may include, other infrared light sources other than IR LEDs, such as IR laser diodes or other IR sources. However, for ease of understanding the following description, the term IR LED may be used throughout the description as a representation of both IR LEDs and such other emitters of infrared light.

[0027] Referring now to FIG. 1, an example of a setting including the present invention and / or a device implementing the present invention is shown. The camera 10 is arranged to capture an image of the area 12 within the scene 13 represented by the square 12 drawn with a dotted line in FIG. 1. In this description, the scene 13 should be interpreted as the surrounding view as seen from the position of the camera. The IR illumination source 14 is arranged to emit IR light towards the scene 13 in order to "illuminate" the area captured by the camera 10 during low light conditions. The area 16 that receives the radiation from the IR illumination source may, in some cases, be larger than the area 12 captured by the camera 10, and in some cases, smaller than the area 12 of the scene captured by the camera. The illuminated area can include the entire scene 13, a part of the scene 13, the area 12 of the scene 13 captured by the camera 10, or a part of the area 12 of the scene 13 captured by the camera. According to some embodiments, the area 16 that receives the IR radiation is represented by the square 16 drawn as a dashed line in FIG. 1. In the exemplary setting of FIG. 1, the camera 10 and the IR illumination source 14 are shown as separate devices, but in some embodiments, the IR illumination source 14 may be included in the housing of the camera 10 that emits IR light in the same direction as the camera is directed.

[0028] Camera 10 may be a camera that captures a video sequence or a continuous stream of video, i.e., captures a video sequence or a continuous video stream. Camera 10 may be a video camera 10 connected to a network of digital video cameras, and / or a video camera 10 connected to a computer network such as, for example, a LAN, WAN, the Internet, or a video camera 10 that is not network-connected but includes local storage for storing video and / or video captured by camera 10.

[0029] In some embodiments, camera 10 is a digital video camera 10 as shown in FIG. 2. The camera includes a lens 22 that focuses an image onto an image sensor 24 via the optical path of the camera. Camera 10 also includes an infrared cut filter 26 that is movable from a position within the optical path in front of the image sensor 24 to a position that does not interfere with the light guided through the optical path to the image sensor 24, i.e., from a position in front of the image sensor 24 to a position not in front of the image sensor 24. The infrared cut filter 26 is also arranged to move in the opposite direction, i.e., to return from a position that does not interfere with the light guided through the optical path to the image sensor 24 to a position within the optical path in front of the image sensor 24.

[0030] Camera 10 may also include an image processor 28 connected to the image sensor 24 for processing the image data generated by the image sensor 24. Alternatively, the image processor 28 may be disposed within a device (not shown) that is separate from the camera 10 and connected to the camera via a network or a direct communication line. In some embodiments, the image processor 28 is connected to a central processing unit CPU 30 that executes software code for controlling the positioning of the infrared cut filter 26 and software code for controlling the IR illumination source 14. In some embodiments, the CPU 30 transmits control instructions / signals to an actuator 32 physically connected to the infrared cut filter 26 to control the positioning of the infrared cut filter 26. In some embodiments, the CPU 30 transmits control instructions / signals to an interface 34 for controlling the IR illumination source 14, which is a device external to the camera. Further, the camera 10 includes a non-volatile memory 36 and a volatile memory 38, both of which are connected to the CPU 30 and the image processor 28 via a data bus 40. The non-volatile memory 36 can store programs related to the camera 10 and operations associated with the camera 10, such as program code for controlling the IR illumination source 14. The control of the IR illumination source can include, for example, precise control of the emission intensity of the IR illumination source and / or switching between on and off states. The control of the IR illumination source may also be synchronized with the exposure interval of the image sensor. The non-volatile memory 36 may be, for example, a read-only memory (ROM), PROM, EPROM, EEPROM, mask ROM, flash memory, ferroelectric RAM, magnetoresistive RAM, phase change RAM, FeFET memory, RRAM memory, etc. The volatile memory 38 may operate as the working memory of the processors 28, 30 of the camera 10 and may be, for example, a random access memory (RAM), SRAM, or DRAM. Further, the camera can include a network interface 42 for connecting the camera 10 to a computer network as described above.

[0031] FIG. 3 shows a camera according to an alternative embodiment. One difference between the camera 10 of FIG. 3 and the camera of FIG. 2 is that the IR illumination source 14 is included as part of the camera. As a result of the short distance between the IR illumination source and the lens, the IR illumination source is arranged to emit IR in a direction that coincides with the direction in which the lens of the camera 10 captures an image in order to illuminate the area captured by the camera 12 with IR radiation.

[0032] When the infrared cut-off filter 26 is in a position outside the optical path, that is, when it does not remove infrared light before the light / radiation reaches the image sensor 24, the image sensor 24 detects both visible light and infrared light, at least near-infrared light, which is generally considered to refer to wavelengths in the range of 800 to 2500 nm. The human eye is generally considered to be sensitive to light in the wavelength range of 380 nm to 800 nm. However, a typical image sensor 24 today without an infrared cut-off filter may be able to detect wavelengths up to 1100 nm. Therefore, an image sensor 24 without a filter that removes infrared radiation detects electromagnetic waves having wavelengths longer than 800 nm.

[0033] A radiation level indicator, e.g., a value indicating radiation received by a camera and / or sensor, can be determined within the camera by either the image processor 28 or the CPU 30. The radiation level indicator is a value registered by an image sensor that indicates the level of radiation reflected from the scene 12 to the image sensor 24, i.e., the levels of visible light and infrared light. In some embodiments, this value is calculated from all the pixels of the image sensor 24 or from the pixels of a limited area on the image sensor 24. Such calculations can include calculating the average intensity received within a pixel, the median intensity received within a pixel, the upper percentile intensity received within a pixel, e.g., the upper quartile, the sum of pixel values, etc. for the captured image frame or the limited pixel area. The pixels used in the calculation can be pixels representing all three color channels, i.e., red, green, and blue, a combination of two color channels, or only one of the channels, e.g., the red channel. In some embodiments, the value representing the radiation level indicator can include values already generated by the camera for other purposes, such as exposure values, gain, exposure time, etc. In some cameras, the value of the luminance of the radiation captured by the camera is automatically generated by the camera. Using values already generated by the camera is advantageous because it does not require using additional processing power to achieve these values and does not reduce the processing speed of the camera by separate processing to obtain the data necessary for the function. The gain mentioned above is a value indicating the amplification of the signal from the camera sensor and is automatically set for the camera in relation to the luminance captured by the image sensor. The term luminance in relation to the acquisition of the radiation level indicator is intended to represent the radiation recorded by the image sensor, i.e., at least some frequencies of visible light and IR light.

[0034] A camera, camera system, or camera facility that utilizes an IR illumination source, and / or the act of removing an infrared cut filter 26 from the optical path of camera 10 to enhance the image quality of images captured in low or no illumination conditions, detecting changing illumination conditions can be important for energy conservation and image quality reasons. Energy wasted on unnecessary illumination is avoided by turning off additional illumination from the IR illumination source when not needed, and image quality, particularly color quality, is increased by inserting an infrared cut filter into the optical path under appropriate illumination conditions. Thus, information is obtained as to whether the illumination conditions within the area captured by the camera are sufficient for the camera 10 or system to operate in day mode, or whether the illumination conditions are so poor that the camera must operate in night mode. The night mode of camera 10 or the system includes an IR illumination source activated for illuminating the area 12 captured by camera 10 and / or for removing the infrared cut filter 26 from the optical path of camera 10. The day mode of camera 10 or the system includes a non-active IR illumination source for illuminating the area captured by the camera, and / or inserting the infrared cut filter 26 into the optical path of camera 10, i.e., without additional radiation from the IR illumination source 14 and with the radiation reaching the image sensor being removed by the infrared cut filter 26.

[0035] Operating the camera or system in day mode means that the camera is set to operate in so-called normal lighting conditions. For example, a day mode camera has an infrared cut filter 26 disposed in the optical path in front of the image sensor. When the lighting conditions deteriorate, i.e., become darker, it may become difficult or impossible to capture an image of acceptable quality by simply adjusting the normal camera settings, and the camera is set to operate in night mode. In night mode, specific settings are used to enable the capture of usable images. Such settings may include removing the infrared cut filter 26 from the optical path and / or providing additional illumination of the scene in order to enable the camera's image sensor to capture an extended wavelength range of electromagnetic radiation such as infrared or near-infrared in addition to visible light.

[0036] Detecting when to switch from night mode to day mode can be achieved by detecting the light coming from the environment without registering the light from the IR illumination source 14. This can be easily done by using a separate light meter that is screened from the light of the IR illumination source 14. However, such a solution requires a separate light meter, as described above, and furthermore, it needs to be positioned in the scene 12 in order to represent the lighting conditions within the scene 12. This can be particularly difficult and expensive when the camera 10 capturing the scene 12 is far from the scene 12, for example, in the case of a camera 10 having a telephoto lens.

[0037] Accordingly, the inventors have conceived an alternative method of detecting the transition from a low-light situation having a scene 12 illuminated by the IR illumination source 14 to a lighting situation where the illumination from the IR illumination source 14 is no longer required, and controlling the IR illumination source accordingly.

[0038] According to an embodiment, a method for controlling an IR illumination source 14, e.g., an IR LED, that currently illuminates an area 12 captured by a camera 10 includes generating a test sequence 400 (see FIG. 4) intended to identify whether IR illumination is no longer required. The test sequence 400 begins, in step 402, with an operation of modulating the light emission from the IR illumination source between a first emission intensity and a second emission intensity at a frequency f1 during a first predetermined period. In step 404, the camera continuously captures images of the area of the scene, and then, in step 406, the plurality of images captured during the first predetermined period are used to determine a radiation level indicator for each of the plurality of captured images. In step 408, when the radiation level indicators are determined, a sequence of these determined radiation level indicators is evaluated to determine whether a predicted frequency fp is detected in the sequence of radiation level indicators. The frequency fp can be regarded as the frequency f1 or as a frequency alias of f1 due to aliasing, and can be seen as a sampling frequency predicted from the frequency of the test sequence taking into account the image rate of the camera 10. This will be described in more detail below. If it is determined that the evaluation result at the frequency fp is not detected in the sequence of radiation level indicators, in step 410, the IR illumination source 14 stops illuminating the scene in the area captured by the camera, i.e., ends the night mode. Further, in some embodiments, this evaluation of not being able to detect the frequency fp in the sequence of radiation level indicators indicates to the camera that the infrared cut filter should be reinserted into the optical path. However, if the evaluation result at the frequency fp is not detected in the sequence of radiation level indicators, the camera remains in the night mode and needs to continue illuminating the scene with the IR illumination source 14.

[0039] The general idea behind this test sequence is that if the fluctuations of the modulated IR light emission cannot be distinguished from noise, the illumination of the scene by ambient light is high enough, i.e., the addition of radiation from the IR illumination source 14 is no longer necessary.

[0040] As an example, the modulation of the emission intensity may resemble the graph of FIG. 5, and the emission intensity varies between a first emission intensity and a second emission intensity, i.e., it is 100% and 90% of the high emission intensity during a predetermined period T and at a frequency f1. The high emission intensity may be referred to as the operating emission intensity in some embodiments, i.e., the emission intensity suitable for illuminating the current scene. However, in some embodiments, the high emission intensity associated with 100% is higher than the operating intensity. The lower level of the emission intensity may be a value other than 90%, for example, the value may be any value such as 50% - 95%, 60% - 90%, 75% - 90%. Alternatively, the modulation may start at the lower of the two emission intensities, i.e., this can be considered such that the first emission intensity is the lower emission intensity and the second emission intensity is the high emission intensity. The emission intensity from the IR illumination source 14 can be determined by the capacity of the IR illumination source 14, the environment in which the IR illumination source 14 is set, and / or the user's preference. Further, the amplitude of the modulation, i.e., the difference between the high value and the low value, may be determined by testing. The test may be performed for the purpose of ensuring that there is a sufficiently detectable signal until the actual mode switch is made when the camera is installed at the operating location and the camera system is in night mode and about to switch to day mode. The test of the camera can be performed as a self-test procedure, for example, over a plurality of consecutive nights. Different high and low values are tested and the IR filter is inserted and compared with the light level in day mode. When the camera detects sufficient light in day mode, the high and low values can be set accordingly. The test procedure may be performed during the installation or configuration phase, or the self-test procedure may be designed to limit interference with image capture and be executable at any time. As another example, the test may be performed during manufacturing.

[0041] The period during which the IR illumination source 14 is modulated may be from 2 seconds to 30 seconds, and in some embodiments, the period is set from 10 seconds to 20 seconds. Determining whether the frequency f1 is detectable in the sequence of determined radiation levels can include an improvement in resolution with a longer period and thus more data points. However, the modulation can become noticeable to the human eye and thus to a person looking at the monitoring monitor presenting the image of the scene. Such a noticeable change can make the person looking at the monitor uncomfortable if modulated over a longer period. Therefore, when determining the period T of how long the IR illumination source 14 is modulated, the risk of modulation prolongation is troublesome, and thus the harmfulness in the monitoring operation must be considered in some cases.

[0042] In some embodiments, the modulation frequency f1 is related to the image capture frequency, i.e., the frame rate fr measured in frames per second (fps), and must be at most half the frequency with respect to the frame rate fr of the camera so that the image sensor of the camera can directly detect the frequency. Thus, when f1≦fr / 2, the frequency of the modulated signal can be directly identified according to the Nyquist-Shannon sampling theorem.

[0043] In an alternative embodiment, the modulation frequency f1 is set to a frequency higher than half the frame rate of the camera, i.e., f1>fr / 2, resulting in a captured optical fluctuation frequency low enough to be captured by the camera due to the aliasing effect. Thus, by utilizing the aliasing effect, even a high frequency f1 can be identified by the frequency resulting from the aliasing effect. For example, if the frequency f1 of the modulated light emission from the IR illumination source 14 is 70 Hz and the frame rate of the camera is 60 fp. Then, due to the aliasing effect, the frequency representing f1 in the captured image to be searched for is 10 Hz.

[0044] Therefore, the frequency to be searched for in the sequence of the radiation level indicator is the predicted frequency fp. The predicted frequency corresponds to f1 when f1 ≤ fr / 2, and corresponds to the frequency resulting from the aliasing effect when f1 > fr / 2. Therefore, for the ease of explaining the present invention, the predicted frequency fp represents either the modulation frequency f1 or one of the corresponding frequencies resulting from aliasing according to the relationship between the modulation frequency of the camera and the frame rate.

[0045] The evaluation of the determined radiation level indicator sequence for determining whether the frequency fp of the modulated light emission from the IR illumination source or the frequency resulting from the aliasing effect is detectable includes, according to some embodiments, applying a conversion to the determined radiation level sequence that converts the radiation level indicator sequence to the frequency domain. Therefore, the evaluation of the determined radiation level sequence includes an evaluation in the frequency domain. Some examples of conversions that can be used are Fourier transform, Laplace transform, cosine transform, sine transform, etc. In some embodiments, for example, discrete conversions such as discrete Fourier transform, discrete cosine transform, discrete sine transform, fast Fourier transform (FFT), etc. can be used. The determined radiation level indicator sequence for evaluation can, in some embodiments, include the radiation level indicators from all the image frames captured during the period of the modulated IR radiation emitted towards the scene. In some embodiments, the determined radiation level indicator sequence for evaluation may be determined for a subset of the image frames captured during the period of the modulated IR radiation, and in other embodiments, the determined radiation level indicator sequence for evaluation may be determined for the image frames captured during the period when the modulated IR radiation is not emitted towards the scene in addition to the period when the image frames captured during the period of the modulated IR radiation are emitted towards the scene.

[0046] The sequence of radiation level indicators converted to the frequency domain using the above-described conversion can result in data corresponding to the graph shown in FIG. 6. The graph shows a situation where the energy level 604 of the radiation level indicator at a frequency of approximately fp is greater than that at any other frequency. As described above, the frequency fp may be the frequency f1 of the modulated IR radiation or the resulting frequency alias. The lower energy levels 602 at other frequencies can be considered as noise levels representing the remaining frequencies registered in the process of capturing image data during the modulation period of the emission intensity. In some embodiments, the evaluation of whether the frequency fp is detected is implemented by calculating the energy level in the frequency fp or a range including fp, calculating the noise level, and comparing the frequency fp energy level with the noise energy level. If the ratio between the noise energy level and the energy level at the frequency fp is less than the detection threshold, the frequency fp modulation of the emission from the IR illumination source is considered not to be detected. Alternatively, if the difference in the energy levels between the frequency fp and the noise is less than the detection threshold, the frequency fp modulation of the emission from the IR illumination source is considered not to be detected. On the other hand, if the ratio or the difference in the energy levels between the above energy levels is greater than or equal to the detection threshold, the frequency fp modulation of the emission from the IR illumination source is considered to be detected.

[0047] In some embodiments, a transformation, e.g., an FFT of a sequence of determined radiation level indicators, results in a sequence of values each representing signal energy associated with discrete and adjacent frequency ranges. Each discrete frequency range is referred to as a frequency bin. Next, a first value representing the energy level of the frequency bin containing the frequency fp, i.e., the energy value, is compared with a second value representing the energy level of a bin containing background noise signal energy. The second value may be obtained by selecting one of the frequency bins not representing the frequency fp, or may be obtained by calculating an average value or a median value of the frequency bins not representing the frequency fp. If the difference between the first value and the second value is greater than or equal to a threshold, frequency f1 modulation of the light emission from the IR illumination source is considered to be detected. If the difference between the first value and the second value is less than the threshold, frequency f1 modulation of the light emission from the IR illumination source is considered not to be detected.

[0048] In some embodiments, the frequency fp within the frequency range containing the frequency fp, or the energy level of the bin containing the frequency fp, is compared with a threshold. This threshold may be predetermined, e.g., during system setup, or may be adaptive and based on measured or estimated noise in the frequency domain.

[0049] According to some embodiments, a test sequence as described in connection with FIG. 4 is started based on time, e.g., at a time when the lighting conditions may be favorable for capturing an image of the scene without illuminating the scene with the IR illumination source 14, or shortly before such a time. Such a time may vary by season because the time of sunrise varies throughout the year when the illuminance condition is appropriate for turning off the IR illumination source. Such a time can be calculated using well-known sunrise equations or a look-up table containing pre-calculated time values. Access to the date and time of the camera's location is required to use any of these methods.

[0050] Next, if the test sequence does not indicate that the lighting conditions are good enough to turn off the IR illumination source, the IR illumination source 14 remains in an active operation, e.g., providing illumination according to the night mode, for a delay period before the test sequence is restarted to evaluate whether the lighting conditions are still changing. The reason why the lighting conditions are not good enough to turn off the IR illumination source may be, for example, that the time point of the first test is set a little earlier than the predicted time point with a certain error. Another reason for the lighting conditions not being good enough to turn off the IR illumination source may be, for example, that the scene 12 is outdoors and cloudy, so a part of the sunlight is blocked from passing through the atmosphere. Yet another reason for the initial test sequence indicating that the lighting conditions are not good enough to swivel the IR illumination source 14 is that the area 12 captured by the camera 10 may be blocked from light at sunrise, e.g., a building, a tree, or other fixed obstacles may be blocking the sun. The light may also be blocked by a temporary obstacle, e.g., a truck parked in a position blocking the light from reaching the scene 12. The delay period can be from a few minutes to 30 minutes, depending on how the system administrator wants to set it.

[0051] The control of the IR illumination source 14 and / or the infrared cut filter 26 can be implemented and controlled in a more complex manner than the manner described above. However, the above-described test sequence may be included as part of such a more complex manner.

[0052] Next, refer to the modulation method described in FIG. 7 and the flowchart of FIG. 8. In some embodiments of the more complex scheme 800, the initial test sequence (step 801) that sets the iteration counter, iteration X, to the first iteration may be started as described above, and steps 802-808 are steps corresponding to steps 402-408 described in connection with FIG. 4 using a modulation with a small change in radiation intensity. For example, the IR illumination source 14 may be modulated between a normal intensity and a lower intensity that is not easily noticed by a person watching the video, such as between 100% and 95-90% (step 802). When the ambient light is low, i.e., dark, this modulation using a low intensity difference can even be detected without disturbing a person looking at a monitor presenting a video stream from the camera. If the frequency f1 of the modulation is detected, i.e., if the predicted frequency fp is detected (step 808), the ambient light is not strong enough to consider the currently running iteration as completed. Thus, before the process starts a new modulation sequence with the same iteration settings as the previous iteration, the process delays for M minutes (step 810), i.e., returns to step 802 without incrementing or decrementing the iteration counter iteration X. If the predicted frequency fp representing the frequency f1 of the modulation is not detected, the number of iterations is checked by checking the iteration counter iteration X. In this check, if it is determined that the iteration is not the last iteration, the process sets a test sequence for the next iteration using a new lower radiation intensity as the lower intensity of the modulation (step 812). However, if the test sequence is the last night mode, i.e., the IR irradiation source is turned off and the infrared cut filter 26 is inserted into the optical path (step 814). The value represented by the number of iterations, i.e., iteration X, for the last iteration can be set to any value. The more iterations there are, the finer the increase in the amplitude of the modulation becomes.

[0053] In some embodiments of the above-described iterative tests, a low-amplitude, i.e., a modulated initial test sequence with little difference between high and low emission intensities, is continuously emitted from the IR illumination source 14 during low illumination to detect when the lighting conditions are improving. Such a test sequence may alternatively be emitted at discrete and repetitive time intervals.

[0054] Small variations in the emission intensity in the modulation do not necessarily mean that the modulation frequency f1 in the converted sequence of the emission level indicator, or the non-detection of the frequency alias of f1, is good enough for the ambient light to turn off the IR illumination source 14 and remove the infrared cut filter 26. Rather, it may be selected at a level such that it does not have to be the time for another test sequence where the difference between the normal intensity and the low intensity of the modulated emission intensity is greater, for example, between 100% and 90 - 75%. A next test sequence with a larger modulation amplitude, i.e., a difference between the normal emission intensity of the modulated emission and a lower emission intensity, may be initiated in response to the non-detection of the frequency representing the modulation frequency in the converted sequence of the emission level indicator. The number of repetitions including an increase in the difference between the high and low intensities of the modulated emission intensity may be at least 2, and in some embodiments may be 3. The number of repetitions may be even more. The advantage of additional repetitions may be in an environment where the difference between the high and low emission intensities can be a problem. By increasing the difference in smaller increments, disturbances from variations can be reduced and even avoided.

[0055] If the modulation frequency f1 or the frequency representing f1 due to aliasing is detected in one of the repetitions, the modulation method may be paused for a predetermined period, for example, 5 - 10 minutes. Then, after the pause, the repetition may be initiated with a repetition presenting the emission intensity difference at which the modulation frequency was detected. The advantage of starting with this emission intensity difference is that it can save time in that it is not necessary to repeat the already passed repetitions.

[0056] The repetition can have the following difference between a high emission intensity and a low emission intensity, and the high emission intensity is 100%. TIFF2025090525000002.tif22170

[0057] In a networked camera system, a device that controls the IR illumination source 14, such as camera 10, may be configured to send a message stating that a modulation sequence has been initiated for the scene. The message may be broadcast, addressed to a specific camera, or addressed to a central server that manages the cameras of the camera installation. By notifying other cameras that a modulation sequence has been initiated, it is possible to cause other cameras to ignore the modulated light emission, thereby avoiding these cameras reacting to test sequences that they do not intend.

[0058] In some embodiments, the modulation frequency is changed between test sequences. Thus, the first test sequence may be modulated at frequency f1, and the next test sequence may be modulated at frequency f2, etc. Such a system may be set up to require two consecutive tests indicating that the lighting conditions are good enough to operate the camera in day mode. This advantage is that the system does not suffer from illusions or make incorrect decisions based on an unrelated light source that emits light at the frequency used in a single frequency test sequence. This advantage can also be achieved by having the camera analyze an image sequence when no test sequence is present and detecting possible interfering frequencies. In such an embodiment, the detected interfering frequencies can be used to prevent the IR illumination source 14 from sending test sequences at such frequencies. The analysis of the image sequence without intentionally modulated IR radiation can be performed in the same way as during the test sequence.

[0059] The above various embodiments are provided only by way of example and should not be construed as limiting the present invention. For example, the principles described herein can be applied to an illumination system or a camera system in any scene. Those skilled in the art will readily recognize various modifications and changes that can be made to the present invention without following the exemplary embodiments and applications illustrated and described herein and without departing from the scope of the present disclosure.

Claims

1. A method for controlling at least one infrared illumination source, an IR illumination source, for illuminating an area captured by a camera, comprising: modulating emission from the IR illumination source between a first emission intensity and a second emission intensity at a first frequency during a first predetermined time period; capturing an image using an image sensor of the camera during the first predetermined period; determining a radiation level indicator for each of a plurality of said images captured by said image sensor; evaluating the determined sequence of radiation level indicators to determine whether a frequency attributable to the emission modulated at the first frequency is detected in the sequence of radiation level indicators; and ceasing the IR illumination source from illuminating the area captured by the camera if it is determined that the evaluation result at the frequency due to the modulated emission at the first frequency is not detected in the sequence of radiation level indicators.

2. The method of claim 1 , wherein the evaluating the sequence of radiation level indicators comprises applying a transform to the sequence of radiation level indicators that transforms the sequence of radiation level indicators into the frequency domain.

3. 3. The method of claim 2, wherein the determining of detection or no detection of the frequency attributable to the modulated emission at the first frequency in the sequence of radiation level indicators comprises determining no detection of the frequency attributable to the modulated emission at the first frequency if the converted sequence of radiation level indicators results in a value of the frequency attributable to the modulated emission at the first frequency that is less than a threshold value.

4. The method of claim 1 , wherein the second emission intensity is between 50 and 95 percent of the first emission intensity of the IR illumination source.

5. The method of claim 1 , wherein the first predetermined period during which the IR illumination source is modulated is between 1 second and 10 minutes.

6. The method of claim 1 , wherein determining the radiation level indicator of a captured image comprises determining an upper quartile luminance value of a region of the captured image.

7. 2. The method of claim 1, further comprising: generating a signal instructing the camera to insert an IR filter in an optical path leading to an image sensor of the camera when it is determined that the evaluation result at the frequency resulting from the modulated emission at the first frequency is not detected in the sequence of radiation level indicators.

8. The method of claim 1, further comprising operating an IR illumination source in a normal mode for a delay period if it is determined that the evaluation result at the frequency attributable to the modulated emission at the first frequency is detected in the sequence of radiation level indicators, the normal mode being that the IR illumination source is operated as it was operated prior to the first period during which the IR illumination source was modulated before the process claimed in claims 1 to 7 is initiated again.

9. The control method of claim 1 includes at least a two-step process associated with a final step, the initial step being: modulating emission from the IR illumination source between a first initial emission intensity and a second initial emission intensity at a second frequency during a second predetermined period of time; capturing an image using the camera during the second predetermined period of time; determining a radiation level indicator for each of a plurality of said captured images; generating a sequence of initial radiation level indicators including radiation level indicators determined from the plurality of the captured images; evaluating the sequence of initial radiation level indicators to determine whether a frequency attributable to the modulated emission at the second frequency is detected in the sequence of initial radiation level indicators; 2. The method of claim 1, wherein if it is determined that the evaluation of the frequency resulting from the light emission modulated at the second frequency is not detected in the sequence of radiation level indicators, a next step in at least a two-stage process is initiated.

10. The method of claim 9 , wherein the first frequency and the second frequency are the same frequency.

11. 10. The method of claim 9, wherein the control method is a two-stage process, and the next step in the at least two-stage process is the final step.

12. 10. The method of claim 1, further comprising: evaluating an image captured from the camera for motion within the area captured by the camera; and initiating modulation of the light emission from the IR illumination source when the motion evaluation indicates a motion value below a motion threshold.

13. 2. The method of claim 1, wherein evaluating the determined sequence of radiation level indicators to determine whether the frequency due to the modulated emission at the first frequency is detected in the sequence of radiation level indicators further comprises determining whether the first frequency, or a frequency due to aliasing of the first frequency and a frame rate of the captured image, is detected in the sequence of radiation level indicators.

14. 13. A video capture system configured to perform the method of claim 1.

15. A computer readable medium containing instructions that, when executed by a computer, cause the computer to perform the method of claim 1.

Images