Method for controlling infrared illumination source

Abstract

A method for controlling at least one infrared illumination source (IR illumination source) that illuminates an area captured by a camera is provided. The method includes modulating emission from the IR illumination source between a first emission intensity and a second emission intensity at a first frequency during a first predetermined time period. When images are captured using an image sensor of a camera during a first predetermined period of time, a radiation level indicator is determined for each of a plurality of images captured by the image sensor. Furthermore, the determined sequence of radiation level indicators is evaluated to determine whether a frequency resulting from the emission modulated at the first frequency is detected in the sequence of radiation level indicators. If the evaluation result is that the frequency resulting from the emission modulated at the first frequency is determined not to be detected in the sequence of radiation level indicators, the IR illumination source is stopped illuminating the area captured by the camera.

Description

Technical Field

[0001] The present invention relates to the control of infrared light emitting diodes, and in particular, the control of infrared light emitting diodes for illuminating an area captured by a camera. Background Art

[0002] Cameras for capturing moving images (i.e., video) are often used to monitor and / or surveil areas of interest. These areas of interest may be exposed to lighting conditions that vary over time and in particular over a 24-hour period, when the scene to be captured will be in daylight during a certain portion of a 24-hour span and in low to no light during another portion of the 24-hour span (i.e., nighttime). In order to enable the camera to capture relevant video during daytime as well as during nighttime, many cameras today are provided with at least two different capture modes (day mode and night mode).

[0003] The image sensor of the camera is sensitive to light in the infrared spectrum (at least light in the near infrared spectrum), and in order to reproduce the colors as seen by the human eye in the captured image, infrared light is blocked from reaching the image sensor by an infrared cut filter located in the light path between the lens and the image sensor. However, in low light conditions (e.g., at night), all light that can be detected by the image sensor contributes to the image quality, and therefore the infrared cut filter is removed from the light path to allow infrared light to be detected by the image sensor. By utilizing light in this low spectrum, the practicality of the camera is also extended to low light environments. Low light images from the camera can be enhanced by illuminating the captured scene using infrared light emitting diodes. Using such infrared light emitting diodes to illuminate the scene will increase the camera's ability to deliver quality images using infrared light and / or near infrared light. The infrared light and / or near infrared light from this illumination is invisible to the human eye, but the image sensor without an infrared cut filter will detect it.

[0004] However, the LEDs should only be active when needed (e.g., when the area captured by the camera is experiencing low light), and the IR cut filter should be removed from the light path in low light situations and reinserted into the light path when lighting conditions improve. The activation and deactivation of the LEDs and the removal and reinsertion of the IR cut filter depend on whether there is sufficient light in the scene.

[0005] In EP3886539A1 (Axis AB) of September 29, 2021, a method for evaluating ambient light levels during video acquisition with a video camera in order to control the positioning of an infrared cut filter to filter out or not filter out infrared light is described. The method comprises capturing an image stream when the scene is illuminated by an IR illuminator having a first output level and then capturing an image stream when the scene is illuminated by an IR illuminator having a reduced output level. The two image streams are then evaluated to represent a measure of the ambient light level. Although the method provides an excellent method for determining when to switch between night mode and day mode in a camera, there are situations and implementations where it may be difficult to correctly evaluate these streams due to, for example, the level of noise components in the detected signal or because the level will not be allowed to be reduced enough to trigger detection.

[0006] From the above, it will be appreciated that there is room for further facilitating solutions that determine switching of additional illumination using IR illuminators when an infrared cut filter is no longer required and / or removed from the optical path. Summary of the invention

[0007] The present invention is defined by the appended independent claims. Additional features and advantages of the concepts disclosed herein are set forth in the following description, and in part will be apparent from the description, or may be learned by practicing the described techniques. These features and advantages of the concepts may be realized and obtained by the tools and combinations particularly pointed out in the appended claims. These and other features of the described techniques will become more fully apparent from the following description and the appended claims, or may be learned by practicing the disclosed concepts set forth herein.

[0008] In a first aspect, a method for controlling at least one infrared illumination source (IR illumination source), the IR illumination source illuminating an area captured by a camera, the method comprising: modulating emissions from the IR illumination source between a first emission intensity and a second emission intensity at a first frequency during a first predetermined time period; capturing images using an image sensor of the camera during the first predetermined time period; determining a radiation level indicator for each of a plurality of images captured by the image sensor; evaluating a sequence of determined radiation level indicators to determine whether a frequency generated by the emissions modulated at the first frequency is detected in the sequence of radiation level indicators; if the evaluation results in a determination that a frequency generated by the emissions modulated at the first frequency is not detected in the sequence of radiation level indicators, stopping the IR illumination source from illuminating the area captured by the camera. An advantage of the method is that it is a simple, robust and effective way of controlling an IR illumination source that illuminates a scene to be captured by the camera. Another advantage is that the method may allow the IR illumination source to be turned off when there is no longer a need for illumination, thereby enabling energy conservation.

[0009] In some embodiments, the evaluation of the sequence of radiation level indicators comprises applying a transform to the sequence of radiation level indicators, transforming the sequence of radiation level indicators into the frequency domain. The use of a frequency transform facilitates the evaluation of received radiation and is easy to implement and operate.

[0010] In some further embodiments, determining whether a frequency generated by transmission modulated at a first frequency is detected or not detected in a sequence of radiation level indicators includes: if the value of the transformed sequence of radiation level indicators at the frequency generated by transmission modulated at the first frequency is less than a threshold value, determining that the frequency generated by transmission modulated at the first frequency is not detected.

[0011] In some embodiments, the second emission intensity is 50 percent to 95 percent of the first emission intensity of the IR illumination source, and in some embodiments, the first predetermined time period during which the IR illumination source is modulated is one second to 10 minutes.

[0012] In yet other embodiments, the determination of the radiation level indicator of the captured image includes determining an upper quartile brightness value for a region of the captured image. An advantage of this feature is that detection of changes in introduced illumination can be facilitated as the focus of detection shifts toward changes at higher detected radiation intensities.

[0013] In some embodiments, the method further comprises generating a signal instructing the camera to insert an IR filter in an optical path to an image sensor of the camera if the evaluation results in a determination that a frequency produced by emissions modulated at the first frequency is not detected in the sequence of radiation level indicators.

[0014] In some additional embodiments, the method includes: if the evaluation results in a determination that a frequency generated by emissions modulated at a first frequency is detected in the sequence of radiation level indicators, operating the IR illumination source in a normal mode for a delay period, the normal mode being that the IR illumination source is operated as the IR illumination source was operated prior to the first period during which the IR illumination source was modulated before the test process begins again. An advantage of this procedure is that the method will detect a switch from a harsh ambient lighting condition to an acceptable ambient lighting condition without continuously modulating emissions from the IR illumination source.

[0015] In still other embodiments, the control method comprises at least a two-step process, in which the process of stopping the IR illumination source from illuminating the area captured by the camera is associated with a final step, and wherein the initial step comprises: modulating the emission from the IR illumination source between a first initial emission intensity and a second initial emission intensity at a second frequency during a second predetermined time period; capturing images using the camera during the second predetermined time period; determining a radiation level indicator for each of a plurality of captured images; creating an initial sequence of radiation level indicators including the determined radiation level indicators from the plurality of captured images; evaluating the initial sequence of radiation level indicators to determine whether a frequency generated by the emission modulated at the second frequency is detected in the initial sequence of radiation level indicators; initiating the next step in the at least two-step process if the evaluation results in a determination that a frequency generated by the emission modulated at the second frequency is not detected in the sequence of radiation level indicators. An advantage of this approach is that the modulated emission from the IR illumination source can be less intrusive and / or disruptive to a user viewing the actual scene or the scene captured by the camera. One reason for this is that in lighting conditions where the variation can be significant, such as in darker ambient lighting conditions, the variation can be kept low in magnitude or variation.

[0016] In some embodiments, the first frequency and the second frequency are the same frequency, and in still other embodiments, the control method is a two-step process, and at least the next step in the two-step process is the final step.

[0017] Some embodiments may include evaluating captured images from a camera for motion in an area captured by the camera, and initiating modulation of emissions from an IR illumination source when the motion evaluation indicates a motion value below a motion threshold. One advantage of this is that the risk of moving objects in the scene affecting the correct detection of changes in the scene is reduced. Another advantage may be that the risk of modulation of the IR affecting motion detection is reduced.

[0018] In some embodiments, evaluating the determined sequence of radiation level indicators to determine whether a frequency generated by emissions modulated at a first frequency is detected in the sequence of radiation level indicators further includes determining whether the first frequency or a frequency generated by aliasing of the first frequency and a frame rate of a captured image is detected in the sequence of radiation level indicators.

[0019] According to a second aspect, the method is performed in a video capture system configured for this purpose. The advantages described with respect to the method apply to corresponding features of the video capture system.

[0020] According to a third aspect, a computer readable medium comprising instructions, when executed by a computer, causes the computer to perform the method. Accordingly, the method can be implemented as a computer executable code that can be made available in various ways. The advantages described with respect to the method apply to the corresponding features of the computer readable medium.

[0021] According to the detailed description given below, the further scope of application of the present invention will become apparent. However, it should be understood that, although the preferred embodiments of the present invention are indicated, since various changes and modifications within the scope of the present invention will become apparent to those skilled in the art from the detailed description, the detailed description and specific examples are given only by way of illustration. Therefore, it is to be understood that, since the specific components of the described device or the steps of the described method can be changed, the present invention is not limited to such devices and methods. It should also be understood that the terms used in this article are only used for the purpose of describing specific embodiments and are not intended to be limited. It must be noted that, as used in the specification and the appended claims, the words "one", "the", "the" and "said" are intended to mean that there are one or more elements, unless the context clearly stipulates otherwise. Therefore, for example, a reference to "sensor" or "the sensor" may include several sensors, etc. In addition, the word "comprising" does not exclude other elements or steps. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to best describe the manner in which the above-described embodiments are implemented, as well as to define other advantages and features of the present disclosure, a more particular description is provided below and is illustrated in the accompanying drawings. Understanding that these drawings depict only exemplary embodiments of the invention and are therefore not to be considered limiting in scope, these examples will be described and explained with additional specificity and detail through the use of the accompanying drawings, in which:

[0023] Figure 1 A schematic diagram depicting a video capture system in which an embodiment of the present invention may be implemented;

[0024] Figure 2 is a schematic block diagram of a camera and an IR illumination source according to some embodiments of the present invention;

[0025] Figure 3 is a schematic block diagram of a camera and an IR illumination source according to some embodiments of the present invention;

[0026] Figure 4 is a flow chart of a method for controlling an infrared illumination source including a test sequence according to an embodiment of the present invention;

[0027] Figure 5 is a schematic diagram illustrating a graph representing modulation of emissions from an IR illumination source according to an embodiment of the present invention;

[0028] Figure 6 is a schematic diagram showing a graph representing results from a sequence of transformed radiation level indicators according to an embodiment of the present invention;

[0029] Figure 7 is a schematic diagram illustrating an alternative graph representing modulation of emissions from an IR illumination source in accordance with an embodiment of the present invention; and

[0030] Figure 8 is a flow chart of a method for controlling an infrared illumination source including a plurality of test sequences according to an embodiment of the present invention.

[0031] Furthermore, in the drawings, like reference numerals designate like or corresponding parts throughout the several views. DETAILED DESCRIPTION

[0032] Hereinafter, certain embodiments will be described more fully with reference to the accompanying drawings. It will be apparent to those skilled in the art that various modifications and variations can be made without departing from the inventive concept. In view of the description and practice disclosed herein, other embodiments will be apparent to those skilled in the art. The embodiments herein are provided by way of example so that the disclosure will be thorough and complete and will fully convey the scope of the inventive concept, and the claims are interpreted as containing all equivalents of the inventive concept that are apparent to those skilled in the art to which the inventive concept belongs. If not otherwise specified, different embodiments may be combined with each other.

[0033] The present invention relates to methods, devices and systems for controlling at least one infrared illumination source, which is also referred to as an IR illumination source hereinafter. The IR illumination source may be an infrared light emitting diode (hereinafter referred to as an IR LED) arranged to emit IR light or near IR light onto an area of ​​interest to capture an image thereof, or a device comprising a plurality of IR LEDs. The IR illumination source may be or include a source of other infrared light other than an IR LED, such as an IR laser diode or other IR source. However, to facilitate understanding of the following description, throughout the description, the term IR LED may be used as a representative of both an IR LED and an emitter of such other infrared light.

[0034] Now refer to Figure 1 , depicting an example of an arrangement including the present invention and / or an apparatus implementing the present invention. A camera 10 is arranged to capture an image of an area 12 in a scene 13, the area 12 being composed of Figure 110 is represented by a square 12 drawn with a dotted dashed line. In this specification, scene 13 should be interpreted as a view of the surrounding environment as seen from the position of the camera. IR illumination source 14 is arranged to emit IR light towards scene 13 so as to "illuminate" the area captured by camera 10 for enhanced image capture during low light conditions. Area 16 receiving radiation from the IR illumination source is in some cases larger than area 12 captured by camera 10, and in some cases smaller than area 12 of the scene captured by the camera. The illuminated area may include the entire scene 13, a portion of the scene 13, an area 12 of the scene 13 captured by camera 10, or a portion of an area 12 of the scene 13 captured by the camera. According to some embodiments, area 16 receiving IR radiation is Figure 1 The dotted square 16 is shown in FIG. Figure 1 In the example arrangement of FIG. 1 , the camera 10 and the IR illumination source 14 are depicted as separate devices, however, the IR illumination source 14 may in some embodiments be included in the housing of the camera 10 to radiate IR light in the same direction as the direction in which the camera is aimed.

[0035] The camera 10 may be a camera that captures a sequence of moving images or a continuous stream of moving images (i.e., captures a video sequence or a continuous video stream). The camera 10 may be a video camera 10 connected to a network of digital video cameras and / or a video camera 10 connected to a computer network (e.g., a LAN, a WAN, the Internet, etc.) or a video camera 10 that is not networked at all but includes a local storage for storing footage and / or video captured by the camera 10.

[0036] In some embodiments, Figure 2 As shown in , the camera 10 is a digital video camera 10. The camera includes a lens 22 that focuses an image onto an image sensor 24 via an optical path of the camera. The camera 10 also includes an infrared cut filter 26 that is movable from a position in the optical path in front of the image sensor 24 to a position that does not interfere with light directed to the image sensor 24 via the optical path (i.e., from a position in front of the image sensor 24 to a position that is not in front of the image sensor 24). The infrared cut filter 26 is also arranged to move in the opposite direction (i.e., from a position in which it does not interfere with light directed to the image sensor 24 via the optical path back to a position in the optical path in front of the image sensor 24).

[0037] The camera 10 may also include an image processor 28 connected to the image sensor 24 to process image data generated by the image sensor 24. Alternatively, the image processor 28 may be arranged in a device not shown, separate from the camera 10 and connected to the camera via a network or a direct communication line. The image processor 28 is connected to a central processing unit (CPU 30) that runs software code for controlling the positioning of the infrared cut filter 26 and software code for controlling the IR illumination source 14 in some embodiments. In some embodiments, the CPU 30 transmits control instructions / signals to an actuator 32 that is physically connected to the infrared cut filter 26 to control the positioning of the infrared cut filter 26. In some embodiments, the CPU 30 transmits control instructions / signals to an interface 34 for controlling the IR illumination source 14, which is a device external to the camera. In addition, the camera 10 includes a non-volatile memory 36 and a volatile memory 38, both of which are connected to the CPU 30 and the image processor 28 via a data bus 40. The non-volatile memory 36 can store program codes for controlling the camera 10 and operations related to the camera 10 (e.g., controlling the IR illumination source 14). Control of the IR illumination source may, for example, include precise control of the emission intensity and / or turning the IR illumination source on and off. Control of the IR illumination source may also be synchronized with the exposure interval of the image sensor. The non-volatile memory 36 may, for example, be a read-only memory (ROM), a PROM, an EPROM, an EEPROM, a mask ROM, a flash memory, a ferroelectric RAM, a magnetoresistive RAM, a phase change RAM, a FeFET memory, an RRAM memory, etc. The volatile memory 38 may serve as a working memory for the processor 28, 30 of the camera 10, and may, for example, be a random access memory (RAM), an SRAM, a DRAM. In addition, as discussed above, the camera may include a network interface 42 for connecting the camera 10 to a computer network.

[0038] exist Figure 3 A camera according to an alternative embodiment is shown in FIG. Figure 3 The camera 10 and Figure 2 One difference compared to the camera of 10 is that the IR illumination source 14 is included as part of the camera. Due to the short distance between the IR illumination source and the lens, the IR illumination source is arranged to emit IR in a direction consistent with the direction in which the lens of the camera 10 captures the image so that the IR radiation illuminates the area captured by the camera 12.

[0039] When the infrared cut filter 26 is located outside the optical path (i.e., infrared light is not filtered out before the light / radiation reaches the image sensor 24), the image sensor 24 will detect both visible light and infrared light (at least near infrared light), which is generally considered to refer to wavelengths from 800nm ​​to 2500nm. The human eye is generally considered to be sensitive to light in the wavelength range of 380nm to 800nm. However, today's ordinary image sensors 24 without infrared cut filters may be able to detect wavelengths up to 1100nm. Accordingly, the image sensor 24 without any filter to remove infrared radiation detects electromagnetic waves with wavelengths longer than 800nm.

[0040] A radiation level indicator, for example, indicating a value of radiation received at the camera and / or sensor, may be determined in the camera by the image processor 28 or by the CPU 30. The radiation level indicator is a value recorded by the image sensor indicating the level of radiation (i.e., visible light and infrared light) reflected from the scene 12 onto the image sensor 24. In some embodiments, the value is calculated from all pixels of the image sensor 24 or from a limited area of ​​pixels on the image sensor 24. Such calculations may include calculating the average intensity received in the pixel, the median intensity received in the pixel, the upper quartile intensity received in the pixel, such as the upper quartile, the sum of the pixel values, etc. for the captured image frame or the pixels of the limited area. The pixels used for the calculation may be pixels representing all three color channels (i.e., red, green, and blue), a combination of two color channels, or only one of the channels (e.g., the red channel). In some embodiments, the value representing the radiation level indicator may include values ​​that have been generated by the camera for other purposes, such as exposure value, gain, exposure time, etc. In some cameras, the value of the brightness of the radiation captured by the camera is automatically generated by the camera. Using values ​​already generated by the camera is advantageous since no additional processing power needs to be used to implement these values ​​and the processing speed of the camera is not reduced by separate processing for acquiring the data required for this function. The gain mentioned above is a value indicating the amplification of the signal from the camera sensor and is automatically set in the camera with respect to the brightness captured by the image sensor. The term brightness in connection with the acquisition of radiation level indicators is intended to mean the radiation recorded by the image sensor (i.e. visible light and at least some frequencies of IR light).

[0041] In an active camera, camera system or camera installation that utilizes an IR illumination source and / or removes an infrared cut filter 26 from the optical path of the camera 10 to improve the image quality of images captured in low light or no light conditions, the detection of changing lighting conditions may be important for energy saving and image quality reasons. By turning off the additional illumination from the IR illumination source when not needed, energy wasted on unnecessary illumination is avoided, and by inserting an infrared cut filter in the optical path under appropriate lighting conditions, image quality, in particular color quality, is improved. Thus, information is obtained indicating whether the lighting conditions in the area captured by the camera are sufficient for the camera 10 or system to operate in day mode or whether the lighting conditions are so poor that the camera must operate in night mode. The night mode of the camera 10 or system includes an activated IR illumination source 14 for illumination of the area 12 captured by the camera 10 and / or removing the infrared cut filter 26 from the optical path of the camera 10. The day mode of the camera 10 or system includes a deactivated IR illumination source for illumination of the area captured by the camera and / or an infrared cut filter 26 inserted in the optical path of the camera 10 (i.e., there is no additional radiation from the IR illumination source 14 and the radiation reaching the image sensor is filtered by the infrared cut filter 26).

[0042] Operating a camera or system in day mode means setting the camera to operate under so-called normal lighting conditions. For example, a camera in day mode will have an infrared cut filter 26 arranged in the light path in front of the image sensor. As lighting conditions deteriorate (i.e., become increasingly darker), at a certain stage it will be difficult or even impossible to capture images of acceptable quality by simply adjusting the normal camera settings, and the camera will be set to operate in night mode. In night mode, specific settings are used to enable the capture of usable images. Such settings may be to remove the infrared cut filter 26 from the light path so as to allow the image sensor of the camera to capture electromagnetic radiation of an extended wavelength range (e.g., infrared or near infrared in addition to visible light) and / or to provide additional illumination of the scene.

[0043] Detecting when to switch from night mode to day mode can be achieved by detecting light from the environment without recording light from the IR illumination source 14. This can be easily accomplished by using a separate light meter that is shielded from the light of the IR illumination source 14. However, as described above, this solution requires a separate light meter, and furthermore, in order to represent the lighting conditions in the scene 12, it should be located at the scene 12. This becomes particularly difficult and expensive if the camera 10 capturing the scene 12 (e.g., a camera 10 with a telephoto lens) is far away from the scene 12.

[0044] Therefore, the inventors have proposed an alternative way of detecting a transition from a low light situation with a scene 12 illuminated by an IR illumination source 14 to a lighting situation where illumination from the IR illumination source 14 is no longer required, and for controlling the IR illumination source accordingly.

[0045] According to the embodiment, see Figure 4 , a method for controlling an IR illumination source 14 (e.g., an IR LED) that is currently illuminating an area 12 captured by a camera 10 includes generating a test sequence 400 intended to confirm whether IR illumination is no longer required. The test sequence 400 begins with an act of modulating emissions from the IR illumination source between a first emission intensity and a second emission intensity at a frequency f1 during a first predetermined time period (step 402). The camera continuously captures images of an area of ​​a scene (step 404), and then a plurality of images captured during the first predetermined time period are used to determine a radiation level indicator for each of the plurality of captured images (step 406). When the radiation level indicator has been determined, the sequence of these determined radiation level indicators is evaluated to determine whether a predicted frequency fp is detected in the sequence of radiation level indicators (step 408). The frequency fp represents the frequency f1 as the frequency f1 or as a frequency alias due to aliasing, and can be considered as a sampling frequency predicted from the frequency of the test sequence according to the image rate of the camera 10. This is discussed in more detail below. If the evaluation results in that frequency fp is determined to not be detected in the sequence of radiation level indicators, then the IR illumination source 14 is stopped from illuminating the scene at the area captured by the camera (i.e., exiting night mode) (step 410). In addition, in some embodiments, this evaluation of failure to detect frequency fp in the sequence of radiation level indicators indicates to the camera that an infrared cut filter is to be reinserted in the optical path. However, if the evaluation results in that frequency fp is detected in the sequence of radiation level indicators, then the camera should remain in night mode and continue to illuminate the scene with the IR illumination source 14.

[0046] The general idea behind this test sequence is that if the variations in the modulated IR emissions cannot be distinguished from noise, then the illumination of the scene by ambient light is sufficiently high (ie, no more radiation from the IR illumination source 14 needs to be added).

[0047] As an example, the modulation of the emission intensity may look like Figure 5A graph in which the emission intensity varies between a first emission intensity and a second emission intensity (i.e., 100% and 90% of the high emission intensity) during a predetermined time period T and at a frequency f1. In some embodiments, the high emission intensity may be referred to as an operating emission intensity (i.e., an emission intensity suitable for illuminating the current scene). However, in some embodiments, the high emission intensity associated with 100% is higher than the operating intensity. The lower level of emission intensity may be another value different from 90%, for example, the value may be any value between 50% and 95%, between 60% and 90%, between 75% and 90%. Alternatively, the modulation starts with the lower of the two emission intensities (i.e., this may be considered that the first emission intensity is a low emission intensity and the second emission intensity is a high emission intensity). The emission intensity from the IR illumination source 14 may be determined by the capacity of the IR illumination source 14, the environment in which the IR illumination source 14 is set, and / or the user's preference. In addition, the amplitude of the modulation (i.e., the difference between the high value and the low value) may be determined by testing. When the camera is installed in an operational location, a test can be performed when the camera system is in night mode and is about to switch to day mode, with the aim of ensuring that there is a sufficient detectable signal until the actual switch of modes will occur. The testing of the camera can be performed, for example, as a self-test procedure during several consecutive nights. With an IR filter inserted, different high and low values ​​are tested and compared to the light levels in day mode. When the camera detects sufficient light in day mode, the high and low values ​​can be set accordingly. The test procedure can be performed during the installation or configuration phase, or the self-test procedure can be designed to limit interference with the capture of the image, enabling it to be performed at any time. As another example, the test can be performed during production.

[0048] The time period during which the IR illumination source 14 is modulated may be 2s to 30s, and in some embodiments the time period is set to 10s to 20s. Determining whether the frequency f1 is detectable in the sequence of determined radiation level indicators may include a resolution that increases with a longer time period and thereby with more data points. However, the modulation may be noticeable by the human eye and thereby by a person viewing a monitoring monitor presenting an image of the scene. If modulated for a longer time period, such noticeable changes may be a disturbance to a person viewing the monitor. Therefore, when determining the time period T for which the IR illumination source 14 is modulated, the risk of the extended modulation becoming an annoyance and thereby potentially becoming a hidden danger in the monitoring operation must be considered.

[0049] In some embodiments, the frequency f1 of the modulation is related to the image capture frequency (i.e., frame rate) fr measured in frames per second (fps), and should be at most half the frequency related to the frame rate fr of the camera's frames, so that it is possible for the camera's image sensor to directly detect the frequency. Accordingly, according to the Nyquist-Shannon sampling theorem, if f1≤fr / 2, it is possible to directly determine the frequency of the modulated signal.

[0050] In an alternative embodiment, the modulation frequency f1 is set to a frequency higher than half the frame rate of the camera (i.e., f1>fr / 2), which will result in the captured light changes being low enough in frequency to be captured by the camera due to the aliasing effect. Thus, by utilizing the aliasing effect, even high frequencies f1 can be identified by the frequencies produced by the aliasing effect. For example, if the frequency f1 of the modulated emission from the IR illumination source 14 is 70 Hz, and the frame rate of the camera is 60 fps, then due to the aliasing effect, the frequency representing f1 in the captured image to be sought is 10 Hz.

[0051] Therefore, the frequency to be found in the sequence of radiation level indicators is the predicted frequency fp. If f1≤fr / 2, the predicted frequency corresponds to f1; if f1>fr / 2, the predicted frequency corresponds to the frequency generated by the aliasing effect. Therefore, for the convenience of describing the present invention, depending on the relationship between the modulation frequency and the frame rate of the camera, the predicted frequency fp represents any one of the modulation frequency f1 or the corresponding frequency generated by aliasing.

[0052] According to some embodiments, evaluating the sequence of determined radiation level indicators to determine whether the frequency fp of the modulated emission from the IR illumination source or the frequency caused by the aliasing effect is detectable includes: applying a transformation to the sequence of determined radiation level indicators to transform the sequence of radiation level indicators to the frequency domain. Therefore, evaluating the sequence of determined radiation level indicators includes an evaluation in the frequency domain. Some examples of transformations that can be used are Fourier transforms, Laplace transforms, cosine transforms, sine transforms, etc. In some embodiments, discrete transforms such as discrete Fourier transforms, discrete cosine transforms, discrete sine transforms, fast Fourier transforms (FFTs) can be used. In some embodiments, the sequence of determined radiation level indicators used for evaluation may include radiation level indicators captured during a time period when modulated IR radiation is emitted toward the scene from all image frames. In some embodiments, a sequence of determined radiation level indicators for evaluation may be determined for a subset of image frames captured during periods of time in which modulated IR radiation is emitted toward a scene, and in other embodiments, a sequence of determined radiation level indicators for evaluation may be determined for image frames captured during periods of time in which modulated IR radiation is not emitted toward a scene, in addition to image frames captured during periods of time in which modulated IR radiation is not emitted toward a scene.

[0053] The sequence of radiation level indicators transformed to the frequency domain using the transforms discussed above can be produced with Figure 6 . The graph depicts a situation where the energy level 604 of the radiation level indicator at a frequency of about fp is greater than the energy level at any other frequency. As previously explained, the frequency fp may be the frequency f1 of the modulated IR radiation or a frequency aliasing produced. The lower level 602 of energy at other frequencies may be considered to represent the noise level of the remaining frequencies recorded in the process of capturing image data during the time period of modulation of the emission intensity. In some embodiments, the evaluation of whether the frequency fp is detected is implemented by calculating the energy level at the frequency fp or at a range including fp, calculating the noise level, and comparing the frequency fp energy level with the noise energy level. If the ratio between the energy level of the noise and the energy level at the frequency fp is less than the detection threshold, the frequency fp modulation of the emission from the IR illumination source is considered not to be detected. Alternatively, if the difference in energy level between the frequency fp and the noise is less than the detection threshold, the frequency fp modulation of the emission from the IR illumination source is considered not to be detected. On the other hand, if the above ratio between the energy levels or the difference in energy levels is equal to or greater than the detection threshold, the frequency fp modulation of the emission from the IR illumination source is considered to be detected.

[0054] In some embodiments, a transformation (e.g., FFT) of a sequence of determined radiation level indicators produces a sequence of values ​​each representing a signal energy associated with a discrete and adjacent frequency range. Each discrete frequency range is referred to as a frequency bin. A first value (i.e., energy value) representing the energy level of a frequency bin including frequency fp is then compared with a second value representing the energy level of a bin including background noise signal energy. The second value can be obtained by selecting one of the frequency bins that does not represent frequency fp, or can be obtained by calculating the average or median of the frequency bins that do not represent frequency fp. If the difference between the first value and the second value is equal to or greater than a threshold value, it is considered that frequency f1 modulation of the emission from the IR illumination source is detected. If the difference between the first value and the second value is less than a threshold value, it is considered that frequency f1 modulation of the emission from the IR illumination source is not detected.

[0055] In some embodiments, the energy level at frequency fp in the frequency range including frequency fp or the energy level of a bin including frequency fp is compared to a threshold value. The threshold value may be predetermined (e.g., during setup of the system), or it may be adaptive and based on measured or estimated noise in the frequency domain.

[0056] According to some embodiments, the image capture method may be initiated based on the time of day (e.g., a time of day when there is a chance that lighting conditions will be favorable for capturing an image of the scene without IR illumination source 14 illuminating the scene, or some time before such a time). Figure 4 The test sequence described. When light conditions are suitable for turning off the IR illumination source, the time of day like this may vary during different seasons of the year because the sunrise time will be different throughout the year. Such a time of day can be calculated using the well-known sunrise formula or a lookup table including pre-calculated time values. Using any of these methods requires access to the date and time at the location of the camera.

[0057] Then, if the test sequence does not indicate that the lighting conditions are good enough to turn off the IR illumination source, the IR illumination source 14 remains in an activated operation, for example, providing illumination in a night mode, for a delay period of time before the test sequence is started again, in order to evaluate whether the lighting conditions have changed so far. The reason why the lighting conditions are not good enough to turn off the IR illumination source may be, for example, that the time point of the first test is set slightly earlier than the predicted time point in order to have a certain margin of error. Another reason why the lighting conditions are not good enough to turn off the IR illumination source 14 may be, for example, that the scene 12 is outdoors and it is cloudy, thus blocking some of the sunlight from passing through the atmosphere. Yet another reason why the initial test sequence indicates that the lighting conditions are not good enough to turn off the IR illumination source 14 may be that the area 12 to be captured by the camera 10 is blocked from light at sunrise, for example, buildings, trees or other fixed obstacles may be blocking the sun. The light may also be blocked by temporary obstacles (for example, a truck parked in a position that blocks the light from reaching the scene 12). Depending on how the system administrator wants to set it, the delay period of time may be a few minutes to half an hour.

[0058] The control of the IR illumination source 14 and / or the infrared cut filter 26 may be controlled by implementing a more complex scheme than that described above. However, the test sequence described above may be included as part of such a more complex scheme.

[0059] Now refer to Figure 7 The modulation scheme depicted in Figure 8 In some embodiments of the more complex scheme 800, the initial test sequence may be started as described above, with the iteration counter iteration X set to the first iteration (step 801), and steps 802 to 808 are related to Figure 4Steps 402 to 408 are described corresponding to steps using modulation with small variations in radiation intensity, for example, the IR illumination source 14 may be modulated between a normal intensity and a lower intensity (e.g., between 100% intensity and 95% to 90% intensity) that is not easily noticed by a person viewing the video (step 802). If the ambient light is low (i.e., it is dark), even this modulation using a low intensity difference will be detected without disturbing a person viewing a monitor presenting a video stream from a camera. If the modulated frequency f1 is detected (i.e., the predicted frequency fp is detected) (step 808), the ambient light is not strong enough to consider the currently running iteration to be complete, and therefore the process is delayed for M minutes (step 810) before the process starts a new modulation sequence with the same iteration settings as the previous iteration (i.e., returns to step 802 without increasing or decreasing the iteration counter Iteration X). If the predicted frequency fp representing the modulated frequency f1 is not detected, the number of iterations is checked by checking the iteration counter Iteration X. If it is determined in this check that the iteration is not the last iteration, the process sets up a test sequence for the next iteration using the new and lower radiation intensity as the lower intensity in the modulation (step 812). However, if the test sequence is the last, the night mode is exited (step 814) (i.e., the IR illumination source is turned off), and the infrared cut filter 26 is inserted in the light path. The number of iterations (i.e., the value represented by Iteration X in the last iteration) can be set to any value. The greater the number of iterations, the more subtle the increase in the amplitude of the modulation.

[0060] In some embodiments of the above-described iterative testing, a modulated initial test sequence at low amplitude (i.e., a small difference between high and low emission intensities) is continuously emitted from the IR illumination source 14 during low light conditions to detect when the lighting conditions are improving. Thus, alternatively, the test sequence may be emitted at discrete and repeated time intervals.

[0061] Small changes in the radiant intensity in the modulation may be selected at a level where failure to detect the modulation frequency f1 (alternatively, a frequency alias of f1) in the transformed sequence of radiation level indicators does not necessarily imply that the ambient light is good enough to turn off the IR illumination source 14 and remove the infrared cut filter 26, but rather implies that it is time to proceed with another test sequence with a larger difference between the normal intensity and the lower intensity of the modulated radiant intensity (e.g., between 100% and 90% to 75%). In response to failure to detect a frequency representing the modulation frequency in the transformed sequence of radiation level indicators, the next test sequence with an even larger modulation amplitude (i.e., the difference between the normal radiant intensity and the lower radiant intensity of the modulated radiation) may be started. The number of iterations including increasing the difference between the high and low intensities of the modulated radiant intensity may be at least two, and in some embodiments may be three. The number of iterations may be even greater. The advantage of additional iterations may exist in environments where the difference between high and low radiant intensities may be a problem. By increasing the difference in smaller increments, interference from changes may be reduced and even avoided.

[0062] If the modulation frequency f1 or a frequency representing f1 due to aliasing is detected in one of the iterations, the modulation scheme can be paused for a predetermined period of time (e.g., 5 to 10 minutes). Then, after the pause, the iteration can be started at the iteration presenting the radiation intensity difference at which the modulation frequency was detected. The advantage of starting at this radiation intensity difference is that time can be saved because it will not be necessary to repeat the iteration that has already passed.

[0063] The iterations may have the following differences between high and low radiation intensities, where the high radiation intensity is 100%:

[0064] Lower radiation intensity Iteration 1 90% to 95% Iteration 2 75% to 90% Iteration 3 50% to 85%

[0065] In a system of networked cameras, the device controlling the IR illumination source 14 (e.g., camera 10) can be configured to send a message indicating that a modulation sequence has been started for a scene. The message can be broadcast, addressed to a specific camera, or to a central server that manages the cameras of a camera installation. By notifying other cameras that a modulation sequence has been started, it is possible to cause other cameras to ignore the emission of the modulation and thereby prevent these cameras from reacting to a test sequence that is not intended for them.

[0066] In some embodiments, the modulation frequency is changed between test sequences. Accordingly, the first test sequence may be modulated at frequency f1 and the next test sequence at frequency f2, and so on. Such a system may be arranged to require two consecutive tests to indicate that the lighting conditions are good enough for the camera to operate in day mode. The advantage of this is that the system will not be deceived, or will not make erroneous decisions based on irrelevant light sources emitting light at the frequency used in a single frequency test sequence. This advantage may also be achieved by having the camera analyze the image sequence and detect possible interfering frequencies when no test sequence is present. In such an embodiment, the detected interfering frequencies may be used to cause the IR illumination source 14 to avoid sending test sequences at such frequencies. Analysis of image sequences that do not include intentionally modulated IR emissions may be performed in the same manner as during the test sequence.

[0067] The various embodiments described above are provided by way of illustration only and should not be construed as limiting the present invention. For example, the principles herein may be applied to any scene lighting system or camera system. Those skilled in the art will readily appreciate that various modifications and changes may be made to the present invention without following the example embodiments and applications illustrated and described herein and without departing from the scope of the present disclosure.

Claims

1. A method for controlling at least one infrared (IR) illumination source, said IR illumination source illuminating an area captured by a camera, said method comprising: modulating emissions from the IR illumination source between a first emission intensity and a second emission intensity at a first frequency during a first predetermined time period; capturing an image using an image sensor of the camera during the first predetermined time period; determining a radiation level indicator for each of a plurality of images captured by the image sensor; evaluating the determined sequence of radiation level indicators to determine whether a frequency resulting from the transmission modulated at the first frequency is detected in the sequence of radiation level indicators; If the evaluation results in a determination that the frequency produced by the emissions modulated at the first frequency is not detected in the sequence of radiation level indicators, the IR illumination source is stopped from illuminating the area captured by the camera.

2. The method according to claim 1, wherein: The evaluating of the sequence of radiation level indicators comprises applying a transformation to the sequence of radiation level indicators, transforming the sequence of radiation level indicators into the frequency domain.

3. The method according to claim 2, wherein: The determination of whether the frequency generated by the transmission modulated at the first frequency is detected or not detected in the sequence of the radiation level indicators includes: if the value of the transformed sequence of the radiation level indicators at the frequency generated by the transmission modulated at the first frequency is less than a threshold value, then determining that the frequency generated by the transmission modulated at the first frequency is not detected.

4. The method according to claim 1, wherein: The second emission intensity is 50 percent to 95 percent of the first emission intensity of the IR illumination source.

5. The method according to claim 1, wherein: The first predetermined period of time during which the IR illumination source is modulated is from one second to ten minutes.

6. The method according to claim 1, wherein: The determining of the radiation level indicator of a captured image comprises determining an upper quartile brightness value for a region of the captured image.

7. The method according to claim 1, further comprising: If the evaluation results in a determination that the frequency produced by the emissions modulated at the first frequency is not detected in the sequence of radiation level indicators, a signal is generated instructing the camera to insert an IR filter in the optical path to the camera's image sensor.

8. The method according to claim 1, further comprising: If the evaluation results in a determination that the frequency generated by the emission modulated at the first frequency is detected in the sequence of the radiation level indicators, the IR illumination source is operated in a normal mode during a delay time period, the normal mode being that the IR illumination source is operated as it was operated before the first time period during which the IR illumination source was modulated, before the process as described in claims 1 to 7 is started again.

9. The method according to claim 1, wherein: The control method comprises at least a two-step process, in which the process of claim 1 involves a final step, and wherein an initial step comprises: modulating emissions from the IR illumination source between a first initial emission intensity and a second initial emission intensity at a second frequency during a second predetermined time period; capturing images using the camera during the second predetermined time period; determining a radiation level indicator for each of a plurality of said captured images; creating an initial sequence of radiation level indicators comprising determined radiation level indicators from said plurality of captured said images; evaluating the initial sequence of radiation level indicators to determine whether a frequency resulting from the transmission modulated at the second frequency is detected in the initial sequence of radiation level indicators; If the evaluation results in a determination that the frequency produced by the transmission modulated at the second frequency is not detected in the sequence of radiation level indicators, then the next step in the at least two-step process is initiated.

10. The method according to claim 9, wherein: The first frequency and the second frequency are the same frequency.

11. The method according to claim 9, wherein: The control method is a two-step process, and the next step in the at least two-step process is the final step.

12. The method according to claim 1, further comprising: A captured image from the camera is evaluated for motion in the area captured by the camera, and the modulation of the emission from the IR illumination source is initiated when the motion evaluation indicates a motion value below a motion threshold.

13. The method according to claim 1, wherein: Evaluating the determined sequence of radiation level indicators to determine whether the frequency produced by the emission modulated at the first frequency is detected in the sequence of radiation level indicators further includes: determining whether the first frequency or a frequency produced by aliasing of the first frequency and the frame rate of the captured image is detected in the sequence of radiation level indicators.

14. A video capture system configured to perform the method of claim 1.

15. A computer readable medium comprising instructions which, when executed by a computer, cause the computer to perform the method of claim 1.

Images