Method for imaging a sequence of image frames and thermal camera having a microbolometer detector

Abstract

To provide a method for imaging the sequence of an image frame in a thermal camera including a microbolometer detector, the thermal camera, and a computer readable medium.SOLUTION: In the method, the first sequence and second sequence of the image frame are imaged in the closing state and opening state of the shutter of the thermal camera respectively. While the first sequence and the second sequence are imaged respectively, the integral time of the microbolometer detector is switched according to the repetition of one or a plurality of the temporal patterns of the integral time among a plurality of integral times. Further, in the method, the image frame in the second sequence imaged when the integral time is switched to a specific position in the temporal pattern of the integral time is corrected by using the image frame in the first sequence imaged when the integral time is switched to the same specific position in the temporal pattern of the integral time.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present invention relates to a thermal camera having a microbolometer detector. In particular, it relates to imaging a sequence of image frames using such a thermal camera.

Background Art

[0002] A thermal camera may image an image frame using a microbolometer sensor which is an uncooled sensor. To compensate for imperfections in the sensor, it is common to subtract a shutter image from the image frame imaged by the image sensor. The shutter image is the image imaged by the sensor when the shutter of the thermal camera is closed, i.e., when heat radiation from the scene does not reach the sensor, and it provides a measurement of non-uniform errors in the sensor. If there were no imperfections in the sensor, the shutter image would be a completely uniform image. However, due to the dark current in the sensor, this may not be the case and differences may occur between the individual pixels in the shutter image. Additionally, since the dark current is temperature-dependent, those differences vary with temperature. Therefore, when the temperature of the sensor begins to fluctuate, or there are other thermal fluctuations in the sensor, it is desirable to image a new shutter image. For example, a new shutter image may be imaged every 10 minutes, or when a change in temperature is detected.

[0003] In some situations, it is desirable to extend the dynamic range of the thermal camera. This can be achieved by imaging image frames while cycling through different integration times. When cycling through different integration times, the heating of the microbolometer sensor will change between frames, and as a result, the dark current will change between frames. As a result, as described in US7885536B1, the dark current value changes with the integration time, and therefore it is desirable for each integration time to have an associated shutter image.

[0004] However, the temperature change of the microbolometer sensor while imaging image frames with different integration times not only affects the dark current but also results in introducing a bias in the imaged image frames. More specifically, when the microbolometer sensor switches from imaging an image frame with a first integration time to imaging an image frame with a second integration time, it takes time for the temperature of the microbolometer to settle to the temperature level associated with the second integration time. This introduces a bias in the sensor when reading out the first frame imaged after the switch to the second integration time. In order to have dark current compensation that does not introduce further systematic errors in the imaged image frames, it is desirable that this bias be taken into account when generating the shutter image. Therefore, there is room for improvement.

Summary of the Invention

[0005] In view of the above, an object of the present invention is therefore to mitigate the above problems. In particular, the object is to correct the dark current of the microbolometer sensor in a way that takes into account the sensor readout bias introduced after switching between different integration times.

[0006] The present invention is defined by the independent claims, and its various embodiments are defined by the dependent claims.

[0007] In particular, the above object is achieved by a method of imaging a sequence of image frames in a thermal camera having a microbolometer detector. This method comprises imaging a first sequence of image frames while the shutter of the thermal camera is closed and switching the integration time of the microbolometer detector among a plurality of integration times according to one or more first repetitions of a temporal pattern of the integration times; imaging a second sequence of image frames while the shutter of the thermal camera is open and switching the integration time of the microbolometer detector among a plurality of integration times according to one or more second repetitions of the same temporal pattern of integration times; correcting an image frame in the second sequence imaged when the integration time is switched to a specific position within the temporal pattern of integration times using an image frame in the first sequence imaged when the integration time is switched to the same specific position within the temporal pattern of integration times; comprising.

[0008] The inventors have recognized that it is preferable to switch the integration time according to the same repetitive temporal pattern as that when imaging the frame of the shutter image and the normal image frame. In this way, the bias in the normal image frame and the frame of the shutter image having the same relative position within the repetition of the temporal pattern becomes the same. Further, the normal image frame and the shutter image used for the correction correspond to the same relative position within the repetition of the temporal pattern. This implies that the normal image frame is not only corrected using the frame of the shutter image imaged with the same integration time as that of the normal image frame, but also has the same bias as that of the normal image frame. As a result, the dark current compensation becomes more accurate and does not introduce systematic errors into the imaged image frames.

[0009] According to a second aspect, the above object is achieved by a thermal camera. The thermal camera controls a microbolometer detector, a shutter, the microbolometer detector and the shutter, imaging a first sequence of image frames while the shutter is closed and switching the integration time of the microbolometer detector among a plurality of integration times according to one or more first repetitions of the temporal pattern of integration times; A controller configured to capture an image while switching the second sequence of image frames while the shutter is open and the integration time of the microbolometer detector is switched according to one or more second repetitions of the same temporal pattern among a plurality of integration times; An image processor configured to correct an image frame in the second sequence captured when the integration time is switched to a specific position within the temporal pattern of the integration time using an image frame in the first image sequence captured when the integration time is switched to the same specific position within the temporal pattern of the integration time; including.

[0010] According to a third aspect, the above object is achieved by a non-transitory computer-readable medium including computer code instructions adapted to cause a thermal camera having a microbolometer detector to execute the method of the first aspect when executed by a device having processing capabilities.

[0011] The second and third aspects may generally have the same features and advantages as the first aspect. It should be noted that the present invention further relates to all possible combinations of features unless otherwise expressly stated.

[0012] The above and further objects, features, and advantages of the present invention are well understood through the following exemplary and non-limiting detailed description of embodiments of the present invention with reference to the accompanying drawings. Here, the same reference numerals are used for similar components.

Brief Description of the Drawings

[0013]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

DETAILED DESCRIPTION OF THE INVENTION

[0014] The present invention will be described in further detail below with reference to the accompanying drawings. Here, embodiments of the present invention are shown.

[0015] FIG. 1 shows a thermal camera 10 arranged to monitor a scene 12. The thermal camera 10 detects radiation in the infrared portion of the spectrum over time using a microbolometer detector and generates a sequence of image frames from the detected radiation. In scene 12, there are objects with different temperatures, which act as sources of infrared radiation. For example, there may be vehicles, people, trees, and fire as shown here. The temperature range spanned by these objects can be wide, which makes it difficult for the thermal camera 10 to properly image all the objects using a single integration time of the microbolometer detector. For illustrative purposes, objects with low temperatures, such as a person or a vehicle, may require a longer integration time than objects with high temperatures, such as a fire. If a longer integration time is used, the temperature of the low-temperature object can be resolved, but the microbolometer detector will saturate for the high-temperature object, so it cannot be done. The reverse is also true. To fully cover the temperature range, the thermal camera 10 may instead have to operate using several different integration times that are optimal for different parts of that temperature range. For example, the thermal camera 10 may switch the integration time of the microbolometer detector while imaging subsequent image frames.

[0016] In some embodiments, the thermal camera 10 may combine subsequent image frames captured with different integration times into a single image frame to generate an image with a high dynamic range. In other embodiments, the thermal camera 10 extracts an image frame having a first integration time from a sequence of image frames to generate a first video stream, and extracts an image frame having a second different integration time from the sequence of image frames to generate a second video stream. In the latter case, the thermal camera will thus generate several separate video streams corresponding to different integration times. These separate video streams may be subject to different types of processing or post-processing, such as different types of analysis including object detection and motion detection. For example, these separate video streams may be suitably used to implement an early fire warning system. The integration time of the first stream may be selected to be suitable for detecting objects in a lower temperature range, such as the temperature range of a human. The integration time of the second stream may be selected to be suitable for detecting objects in a higher temperature range, such as a temperature range suitable for detecting newly ignited fires (temperatures above 250 °C). The first video stream may subsequently be used to monitor people, for example, to perform analysis to detect a target person, predict the movement of the target person, and track the target person. The second stream may be used to monitor for the occurrence of a fire. For example, the second stream may be monitored to issue an alarm when the pixel values in the second video stream exceed an alarm threshold.

[0017] FIG. 2 shows the thermal camera 10 in more detail. It includes a microbolometer detector 20, a shutter 21, an image processor 22, and a controller 23. The microbolometer detector 20 may also sometimes be referred to as a microbolometer sensor 20. The microbolometer detector 20 may also include a temperature sensor 24 that measures the temperature of the microbolometer detector 20.

[0018] As is known to those skilled in the art, the microbolometer detector 20 or the microbolometer sensor is arranged to detect infrared radiation from the scene 12. When generating an image frame, the microbolometer detector 20 integrates the infrared radiation detected over a period called the integration time. The integration time used for each frame is set by the controller 23. In particular, the controller 23 may control the microbolometer detector 20 to switch between a plurality of integration times so that subsequent image frames have different integration times. The switching may be according to a predefined temporal pattern that is repeated. The repetition of the temporal pattern may be regarded as a sequence of integration times that defines which integration time should be used for a number of subsequent image frames. Generally, the temporal pattern of the integration time may be referred to as indicating the frequency and order of switching between a plurality of integration times. This is further illustrated in FIG. 3. This shows a sequence of image frames I1 to I14 captured using the microbolometer detector 20. While capturing the image frames I1 to I14, the microbolometer detector 20 is controlled to switch the integration time according to a plurality of repetitions R1 to R7 of the temporal pattern of the integration time. The temporal pattern is the same in each of the repetitions R1 to R7. The plurality of repetitions R1 to R7 of the temporal pattern may thus be referred to as forming a repetitive temporal pattern. In this example, the plurality of integration times includes two integration times T1 and T2, and each repetition of the temporal pattern is given by T1 followed by T2, that is, given by the sequence of integration times {T1, T2}. The integration time T1 is placed at the first position within the temporal pattern, that is, the first in the sequence of integration times, and the integration time T2 is placed at the second position within the temporal pattern, that is, the second in the sequence of integration times. As used herein, the position within the temporal pattern of the integration time thus refers to consecutive positions within the sequence of integration times defined by the repetition of the temporal pattern.

[0019] In the example of FIG. 3, the temporal pattern indicates that each of the integration times T1 and T2 is switched every second frame. This example may be generalized to the situation where there are a number of integration times with n≧2. Specifically, there may be n integration times, and the temporal pattern may indicate that each of the integration times is switched every nth frame. In the above example, the temporal pattern indicates that the integration time of the microbolometer detector is switched between image frames. However, the temporal pattern may also indicate that one or more of the multiple integration times are repeated for more than one frame in a row. For example, in the case of two integration times, the repetition of the temporal pattern may be {T1, T2, T2}, as shown in FIG. 4. In that case, the integration time T1 is placed at the first position within the temporal pattern, and the integration time T2 is placed at the second and third positions within the temporal pattern. Another example of a temporal pattern with two integration times is {T1, T1, T2, T2}. As will be appreciated by those skilled in the art, iterative temporal patterns can be formed in many ways from multiple integration times, and the examples given above are only some of the many possible temporal patterns.

[0020] When switching between different integration times, different levels of current flow through the pixels of the microbolometer detector 20. A longer integration time gives a rise to a higher current than that of a shorter integration time. These different currents affect the temperature of the microbolometer detector 20 and ultimately affect the signal exiting the microbolometer detector 20. For example, when switching from T1 to T2, where T1 > T2, the higher current during T1 will give the microbolometer 20 a higher temperature than it would have had if the integration time had been T2 instead, prior to that switch. As a result, the signal output of subsequent frames will be higher than it would have been otherwise. In other words, the switch introduces a bias in subsequent image frames captured with the integration time. The same is true when switching back again from T2 to T1, in which case the bias is in the negative direction, resulting in a lower signal output than it would have been otherwise. This bias will decrease over time when several frames in one row are captured with the same integration time. This is because the temperature of the microbolometer detector 20 stabilizes over time. This is the case, for example, in the example of FIG. 4 where two image frames in one row are captured using the integration time T2. In that case, it can be seen that the bias will be different for subsequent image frames captured with the same integration time T2. Specifically, the first image frame captured after the switch from T1 to T2 will have a greater bias than that of the second image frame captured after the switch. However, it can also be seen that the bias will be the same for two image frames having the same position within the temporal pattern repetition. For example, the image frames I2, I4, I6, I8, I10, I12, and I14 in FIG. 3 are associated with the first bias, while the image frames I1, I3, I5, I7, I9, I11, I13 are associated with a second different bias.Similarly, in FIG. 4, image frames J1, J4, J7, J10, J13, J16, J19 will have a first bias, image frames J2, J5, J8, J11, J14, J17, J20 will have a second bias, and image frames J3, J6, J9, J12, J15, J18, J21 will have a third bias. The embodiments described herein make use of this fact and cancel the bias when correcting the image frames.

[0021] The controller 23 may further control the shutter 21 and set it to an open or closed state. In particular, it may control the shutter 21 to keep it closed during one or more first repetitions R1 to R2 of the temporal pattern of the integration time, and open it during one or more other second repetitions R3 to R7 of the temporal pattern. In contrast to the open state, when the shutter 21 is in the closed state, heat radiation from the scene 12 does not enter the thermal camera 10 and does not reach the microbolometer detector 20. Images captured when the shutter is closed are sometimes called shutter images. This is because they depict the closed shutter 21 rather than the scene 12. Other names for them are flat field correction images or non-uniformity correction images. If there were no imperfections in the microbolometer detector 20, the shutter images would be completely uniform, i.e., would have the same pixel value at all pixels. However, in reality, due to imperfections in the microbolometer detector 20 known as dark current, they are not uniform. The role of the shutter images is to measure these imperfections so that they can be canceled from the image frames captured when the shutter 21 is in the open state.

[0022] As used herein, one or more first repetitions of the temporal pattern of the integration time refer to one or more subsequent repetitions of the temporal pattern. Similarly, one or more second repetitions of the temporal pattern refer to one or more subsequent repetitions of the temporal pattern. This is shown in FIG. 3. Here, the shutter is in the closed state during repetitions R1 to R2 of the temporal pattern and in the open state during repetitions R3 to R7 of the temporal pattern. Image frames I1 to I4 captured while the shutter is in the closed state form a first sequence S1 of image frames, and image frames I5 to I14 captured while the shutter is in the open state form a second sequence S2 of image frames. The first and second sequences may be viewed as the first and second subsequences of the sequence of image frames captured by the thermal camera 10.

[0023] It is understood that the number of first repetitions and the number of second repetitions are not limited to those shown in FIG. 3 and may vary in different embodiments. However, it is beneficial for one or more first repetitions of the temporal pattern of the integration time to include fewer repetitions than one or more second repetitions of the temporal pattern of the integration time. In that way, many of the captured images will be useful images depicting the scene rather than a closed shutter. Still, it may be beneficial to include more than one repetition of the temporal pattern in the first repetition in order to have the opportunity to average out random noise in the images captured while the shutter is closed.

[0024] The thermal camera 10 thus includes an image processor 22 and a controller 23, which are configured to implement the various functions of the thermal camera. Generally, the thermal camera may include the controller 23 and the image processor 22, and more specifically, the circuits configured to implement their functions.

[0025] In a hardware implementation, the image processor 22 and the controller 23 may correspond to specialized, specifically designed circuits that provide the functions of their respective components. This circuit may be in the form of one or more integrated circuits, such as one or more application-specific integrated circuits, or one or more field-programmable gate arrays. For purposes of illustration, the controller 23 may include, in use, a circuit that causes the thermal camera 10 to image with the shutter 21 in each of the closed and open states while switching the integration time of the microbolometer detector 20 for the first and second sequences of image frames. Similarly, the image processor 22 may include, in use, a circuit that causes an image frame in the second sequence to be corrected using an image frame in the first sequence. For example, the image processor 22 may form part of the image processing pipeline of the thermal camera 10.

[0026] In a software implementation, instead, the circuit may be in the form of a processor, such as a microprocessor, that causes the device 104 to implement any of the methods disclosed herein, in association with computer code instructions stored on a (non-transitory) computer-readable medium, such as non-volatile memory. Examples of non-volatile memory include read-only memory, flash memory, ferroelectric random access memory (RAM), magnetic computer storage devices, optical disks, and the like. In the case of software, each of the controller 22 and the image processor may thus correspond to a portion of the computer code instructions stored on a computer-readable medium that, when executed by the processor, causes the device thermal camera 10 to implement the functions of the components.

[0027] It will be understood that a combination of hardware implementation and software implementation is also possible. This means that one of the functions in the controller 23 or the image processor 22 is implemented in hardware and the other is implemented in software.

[0028] A method of imaging a sequence of image frames in a thermal camera 10 having a microbolometer detector 20 will be described with reference to the flowcharts of FIGS. 2 to 4 and FIG. 6. The method steps of any feature are indicated by dashed lines in FIG. 6.

[0029] In step S102, the thermal camera 10 captures a first sequence S1 of image frames I1 to I4 while switching the integration time of the microbolometer detector 20 according to one or more first repetitions R1 to R2 of a temporal pattern of integration times among a plurality of integration times I1, I2. More specifically, the controller 23 of the thermal camera 10 controls both the microbolometer detector 20 and the shutter 21 to capture the first image sequence S1.

[0030] In step S104, the thermal camera 10 captures a second sequence S2 of image frames I5 to I14 while switching the integration time of the microbolometer detector 20 according to one or more second repetitions R3 to R7 of the same temporal pattern among a plurality of integration times I1, I2.

[0031] As shown in FIG. 3, the second sequence S2 of image frames may be captured after the first sequence S1 of image frames. This is suitable in that the shutter image in the first sequence S1, which is used for correcting the images in the second sequence S2, is already available when the images in the second sequence S2 are captured. Thus, the images in the second sequence S2 can be corrected immediately without having to wait for the shutter image to be captured. However, embodiments in which the first sequence S1 is captured after the second sequence S2 can also be envisioned.

[0032] When the image frames of the first sequence S1 and the second sequence S2 are captured, they are input into the image processor 22. The image processor 22 corrects the image frames I5 to I14 in the second sequence S2 using the image frames in the first sequence S1. The correction is performed to reduce the influence of the dark current in the microbolometer detector 20, which could otherwise result in some pixels having incorrect pixel values. More specifically, in step S106, the image processor 22 corrects the image frames in the second sequence S2 captured when the integration time is switched to a specific position within the temporal pattern of the integration time, using the image frames in the first sequence S1 captured when the integration time is switched to the same specific position within the temporal pattern of the integration time.

[0033] Referring back to FIG. 3, the temporal pattern of the integration time corresponds to the sequence {T1, T2} of the integration time, where T1 is placed at the first position in the temporal pattern and T2 is placed at the second position in the temporal pattern. All of the image frames I5, I7, I9, I11, and I13 in the second sequence S2 are captured when the integration time is switched to the first position in the temporal pattern. Similarly, the image frames I1, I3 in the first sequence S1 are captured when the integration time is switched to the first position in the temporal pattern. In this example, the image processor 22 will thus correct the image frames I5, I7, I9, I11, and I13 in the second sequence S2 using the image frames I1, I3 in the first sequence S1. Similarly, the image processor 22 will correct the image frames I6, I8, I10, I12, and I14 in the second sequence S2 using the image frames I2 and I4 in the first sequence S1. In particular, the image frames I5, I7, I9, I11, and I13, and the image frames I1, I3, which are used for the correction, have the same positions within the repetition of the temporal pattern. The same can be said for the image frames I6, I8, I10, I12, and I14, and the image frames I2, I4, which are used for the correction.

[0034] FIG. 4 shows another example. Here, the temporal pattern of the integration time corresponds to a sequence of integration times {T1, T2, T2}, where T1 is placed at the first position in the temporal pattern and T2 is placed at the second and third positions in the temporal pattern. In this case, the image frames J7, J10, J13, J16, J19 in the second sequence S2 are captured when the integration time is switched to the first position in the temporal pattern, and thus are corrected using the image frames J1, J4 in the first sequence S1. The image frames J8, J11, J14, J17, J20 in the second sequence S2 are captured when the integration time is switched to the second position in the temporal pattern, and thus are corrected using the image frames J2, J5 in the first sequence S1. The image frames J9, J12, J15, J18, J21 in the second sequence S2 are captured when the integration time is switched to the second position in the temporal pattern, and thus are corrected using the image frames J3, J6 in the first sequence S1.

[0035] The image frames in the second sequence S2 are thus corrected not only using the image frames in the first sequence S1 having the same integration time, but this integration time also has the same position within the temporal pattern of the integration time. Since they have the same position within the temporal pattern of the integration time, the image frames to be corrected and the image frames used for correction are associated with the same bias. This makes it possible to cancel out the bias when performing the correction.

[0036] To perform correction on the image frames in the second sequence 22, the image processor 22 may first form the average of the image frames in the first image sequence S1 used for correction. When the image processor 22 receives the image frames of the first sequence S1, it may calculate and temporarily store this average. In this way, it becomes available for use by the image processor 22 when the image frames of the second sequence 22 are captured. In particular, the image frames in the second sequence S2 captured when the integration time is switched to a specific position within the temporal pattern of the integration time may be corrected using the average of the image frames in the first image sequence S1 captured when the integration time is switched to the same specific position within the temporal pattern of the integration time. This average is typically the average with respect to pixels, and this correction may be performed by subtracting this average with respect to pixels from the image frames in the second sequence S2. By averaging, the sensitivity to measurement noise is reduced. Further, by performing the subtraction, the above-mentioned bias is canceled. For illustrative purposes, the image processor 22 may form the average with respect to the first pixel of the image frames I1 and I3 and the average with respect to the second pixel of the image frames I2 and I4. The image processor 22 may subsequently subtract the average thus formed from the image frames to perform the correction on the image frames in the second sequence S2. In the example of FIG. 3, the image processor 22 will thus subtract the average with respect to the pixels of the frames I1 and I3 from each of the image frames I5, I7, I9, I11, and I13. Similarly, the image processor 22 will subtract the average with respect to the pixels of the frames I2 and I4 from each of the image frames I6, I8, I10, I12, and I14.

[0037] Subtraction from an image frame in the second sequence S2 of the average with respect to pixels may affect the overall signal level in the image frame in the second sequence S2. To maintain the overall signal level, the total average of the relevant shutter images from the first sequence S1 may be added to the image frames in the second sequence S2. The total average means the average of all pixels in all relevant shutter images. By way of example, the image processor 21 may form a first total average by averaging all pixel values in frames I1 and I3, and add that first total average value to image frames I5, I7, I9, I11, and I13. Similarly, the image processor 22 may form a second total average by averaging all pixel values in frames I2 and I4, and add that second total average value to image frames I6, I8, I10, I12, and I14. Similarly, as described above for the average with respect to pixels, when the image processor 22 receives the image frames of the first sequence S1, it may calculate and temporarily store their total average. Alternatively, instead of first subtracting the average with respect to pixels and subsequently adding the total average of the shutter images, the image processor 22 may pre - combine the average with respect to pixels and the total average into a corrected image, for example, by subtracting the total average from the average with respect to pixels. The corrected image may subsequently be used for correcting the image frames in the second sequence S2, for example, by subtracting the corrected image from the image frames in the second sequence S2 in a single operation.

[0038] As described above, the image processor 22 corrects the image frames in the second sequence S2 using the shutter image of the first image sequence S1. However, sometimes it is desirable for the thermal camera 10 to capture a new sequence of shutter images. Thus, in some embodiments, the controller 23 controls the microbolometer detector 20 and the shutter 21 in step S108 to capture a third sequence of image frames while the shutter 21 of the thermal camera 10 is closed and the integration time of the microbolometer detector 20 is switched according to one or more third repetitions of the same temporal pattern of integration times among a plurality of integration times. The third sequence of image frames may be captured after the second sequence S2 of image frames. The third sequence of image frames may be used to correct the image frames captured after the third sequence of image frames is captured. This is further illustrated in FIG. 5. This shows the third sequence S3 of image frames captured after the second sequence S2 of image frames when the shutter is in the closed state, and the fourth sequence S4 of image frames captured after the third sequence S3 when the shutter is in the open state. Similar to what is described above, the image frames of the third sequence S3 may be used to correct the image frames in the fourth sequence S4. As shown in FIG. 5, this procedure is typically repeated so that the thermal camera 10 can continue to capture sequences of image frames with the shutter alternately closed and open.

[0039] One reason for occasionally capturing a new shutter image is that the dark current in the microbolometer detector 20 increases with the sensor temperature. If there is a significant change in the sensor temperature after the last shutter image was captured, the shutter image will thus become old, and it is desirable to capture a new shutter image. Different strategies may be used to trigger the capture of a new shutter image. One simple strategy is to capture a new shutter image when a predetermined time has elapsed since the previous shutter image was captured. Thus, capturing the third sequence S3 may be triggered by a predetermined time elapsed since the first sequence of image frames was captured. For illustrative purposes, a new shutter image may be captured every 10 minutes. Another more sophisticated strategy is to capture a new shutter image when it is found that the temperature of the microbolometer detector 20 has varied beyond a specific value since the last shutter image was captured. Thus, capturing the third sequence S3 may be triggered by the temperature of the microbolometer detector that has changed beyond the temperature threshold since the first sequence of image frames was captured. For illustrative purposes, the temperature threshold may be set to 0.2 °C. However, the value used for the temperature threshold may depend on the sensitivity of the microbolometer detector 20 and the type of camera module used. To implement this strategy, the controller 23 enables the use of the temperature sensor 24 of the microbolometer detector 20. In particular, it can monitor the temperature of the microbolometer detector 20 and trigger the capture of a new sequence S3 of shutter images when the temperature of the detector 20 has varied beyond the temperature threshold since the preview sequence S1 of shutter images was captured. For example, it can use the temperature of the microbolometer detector 20 as a reference for comparison when the first image frame I1 of the previous sequence S1 is captured. The temperature threshold may be a predetermined value. Alternatively, the temperature threshold may change according to a predefined relationship with the ambient temperature of the thermal camera.For example, a predefined relationship may specify using a lower temperature threshold for ambient temperatures lower than a higher ambient temperature. One reason for varying the temperature threshold along with the ambient temperature is that the sensitivity of the microbolometer detector 20 changes with the ambient temperature. The temperature sensor 24 of the microbolometer detector 20 may be used to provide an estimate of the ambient temperature of the camera, since the temperature of the detector 20 essentially reflects the ambient temperature.

[0040] In the above, it is assumed that the integration time among the plurality of integration times is fixed through the first sequence S1 and the second sequence S1. However, similar to the exposure time of a visible light camera, the integration time is sometimes adjusted to provide an appropriate exposure in the image frame generated by the thermal camera 10. The control unit 23 may thus adjust, in step S110, one of the plurality of integration times. This adjustment may be made in response to detecting a change in the ambient temperature of the thermal camera 10. For example, this adjustment may be made when the ambient temperature of the thermal camera 10 changes beyond a second temperature threshold since the integration time was last adjusted. Again, the temperature sensor 24 of the thermal camera 10 may be used to estimate the ambient temperature. This adjustment may also be made in response to detecting that the number of saturated or black pixels in the image frame of the second sequence S2 exceeds a saturation threshold. In that case, the image frame is not properly exposed and it is desirable to adjust the integration time used when capturing that image.

[0041] Since the dark current value in the microbolometer detector 20 changes with the integration time, it is preferred to trigger imaging a new sequence of shutter images after adjusting the integration time. Thus, according to yet another strategy, imaging the third sequence S3 may be triggered by adjusting at least one of the plurality of integration times.

[0042] It is understood that the above strategies for triggering the imaging of a new sequence of shutter images may be combined. For example, as a basic rule, a new sequence of shutter images may be triggered when a predetermined time has elapsed. However, it may be triggered when the temperature of the microbolometer detector 20 has changed beyond a temperature threshold from the last sequence of shutter images, or following one or more adjustments of the integration time value.

[0043] After performing the correction, the image processor 22 may proceed to generate two or more video streams from the images captured while the shutter 21 of the camera 10 is open. In particular, in step S112, the image processor 22 may extract an image frame having a first integration time from the second sequence S2 of image frames to generate a first video stream, and extract an image frame having a second integration time shorter than the first integration time from the second sequence of image frames to generate a second video stream. It is understood that this generalizes such that one video stream is formed for each integration time. It is further understood that this is not limited to extracting image frames from the second sequence S2, but also applies to any subsequent sequences S4, S6, etc. captured while the shutter 21 is open. Video streams corresponding to different integration times may preferably undergo different types of processing or post-processing, such as different types of analysis including object detection or motion detection. In one example, in step S114, the image processing unit 22 may monitor one of the video streams, such as the second video stream, and issue an alarm when the pixel values in that video stream exceed an alarm threshold. As described further above, this may be preferably used to implement an early fire alarm system.

[0044] Those skilled in the art will understand that the above embodiments can be modified in many ways and that the advantages of the present invention as shown in the above embodiments can still be used. The present invention should therefore not be limited to the embodiments shown herein but should be defined only by the claims. Furthermore, as will be understood by those skilled in the art, the embodiments shown herein can also be combined.

Claims

1. A method for imaging a sequence of image frames in a thermal camera having a microbolometer detector, comprising: imaging a first sequence of image frames while the shutter of the thermal camera is closed and while switching the integration time of the microbolometer detector among a plurality of integration times according to one or more first repetitions of a temporal pattern of the integration time; imaging a second sequence of image frames while the shutter of the thermal camera is open and while switching the integration time of the microbolometer detector among the plurality of integration times according to one or more second repetitions of the same temporal pattern as the temporal pattern of the integration time; correcting, by subtraction, an image frame in the second sequence imaged when the integration time is switched to a specific position within the temporal pattern of the integration time, using an image frame in the first sequence imaged when the integration time is switched to the same specific position within the temporal pattern of the integration time; wherein the method includes the above steps.

2. The method according to claim 1, wherein the temporal pattern of the integration time indicates a frequency and an order of switching among the plurality of integration times.

3. The method according to claim 1, wherein the temporal pattern indicates that the integration time of the microbolometer detector is switched between image frames.

4. The method according to claim 1, wherein there are n integration times, and the temporal pattern indicates that each of the integration times is switched every nth image frame.

5. The method according to claim 1, wherein the image frame in the second sequence imaged when the integration time is switched to a specific position within the temporal pattern of the integration time is corrected by subtracting an average of the image frames in the first sequence imaged when the integration time is switched to the same specific position within the temporal pattern of the integration time.

6. After imaging the second sequence of the image frames, while the shutter of the thermal camera is in the closed state and in accordance with one or more third repetitions of the same temporal pattern of the integration time, during the switching of the integration time of the microbolometer detector among the plurality of integration times, further including imaging a third sequence of the image frames, Imaging the third sequence includes the temperature of the microbolometer detector having changed beyond a temperature threshold since imaging the first sequence of the image frames, a predetermined time having elapsed since imaging the first sequence of the image frames, adjustment of at least one of the plurality of integration times, The method according to claim 1, triggered by at least one of.

7. The third sequence of the image frames is used to correct the image frames imaged after imaging the third sequence of the image frames, the method according to claim 6.

8. The temperature threshold changes according to a predefined relationship together with the ambient temperature of the thermal camera, the method according to claim 6.

9. Further including adjusting one of the plurality of integration times in response to detecting a change in the ambient temperature of the thermal camera or in response to detecting that the number of saturated or black pixels in the image frames of the second sequence exceeds a saturation threshold, the method according to claim 1.

10. The second sequence of the image frames is imaged after the first sequence of the image frames, the method according to claim 1.

11. The one or more first repetitions of the temporal pattern of the integration time include fewer repetitions than the one or more second repetitions of the temporal pattern of the integration time, the method according to claim 1.

12. Extracting an image frame having a first integration time from the second sequence of the image frames to generate a first video stream, extracting an image frame having a second integration time shorter than the first integration time from the second sequence of the image frames to generate a second video stream, The method according to claim 1, further including.

13. The method according to claim 12, comprising monitoring the second video stream and issuing an alarm when a pixel value in the second video stream exceeds an alarm threshold value.

14. A microbolometer detector, A shutter, A controller for controlling the microbolometer detector and the shutter, While the shutter is in a closed state and the integration time of the microbolometer detector is switched among a plurality of integration times according to one or more first repetitions of a temporal pattern of the integration time, imaging a first sequence of image frames, A controller configured to image a second sequence of image frames while the shutter is in an open state and the integration time of the microbolometer detector is switched among the plurality of integration times according to one or more second repetitions of the same temporal pattern as the temporal pattern, An image processor configured to correct, by subtraction, an image frame in the second sequence imaged when the integration time is switched to a specific position within the temporal pattern of the integration time, using an image frame in the first sequence imaged when the integration time is switched to the same specific position within the temporal pattern of the integration time, A thermal camera comprising:

15. A non-transitory computer-readable medium including computer code instructions adapted to cause a thermal camera having a microbolometer detector to execute the method according to any one of claims 1 to 13 when executed by a device having processing capabilities.

Images