Method for imaging sequence of image frame and thermal camera including microbolometer detector

Abstract

To provide a method for imaging the sequence of an image frame in a thermal camera including a microbolometer detector, the thermal camera, and a computer readable medium.SOLUTION: In the method, the first sequence and second sequence of the image frame are imaged in the closing state and opening state of the shutter of the thermal camera respectively. While the first sequence and the second sequence are imaged respectively, the integral time of the microbolometer detector is switched according to the repetition of one or a plurality of the temporal patterns of the integral time among a plurality of integral times. Further, in the method, the image frame in the second sequence imaged when the integral time is switched to a specific position in the temporal pattern of the integral time is corrected by using the image frame in the first sequence imaged when the integral time is switched to the same specific position in the temporal pattern of the integral time.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present invention relates to a thermal camera having a microbolometer detector. In particular, it relates to imaging a sequence of image frames using such a thermal camera.

Background Art

[0002] A thermal camera may image an image frame using a microbolometer sensor which is a non-cooled sensor. To compensate for imperfections in the sensor, it is common to subtract a shutter image from the image frame imaged by the image sensor. The shutter image is an image imaged by the sensor when the shutter of the thermal camera is closed, i.e., when heat radiation from the scene does not reach the sensor, and it provides a measurement of non-uniform errors in the sensor. If there were no imperfections in the sensor, the shutter image would be a completely uniform image. However, due to dark current in the sensor, this may not be the case and differences may occur between separate pixels in the shutter image. Additionally, since dark current is temperature-dependent, those differences change with temperature. Therefore, when the temperature of the sensor begins to fluctuate, or there are other thermal fluctuations in the sensor, it is desirable to image a new shutter image. For example, a new shutter image may be imaged every 10 minutes, or when a change in temperature is detected.

[0003] In some situations, it is desirable to extend the dynamic range of a thermal camera. This can be achieved by imaging image frames while cycling through different integration times. When cycling through different integration times, the heating of the microbolometer sensor will change between frames, and as a result, the dark current will change between frames. As a result, as described in US7885536B1, the dark current value changes with the integration time, and therefore it is desirable for each integration time to have an associated shutter image.

[0004] However, temperature changes in the microbolometer sensor between image frames being captured with different integration times not only affect the dark current but also result in bias in the captured image frames. More specifically, when the microbolometer sensor switches from capturing an image frame with a first integration time to capturing an image frame with a second integration time, it takes time for the microbolometer temperature to settle to the temperature level associated with the second integration time. This introduces bias in the sensor reading out the first frame captured after the switch to the second integration time. It is desirable that this bias be taken into account when generating the shutter image in order to have dark current compensation that does not introduce further systematic errors in the captured image frames. Therefore, there is room for improvement. [Overview of the project]

[0005] In view of the above, the object of the present invention is therefore to mitigate the above-mentioned problems. In particular, the object is to correct the dark current of a microbolometer sensor in a manner that takes into account the sensor readout bias that is introduced after switching between different integration times.

[0006] The present invention is defined by the independent claims, and various embodiments thereof are defined by the dependent claims.

[0007] In particular, the above objective is achieved by a method for imaging a sequence of image frames in a thermal camera having a microbolometer detector. This method is The first sequence of image frames is captured with the shutter of the thermal camera closed, while the integration time of the microbolometer detector is switched between multiple integration times according to one or more first repetitions of a temporal pattern of integration times. The second sequence of image frames is imaged with the thermal camera shutter open, while the integration time of the microbolometer detector is switched between multiple integration times according to one or more second repetitions of the same temporal pattern of integration time. Correcting the image frame in the second sequence, captured when the integration time is switched to a specific position within the temporal pattern of the integration time, using the image frame in the first sequence, captured when the integration time is switched to the same specific position within the temporal pattern of the integration time, Includes.

[0008] The inventors recognized that it is preferable to switch the integration time according to the same repetitive temporal pattern as when capturing the shutter image frame and the normal image frame. In this way, the biases in the normal image frame and the shutter image frame that have the same relative position within the repetition of the temporal pattern are the same. Furthermore, the normal image frame and shutter image used for the correction correspond to the same relative position within the repetition of the temporal pattern. This implies that the normal image frame is not only corrected using the shutter image frame captured with the same integration time as that of the normal image frame, but also has the same bias as that of the normal image frame. As a result, dark current compensation becomes more accurate and does not introduce systematic errors into the captured image frame.

[0009] According to a second aspect, the above objective is achieved by a thermal camera. The thermal camera includes a microbolometer detector, a shutter, and controls the microbolometer detector and the shutter. A first sequence of image frames is captured with the shutter closed, while the integration time of the microbolometer detector is switched between multiple integration times according to one or more first repetitions of a temporal pattern of integration times. A controller configured to capture a second sequence of image frames with the shutter open, while switching the integration time of the microbolometer detector between multiple integration times according to one or more second repetitions of the same temporal pattern, An image processor configured to correct an image frame in a second sequence captured when the integration time is switched to a specific position within the temporal pattern of the integration time, using an image frame in a first image sequence captured when the integration time is switched to the same specific position within the temporal pattern of the integration time, Includes.

[0010] According to a third aspect, the above objective is achieved by a non-temporary computer-readable medium which, when executed by a device having processing capabilities, computer code instructions are adapted to cause a thermal camera having a microbolometer detector to perform the method of the first aspect.

[0011] The second and third embodiments may generally have the same features and advantages as the first embodiment. It should be noted that the present invention also relates to all possible combinations of features unless otherwise explicitly stated.

[0012] The above and further objects, features, and advantages of the present invention will be better understood through the following exemplary and non-limiting detailed description of embodiments of the invention with reference to the accompanying drawings. Herein, the same reference numerals are used for similar components. [Brief explanation of the drawing]

[0013] [Figure 1] A schematic diagram shows thermal cameras positioned to monitor the scene. [Figure 2] A schematic representation of a thermal camera according to several embodiments is shown. [Figure 3]In some embodiments, a schematic representation shows a first sequence of image frames captured when the shutter of a thermal camera is in the closed and open positions, respectively. [Figure 4] In some embodiments, a schematic diagram shows a second sequence of image frames captured when the shutter of the thermal camera is in the closed and open positions, respectively. [Figure 5] The following are six sequences of image frames captured when the shutter of a thermal camera is interchangeably in the closed and open positions according to several embodiments. [Figure 6] This is a flowchart of a method for capturing a sequence of image frames in a thermal camera having a microbolometer detector, according to some embodiments. [Modes for carrying out the invention]

[0014] The present invention will be described in further detail below with reference to the accompanying drawings. Hereinafter, embodiments of the present invention will be shown.

[0015] Figure 1 shows a thermal camera 10 positioned to monitor Scene 12. The thermal camera 10 detects radiation in the infrared portion of the spectrum over time using a microbolometer detector and generates a sequence of image frames from the detected radiation. In Scene 12, there are objects with different temperatures, which act as sources of infrared radiation. For example, there may be vehicles, people, trees, and fire, as shown here. The temperature range spread by these objects can be wide, making it difficult for the thermal camera 10 to properly image all objects using a single integration time of the microbolometer detector. For illustrative purposes, objects with lower temperatures, such as people or vehicles, may require longer integration times than objects with higher temperatures, such as fire. When longer integration times are used, the temperature of lower-temperature objects may be resolved, but this will not be possible for higher-temperature objects because the microbolometer detector will saturate. The reverse is also true. To fully cover the temperature range, the thermal camera 10 may instead have to operate with several different integration times that are optimized for different parts of that temperature range. For example, the thermal camera 10 may switch the integration time of the microbolometer detector while capturing subsequent image frames.

[0016] In some embodiments, the thermal camera 10 may combine subsequent image frames captured with different integration times into a single image frame to produce an image with a high dynamic range. In other embodiments, the thermal camera 10 generates a first video stream by extracting image frames having a first integration time from a sequence of image frames, and generates a second video stream by extracting image frames having a second different integration time from a sequence of image frames. In the latter case, the thermal camera will thus generate several separate video streams corresponding to different integration times. These separate video streams may undergo different types of processing or post-processing, such as different types of analysis including object detection and motion detection. For example, these separate video streams may be suitably used to implement an early fire alarm system. The integration time of the first stream may be selected to be suitable for detecting objects in a lower temperature range, such as the human temperature range. The integration time of the second stream may be selected to be suitable for detecting objects in a higher temperature range, such as a temperature range suitable for detecting a newly ignited fire (temperatures above 250°C). The first video stream may then be used to monitor people, for example, to detect a person, predict a person's movements, and perform tracking analyses of the person. The second stream may be used to monitor for the occurrence of a fire. For example, the second stream may be monitored to trigger an alarm when the pixel value in the second video stream exceeds an alarm threshold.

[0017] Figure 2 shows the thermal camera 10 in more detail. It includes a microbolometer detector 20, a shutter 21, an image processor 22, and a controller 23. The microbolometer detector 20 may also be called a microbolometer sensor 20. The microbolometer detector 20 may also include a temperature sensor 24 that measures the temperature of the microbolometer detector 20.

[0018] As is known to those skilled in the art, a microbolometer detector 20 or microbolometer sensor is configured to detect infrared radiation from scene 12. When generating an image frame, the microbolometer detector 20 integrates the detected infrared radiation over a period called an integration time. The integration time used for each frame is set by the controller 23. In particular, the controller 23 may control the microbolometer detector 20 to switch between multiple integration times so that subsequent image frames have different integration times. The switching may follow a repeating, predetermined temporal pattern. The repetition of the temporal pattern may be viewed as a sequence of integration times that defines which integration time should be used for a number of subsequent image frames. Generally, the temporal pattern of integration times may be referred to as the frequency and order of switching between multiple integration times. This is further illustrated in Figure 3, which shows a sequence of image frames I1 through I14 captured using the microbolometer detector 20. While image frames I1 to I14 are acquired, the microbolometer detector 20 is controlled to switch the integral times according to multiple repetitions R1 to R7 of a temporal pattern of integral times. The temporal pattern is the same in each of the repetitions R1 to R7. Multiple repetitions R1 to R7 of the temporal pattern may therefore be called forming a repeating temporal pattern. In this example, the multiple integral times include two integral times T1 and T2, and each repetition of the temporal pattern is given by T1 followed by T2, i.e., given by a sequence of integral times {T1, T2}. Integral time T1 is placed in the first position in the temporal pattern, i.e., it is the first in the sequence of integral times, and integral time T2 is placed in the second position in the temporal pattern, i.e., it is the second in the sequence of integral times. As used here, the position of an integral time in the temporal pattern therefore refers to a consecutive position in the sequence of integral times defined by the repetition of the temporal pattern.

[0019] In the example in Figure 3, the temporal pattern shows that each of the integral times T1 and T2 is switched every second frame. This example can be generalized to a situation where there are n ≥ 2 integral times. Specifically, there may be n integral times, and the temporal pattern can show that each of the integral times is switched every n frames. In the example above, the temporal pattern shows that the integral times of the microbolometer detector are switched between image frames. However, the temporal pattern can also show that one or more of the integral times are repeated for more than one frame in a row. For example, in the case of two integral times, the repetition of the temporal pattern may be {T1, T2, T2}, as shown in Figure 4. In that case, integral time T1 is placed in the first position in the temporal pattern, and integral time T2 is placed in the second and third positions in the temporal pattern. Another example of a temporal pattern with two integral times is {T1, T1, T2, T2}. As those skilled in the art will recognize, repetitive temporal patterns can be formed in many ways from multiple integral times, and the examples given above are only a few of the many possible temporal patterns.

[0020] When switching between different integration times, different levels of current flow through the pixels of the microbolometer detector 20. Longer integration times result in a higher current rise than shorter integration times. These different currents affect the temperature of the microbolometer detector 20, and ultimately affect the signal exiting the microbolometer detector 20. For example, if switching from T1 to T2, where T1 > T2, the higher current in T1 will give the microbolometer 20 a higher temperature than it would have had when the integration time was instead T2 before the switch. As a result, the signal output of the subsequent frame will be higher than it would have been up to that point. In other words, the switch introduces a bias into the subsequent image frame acquired with the same integration time. The same is true when switching back from T2 to T1, but in that case, the bias is negative, resulting in a lower signal output than it would have been. This bias decreases over time if several frames in a row are acquired with the same integration time. This is because the temperature of the microbolometer detector 20 stabilizes over time. This is the case, for example, in the example in Figure 4, where two image frames in a single row are acquired using integration time T2. In this case, we can see that the bias will be different for subsequent image frames acquired with the same integration time T2. Specifically, the first image frame acquired after the switch from T1 to T2 will have a greater bias than that of the second image frame acquired after the switch. However, we can also see that the bias will be the same for two image frames that are in the same position within the repetition of the temporal pattern. For example, image frames I2, I4, I6, I8, I10, I12, and I14 in Figure 3 are associated with a first bias, while image frames I1, I3, I5, I7, I9, I11, and I13 are associated with a second different bias.Similarly, in FIG. 4, image frames J1, J4, J7, J10, J13, J16, J19 will have a first bias, image frames J2, J5, J8, J11, J14, J17, J20 will have a second bias, and image frames J3, J6, J9, J12, J15, J18, J21 will have a third bias. The embodiments described herein enable use of this fact to cancel the bias when correcting the image frames.

[0021] The controller 23 may further control the shutter 21 and set it to an open or closed state. In particular, it may control the shutter 21 to be in the closed state during one or more first repetitions R1 to R2 of the temporal pattern of the integration time and in the open state during one or more other second repetitions R3 to R7 of the temporal pattern. In contrast to the open state, if the shutter 21 is in the closed state, heat radiation from the scene 12 does not enter the thermal camera 10 and does not reach the microbolometer detector 20. Images captured when the shutter is in the closed state are sometimes referred to as shutter images. This is because they depict the closed shutter 21 rather than the scene 12. Other names for them are flat field correction images or non-uniformity correction images. If there were no imperfections in the microbolometer detector 20, the shutter images would be completely uniform, that is, they would have the same pixel value at all pixels. However, in reality, due to imperfections in the microbolometer detector 20 known as dark current, they are not uniform. The role of the shutter images is to measure those imperfections so that they can be canceled from the image frames captured when the shutter 21 is in the open state.

[0022] As used herein, one or more first repetitions of the temporal pattern of the integration time refer to one or more subsequent repetitions of the temporal pattern. Similarly, one or more second repetitions of the temporal pattern refer to one or more subsequent repetitions of the temporal pattern. This is shown in FIG. 3. Here, the shutter is in the closed state during repetitions R1 to R2 of the temporal pattern and in the open state during repetitions R3 to R7 of the temporal pattern. Image frames I1 to I4 captured while the shutter is in the closed state form a first sequence S1 of image frames, and image frames I5 to I14 captured while the shutter is in the open state form a second sequence S2 of image frames. The first and second sequences may be viewed as the first and second subsequences of the sequence of image frames captured by the thermal camera 10.

[0023] It is understood that the number of first repetitions and the number of second repetitions are not limited to those shown in FIG. 3 and may vary in different embodiments. However, it is beneficial for one or more first repetitions of the temporal pattern of the integration time to include fewer repetitions than one or more second repetitions of the temporal pattern of the integration time. In that way, many of the captured images will be useful images depicting the scene rather than a closed shutter. Still, it may be beneficial to include more than one repetition of the temporal pattern in the first repetition in order to have an opportunity to average out random noise in the images captured while the shutter is closed.

[0024] The thermal camera 10 thus includes an image processor 22 and a controller 23, which are configured to implement the various functions of the thermal camera. Generally, the thermal camera may include the controller 23 and the image processor 22, more specifically, the circuits configured to implement their functions.

[0025] In hardware implementations, the image processor 22 and controller 23 may correspond to specialized, specifically designed circuits that provide the functions of their respective components. These circuits may take the form of one or more application-specific integrated circuits, or one or more field-programmable gate arrays, or other integrated circuits. For illustrative purposes, the controller 23 may include, in use, a circuit that causes the thermal camera 10 to capture first and second sequences of image frames, with the shutter 21 in closed and open states, respectively, while switching the integration time of the microbolometer detector 20. Similarly, the image processor 22 may include, in use, a circuit that corrects the image frames in the second sequence using the image frames in the first sequence. For example, the image processor 22 may form part of the image processing pipeline of the thermal camera 10.

[0026] In a software implementation, the circuit may instead take the form of a processor, such as a microprocessor, that causes device 104 to perform any of the methods disclosed herein in relation to computer code instructions stored on a (non-temporary) computer-readable medium, such as non-volatile memory. Examples of non-volatile memory include read-only memory, flash memory, ferroelectric random-access memory (RAM), magnetic computer storage devices, and optical disks. In the software case, the controller 22 and the image processor may, when executed by the processor, correspond to parts of computer code instructions stored on a computer-readable medium that cause device thermal camera 10 to perform the functions of its components.

[0027] It will be understood that a combination of hardware and software implementations is also possible. This means that one function of the controller 23 or image processor 22 may be implemented in hardware, while the others may be implemented in software.

[0028] A method for capturing a sequence of image frames in a thermal camera 10 having a microbolometer detector 20 will be described with reference to the flowcharts in Figures 2 to 4 and Figure 6. Method steps for any feature are indicated by dashed lines in Figure 6.

[0029] In step S102, the thermal camera 10 captures a first sequence S1 of image frames I1 to I4 while switching the integration time of the microbolometer detector 20 between multiple integration times I1 and I2 according to one or more first repetitions R1 to R2 of the temporal patterns of the integration time. More specifically, the controller 23 of the thermal camera 10 controls both the microbolometer detector 20 and the shutter 21 to capture the first image sequence S1.

[0030] In step S104, the thermal camera 10 images the second sequence S2 of image frames I5 to I14 while switching the integration time of the microbolometer detector 20 between multiple integration times I1 and I2 according to one or more second repetitions R3 to R7 of the same temporal pattern.

[0031] As shown in Figure 3, the second sequence S2 of the image frame may be captured after the first sequence S1 of the image frame. This is preferable in that the shutter image from the first sequence S1, which is used to correct the image in the second sequence S2, is already available when the image in the second sequence S2 is captured. Therefore, the image in the second sequence S2 can be corrected immediately without having to wait for the shutter image to be captured. However, embodiments in which the first sequence S1 is captured after the second sequence S2 are also conceivable.

[0032] When image frames from the first sequence S1 and the second sequence S2 are acquired, they are input to the image processor 22. The image processor 22 corrects image frames I5 to I14 in the second sequence S2 using the image frames from the first sequence S1. The correction is performed to reduce the effect of dark current in the microbolometer detector 20, which otherwise could result in some pixels having incorrect pixel values. More specifically, in step S106, the image processor 22 corrects image frames from the second sequence S2, which were acquired when the integration time was switched to a specific position in the temporal pattern of the integration time, using image frames from the first sequence S1, which were acquired when the integration time was switched to the same specific position in the temporal pattern of the integration time.

[0033] Referring again to Figure 3, the temporal pattern of the integration time corresponds to the sequence of integration time {T1, T2}, where T1 is at the first position in the temporal pattern and T2 is at the second position in the temporal pattern. Image frames I5, I7, I9, I11, and I13 in the second sequence S2 are all captured when the integration time switches to the first position in the temporal pattern. Similarly, image frames I1 and I3 in the first sequence S1 are captured when the integration time switches to the first position in the temporal pattern. In this example, the image processor 22 will therefore correct image frames I5, I7, I9, I11, and I13 in the second sequence S2 using image frames I1 and I3 in the first sequence S1. Similarly, the image processor 22 corrects image frames I6, I8, I10, I12, and I14 in the second sequence S2 using image frames I2 and I4 in the first sequence S1. In particular, image frames I5, I7, I9, I11, and I13 and image frames I1 and I3, which are used for the correction, have the same positions within the repetition of the temporal pattern. The same is true for image frames I6, I8, I10, I12, and I14 and image frames I2 and I4, which are used for the correction.

[0034] Figure 4 shows another example. Here, the temporal pattern of integration time corresponds to the sequence of integration time {T1, T2, T2}, where T1 is in the first position in the temporal pattern, and T2 is in the second and third positions in the temporal pattern. In this case, image frames J7, J10, J13, J16, and J19 in the second sequence S2 are captured when the integration time is switched to the first position in the temporal pattern, and are therefore corrected using image frames J1 and J4 in the first sequence S1. Image frames J8, J11, J14, J17, and J20 in the second sequence S2 are captured when the integration time is switched to the second position in the temporal pattern, and are therefore corrected using image frames J2 and J5 in the first sequence S1. Image frames J9, J12, J15, J18, and J21 in the second sequence S2 were acquired when the integration time was switched to a second position in the temporal pattern, and are therefore corrected using image frames J3 and J6 from the first sequence S1.

[0035] The image frame in the second sequence S2 is therefore corrected using the image frame in the second sequence S1, which has the same integration time, and this integration time also occupies the same position within the temporal pattern of integration time. Because they occupy the same position within the temporal pattern of integration time, the image frame being corrected and the image frame used for correction are associated with the same bias. This makes it possible to cancel out the bias when performing the correction.

[0036] To perform correction on the image frames in the second sequence 22, the image processor 22 may first form an average of the image frames in the first image sequence S1 to be used for correction. When the image processor 22 receives the image frames of the first sequence S1, it may calculate and temporarily store this average. In this way, it becomes available to the image processor 22 for use when the image frames of the second sequence 22 are captured. In particular, the image frames in the second sequence S2 captured when the integration time is switched to a specific position in the temporal pattern of the integration time may be corrected using the average of the image frames in the first image sequence S1 captured when the integration time is switched to the same specific position in the temporal pattern of the integration time. This average is typically an average with respect to pixels, and this correction may be performed by subtracting this pixel-respected average from the image frames in the second sequence S2. Averaging reduces sensitivity to measurement noise. Furthermore, subtraction cancels out the above bias. For illustrative purposes, the image processor 22 may form averages for the first pixels of image frames I1 and I3, and averages for the second pixels of image frames I2 and I4. The image processor 22 may then perform a correction by subtracting the averages thus formed from the image frames from the image frames in the second sequence S2. In the example in Figure 3, the image processor 22 would therefore subtract the averages for the pixels of frames I1 and I3 from image frames I5, I7, I9, I11, and I13, respectively. Similarly, the image processor 22 would subtract the averages for the pixels of frames I2 and I4 from image frames I6, I8, I10, I12, and I14, respectively.

[0037] Subtracting the pixel average from the image frame in the second sequence S2 may affect the overall signal level in the image frame in the second sequence S2. To maintain the overall signal level, the summation average of the associated shutter images from the first sequence S1 may be added to the image frame in the second sequence S2. The summation average means the average of all pixels in all associated shutter images. For example, image processor 21 may form a first summation average by averaging all pixel values ​​in frames I1 and I3, and add this first summation average to image frames I5, I7, I9, I11, and I13. Similarly, image processor 22 may form a second summation average by averaging all pixel values ​​in frames I2 and I4, and add this second summation average to image frames I6, I8, I10, I12, and I14. Similarly, as described above regarding the pixel average, when image processor 22 receives the image frames from the first sequence S1, it may calculate and temporarily store their summation averages. Alternatively, instead of first subtracting the pixel average and then adding the total average of the shutter images, the image processor 22 may pre-combine the pixel average and the total average into the corrected image, for example, by subtracting the total average from the pixel average. The corrected image may then be used to correct the image frame in the second sequence S2 in a single operation, for example, by subtracting the corrected image from the image frame in the second sequence S2.

[0038] As described above, the image processor 22 corrects the image frame in the second sequence S2 using the shutter image of the first image sequence S1. However, it is sometimes desirable for the thermal camera 10 to capture a new sequence of shutter images. Therefore, in some embodiments, the controller 23 may, in step S108, control the microbolometer detector 20 and the shutter 21 to capture a third sequence of image frames while the shutter 21 of the thermal camera 10 is closed and the integration time of the microbolometer detector 20 is switched between multiple integration times according to one or more third repetitions of the same temporal pattern of integration time. The third sequence of image frames may be captured after the second sequence S2 of image frames. The third sequence of image frames may be used to correct image frames captured after the third sequence of image frames has been captured. This is further illustrated in Figure 5. This shows a third sequence of image frames S3, captured after the second sequence S2 of image frames while the shutter is closed, and a fourth sequence of image frames S4, captured after the third sequence S3 while the shutter is open. Similar to the above, the image frames of the third sequence S3 may be used to correct the image frames in the fourth sequence S4. As shown in Figure 5, this procedure is typically repeated so that the thermal camera 10 can continue to capture sequences of image frames with the shutter interchangeably closed and open.

[0039] One reason for occasionally capturing new shutter images is that the dark current in the microbolometer detector 20 increases with the sensor temperature. If there is a large change in sensor temperature since the last shutter image was captured, the shutter image becomes old, and it is desirable to capture a new shutter image. Different strategies may be used to trigger the capture of a new shutter image. One simple strategy is to capture a new shutter image when a predetermined time has elapsed since the previous shutter image was captured. Thus, capturing a third sequence S3 may be triggered by a predetermined time that has elapsed since the first sequence of the image frame was captured. For illustrative purposes, a new shutter image may be captured every 10 minutes. Another, more sophisticated strategy is to capture a new shutter image when it is found that the temperature of the microbolometer detector 20 has changed by more than a certain value since the last shutter image was captured. Thus, capturing a third sequence S3 may be triggered by the temperature of the microbolometer detector changing by more than a temperature threshold since the first sequence of the image frame was captured. For illustrative purposes, the temperature threshold may be set to 0.2°C. However, the value used for the temperature threshold may depend on the sensitivity of the microbolometer detector 20 and the type of camera module used. To implement this strategy, the controller 23 makes the temperature sensor 24 of the microbolometer detector 20 available. In particular, it can monitor the temperature of the microbolometer detector 20 and trigger the acquisition of a new sequence of shutter images S3 when the temperature of the detector 20 has changed beyond the temperature threshold since the acquisition of the preview sequence S1 of the shutter images. For example, it can use the temperature of the microbolometer detector 20 as a reference for comparison when acquiring the first image frame I1 of the previous sequence S1. The temperature threshold may be a predetermined value. Alternatively, the temperature threshold may vary according to a predetermined relationship with the ambient temperature of the thermal camera.For example, a predetermined relationship may specify that a lower temperature threshold should be used for lower ambient temperatures than for higher ambient temperatures. One reason for varying the temperature threshold with ambient temperature is that the sensitivity of the microbolometer detector 20 changes with ambient temperature. The temperature sensor 24 of the microbolometer detector 20 may be used to give an estimate of the ambient temperature of the camera, because the temperature of the detector 20 essentially reflects the ambient temperature.

[0040] In the above, it is assumed that the integration time among the multiple integration times is fixed throughout the first sequence S1 and the second sequence S1. However, similar to the exposure time of a visible light camera, the integration time needs to be adjusted from time to time to provide proper exposure in the image frame generated by the thermal camera 10. The control unit 23 may therefore adjust one of the multiple integration times in step S110. This adjustment may be made in response to the detection of a change in the ambient temperature of the thermal camera 10. For example, this adjustment may be made if the ambient temperature of the thermal camera 10 has changed by more than a second temperature threshold since the integration time was last adjusted. Again, the temperature sensor 24 of the thermal camera 10 may be used to predict the ambient temperature. This adjustment may also be made in response to the detection that the number of saturated or black pixels in the image frame of the second sequence S2 exceeds a saturation threshold. In that case, the image frame is not properly exposed, and it is desirable that the integration time used to capture the image be adjusted.

[0041] Since the dark current value in the microbolometer detector 20 changes with integration time, it is preferable to trigger the acquisition of a new sequence of shutter images after adjusting the integration time. Therefore, according to yet another strategy, the acquisition of a third sequence S3 may be triggered by adjusting at least one of the integration times among multiple integration times.

[0042] It is understood that the above strategies for triggering the acquisition of a new sequence of shutter images may be combined. For example, as a basic rule, a new sequence of shutter images may be triggered after a predetermined time has elapsed. However, it may also be triggered if the temperature of the microbolometer detector 20 has changed by more than a temperature threshold since the last sequence of shutter images, or following one or more adjustments to the integral time value.

[0043] After correction, the image processor 22 may proceed to generate two or more video streams from images captured while the shutter 21 of the camera 10 is open. In particular, in step S112, the image processor 22 may generate a first video stream by extracting image frames having a first integration time from a second sequence of image frames S2, and generate a second video stream by extracting image frames having a second integration time shorter than the first integration time from a second sequence of image frames. It is understood that this generalizes so that one video stream is formed for each integration time. It is further understood that this is not limited to extracting image frames from the second sequence S2, but also applies to any subsequent sequences S4, S6, etc., captured while the shutter 21 is open. Video streams corresponding to different integration times may suitably undergo different types of processing or post-processing, such as different types of analysis including object detection and motion detection. In one example, the image processing unit 22 may, in step S114, monitor one of the video streams, such as a second video stream, and issue an alarm when the pixel value in that video stream exceeds an alarm threshold. Furthermore, as described above, this may be suitably used to implement an early fire alarm system.

[0044] Those skilled in the art will understand that the above embodiments can be modified in many ways, and the advantages of the present invention, as shown in the above embodiments, can still be used. The present invention should therefore not be limited to the embodiments shown herein, but should be defined solely by the claims. Furthermore, as those skilled in the art will understand, the embodiments shown herein can also be combined.

Claims

1. 1. A method for capturing a sequence of image frames in a thermal camera having a microbolometer detector, comprising: capturing a first sequence of image frames while a shutter of the thermal camera is closed and switching an integration time of the microbolometer detector among a plurality of integration times according to one or more first repetitions of a temporal pattern of integration times; capturing a second sequence of image frames while the shutter of the thermal camera is open and switching the integration time of the microbolometer detector among the plurality of integration times according to one or more second repetitions of the same temporal pattern as the temporal pattern of integration times; correcting, by subtraction, an image frame in the second sequence captured when the integration time is switched to a specific position in the temporal pattern of integration times using an image frame in the first sequence captured when the integration time is switched to the same specific position in the temporal pattern of integration times; The method includes:

2. The method of claim 1 , wherein the temporal pattern of the integration times indicates a frequency and order of switching between the integration times.

3. The method of claim 1 , wherein the temporal pattern indicates that the integration time of the microbolometer detector is switched between every image frame.

4. The method of claim 1 , wherein there are n integration times, and the temporal pattern indicates that each of the integration times is switched every nth image frame.

5. 2. The method of claim 1, wherein image frames in the second sequence captured when the integration time was switched to a particular position in the temporal pattern of integration times are corrected by subtracting an average of the image frames in the first sequence captured when the integration time was switched to the same particular position in the temporal pattern of integration times.

6. after capturing the second sequence of image frames, capturing a third sequence of image frames while the shutter of the thermal camera is closed and switching an integration time of the microbolometer detector among the plurality of integration times according to one or more third repetitions of the same temporal pattern of integration times; Imaging the third sequence includes: the temperature of the microbolometer detector has changed beyond a temperature threshold since capturing the first sequence of image frames; and a predetermined time has elapsed since capturing the first sequence of image frames; and adjusting at least one integral time among the plurality of integral times; The method of claim 1 , wherein the method is triggered by at least one of:

7. The method of claim 6 , wherein the third sequence of image frames is used to correct image frames captured after capturing the third sequence of image frames.

8. The method of claim 6 , wherein the temperature threshold varies with an ambient temperature of the thermal camera according to a predefined relationship.

9. The method of claim 1, further comprising adjusting one of the plurality of integration times in response to detecting a change in ambient temperature of the thermal camera or in response to detecting that a number of saturated or black pixels in an image frame of the second sequence exceeds a saturation threshold.

10. The method of claim 1 , wherein the second sequence of image frames is captured after the first sequence of image frames.

11. The method of claim 1 , wherein the one or more first repetitions of the temporal pattern of integration times include fewer repetitions than the one or more second repetitions of the temporal pattern of integration times. Extracting image frames having a first integration time from the second sequence of image frames to generate a first video stream; extracting image frames from the second sequence of image frames having a second integration time shorter than the first integration time to generate a second video stream; The method of claim 1 further comprising:

13. 13. The method of claim 12, comprising monitoring the second video stream and issuing an alarm when a pixel value in the second video stream exceeds an alarm threshold.

14. a microbolometer detector; The shutter, a controller for controlling the microbolometer detector and the shutter, capturing a first sequence of image frames while the shutter is closed and switching an integration time of the microbolometer detector among a plurality of integration times according to one or more first repetitions of an integration time temporal pattern; a controller configured to capture a second sequence of image frames while the shutter is open and switching the integration time of the microbolometer detector among the plurality of integration times according to one or more second repetitions of the same temporal pattern; an image processor configured to correct, by subtraction, an image frame in the second sequence captured when the integration time is switched to a specific position in the temporal pattern of integration times using an image frame in the first sequence captured when the integration time is switched to the same specific position in the temporal pattern of integration times; Including, a thermal camera.

15. A non-transitory computer readable medium comprising computer code instructions adapted, when executed by a device having processing capability, to cause a thermal camera having a microbolometer detector to perform the method of any one of claims 1 to 13.

Images