Method and thermal imager for capturing a sequence of image frames

Abstract

A method and a thermal imager for capturing a sequence of image frames are provided. The method includes capturing a first sequence and a second sequence of image frames with a shutter of the thermal imager in a closed state and an open state, respectively. When capturing each of the first sequence and the second sequence, an integration time of a microbolometer detector is switched between a plurality of integration times according to one or more repetitions of a temporal pattern of integration times. The method further includes using image frames in the first sequence captured when the integration time is switched to a specific position within the temporal pattern of integration times to correct image frames in the second sequence captured when the integration time is switched to the same specific position within the temporal pattern of integration times.

Description

Technical Field

[0001] The present invention relates to a thermal imager having a microbolometer detector. In particular, it relates to capturing a sequence of image frames using such a thermal imager. Background Art

[0002] Thermal imagers may use microbolometer sensors (as uncooled sensors) to capture image frames. In order to compensate for imperfections in the sensor, it is common practice to subtract a shutter image from the image frames captured by the image sensor. The shutter image is an image captured by the sensor when the shutter of the thermal imager is closed (i.e., when no radiation from the scene reaches the sensor), and it provides a measure of the non-uniform errors in the sensor. If there were already no defects in the sensor, the shutter image would already be a completely uniform image. However, due to the dark current in the sensor, this is not the case and there will be differences between different pixels in the shutter image. Furthermore, because the dark current is temperature dependent, these differences vary with temperature. Therefore, as the temperature of the sensor begins to drift or when there are other thermal fluctuations in the sensor, it is preferred to capture a new shutter image. For example, a new shutter image may be captured every 10 minutes or when a temperature drift is detected.

[0003] In some cases, it is desirable to expand the dynamic range of the thermal imager. That can be achieved by capturing image frames while cycling through different integration times. When cycling through different integration times, the heating of the microbolometer sensor will vary between frames, and therefore the dark current will also vary. Accordingly, as described in US7885536B1, the dark current value changes with the integration time, and therefore each integration time preferably has an associated shutter image.

[0004] However, the temperature change of the microbolometer sensor between capturing image frames with different integration times not only affects the dark current, but also has the result of introducing bias in the captured image frames. More specifically, when the microbolometer sensor switches from capturing image frames with a first integration time to capturing image frames with a second integration time, it takes some time for the temperature of the microbolometer to stabilize to the temperature level associated with the second integration time. This can introduce a bias in the sensor readout of the first frame captured after switching to the second integration time. In order to have a dark current compensation that does not introduce more systematic errors in the captured image frames, this bias is preferably taken into account when generating the shutter image. There is therefore room for improvement. Summary of the invention

[0005] In view of the above, it is therefore an object of the present invention to alleviate the above mentioned problems. In particular, it is an object to correct the dark current of a microbolometer sensor in a way that takes into account sensor readout deviations introduced when switching between different integration times.

[0006] The invention is defined by the independent claim and its various embodiments are defined by the dependent claims.

[0007] In particular, the above object is achieved by a method for capturing a sequence of image frames in a thermal imager having a microbolometer detector, the method comprising:

[0008] capturing a first sequence of image frames with a shutter of the thermal imager in a closed state and simultaneously switching an integration time of the microbolometer detector between a plurality of integration times according to one or more first repetitions of the temporal pattern of integration times,

[0009] capturing a second sequence of image frames with the shutter of the thermal imager open and simultaneously switching the integration time of the microbolometer detector between the plurality of integration times according to one or more second repetitions of the same temporal pattern of integration times, and

[0010] Image frames in a first sequence captured when the integration time was switched to a specific position within the temporal pattern of integration times are used to correct image frames in a second sequence captured when the integration time was switched to the same specific position within the temporal pattern of integration times.

[0011] The inventors have appreciated that it is advantageous to switch the integration time according to the same repetitive time pattern when capturing shutter image frames and normal image frames. In that way, the deviations in normal image frames and shutter image frames having the same relative position within the repetition of the time pattern will be the same. Furthermore, the normal image frame and the shutter image used for its correction correspond to the same relative position within the repetition of the time pattern. What this means is that the normal image frame is corrected using a shutter image frame that is not only captured with the same integration time as the normal image frame but also has the same deviation as the deviation of the normal image frame. Accordingly, dark current compensation will be more accurate and will not introduce systematic errors in the captured image frames.

[0012] According to a second aspect, the above object is achieved by a thermal imager, the thermal imager comprising a microbolometer detector, a shutter, and a controller, wherein the controller is configured to control the microbolometer detector and the shutter to:

[0013] capturing a first sequence of image frames with the shutter in a closed state and simultaneously switching an integration time of the microbolometer detector between a plurality of integration times according to one or more first repetitions of the temporal pattern of integration times,

[0014] capturing a second sequence of image frames with the shutter in an open state and simultaneously switching the integration time of the microbolometer detector between the plurality of integration times according to one or more second repetitions of the same temporal pattern, and

[0015] An image processor is configured to use image frames in a first image sequence captured when the integration time is switched to a specific position within a time pattern of the integration time to correct image frames in a second sequence captured when the integration time is switched to the same specific position within the time pattern of the integration time.

[0016] According to a third aspect, the above object is achieved by a non-transitory computer readable medium comprising computer code instructions which, when executed by a device having processing capabilities, are suitable for causing a thermal imager having a microbolometer detector to perform the method of the first aspect.

[0017] The second and third aspects may generally have the same features and advantages as the first aspect.It is further noted that the invention relates to all possible combinations of features unless explicitly stated otherwise. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and additional objects, features and advantages of the present invention will be better understood through the following illustrative and non-limiting detailed description of embodiments of the present invention with reference to the accompanying drawings, in which the same reference numerals will be used for similar elements, and in which:

[0019] Figure 1 A thermal imager arranged to monitor a scene is schematically shown.

[0020] Figure 2 A thermal imager according to an embodiment is schematically shown.

[0021] Figure 3 and Figure 4 Schematically illustrated are a first sequence and a second sequence of image frames captured when a shutter of a thermal imager is in a closed position and an open position, respectively, according to an embodiment.

[0022] Figure 5 Six sequences of image frames captured when a shutter of a thermal imager is interchangeable between a closed position and an open position according to an embodiment are shown.

[0023] Figure 6 is a flow chart of a method for capturing a sequence of image frames in a thermal imager having a microbolometer detector according to an embodiment. DETAILED DESCRIPTION

[0024] The invention will be described more fully hereinafter with reference to the accompanying drawings, in which embodiments of the invention are shown.

[0025] Figure 1 A thermal imager 10 is shown arranged to monitor a scene 12. The thermal imager 10 detects radiation in the infrared portion of the spectrum over time using a microbolometer detector and generates a sequence of image frames from the detected radiation. In the scene 12, there are objects of different temperatures that act as infrared radiation sources. For example, as shown, there may be cars, people, trees, and fires. The temperature range spanned by these objects may be large, which makes it difficult for the thermal imager 10 to correctly image all objects using a single integration time of the microbolometer detector. For example, objects of lower temperatures such as people or cars may require longer integration times than objects of higher temperatures such as fire. If a longer integration time is used, the temperature will be able to be resolved for objects of lower temperatures, but not for objects of higher temperatures because the microbolometer detector will be saturated, and vice versa. In order to cover the entire temperature range, the thermal imager 10 may instead have to operate with several different integration times that are optimal for different parts of the temperature range. For example, the thermal imager 10 may switch the integration time of the microbolometer detector between capturing subsequent image frames.

[0026] In some embodiments, the thermal imager 10 may combine subsequent image frames captured with different integration times into a single image frame to produce a high dynamic range image. In other embodiments, the thermal imager 10 extracts image frames with a first integration time from a sequence of image frames to produce a first video stream, and extracts image frames with a different second integration time from a sequence of image frames to produce a second video stream. In the latter case, the thermal imager will thus produce several separate video streams corresponding to different integration times. These separate video streams may be subjected to different types of processing or post-processing, such as different types of analysis including object detection, motion detection, etc. For example, these separate video streams may be advantageously used to implement an early fire alarm system. The integration time of the first stream may be selected so that it is suitable for detecting objects in a lower temperature range such as the temperature range of humans. The integration time of the second stream may be selected so that it is suitable for detecting objects in a higher temperature range such as the temperature range suitable for detecting a newly ignited fire (temperatures above 250°C). The first video stream may then be used to monitor humans, such as performing analysis to detect human objects, estimate the movement of human objects, and track human objects. The second stream may be used to monitor the occurrence of a fire. For example, the second stream may be monitored to issue an alarm when pixel values ​​in the second video stream exceed an alarm threshold.

[0027] Figure 2The thermal imager 10 is shown in more detail. It includes a microbolometer detector 20, a shutter 21, an image processor 22, and a controller 23. The microbolometer detector 20 may also be referred to as a microbolometer sensor 20. The microbolometer detector 20 may also include a temperature sensor 24 that measures the temperature of the microbolometer detector 20.

[0028] As is known in the art, the microbolometer detector 20 or microbolometer sensor is arranged to detect infrared radiation from the scene 12. When an image frame is generated, the microbolometer detector 20 integrates the detected infrared radiation during a time period referred to as an integration time. The integration time used for each frame is set by the controller 23. Specifically, the controller 23 can control the microbolometer detector 20 to switch between multiple integration times so that subsequent image frames have different integration times. The switching can follow a repeated predefined time pattern. The repetition of the time pattern can be regarded as a sequence of integration times that defines which integration time should be used for many subsequent image frames. In general, the time pattern of the integration times can be said to indicate the frequency and order of switching between multiple integration times. This is in Figure 3 It is further shown in Figure 3 A sequence of image frames I1-I14 captured using a microbolometer detector 20 is shown. When capturing image frames I1-I14, the microbolometer detector 20 is controlled to switch the integration time according to a plurality of repetitions R1-R7 of the time pattern of the integration time. The time pattern is the same in each repetition R1-R7. The plurality of repetitions R1-R7 of the time pattern can therefore be said to form a repetitive time pattern. In this example, the plurality of integration times includes two integration times T1 and T2, and each repetition of the time pattern is given by T1 followed by T2 (i.e., by a sequence of integration times {T1, T2}). The integration time T1 is located at a first position within the time pattern, i.e., it is the first in order within the sequence of integration times, and the integration time T2 is located at a second position within the time pattern, i.e., it is the second in order within the sequence of integration times. As used herein, the position within the time pattern of the integration time therefore refers to the sequential position within the sequence of integration times defined by the repetition of the time pattern.

[0029] exist Figure 3In the example, the time pattern indicates that the individual integration times T1 and T2 are switched every two frames. This example can be generalized to the case where there are n ≥ 2 integration times. Specifically, there may be n integration times, and the time pattern may indicate that the individual integration times are switched every n frames. In the above example, the time pattern indicates that the integration time of the microbolometer detector is to be switched between each image frame. However, it is also possible that the time pattern indicates that one or more of the multiple integration times are to be repeated continuously for more than one frame. For example, in the case of two integration times, the repetition of the time pattern may be {T1, T2, T2}, such as Figure 4 . In that case, integration time T1 is located at a first position within the time pattern, and integration time T2 is located at a second and third position within the time pattern. Another example of a time pattern with two integration times is {T1, T1, T2, T2}. As those skilled in the art will appreciate, a repeating time pattern can be formed from multiple integration times in many ways, and the examples given above are only some of many possible time patterns.

[0030] When switching between different integration times, different levels of current are driven through the pixels of the microbolometer detector 20. Longer integration times produce higher currents than shorter integration times. These different currents have an effect on the temperature of the microbolometer detector 20, and will ultimately affect the signal output of the microbolometer detector 20. For example, when switching from T1 to T2 (where T1>T2), the higher current during T1 will provide the microbolometer 20 with a higher temperature before the switch than it would have if the integration time had been replaced with T2. As a result, the signal output of the next frame will be higher than it should be. In other words, the switch introduces a deviation in the next image frame captured using the integration time. The same applies when switching back from T2 to T1 again, although in that case the deviation is in the negative direction, resulting in a signal output that is lower than it should be. If a number of consecutive frames are captured with the same integration time, the deviation will decrease over time because the temperature of the microbolometer detector 20 will stabilize over time. This will, for example, be Figure 4 In the example of , where two consecutive image frames are captured using integration time T2. In that case, it should be noted that the deviation will be different for subsequent image frames captured with the same integration time T2. Specifically, the first image frame captured after switching from T1 to T2 will have a larger deviation than the second image frame captured after the switch. However, it should also be noted that the deviation will be the same for two image frames having the same position within the repetition of the temporal pattern. For example, in Figure 3The image frames I2, I4, I6, I8, I10, I12 and I14 in will be associated with a first deviation, while the image frames I1, I3, I5, I7, I9, I11, I13 are associated with different second deviations. Figure 4 In the embodiment described herein, image frames J1, J4, J7, J10, J13, J16, J19 will have a first deviation, image frames J2, J5, J8, J11, J14, J17, J20 will have a second, and image frames J3, J6, J9, J12, J15, J18, J21 will have a third deviation. The embodiments described herein take advantage of this fact to counteract the deviations when performing correction of the image frames.

[0031] The controller 23 can further control the shutter 21 to be in an open or closed state. Specifically, it can control the shutter 21 to be in a closed state during one or more first repetitions R1-R2 of the time pattern of the integration time, and to be in an open state during one or more second repetitions R3-R7 of the time pattern. In contrast to the open state, when the shutter 21 is in the closed state, no radiation from the scene 12 enters the thermal imager 10 to reach the microbolometer detector 20. Images captured when the shutter is in the closed state are sometimes referred to as shutter images because they depict the closed shutter 21 instead of the scene 12. Other names are flat field correction images and non-uniformity correction images. If there are no defects in the microbolometer detector 20, the shutter images will be completely uniform, that is, have the same pixel value in all pixels. However, in reality, they will be non-uniform due to defects in the microbolometer detector 20 called dark current. The role of the shutter image is to measure these defects so that they can be offset from the image frames captured when the shutter 21 is in the open state.

[0032] As used herein, one or more first repetitions of a time pattern of an integration time refers to one or more subsequent repetitions of the time pattern. Similarly, one or more second repetitions of a time pattern refers to one or more subsequent repetitions of the time pattern. Figure 3 , where the shutter is in a closed state during repetitions R1-R2 of the temporal pattern and in an open state during repetitions R3-R7 of the temporal pattern. Image frames I1-I4 captured when the shutter is in the closed state form a first sequence S1 of image frames, and image frames I5-I14 captured when the shutter is in the open state form a second sequence S2 of image frames. The first sequence and the second sequence can be regarded as a first subsequence and a second subsequence of a sequence of image frames captured by the thermal imager 10.

[0033] It is understood that the number of first repetitions and the number of second repetitions are not limited to Figure 3, but may vary in different embodiments. However, one or more first repetitions of the temporal pattern of the integration time beneficially include fewer repetitions than one or more second repetitions of the temporal pattern of the integration time. In that way, most of the captured images will be useful images depicting the scene rather than the closed shutter. Nevertheless, it may be beneficial to include more than one repetition of the temporal pattern in the first repetition in order to have a chance to average out random noise in the images captured when the shutter is closed.

[0034] The thermal imager 10 thus includes an image processor 22 and a controller 23 configured to implement various functions of the thermal imager. In general, the thermal imager may include circuits configured to implement the controller 23 and the image processor 22, and more specifically, circuits configured to implement their functions.

[0035] In a hardware implementation, the image processor 22 and the controller 23 may correspond to circuits that are dedicated and specifically designed to provide the functionality of the respective components. The circuits may take the form of one or more integrated circuits, such as one or more application specific integrated circuits or one or more field programmable gate arrays. For example, the controller 23 may include circuits that, when in use, cause the thermal imager 10 to capture a first sequence and a second sequence of image frames, while switching the integration time of the microbolometer detector 20 and causing the shutter 21 to be in a closed state and an open state, respectively. Similarly, the image processor 22 may include circuits that, when in use, cause the image frames in the first sequence to be used to correct the image frames in the second sequence. For example, the image processor 22 may form part of an image processing pipeline of the thermal imager 10.

[0036] In a software implementation, the circuitry may alternatively take the form of a processor such as a microprocessor associated with computer code instructions stored on a (non-transitory) computer readable medium such as a non-volatile memory, causing the device 104 to perform any of the methods disclosed herein. Examples of non-volatile memory include read-only memory, flash memory, ferroelectric RAM, magnetic computer storage devices, and optical disks, etc. In the case of software, the controller 22 and the image processor may therefore each correspond to a portion of the computer code instructions stored on the computer readable medium, which, when executed by the processor, causes the device thermal imager 10 to perform the functions of the components.

[0037] It will be appreciated that it is also possible to have a combination of hardware and software implementation, meaning that the functionality of one of the controller 23 or the image processor 22 is implemented in hardware and the other in software.

[0038] Now refer to Figures 2 to 4 as well as Figure 6A method for capturing a sequence of image frames in a thermal imager 10 having a microbolometer detector 20 is described by referring to a flowchart of FIG. 1 . Optional features of the method steps are Figure 6 Indicated by dotted line.

[0039] In step S102, the thermal imager 10 captures a first sequence S1 of image frames I1-I4 while switching the integration time of the microbolometer detector 20 between a plurality of integration times I1, I2 according to one or more first repetitions R1-R2 of the temporal pattern of integration times. In more detail, the controller 23 of the thermal imager 10 jointly controls the microbolometer detector 20 and the shutter 21 to capture the first image sequence S1.

[0040] In step S104 , the thermal imager 10 captures a second sequence S2 of image frames I5 - I14 while switching the integration time of the microbolometer detector 20 between a plurality of integration times I1 , I2 according to one or more second repetitions R3 - R7 of the same temporal pattern.

[0041] like Figure 3 As shown in , a second sequence S2 of image frames may be captured after a first sequence S1 of image frames. Advantageously, when the images in the second sequence S2 are captured, the shutter images in the first sequence S1 for correcting the images in the second sequence S2 are already available. Thus, the images in the second sequence S2 may be corrected immediately without having to wait for the shutter image to be captured. However, embodiments are also conceivable in which the first sequence S1 is captured after the second sequence S2.

[0042] Once captured, the image frames of the first sequence S1 and the second sequence S2 are input to the image processor 22. The image processor 22 uses the image frames in the first sequence S1 to correct the image frames I5-I14 in the second sequence S2. The correction is made to reduce the effects of dark current in the microbolometer detector 20, which otherwise may cause some pixels to have misleading pixel values. In more detail, in step S106, the image processor 22 uses the image frames in the first sequence S1 captured when the integration time was switched to a specific position within the time pattern of the integration time to correct the image frames in the second sequence S2 captured when the integration time was switched to the same specific position within the time pattern of the integration time.

[0043] Refer again Figure 3, the time pattern of the integration times corresponds to the sequence of integration times {T1, T2}, where T1 is located at a first position in the time pattern and T2 is located at a second position in the time pattern. Image frames I5, I7, I9, I11 and I13 in the second sequence S2 are all captured when the integration time is switched to the first position in the time pattern. Similarly, image frames I1, I3 in the first sequence S1 are captured when the integration time is switched to the first position in the time pattern. In this example, the image processor 22 will therefore use the image frames I1, I3 in the first sequence S1 to correct the image frames I5, I7, I9, I11 and I13 in the second sequence S2. In the same way, the image processor 22 will use the image frames I2 and I4 in the first sequence S1 to correct the image frames I6, I8, I10, I12 and I14 in the second sequence S2. Obviously, the image frames I5, I7, I9, I11 and I13 and the image frames I1, I3 used for their correction have the same position within the repetition of the temporal pattern. The same applies to the image frames I6, I8, I10, I12 and I14 and the image frames I2, I4 used for their correction.

[0044] Figure 4 Another example is shown, in which the time pattern of the integration times corresponds to a sequence of integration times {T1, T2, T2}, in which T1 is located at a first position in the time pattern, and T2 is located at a second position and a third position in the time pattern. In this case, image frames J7, J10, J13, J16, J19 in the second sequence S2 are captured when the integration time is switched to the first position in the time pattern, and are therefore corrected using image frames J1, J4 in the first sequence S1. Image frames J8, J11, J14, J17, J20 in the second sequence S2 are captured when the integration time is switched to the second position in the time pattern and are therefore corrected using image frames J2, J5 in the first sequence S1. Image frames J9, J12, J15, J18, J21 in the second sequence S2 are captured when the integration time is switched to the second position in the time pattern and are therefore corrected using image frames J3, J6 in the first sequence S1.

[0045] Therefore, the image frames in the second sequence S2 are not only corrected using the image frames in the second sequence S1 having the same integration time, but the integration time has the same position within the time pattern of the integration time. Since they have the same position within the time pattern of the integration time, the image frames to be corrected and the image frames used for correction are associated with the same deviation. This makes it possible to cancel the deviation when performing the correction.

[0046] In order to perform correction for the image frames in the second sequence 22, the image processor 22 may first form an average value of the image frames in the first image sequence S1 for correction. The image processor 22 may calculate and temporarily store the average value once it has received the image frames of the first sequence S1. In that way, it is available for use by the image processor 22 when the image frames of the second sequence 22 are captured. Specifically, the image frames in the second sequence S2 captured when the integration time is switched to the same specific position within the time pattern of the integration time may be corrected by using the average value of the image frames in the first image sequence S1 captured when the integration time is switched to a specific position within the time pattern of the integration time. The average value is typically a pixel average value, and correction may be performed by subtracting the pixel average value from the image frames in the second sequence S2. By averaging, the sensitivity to measurement noise is reduced. In addition, by making the subtraction, the above-mentioned deviations are offset. For example, the image processor 22 may form a first pixel average value for image frames I1 and I3, and a second pixel average value for image frames I2 and I4. The image processor 22 may then subtract the average value so formed from the image frames from the image frames in the second sequence S2 to perform correction. Figure 3 In the example, the image processor 22 will therefore subtract the pixel average of frames I1 and I3 from each of image frames I5, I7, I9, I11 and I13. Similarly, the image processor 22 will subtract the pixel average of frames I2 and I4 from each of image frames I6, I8, I10, I12 and I14.

[0047] Subtracting the pixel average from the image frames in the second sequence S2 may affect the overall signal level in the image frames in the second sequence S2. In order to maintain the overall signal level, the grand average of the related shutter images from the first sequence S1 may be added to the image frames in the second sequence S2. The grand average refers to the average value across all pixels in all related shutter images. For example, the image processor 21 may form a first grand average by averaging all pixel values ​​in frames I1 and I3, and add the first grand average to image frames I5, I7, I9, I11, and I13. Similarly, the image processor 22 may form a second grand average by averaging all pixel values ​​in frames I2 and I4, and add the second grand average to image frames I6, I8, I10, I12, and I14. Similarly, for the above content regarding pixel averages, the image processor 22 may calculate and temporarily store these grand averages once the image frames of the first sequence S1 have been received. As an alternative, the image processor 22 may combine the pixel average and the grand average into a correction image in advance (e.g., by subtracting the grand average from the pixel average), rather than first subtracting the pixel average and then adding the grand average of the shutter image. The correction image may then be used to correct the image frames in the second sequence S2 in one operation (e.g., by subtracting the correction image from the image frames in the second sequence S2).

[0048] As described above, the image processor 22 uses the shutter image of the first image sequence S1 to correct the image frames in the second sequence S2. However, sometimes, the thermal imager 10 preferably captures a new sequence of shutter images. Therefore, in some embodiments, the controller 23 may control the microbolometer detector 20 and the shutter 21 in step S108 to capture a third sequence of image frames with the shutter 21 of the thermal imager 10 in a closed state, and simultaneously switch the integration time of the microbolometer detector 20 between multiple integration times according to one or more third repetitions of the same time pattern of integration times. The third sequence of image frames may be captured after the second sequence of image frames S2. The third sequence of image frames may be used to correct image frames captured after the third sequence of image frames is captured. This is Figure 5 It is further shown in Figure 5 A third sequence of image frames S3 captured after the second sequence of image frames S2 when the shutter is in a closed state, and a fourth sequence of image frames S4 captured after the third sequence S3 when the shutter is in an open state are shown. Similar to what has been described above, the image frames of the third sequence S3 can be used to correct the image frames in the fourth sequence S4. Figure 5 As shown in , this process is generally repeated such that the thermal imager 10 keeps capturing a sequence of image frames with the shutter interchangeably in closed and open states.

[0049] The reason for sometimes capturing a new shutter image is that the dark current in the microbolometer detector 20 increases with the sensor temperature. If there has been a considerable drift in the sensor temperature since the last shutter image was captured, the shutter image therefore becomes outdated and a new shutter image is preferably captured. Different strategies can be used to trigger the capture of a new shutter image. A simple strategy is to capture a new shutter image when a predetermined amount of time has passed since the capture of the previous shutter image. Thus, the capture of the third sequence S3 can be triggered by a predetermined amount of time having passed since the capture of the first sequence of image frames. For example, a new shutter image can be captured every ten minutes. Another more sophisticated strategy is to capture a new shutter image when it is found that the temperature of the microbolometer detector 20 has drifted by more than a certain value since the last shutter image was captured. Therefore, the capture of the third sequence S3 can be triggered by the temperature of the microbolometer detector having changed by more than a temperature threshold since the capture of the first sequence of image frames. For example, the temperature threshold can be set to 0.2°C. However, the value used for the temperature threshold may depend on the sensitivity of the microbolometer detector 20 and which type of camera module is used. To implement this strategy, the controller 23 may use the temperature sensor 24 of the microbolometer detector 20. Specifically, it may monitor the temperature of the microbolometer detector 20 and trigger the capture of a new sequence S3 of shutter images when the temperature of the detector 20 has drifted by more than the temperature threshold since the capture of the preview sequence S1 of shutter images. For example, it may use the temperature of the microbolometer detector 20 when capturing the first image frame I1 of the previous sequence S1 as a reference for its comparison. The temperature threshold may be a predetermined value. Alternatively, the temperature threshold may vary with the ambient temperature of the thermal imager according to a predefined relationship. For example, the predefined relationship may specify that for lower ambient temperatures, a lower temperature threshold is used than for higher ambient temperatures. The reason for varying the temperature threshold with ambient temperature is that the sensitivity of the microbolometer detector 20 varies with ambient temperature. The temperature sensor 24 of the microbolometer detector 20 may be used to give an assessment of the ambient temperature of the camera, because the temperature of the detector 20 substantially reflects the ambient temperature.

[0050] In the above, it is assumed that the integration time among the plurality of integration times is fixed throughout the first sequence S1 and the second sequence S2. However, similar to the exposure time of a visible light camera, the integration time must sometimes be adjusted to give a proper exposure in the image frame produced by the thermal imager 10. Therefore, the control unit 23 can adjust the integration time among the plurality of integration times in step S110. The adjustment can be made in response to detecting a change in the ambient temperature of the thermal imager 10. For example, if the ambient temperature of the thermal imager 10 has changed by more than a second temperature threshold since the last adjustment of the integration time, the adjustment can be made. Furthermore, the temperature sensor 24 of the thermal imager 10 can be used to assess the ambient temperature. Adjustments can also be made in response to detecting a saturated or black pixel or number in the image frame of the second sequence S2 exceeding a saturation threshold. In that case, the image frame is not properly exposed, and the integration time used when capturing the image is preferably adjusted.

[0051] Since the dark current value in the microbolometer detector 20 changes with the integration time, it is preferred to trigger the capture of a new shutter image sequence after the integration time is adjusted. Therefore, according to another strategy, the capture of the third sequence S3 can be triggered by adjusting at least one integration time among the multiple integration times.

[0052] It is understood that the above strategies for triggering the capture of a new shutter image sequence may be combined. For example, as a basic rule, a new shutter image sequence may be triggered once a predetermined amount of time has elapsed. However, it may be triggered earlier than that if the temperature of the microbolometer detector 20 has changed by more than a temperature threshold since the last shutter image sequence, or after adjustment of one or more integration time values.

[0053] After performing the correction, the image processor 22 may continue to generate two or more video streams from the images captured when the shutter 21 of the camera 10 is in an open state. Specifically, in step S112, the image processor 22 may extract image frames having a first integration time from the second sequence S2 of image frames to generate a first video stream, and extract image frames having a second integration time shorter than the first integration time from the second sequence of image frames to generate a second video stream. It is understood that this is summarized as forming one video stream for each integration time. It is further understood that this is not limited to extracting image frames from the second sequence S2, but rather extracting image frames from any subsequent sequence S4, S6, etc. captured when the shutter 21 is open. The video streams corresponding to different integration times may advantageously be subjected to different types of processing or post-processing, such as different types of analysis including object detection, motion detection, etc. In one example, the image processing unit 22 may monitor one of the video streams (such as the second video stream) in step S114 to issue an alarm when a pixel value in the video stream exceeds an alarm threshold. As further described above, this may advantageously be used to implement an early fire alarm system.

[0054] It will be appreciated that the above embodiments may be modified in many ways by those skilled in the art and still use the advantages of the invention as shown in the above embodiments. Therefore, the present invention should not be limited to the embodiments shown, but should only be limited by the appended claims. In addition, as the skilled person understands, the embodiments shown may be combined.

Claims

1. A method for capturing a sequence of image frames in a thermal imager having a microbolometer detector, comprising: capturing a first sequence of image frames with a shutter of the thermal imager in a closed state and simultaneously switching the integration time of the microbolometer detector between a plurality of integration times according to one or more first repetitions of the temporal pattern of integration times, capturing a second sequence of image frames with the shutter of the thermal imager in an open state and simultaneously switching the integration time of the microbolometer detector between the plurality of integration times according to one or more second repetitions of the same temporal pattern of integration times, and Using image frames in the first sequence captured when the integration time is switched to a specific position within the time pattern of integration time, image frames in the second sequence captured when the integration time is switched to the same specific position within the time pattern of integration time are corrected by subtraction.

2. The method according to claim 1, wherein: The time pattern of integration times indicates the frequency and order of switching between the plurality of integration times.

3. The method according to claim 1, wherein: The temporal pattern dictates that the integration time of the microbolometer detector is to be switched between each image frame.

4. The method according to claim 1, wherein: There are n integration times, and the time pattern indicates that the respective integration times are switched every n image frames.

5. The method according to claim 1, wherein: The image frames in the second sequence captured when the integration time is switched to a specific position within the time pattern of integration time are corrected by subtracting the average value of the image frames in the first sequence captured when the integration time is switched to the same specific position within the time pattern of integration time.

6. The method according to claim 1, further comprising: after capturing said second sequence of image frames, capturing a third sequence of image frames with said shutter of said thermal imager in a closed state and simultaneously switching the integration time of said microbolometer detector between said plurality of integration times according to one or more third repetitions of the same temporal pattern of integration times, Wherein, the capture of the third sequence is triggered by at least one of the following: the temperature of the microbolometer detector has changed by more than a temperature threshold since the capture of the first sequence of image frames, a predetermined amount of time has elapsed since said capturing of said first sequence of image frames, and Adjustment of at least one integration time among the plurality of integration times.

7. The method according to claim 6, wherein: The third sequence of image frames is used to correct image frames captured after the capturing of the third sequence of image frames.

8. The method according to claim 6, wherein: The temperature threshold varies with the ambient temperature of the thermal imager according to a predefined relationship.

9. The method according to claim 1, further comprising: In response to detecting a change in the ambient temperature of the thermal imager or in response to detecting that the number of saturated or black pixels in the image frames of the second sequence exceeds a saturation threshold, an integration time among the plurality of integration times is adjusted.

10. The method according to claim 1, wherein: The second sequence of image frames is captured after the first sequence of image frames.

11. The method according to claim 1, wherein: The one or more first repetitions of the temporal pattern of integration times comprise fewer repetitions than the one or more second repetitions of the temporal pattern of integration times.

12. The method according to claim 1, further comprising: extracting image frames having a first integration time from said second sequence of image frames to produce a first video stream, Image frames having a second integration time shorter than the first integration time are extracted from the second sequence of image frames to produce a second video stream.

13. The method according to claim 12, comprising: The second video stream is monitored to issue an alarm when a pixel value in the second video stream exceeds an alarm threshold.

14. A thermal imager comprising: Microbolometer detector, shutter, a controller configured to control the microbolometer detector and the shutter to: capturing a first sequence of image frames with the shutter in a closed state and simultaneously switching the integration time of the microbolometer detector between a plurality of integration times according to one or more first repetitions of the temporal pattern of integration times, capturing a second sequence of image frames with the shutter in an open state and simultaneously switching the integration time of the microbolometer detector between the plurality of integration times according to one or more second repetitions of the same temporal pattern, and An image processor is configured to use image frames in the first sequence captured when the integration time is switched to a specific position within the time pattern of integration times to correct image frames in the second sequence captured when the integration time is switched to the same specific position within the time pattern of integration times by subtraction.

15. A non-transitory computer readable medium comprising computer code instructions adapted to cause a thermal imager having a microbolometer detector to perform the method of claim 1 when executed by a device having processing capabilities.

Images