Infrared imaging device

Abstract

The present invention comprises an imaging element (100), an ADC (16) that converts an electric signal to a digital signal, a frame memory (18) that stores the digital signal as a thermal image; a background information memory (20) that stores background information; a state determination unit (22) that determines whether a subtraction value which is obtained by subtracting the background information from the thermal image is greater than thA; a background information update unit (24) that updates the background information memory (20) by storing therein, as new background information, a value obtained by using a number a to calculate a function f(a,T,B) in which the proportion of T among T and B increases along with an increase in the number a, where T is a value of the thermal image and B is a value of the background information; and a thermal image generation unit (26) that subtracts the background information from the thermal image and outputs the revised thermal image. When a1 is defined as the value of the number a when the state determination unit (22) has determined that the subtraction value is greater than thA, and a2 is defined as the value of the number a when the state determination unit (22) has determined that the subtraction value is not greater than thA, a1 and a2 satisfy the expression a1<a2.

Description

Infrared imaging device

[0001] The present disclosure relates to infrared imaging devices.

[0002] Conventionally, infrared imaging devices capable of capturing two-dimensional images have been used for business, industrial, military, etc. in fields such as surveillance cameras, night vision devices, thermography, or forward monitoring devices mounted on vehicles, aircraft, etc. Meanwhile, in recent years, the prices of infrared imaging devices have been falling, and they are now being used for consumer purposes as well, such as infrared imaging devices that use SOI (Silicon On Insulator) diodes and are capable of capturing two-dimensional images being mounted on home air conditioners.

[0003] In thermal infrared imaging devices, an imaging element is constructed by arranging thermally insulating pixels in a two-dimensional array, and infrared images are captured by utilizing the temperature change of the pixels due to incident infrared rays. Specifically, an electric signal is generated by injecting a current or applying a voltage to the pixels. When infrared rays are incident, the temperature of the pixels fluctuates slightly, causing the electric signal to fluctuate slightly. This fluctuation in the electric signal is amplified, converted into a digital signal, and output to the outside.

[0004] Here, the pixel temperature is affected not only by temperature changes caused by incident infrared rays, but also by temperature changes due to self-heating of the pixel itself caused by application of current or voltage, heat generation by components of the image sensor, and changes in ambient temperature. Furthermore, the degree of self-heating varies depending on the thermal insulation performance, power consumption, applied voltage, and current of each pixel, resulting in differences in output level between pixels. Amplifier variations also contribute to output level differences. These output level differences appear as fixed pattern noise on the image sensor, and are therefore called fixed pattern noise (FPN). Since the thermal insulation performance and power consumption of each pixel change depending on the ambient temperature, fluctuations in output level due to changes in ambient temperature become significant.

[0005] As described above, the thermal imaging data captured by an infrared imaging device contains a mixture of actual imaging data capturing the energy emitted from the subject and fixed pattern noise specific to the device. To obtain a correct thermal image, it is necessary to generate fixed pattern noise data and remove the fixed pattern noise data from the captured thermal imaging data.

[0006] Known methods for obtaining fixed pattern noise data include capturing an image with the optical system covered with a shutter to block infrared light from outside, and pointing the visual axis at something that is thought to have a relatively uniform temperature (such as the sky), blurring the focus of the lens, and performing calibration without closing the correction shutter.

[0007] However, low cost is important for consumer applications, and to further reduce the price, it is desirable to eliminate the mechanical shutter used to acquire fixed pattern noise data from the optical system of the infrared imaging device, and it is also desirable for the optical system to have a fixed focus.

[0008] For example, Patent Document 1 discloses an infrared imaging device that acquires fixed pattern noise data without using a mechanical shutter.

[0009] International Publication No. WO2022 / 264390

[0010] The infrared imaging device described in Patent Document 1 generates fixed pattern noise data using information about multiple images. This requires a frame memory that can store multiple images. As a result, the memory capacity required for the frame memory increases.

[0011] The present disclosure has been made to solve the above problems, and has an object to provide an infrared imaging device that can suppress an increase in the memory capacity required for the frame memory.

[0012] The infrared imaging device according to the present disclosure includes an imaging element that receives infrared rays emitted from an imaging target area including a subject and a background and outputs an electrical signal corresponding to the intensity of the infrared rays; an ADC that converts the electrical signal into a digital signal; a frame memory that stores the digital signal as a thermal image; a background information memory that stores background information calculated based on the thermal image; a state determination unit that determines, for each pixel, whether a subtraction value obtained by subtracting the background information from the thermal image is greater than thA, which is a positive threshold value; a background information update unit that calculates, for each pixel, a function f(a, T, B) using a number a, where T is the value of the thermal image and B, and the ratio of T to B increases as a increases, and updates the value obtained by calculating this function f(a, T, B) in the background information memory as new background information and stores the calculated value; and a thermal image generation unit that subtracts the background information from the thermal image for each pixel and outputs a corrected thermal image, wherein the value of a when the state determination unit determines that the subtraction value is greater than thA is a1, and the value of a when the state determination unit determines that the subtraction value is not greater than thA is a2, where a1 and a2 satisfy a1<a2.

[0013] According to the present disclosure, an infrared imaging device can be obtained that can suppress an increase in the memory capacity required for the frame memory.

[0014] 1 is a diagram illustrating a configuration of an infrared imaging device according to a first embodiment. FIG. 2 is a flowchart illustrating an operation of the infrared imaging device according to the first embodiment. FIG. 3 is a diagram illustrating thermal images before and after correction of the infrared imaging device according to the first embodiment. FIG. 4 is a diagram illustrating the frequency of thermal images before and after correction of the infrared imaging device according to the first embodiment. FIG. 5 is a diagram illustrating a configuration of an infrared imaging device according to a second embodiment. FIG. 6 is a flowchart illustrating an operation of the infrared imaging device according to the second embodiment. FIG. 7 is a diagram illustrating the frequency of thermal images before and after correction of the infrared imaging device according to the second embodiment. FIG. 8 is a diagram illustrating a thermal image output by the infrared imaging device according to the second embodiment. FIG. 9 is a diagram illustrating a method for calculating a temperature subtraction value of the infrared imaging device according to the second embodiment. FIG. 10 is a diagram illustrating a method for calculating a temperature subtraction value of the infrared imaging device according to the second embodiment. FIG. 11 is a diagram illustrating a method for calculating a temperature subtraction value of the infrared imaging device according to the second embodiment. FIG. 12 is a diagram illustrating a configuration of an infrared imaging device according to a third embodiment. FIG. 13 is a diagram illustrating a configuration of a modified example of the infrared imaging device according to the third embodiment. FIG. 14 is a flowchart illustrating the operation of the infrared imaging device according to the third embodiment. FIG. 15 is a flowchart illustrating the operation of the infrared imaging device according to the third embodiment. FIG. 16 is a diagram illustrating the frequency of thermal images before and after correction of the infrared imaging device according to the third embodiment. Fig. 10 is a flowchart showing the operation of the infrared imaging device according to embodiment 4. Fig. 11 is a diagram showing the frequency of thermal images before and after correction of the infrared imaging device according to embodiment 4. Fig. 12 is a diagram showing the configuration of the infrared imaging device according to embodiment 5. Fig. 13 is a flowchart showing the operation of the infrared imaging device according to embodiment 5. Fig. 14 is a diagram showing thermal images before and after correction of the infrared imaging device according to embodiment 5 and their frequency.

[0015] 1 is a diagram showing the configuration of an infrared imaging device 100 according to embodiment 1. The infrared imaging device 100 includes an optical system 12, an imaging element 14, an ADC 16, a frame memory 18, a state determination unit 22, a background information update unit 24, a background information memory 20, and a thermal image generation unit 26.

[0016] The optical system 12 is an imaging optical system including one or more lenses, and receives and collects infrared light emitted from an imaging target area including a subject and background.

[0017] The image sensor 14 has a plurality of pixels arranged in a two-dimensional array. The pixels receive infrared light collected by the optical system 12, and each pixel outputs an electrical signal corresponding to the intensity of the infrared light. Each pixel is, for example, a temperature sensor formed of an SOI diode with a thermal insulating structure. Alternatively, the image sensor may have a resistive bolometer or thermopile structure. The pixels receive infrared light via the optical system 12, and output a voltage signal corresponding to the intensity of the infrared light by utilizing the change in pixel temperature caused by the received infrared light. The voltage signal is output as an analog voltage signal. Images are captured at regular intervals.

[0018] The ADC 16 is an A / D converter (Analog to Digital Converter) that converts the electrical signal output from the image sensor 14 into a digital signal and outputs it.

[0019] The frame memory 18 is a memory that stores the digital signal output from the ADC 16 as a thermal image. The data stored in the frame memory 18 is two-dimensional thermal image data corresponding to the intensity of infrared light received by the pixels of the image sensor 14. This thermal image contains not only components corresponding to the intensity of infrared light emitted from the imaging target area, but also fixed pattern noise inherent to the image sensor 14. If the imaging target area includes a moving subject such as a person, the thermal image will contain both components emitted from the subject and components generated from the stationary background. The data that make up one thermal image is called frame data. The data is handled in frame units.

[0020] The state determination unit 22 calculates the difference between the thermal image stored in the frame memory 18 and the background information stored in the background information memory 20 (described later) for each pixel, and determines the subject imaging state, background imaging state, or false detection state. The processing by the state determination unit 22 may be performed for each frame, or may be performed once for multiple frames. These states will be described in detail later.

[0021] The background information update unit 24 weights the thermal image stored in the frame memory 18 and the background information stored in the background information memory 20 for each pixel according to the state determined by the state determination unit 22, and updates and stores the weighted thermal image and background information in the background information memory 20 as new background information.

[0022] The background information memory 20 is a memory for storing background information. The background information is two-dimensional data that can contain information on the background area of ​​the imaging target area as well as the background behind the location of the subject, even if the subject is in that area. The size of this data is the same as the thermal image stored in the frame memory 18. The background information also includes fixed pattern noise generated by the imaging element 14.

[0023] The thermal image generating unit 26 subtracts the background information stored in the background information memory 20 from the thermal image stored in the frame memory 18 for each pixel, and outputs the result as a corrected thermal image.

[0024] Next, a description will be given of the operation of the infrared imaging device 100 according to the first embodiment.

[0025] When the infrared imaging device 100 starts operating, in step S101, an image is captured by the imaging element 14. Next, in step S102, the ADC 16 converts the two-dimensional electrical signals captured by the imaging element 14 into a thermal image, which is stored in the frame memory 18. The background information initially stored in the background information memory 20 immediately after the infrared imaging device 100 starts operating may be the first thermal image stored in the frame memory 18, a thermal image stored in the frame memory 18 after operation has stabilized, or a pre-prepared pattern. Using the first thermal image stored in the frame memory 18 allows for early start of image acquisition. Furthermore, using a thermal image stored in the frame memory 18 after operation has stabilized avoids unstable image output immediately after startup. Immediately after startup, the sensor output image may be distorted due to voltage stability issues, etc. Furthermore, as the sensor temperature stabilizes, temperature drift occurs, and the output stored in the frame memory 18 is likely to fluctuate significantly. Therefore, correction is more likely to be successful after the sensor temperature has stabilized to a certain extent. Furthermore, by using a prepared pattern, the unstable state immediately after startup can be avoided, shortening the time until stable operation.

[0026] Next, in step S103, the state determination unit 22 performs a state determination. In step S103, a subtraction value is calculated for each pixel by subtracting the background information stored in the background information memory 20 from the thermal image stored in the frame memory 18. If the subtraction value is greater than thA, which is a positive threshold value, the pixel is determined to be in a subject imaging state. Generally, the surface temperature of a person, who is considered to be a subject, is 33 to 34°C, which is higher than the background temperature and environmental temperature. Therefore, if the subtraction value is greater than thA, the pixel is considered to be in a state in which a subject has been imaged (subject imaging state).

[0027] If it is determined that the subject is in an imaging state, in step S104, the background information update unit 24 updates the background information memory 20 with a value calculated using the formula a1*T+(1-a1)*B, using a number a1 satisfying 0<a1<1. The value is stored as new background information. Here, T is the value of the target pixel in the thermal image stored in the frame memory 18, and B is the value of the same pixel in the background information. Since the target pixel is in an imaging state, the updated background information should minimize the thermal image components stored in the frame memory 18 and maximize the background information components. Therefore, a1 should be set to a small value. This prevents attenuation of subject information in the corrected thermal image output from the thermal image generator 26, even if the subject remains in one place for a long time. From this point onward (including other embodiments), the background information update unit 24 updates the background information memory 20 with a value calculated using the function f(a, T, B)=a*T+(1-a)*B, using a satisfying 0<a<1. However, the range of a and the function f(a, T, B) are not limited, and any function may be used as long as the ratio of T to T and B increases as a increases.

[0028] If it is not determined in step S103 that the pixel is in a subject imaging state, the process proceeds to step S105, where if the subtracted value is smaller than thB, which is a negative threshold value, the pixel is determined to be in a false detection state. In such a pixel, the temperature is determined to be lower than the background by more than the threshold, and such a pixel is generally considered to be in a false detection state.

[0029] If it is determined that the pixel is in a false detection state, in step S106, the background information update unit 24 calculates the value using the formula a3*T+(1-a3)*B, where a3 satisfies 0<a3<1, and stores the calculated value as new background information in the background information memory 20. At this time, since the target pixel is in a false detection state, it is desirable to increase the components of the thermal image stored in the frame memory 18 and decrease the components of the background information in the updated background information. Therefore, it is preferable to set a3 to a large value.

[0030] If it is not determined in step S105 that the pixel is in a false detection state, the pixel is considered to be in a state in which the background is being captured (a background-captured state), and the process proceeds to step S107, where the background information update unit 24 updates the background information memory 20 with a value calculated by the formula a2*T+(1-a2)*B using a number a2 that satisfies 0<a2<1, and stores the calculated value as new background information. At this time, the target pixel is in a background-captured state, and the value of a2 should be set between a1 and a3.

[0031] To summarize the magnitude relationship between a1, a2, and a3, a relationship of a1 < a2 < a3 is desirable. However, this relationship is not limited to the above, depending on the time constraints before the image is smoothed, the required degree of image smoothness, and so on. Furthermore, although a1 is set to 0 < a1 < 1, it is not limited to this and may include 0 or 1, or may be less than 0 or exceed 1. The same applies to a2 and a3. The reason for setting 0 < a1 < 1 is to easily restore a smooth image even in the event of sudden image disturbance due to a drop in equipment voltage, momentary entrance of ambient light, or the like.

[0032] Furthermore, it is preferable that the values ​​of thA and thB are set so that the subject imaging state, background imaging state, and false detection state can be appropriately distinguished, taking into consideration the output of the thermal image generation unit 26, the amount of noise contained in the output of the thermal image generation unit 26, and the analog-to-digital conversion magnification of the ADC 16, and the absolute values ​​of thA and thB may be the same or may be set to different values. Note that, although thA and thB are described as threshold determinations using fixed values, the number of thresholds may be changed depending on the number of states to be distinguished, or the thresholds themselves may be varied depending on the image state. The same applies to the numbers a1, a2, and a3.

[0033] After completing any of steps S104, S106, and S107, the process proceeds to step S108, where the thermal image generator 26 subtracts the background information stored in the background information memory 20 from the thermal image stored in the frame memory 18 for each pixel, and generates and outputs a corrected thermal image. The above operation increases the ratio of fixed pattern noise and background information in the background information, and by repeating this operation, this ratio increases, so that the contours of the subject are emphasized in the corrected thermal image output from the thermal image generator 26. Then, the process returns to step S101, where the next image is captured.

[0034] Although the series of operations from S102 to S108 is performed for each pixel, the series of operations may be performed for each pixel individually, or one operation or multiple operations may be performed for all pixels, and then the next operation may be performed again for each pixel individually.

[0035] Although the state determination unit 22 determines the subject imaging state, background imaging state, and false detection state for each pixel by comparing the subtracted value obtained by subtracting the background information stored in the background information memory 20 from the thermal image stored in the frame memory 18 with a threshold, the determination method is not limited to this. For example, an image processing method other than subtraction calculation may be used, a different determination method may be used for the threshold, or the threshold may be variable.

[0036] The relationship between the thermal image stored in the frame memory 18 and the corrected thermal image output by the thermal image generator 26 will now be described. FIG. 3(a) is an example of a thermal image stored in the frame memory 18, and FIG. 4(a) is a diagram schematically illustrating the frequency of pixel values ​​in the thermal image of FIG. 3(a). The horizontal axis in FIG. 4 represents the intensity of pixel values. In FIG. 4, the largest area represents fixed pattern noise and background information, and is indicated by "Noise and Background Information" in the figure. In FIG. 4, the area with high pixel value intensity (the right side of the figure, indicated by "Human Information" in the figure) represents information about the subject. This thermal image includes fixed pattern noise, information corresponding to the intensity of infrared light emitted from the imaging target area, information emitted from moving subjects, and information about stationary backgrounds. This makes it difficult to visually identify the subject information. FIG. 4(a) shows how the information of the subject is hidden in the fixed pattern noise and background.

[0037] Figure 3(b) is a modified thermal image output from the thermal image generator 26 after the thermal image of Figure 3(a) has passed through, and Figure 4(b) is a diagram showing the frequency of pixel values ​​in the modified thermal image of Figure 3(b). The modified thermal image output by the thermal image generator 26 is obtained by subtracting background information from the thermal image. Since the background information includes fixed pattern noise in addition to background information, Figure 3(b) is a clearer image with sharper contours than Figure 3(a). In both Figure 4(b) and Figure 4(b), the subject (human information) area is separated and distinguished from other areas.

[0038] Next, we will explain the change over time of the corrected thermal image output by the thermal image generation unit 26. Figure 5(a) is a diagram schematically showing the frequency of pixel values ​​in the corrected thermal image output by the thermal image generation unit 26.

[0039] The subject is in a subject imaging state and is therefore at a higher temperature than the background, which is why it is on the high temperature side (the area marked "Human information") in Figure 5(a). Pixels in this state retain more thermal image information in the frame memory 18 than background information, so even if the subject is moving very slowly, the output from the thermal image generator 26 decays slowly. As a result, subject imaging is possible for a long period of time.

[0040] Conversely, the area in the false detection state is closer to the low temperature side (the area marked "Noise information"). Pixels in this state have a low ratio of the thermal image information in the frame memory 18 reflected in the background information, so the output from the thermal image generator 26 decays quickly. As a result, a smooth image is quickly restored. Figure 5(b) is a schematic diagram showing the decay of a pixel in the false detection state.

[0041] The area where the background is captured is near zero. Pixels in this state have a high ratio of reflecting the thermal image information in the frame memory 18 to the background information, so the output from the thermal image generator 26 decays quickly.

[0042] As described above, the infrared imaging device 100 according to the first embodiment determines the subject imaging state, background imaging state, and false detection state for each pixel from a captured thermal image, and updates the background information according to the determined state. Therefore, only one thermal image needs to be stored in the frame memory 18, which prevents an increase in the memory capacity required for the frame memory.

[0043] The background information stores information on the background image and fixed pattern noise, and the background information is subtracted from the thermal image stored in the frame memory 18 to output a corrected thermal image, so that the contours of the subject are emphasized in the corrected thermal image to be output, and the influence of the fixed pattern noise is reduced.

[0044] Furthermore, since the updating of the background information does not stop even when the subject is moving, an image can be obtained from which the background information that has been updated at any time has been subtracted.

[0045] Furthermore, if the subject is not moving, that portion is determined to be in a subject imaging state, and the reflection of subject information in the background information is reduced, thereby reducing the problem of a sudden drop in the subject output value.

[0046] Furthermore, since the background information containing information about fixed pattern noise is automatically generated during operation of the infrared imaging device, there is no need to provide a process for generating fixed pattern noise during manufacturing.

[0047] Second Embodiment The configuration of an infrared imaging device 200 according to a second embodiment is shown in Fig. 6. The infrared imaging device 200 according to the second embodiment is similar to the infrared imaging device 100 according to the first embodiment, but includes an environment determination unit 30 that determines whether the temperature of the subject is lower than the temperature of the background, and changes its operation depending on the determination result.

[0048] Next, the operation of the infrared imaging device 200 according to the second embodiment will be described. Figures 7 and 8 are flowcharts showing the operation of the infrared imaging device 200. A in Figure 7 connects to A in Figure 8, and B in Figure 8 connects to B in Figure 7. Steps S201 to S202 are the same as steps S101 to S102 in Figure 2 of the first embodiment, and therefore will not be described here.

[0049] In step S211, which is the next step after step S202, the environment determination unit 30 determines whether the temperature of the subject is lower than the temperature of the background. Specifically, a temperature subtraction value corresponding to the value obtained by subtracting the representative value of the background temperature from the representative value of the subject temperature is calculated, and whether the temperature of the subject is lower than the temperature of the background is determined based on whether the temperature subtraction value is a negative value. If it is determined that the temperature of the subject is not lower than the temperature of the background, the process proceeds to step S203. Steps S203-208 are the same as steps S103-108 in FIG. 2 of the first embodiment, and therefore description thereof will be omitted.

[0050] If it is determined in step S211 that the temperature of the subject is lower than the temperature of the background, the process proceeds to step S213, where state determination is performed by state determination unit 22. The process proceeds to step S213 only if it is determined that the temperature of the subject is lower than the temperature of the background. If this is not the case, the process proceeds to the same steps (S203-207) as in embodiment 1, as described above.

[0051] Steps S213 to S217 are similar to steps S103 to S107 in Fig. 2 of embodiment 1. In step S213, if it is determined that the subtracted value obtained by subtracting the background information stored in the background information memory 20 from the thermal image stored in the frame memory 18 for each pixel is smaller than thC, which is a negative threshold value, the pixel is determined to be in a subject imaging state.

[0052] If it is determined that the subject is being imaged, in step S214, the background information update unit 24 updates the background information memory 20 with a value calculated by the formula b1*T+(1-b1)*B using a number b1 that satisfies 0<b1<1, and stores the value as new background information.

[0053] If it is not determined in step S213 that the subject is in an image capturing state, the process proceeds to step S215, where for each pixel, if the subtracted value is greater than thD, which is a positive threshold value, the pixel is determined to be in a false detection state.

[0054] If it is determined that a false detection state exists, in step S216, the background information update unit 24 updates the background information memory 20 with the value calculated by the formula b3*T+(1-b3)*B using the number b3 satisfying 0<b3<1, and stores the value as new background information.

[0055] If it is not determined in step S215 that the pixel is in a false detection state, the pixel is considered to be in a state in which the background is being captured (background captured state), and the process proceeds to step S217, where the background information update unit 24 updates the background information memory 20 with the value calculated by the formula b2*T+(1-b2)*B using the number b2 satisfying 0<b2<1, and stores the value as new background information.

[0056] Here, it is desirable that b1, b2, and b3 have a relationship of b1<b2<b3 for the same reasons as those explained in embodiment 1. However, the above relationship is not limited to this due to time constraints before the image is smoothed, requirements for the degree of image smoothness, etc.

[0057] When any of steps S214, S216, and S217 is completed, the process proceeds to step S208, which is the same as step S108 in FIG. 2 of the first embodiment.

[0058] Next, we will explain the change over time in the corrected thermal image output by the thermal image generation unit 26. Figure 9(a) is a diagram showing the frequency of pixel values ​​in the corrected thermal image output by the thermal image generation unit 26. In this example, the object temperature is lower than the background temperature.

[0059] The subject is in a subject imaging state and has a lower temperature than the background, so it is on the low temperature side (the region indicated as "Human information") in Fig. 9(a). Pixels in this state are capable of imaging the subject for a long period of time for the same reasons as those described in the first embodiment.

[0060] Conversely, the area in the false detection state is closer to the high temperature side (the area marked "Noise information"). For the same reason as explained in the first embodiment, the output from the thermal image generating unit 26 of a pixel in this state decays quickly. As a result, a smooth image is quickly restored. Figure 9(b) is a diagram showing a schematic diagram of a pixel in the false detection state decaying.

[0061] 10 is an example of a corrected thermal image output by the thermal image generating unit 26. The background has a higher intensity than the subject.

[0062] A method for calculating the temperature subtraction value by the environment determining unit 30 will now be described, although the calculation method is not limited to the method described below.

[0063] The first method will be explained using FIG. 11. FIG. 11 is a diagram that schematically shows the frequency of pixel values ​​in the corrected thermal image output by the thermal image generation unit 26. The dashed line indicates the position of the pixel average value, which is the average of the pixel values ​​of the corrected thermal image. If the subject (human information) is cooler than the background, the pixel average value will be closer to the lower temperature side. In other words, the pixel average value will be a negative value. On the other hand, if the subject (human information) is warmer than the background, the pixel average value will be closer to the higher temperature side, and the pixel average value will be a positive value. Therefore, the pixel average value is used as the temperature subtraction value.

[0064] Another method will be explained using FIG. 12. FIG. 12 is a diagram schematically illustrating the frequency of pixel values ​​in the corrected thermal image output by the thermal image generation unit 26. In the diagram, the dashed line on the left represents the position of thP, which is a threshold value having a negative value, and the dashed line on the right represents the position of thQ, which is a threshold value having a positive value. Let n be the number of pixels having a value smaller than thP, and m be the number of pixels having a value larger than thQ. In this case, if n > m, it can be determined that the temperature of the subject is lower than the temperature of the background, and if n ≦ m, it can be determined that the temperature of the subject is not lower than the temperature of the background. Therefore, the value of m - n is used as the temperature subtraction value.

[0065] Another method will be explained using FIG. 13 . FIG. 13 is a diagram that schematically illustrates the frequency of pixel values ​​in the corrected thermal image output by the thermal image generation unit 26. The subject is often a person or the like, and moves within the thermal image. The dashed lines in the diagram schematically represent the movement of the subject. By examining the positive and negative values ​​of the area that fluctuates over time in this way, the magnitude of the temperature of the subject and the temperature of the background can be determined. That is, if the value of the subject area is negative, it is determined that the temperature of the subject is lower than the temperature of the background, and if the value of the subject area is not negative, it is determined that the temperature of the subject is not lower than the temperature of the background. Therefore, among the frequency areas in FIG. 13 , the area that fluctuates most over time is determined to be the subject area, and the average value of the subject area is used as the temperature subtraction value.

[0066] Another method will be described with reference to FIG. 14 . FIG. 14 shows the configuration of a modified infrared imaging device 200, which includes a thermometer 40. The thermometer 40 measures the temperature around the infrared imaging device 200 or near the background. This temperature is referred to as the measured temperature. The measured temperature is transmitted to the environment determination unit 30. The environment determination unit 30 calculates the temperature (thermal image temperature) corresponding to the average value (pixel average value) of each pixel of the thermal image stored in the frame memory 18. A pre-defined relationship between the pixel average value and the thermal image temperature can be used. The measured temperature is considered to be close to the temperature of the background. Furthermore, if a subject is captured in the thermal image stored in the frame memory 18, the thermal image temperature will be closer to the temperature of the subject than the temperature of the background. Therefore, the value obtained by subtracting the measured temperature from the thermal image temperature is used as the temperature subtraction value. The thermometer 40 may be a thermistor, a non-contact radiation thermometer, or other device.

[0067] As another method, the subject may be identified and the temperature of the identified subject may be estimated based on the subject's shape, an object that has entered the image capture area from outside may be identified as the subject, the subject may be identified based on the speed of movement within the image, or any other image determination method.

[0068] As described above, the infrared imaging device 200 according to the second embodiment determines whether the temperature of the subject is lower than the temperature of the background, and if it determines that the temperature of the subject is not lower than the temperature of the background, it operates to appropriately determine the subject imaging state, background imaging state, and false detection state, thereby enabling stable imaging even when the temperature of the subject is lower than the background.

[0069] Third Embodiment The configuration of an infrared imaging device 300 according to a third embodiment is shown in Fig. 15. The infrared imaging device 300 according to the third embodiment is similar to the infrared imaging device 200 according to the second embodiment, but the environment determination unit 30 also determines whether the temperature of the subject and the temperature of the background are close to each other, and changes operation depending on the determination result.

[0070] Next, the operation of the infrared imaging device 300 according to the third embodiment will be described. Figures 16 and 17 are flowcharts showing the operation of the infrared imaging device 300. C in Figure 16 connects to C in Figure 17, and D in Figure 17 connects to D in Figure 16. A in Figure 16 connects to A in Figure 8 of the second embodiment, which has already been described, and B in Figure 16 connects to B in Figure 8. Steps S301 to S302 are the same as steps S201 to S202 in Figure 7 of the second embodiment, and therefore will not be described here.

[0071] In step S321, which is the next step after step S302, the environment determination unit 30 determines whether the temperature of the subject and the temperature of the background are close to each other. Specifically, it determines whether the absolute value of the temperature subtraction value described in the second embodiment is smaller than thU, which is a threshold value of a positive value.

[0072] If it is determined that the absolute value of the temperature subtraction value is not smaller than thU, the process proceeds to step S311. Steps from step S311 are the same as steps from step S211 in FIG. 7 of the second embodiment, and therefore a description thereof will be omitted.

[0073] If it is determined in step S321 that the absolute value of the temperature subtraction value is smaller than thU, the process proceeds to step S323, where the state determination unit 22 performs a state determination. Steps S323 to S327 are similar to steps S103 to S107 in FIG. 2 of embodiment 1. In step S323, if it is determined that the subtraction value obtained by subtracting the background information stored in the background information memory 20 from the thermal image stored in the frame memory 18 for each pixel is larger than thE, which is a positive threshold value, the pixel is determined to be in a subject imaging state.

[0074] If it is determined that the subject is being imaged, in step S324, the background information update unit 24 updates the background information memory 20 with the value calculated by the formula c1*T+(1-c1)*B using a number c1 that satisfies 0<c1<1, and stores the value as new background information.

[0075] If it is not determined in step S323 that the subject is in an image capturing state, the process proceeds to step S325, where for each pixel, if the subtracted value is greater than thF, which is a negative threshold value, the pixel is determined to be in a false detection state.

[0076] If it is determined that a false detection state exists, in step S326, the background information update unit 24 updates the background information memory 20 with the value calculated by the formula c3*T+(1-c3)*B using the number c3 satisfying 0<c3<1, and stores the value as new background information.

[0077] If it is not determined in step S325 that the pixel is in a false detection state, the pixel is considered to be in a state in which the background is being captured (background captured state), and the process proceeds to step S327, where the background information update unit 24 updates the background information memory 20 with the value calculated by the formula c2*T+(1-c2)*B using the number c2 satisfying 0<c2<1, and stores the value as new background information.

[0078] Here, it is desirable that c1, c2, and c3 have a relationship of c1<c2<c3 for the same reasons as those explained in embodiment 1. However, the relationship is not limited to the above due to time constraints before the image is smoothed, requirements for the degree of image smoothness, etc.

[0079] Furthermore, it is desirable to satisfy the relationships c1<a3 and c3<a3. When the temperatures of the subject and background are close, it is difficult to correctly determine the subject imaging state, background imaging state, and false detection state. Therefore, in order to suppress sudden changes in the output image, it is necessary to satisfy the relationships c3<a3 and c3<a3, and the proportion of thermal images stored in the frame memory 18 when updating the background information is reduced.

[0080] When any of steps S324, S326, and S327 is completed, the process proceeds to step S308, which is the same as step S208 in FIG. 7 of the second embodiment.

[0081] Next, we will explain the changes over time in the corrected thermal image output by the thermal image generation unit 26. Figure 18(a) is a diagram showing the frequency of pixel values ​​in the corrected thermal image output by the thermal image generation unit 26. This example shows a case where the temperature difference between the subject and the background is small.

[0082] In the third embodiment, c1 and c3, which are a in the function f(a, T, B) used to update the background information, are both smaller than a3 in the first embodiment. Therefore, the attenuation of the subject information is small. As a result, as shown in FIG. 18(b), the attenuation of the subject information over time is small.

[0083] As described above, the infrared imaging device 300 according to the third embodiment determines whether the temperature of the subject and the temperature of the background are close to each other, and if it is determined that the temperature difference is small, it outputs an image with little attenuation of the subject information, thereby stabilizing the output image.

[0084] 19 shows the configuration of an infrared imaging device 400 according to a fourth embodiment. The infrared imaging device 400 according to the fourth embodiment is similar to the infrared imaging device 200 according to the second embodiment, but the environment determination unit 30 also determines whether the temperature of the subject and the temperature of the background are significantly different, and changes the operation according to the determination result.

[0085] Next, the operation of the infrared imaging device 400 according to the fourth embodiment will be described. Fig. 20 is a flowchart showing the operation of the infrared imaging device 400. A in Fig. 20 is connected to A in Fig. 8 of the second embodiment already described, and B in Fig. 20 is connected to B in Fig. 8. Steps S401 to S402 are the same as steps S201 to S202 in Fig. 7 of the second embodiment, and therefore will not be described here.

[0086] In step S431, which is the next step after step S402, the environment determination unit 30 determines whether the temperature of the subject and the temperature of the background are significantly different from each other. Specifically, it determines whether the absolute value of the temperature subtraction value described in the second embodiment is greater than thV, which is a threshold value for positive values.

[0087] If it is determined that the absolute value of the temperature subtraction value is not greater than thV, the process proceeds to step S411. Steps from step S411 are the same as steps from step S211 in FIG. 7 of the second embodiment, and therefore a description thereof will be omitted.

[0088] If it is determined in step S431 that the absolute value of the temperature subtraction value is greater than thV, the process proceeds to step S433, where the background information update unit 24 updates the background information memory 20 with the value calculated by the formula d*T+(1-d)*B using a number d satisfying 0<d<1, and stores the updated value as new background information.

[0089] It is desirable that the number d satisfy the relationship d>a3. When the infrared imaging device 400 is exposed to direct sunlight or hot air, or when there is a sudden change in the environmental temperature, the image output from the thermal image generating unit 26 changes suddenly. In this case, the subject imaging performance deteriorates significantly, so early image recovery is necessary. To achieve this, it is important to increase the value of d and increase the proportion of the thermal image information stored in the frame memory 18 reflected in the background information stored in the background information memory 20.

[0090] After step S433, the process proceeds to step S408, which is the same as step S208 in FIG. 7 of the second embodiment.

[0091] Next, we will explain the change over time of the corrected thermal image output by the thermal image generation unit 26. Figure 21(a) is a diagram showing the frequency of pixel values ​​in the corrected thermal image output by the thermal image generation unit 26. This example shows a case where it is determined that there is a large temperature difference between the subject and the background.

[0092] Normally, the background area is located near zero (the position of the bar on the vertical axis in the figure), but in situations where the temperature difference between the subject and the background is large, the area including the background will shift significantly to the high or low temperature side, as shown in Figure 21 (a).

[0093] In the fourth embodiment, even in such a situation, by increasing the number d, the information of the thermal image stored in the frame memory 18 is largely reflected in the background information, and the image is quickly restored as shown in FIG. 21(b).

[0094] As described above, the infrared imaging device 400 according to the fourth embodiment determines whether the temperature of the subject and the temperature of the background are significantly different, and if it is determined that they are significantly different, the information of the thermal image stored in the frame memory 18 is largely reflected in the background information, thereby achieving early image recovery.

[0095] Fifth Embodiment The configuration of an infrared imaging device 500 according to a fifth embodiment is shown in Fig. 22. The infrared imaging device 500 according to the fifth embodiment is similar to the infrared imaging device 100 according to the first embodiment, but includes a timing determination unit 50 that determines whether the infrared imaging device 500 is in the middle of starting up, and changes its operation depending on the determination result.

[0096] Next, the operation of the infrared imaging device 500 according to the fifth embodiment will be described. Fig. 23 is a flowchart showing the operation of the infrared imaging device 500. Steps S501 to S502 are the same as steps S501 to S502 in Fig. 2 of the first embodiment, and therefore a description thereof will be omitted.

[0097] In step S541, which is the next step after step S502, the timing determination unit 50 determines whether the infrared imaging device 500 is in the middle of starting up.

[0098] If it is determined that the infrared imaging device 500 is not in the process of starting up, the process proceeds to step S503. Steps from step S503 are the same as steps from step S103 in FIG. 2 of the first embodiment, and therefore a description thereof will be omitted.

[0099] If it is determined in step S541 that the infrared imaging device 500 is in the process of starting up, the process proceeds to step S542, in which the background information update unit 24 updates the background information memory 20 with the value calculated by the formula e*T+(1-e)*B using a number e satisfying 0<e<1, and stores the value as new background information.

[0100] It is desirable that the number e satisfy e>a3. When the infrared imaging device 500 is in the process of starting up, the image output from the thermal image generating unit 26 is unstable. It is also possible that an unwanted subject may be captured in the image. In such cases, the background information stored in the background information memory 20 becomes unstable, significantly degrading the subject imaging performance, and therefore, early image recovery is necessary. To achieve this, it is important to increase the value of e to increase the proportion of the thermal image information stored in the frame memory 18 reflected in the background information stored in the background information memory 20.

[0101] After step S542, the process proceeds to step S508, which is the same as step S108 in FIG. 2 of the first embodiment.

[0102] Next, we will explain the change over time in the corrected thermal image output by the thermal image generation unit 26. Figure 24(a) is a diagram showing the frequency of pixel values ​​in the corrected thermal image output by the thermal image generation unit 26. The corresponding output image is shown to the right of the frequency diagram. This example shows a case where it is determined that the infrared imaging device 500 is in the middle of starting up.

[0103] As described above, when the infrared imaging device 500 is in the process of starting up, it is necessary to quickly recover from the state in which the subject imaging performance has deteriorated. In the fifth embodiment, even in such a situation, by increasing the number e, the information of the thermal image stored in the frame memory 18 is largely reflected in the background information, and the image is quickly restored as shown in Figure 24(b).

[0104] The method by which the timing determination unit 50 determines whether the infrared imaging device 500 is in the middle of starting up will now be described, although the determination method is not limited to the method described below.

[0105] In the first method, the timing determination unit 50 has a function of measuring the elapsed time since the infrared imaging device 500 was started up, and determines whether the infrared imaging device 500 is in the middle of starting up based on whether the elapsed time is smaller than thX, which is a positive threshold value. For example, the elapsed time may be measured using a component with a clock function. Alternatively, the timer function of a microcomputer may be used.

[0106] Another method is to count the number of times a thermal image is stored in the frame memory 18. The timing determination unit 50 counts the number of times a thermal image is stored in the frame memory 18. The count number is regarded as the time since the infrared imaging device 500 was started up, and whether the infrared imaging device 500 is in the middle of starting up is determined based on whether the count number is smaller than thX, which is a positive threshold value.

[0107] Another method is to count the number of operations of the ADC 16. In this method, the number of times the frame memory 18 stores data in the first method is replaced with the number of operations of the ADC 16.

[0108] As described above, the infrared imaging device 500 according to the fifth embodiment determines whether the infrared imaging device 500 is in the process of starting up, and if it is determined that the infrared imaging device 500 is in the process of starting up, the information of the thermal image stored in the frame memory 18 is largely reflected in the background information, thereby achieving early image recovery.

[0109] As a modification of the infrared imaging device 500 according to the fifth embodiment, instead of determining whether the infrared imaging device 500 is in the middle of starting up, the timing determination unit 50 may determine whether a long time has passed since the infrared imaging device 500 was started up, and if it determines that a long time has passed, the information on the thermal image stored in the frame memory 18 may be significantly reflected in the background information, as described above. Specifically, the timing determination unit 50 determines whether the time since the infrared imaging device 500 was started up is longer than thY, which is a positive threshold value, and if it determines that the time is longer, the background information is updated using a number g satisfying 0<g<1 instead of the above-mentioned e. It is desirable that the number g satisfy g>a3.

[0110] When the infrared imaging device 500 is operated for a long time, a large amount of information may be accumulated in the background information stored in the background information memory 20, and it may become necessary to reset the information. In this modified example, in such a case, the background information is quickly restored.

[0111] The timing determination unit 50 may be an independent component, or another component, such as the state determination unit 22, may have the function of the timing determination unit 50.

[0112] The above has described embodiments 1 to 5, but in any of the embodiments, the state determination unit may not only determine the subject imaging state, the background imaging state, and the erroneous detection state, but may also determine only the subject imaging state and the background imaging state without determining the erroneous detection state. Alternatively, the number of states may be further increased, and the number a used in the function f(a, T, B) may be set to a value appropriate for each state.

[0113] Furthermore, the pixel average value, which is the average of the pixel values ​​of the corrected thermal image, may be calculated using all the pixels of the corrected thermal image, or may be calculated using the pixels after sampling.

[0114] The infrared imaging device may be a single completed product, or may be a module that is incorporated into another product for use.

[0115] Although the present disclosure describes various exemplary embodiments and examples, the various features, aspects, and functions described in one or more embodiments are not limited to application to a particular embodiment, but may be applied to the embodiments alone or in various combinations.

[0116] Therefore, countless variations not illustrated are conceivable within the scope of the technology disclosed in the present specification, including, for example, cases where at least one component is modified, added, or omitted, and cases where at least one component is extracted and combined with components of another embodiment.

[0117] 100, 200, 300, 400, 500 Infrared imaging device, 14 Imaging element, 16 ADC, 18 Frame memory, 20 Background information memory, 22 Status determination unit, 24 Background information update unit, 26 Thermal image generation unit, 30 Environment determination unit, 40 Thermometer, 50 Timing determination unit.

Claims

1. An imaging element that receives infrared rays emitted from an imaging target area including a subject and background, and outputs an electrical signal corresponding to the intensity of the infrared rays; an ADC that converts the electrical signal into a digital signal; a frame memory that stores the digital signal as a thermal image; a background information memory that stores background information calculated based on the thermal image; a state determination unit that determines whether a subtraction value obtained by subtracting the background information from the thermal image for each pixel is greater than thA which is a positive threshold value; a background information update unit that updates and stores in the background information memory as new background information a value obtained by calculating a function f(a, T, B) in which the ratio of T to B increases as a value increases, and a thermal image generation unit that subtracts the background information from the thermal image for each pixel, and outputs a corrected thermal image. an infrared imaging device, wherein the value of a when the state determination unit determines that the subtraction value is greater than thA is a1, and the value of a when the state determination unit determines that the subtraction value is not greater than thA is a2, and the a1 and a2 satisfy a1<a2.

2. The infrared imaging device according to claim 1, wherein a satisfies 0<a<1, and the function f(a, T, B) is f(a, T, B)=a*T+(1-a)*B.

3. An infrared imaging device as described in claim 1 or 2, wherein only when the state determination unit determines that the subtraction value is not greater than thA, the state determination unit determines whether the subtraction value is smaller than thB, which is a threshold value having a negative value, the value of a when the state determination unit determines that the subtraction value is smaller than thB is a3, and the value of a when the state determination unit determines that the subtraction value is not smaller than thB is a2, and a1, a2, and a3 satisfy a1<a2<a3.

4. An infrared imaging device as described in any one of claims 1 to 3, further comprising an environment determination unit that calculates a temperature subtraction value corresponding to a value obtained by subtracting a representative value of the temperature of the background from a representative value of the temperature of the subject, wherein the environment determination unit determines whether the temperature subtraction value is a negative value, and only when the environment determination unit determines that the temperature subtraction value is a negative value, the state determination unit determines for each pixel whether the subtraction value is smaller than thC which is a negative value threshold, and sets the value of a when it is determined that the subtraction value is smaller than thC to b1, and sets the value of a when it is determined that the subtraction value is not smaller than thC to b2, wherein b1 and b2 satisfy b1<b2.

5. The infrared imaging device of claim 4, wherein only when the environment determination unit determines that the temperature subtraction value is a negative value and the state determination unit determines that the subtraction value is not smaller than thC, the state determination unit determines for each pixel whether the subtraction value is greater than thD which is a positive value threshold, the value of a when the state determination unit determines that the subtraction value is greater than thD is b3, and the value of a when the state determination unit determines that the subtraction value is not greater than thD is b2, and the b1, b2, and b3 satisfy b1<b2<b3.

6. The infrared imaging device according to claim 4 or 5, wherein the environment determination unit determines whether the absolute value of the temperature subtraction value is smaller than thU which is a threshold value for a positive value, and only when the environment determination unit determines that the absolute value of the temperature subtraction value is smaller than thU, the state determination unit determines for each pixel whether the subtraction value is larger than thE which is a threshold value for a positive value, and sets the value of a when it is determined that the subtraction value is larger than thE to c1, and when it is determined that the subtraction value is not larger than thE, the state determination unit determines for each pixel whether the subtraction value is smaller than thF which is a threshold value for a negative value, and sets the value of a when it is determined that the subtraction value is not smaller than thF to c2, and sets the value of a when it is determined that the subtraction value is smaller than thF to c3, wherein c1 satisfies c1<a3 and c3 satisfies c3<a3.

7. The infrared imaging device according to claim 4 or 5, wherein the environment determination unit determines whether the absolute value of the temperature subtraction value is greater than thV which is a threshold value of a positive value, and only when the environment determination unit determines that the absolute value of the temperature subtraction value is greater than thV, the state determination unit sets the value of a to d, and d satisfies d>a3.

8. An infrared imaging device as described in any one of claims 4 to 7, wherein the environment determination unit calculates a pixel average value which is the average of the values ​​of all pixels or a portion of pixels of the corrected thermal image, and sets the pixel average value as the temperature subtraction value.

9. An infrared imaging device as described in any one of claims 4 to 7, wherein the environment determination unit defines the number of pixels having values ​​smaller than thP, a negative threshold, among all pixels or a portion of the pixels of the corrected thermal image, as n, the number of pixels having values ​​larger than thQ, a positive threshold, as m, and defines the value of m-n as the temperature subtraction value.

10. An infrared imaging device as described in any one of claims 4 to 7, wherein the environment determination unit examines the frequency of values ​​of all pixels or a portion of pixels of the corrected thermal image, determines the area of ​​the frequency that fluctuates the most over time to be the area of ​​the subject, and sets the average value of the area of ​​the subject to the temperature subtraction value.

11. An infrared imaging device as described in any one of claims 4 to 7, further comprising a thermometer for measuring a measured temperature, wherein the environment determination unit determines a temperature corresponding to an average value of the values ​​of all pixels or a portion of pixels of the thermal image as the thermal image temperature, and determines a value obtained by subtracting the measured temperature from the thermal image temperature as the temperature subtraction value.

12. An infrared imaging device as claimed in any one of claims 1 to 11, further comprising a timing determination unit which determines whether the time since the infrared imaging device was started is shorter than thX which is a positive threshold value, and uses a number e which satisfies e>a3 as the value of a which is an argument of the function f(a, T, B) only when the timing determination unit determines that the time since the infrared imaging device was started is shorter than thX.

13. An infrared imaging device as described in any one of claims 1 to 11, further comprising a timing determination unit that determines whether the time since the infrared imaging device was started is longer than thY, which is a positive threshold value, and uses a number g that satisfies g>a3 as the value of a, which is an argument of the function f(a, T, B), only when the timing determination unit determines that the time since the infrared imaging device was started is longer than thY.

Images