Method, device and system for temperature calibration and determination of temperature within a scene

Abstract

To provide a method for temperature calibration and determination of temperature in a scene.SOLUTION: A method includes: collecting, at each time point out of a plurality of time points in a first period, an ambient temperature representing the temperature of a first part of a scene and collecting a plurality of thermal image sensor signal values corresponding to the collected ambient temperatures and related to the first part; determining a calibration function based on the collected ambient temperatures and the thermal image sensor signal values corresponding to the collected ambient temperatures; and capturing, in a second period, a thermal image of the scene including thermal image sensor signal values related to a second part of the scene and determining the temperature of the second part of the scene based on the calibration function and based on the thermal image sensor signal values included in the thermal image.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] FIELD OF THE INVENTION Embodiments herein relate to the field of thermal cameras, and in particular to temperature calibration and determining temperature within a scene. [Background technology]

[0002] A thermal camera, sometimes called a thermographic camera, infrared camera, thermal imaging camera, or infrared thermograph, is an imaging device that uses infrared radiation to form images of heat zones, similar to a typical camera that uses visible light to form an image. Instead of the 380-780 nanometers of a visible light camera, a thermal camera operates at wavelengths from about 3 micrometers up to about 14 micrometers.

[0003] Thermal cameras have applications in a wide range of areas, such as object detection, anomaly detection, fire monitoring, and trend analysis, to name a few. In many of these applications, it is important to accurately measure the temperature(s) within the scene being imaged. However, today, calibrating thermal cameras to accurately measure temperature is difficult and expensive.

[0004] For example, a thermal camera can be calibrated at a manufacturing facility by placing the thermal camera in a calibration chamber or room with a constant chamber / room temperature. Once the thermal camera is acclimatized to a constant temperature in the calibration chamber, various objects of different, well-defined absolute temperatures are placed one at a time in front of the thermal camera to calibrate the thermal camera. Specifically, each object is placed within a scene covered by the thermal camera's field of view. For each object's temperature, the thermal camera captures an image of the scene and records a respective image sensor signal value for each sensor element of the image sensor. Once the sensor signal values ​​have been recorded for all the various objects, a calibration function for each sensor element is determined. The calibration function relates absolute temperature to the sensor signal value. Because multiple objects of different, well-defined temperatures must be placed in the calibration chamber and because the sensor signal value for each sensor element of the image sensor must be recorded to obtain the calibration function, the calibration procedure is very time-consuming. Sometimes, once the calibration function has been determined for each sensor element of the thermal camera, the thermal camera is said to be fully calibrated.

[0005] U.S. Patent No. 6,267,501 describes another example of factory calibration using a calibration chamber. In this example, a thermal camera is placed in the calibration chamber at a controlled ambient temperature. The thermal camera faces a scene that is isothermal across the scene through the walls of the calibration chamber, and this temperature is presented to all sensor elements of the thermal camera. During calibration of the thermal camera, both the chamber temperature and the scene temperature are varied. Specifically, for each chamber temperature, the scene temperature is varied within a temperature interval, and for each scene temperature, the thermal camera records sensor signal values ​​through the window of the calibration chamber. The chamber temperature is then changed, and the thermal camera records sensor signal values ​​for each scene temperature within that temperature interval. Once sensor signal values ​​have been recorded for all temperatures, i.e., for all chamber temperatures combined with all scene temperatures, a calibration function for each sensor element and each chamber temperature is determined. The calibration function for each sensor element relates absolute temperature to the sensor signal value for the thermal camera operating at that chamber temperature. Because a calibration function is again determined for each sensor element in this example, the thermal camera is said to be fully calibrated.

[0006] FIG. 1A schematically illustrates exemplary calibration functions for different sensor elements of a thermal camera that is said to be fully calibrated. The calibration function for the sensor elements marked with "A" is shown as curve A, the calibration function for the sensor elements marked with "B" is shown as curve B, and the calibration function for the sensor elements marked with "C" is shown as curve C. In FIG. 1A, each calibration curve is valid for three sensor elements. However, it should be understood that this is merely an example, and that each calibration function can be valid for a different number of sensor elements. FIG. 1A also illustrates that the calibration is performed over a wide range of temperatures, for example, between 0° C. and 300° C.

[0007] Some drawbacks of at least some fully calibrated thermal cameras are that they take time to calibrate and that they must be recalibrated after several years, sometimes even annually. The thermal camera must then be sent back to the factory and one of the procedures described above must be performed again. This is particularly inconvenient for thermal cameras that are installed far from the factory.

[0008] 1B shows a schematic diagram of exemplary calibration functions for different sensor elements of a thermal camera that is said to be standard calibrated. By the expression "standard calibrate" as used in this disclosure, it is meant that the same type of thermal camera is calibrated using a single calibration function. The calibration function is sometimes referred to as the default calibration function. As can be seen in FIG. 1B, the calibration function is the same for all nine sensor elements.

[0009] One drawback of standard-calibrated thermal cameras is that a single calibration function is used to calibrate thermal cameras of the same type, i.e., thermal cameras with the same type of image sensor. Therefore, variations in the performance of individual sensor elements are not taken into account in the calibration. Another drawback of some standard-calibrated thermal cameras is that they may also need to be recalibrated after several years, sometimes even annually. For example, depending on the age of a standard-calibrated thermal camera, the camera can be calibrated using a new calibration function to account for the aging of the camera, i.e., the aging of the image sensor.

[0010] U.S. Patent Application Publication No. 2018 / 0191967 discloses an infrared (IR) imaging system for determining the concentration of a target species in an object. The imaging system can include an optical system including an optical focal plane array (FPA) unit. The optical system can have components defining at least two optical channels thereof, the at least two optical channels being spatially and spectrally distinct from one another. Each of the at least two optical channels can be positioned to direct IR radiation incident on the optical system toward the optical FPA. The system can include a processing unit including a processor, and the processor can be configured to obtain multispectral optical data indicative of the target species from the IR radiation received by the optical FPA. Summary of the Invention

[0011] In view of the above, it is therefore an object of the embodiments disclosed herein to overcome or at least mitigate the above-discussed drawbacks, and in particular to provide methods, devices, and systems that provide simple temperature calibration of thermal cameras while still providing a sufficiently accurate determination of the temperature within a scene.

[0012] The embodiments disclosed herein are applicable for in-situ calibration of thermal cameras. By the expression "in-situ temperature calibrate" as used in this disclosure, it is meant that the temperature calibration is performed at the same location where the thermal camera is operating or is supposed to operate. Thus, the location can be the same location where the thermal camera is installed when in operation. In-situ calibration allows for on-the-fly calibration of the thermal camera, i.e., calibration of the thermal camera during its normal operation.

[0013] According to one embodiment, the object is achieved by a method for temperature calibration and determination of temperature in a scene.

[0014] The method includes collecting ambient temperatures indicative of the temperature of a first portion of the scene at each of a plurality of time points within a first time period to obtain collected ambient temperatures spanning a range of ambient temperatures, and collecting, with a thermal camera, one or more thermal image sensor signal values ​​corresponding to the collected ambient temperatures and associated with the first portion of the scene.

[0015] A calibration function is determined based on the collected ambient temperatures and one or more thermal image sensor signal values ​​corresponding to each of the collected ambient temperatures, the calibration function providing a temperature as a function of the thermal image sensor values.

[0016] Within a second time period, the thermal camera captures a thermal image of the scene, the thermal camera including thermal image sensor values ​​associated with a second portion of the scene.

[0017] Within a second time period, the temperature of a second portion of the scene is determined based on the calibration function and based on one or more thermal image sensor signal values ​​included in the captured thermal image of the scene and associated with the second portion of the scene.

[0018] With such a method, a calibration function is determined based on collected ambient temperatures indicative of the temperature of a first portion of a scene and one or more thermal image sensor signal values ​​corresponding to each of the collected ambient temperatures. The calibration function can be created, for example, by regression analysis, which finds a mathematical function that best fits the collected data. The function can then be created and adjusted to fit the collected values. The scene can be a scene in which a thermal camera is positioned to monitor it when in operation, and the first portion of the scene can correspond to the field of view of the thermal camera, or the first portion of the scene can correspond to a portion of the field of view. The first portion of the scene can be a background portion of the scene, and the first portion of the scene can be selected to include one or more background areas having a certain temperature, for example, a common temperature, such that the ambient temperature is indicative of this temperature. It should be understood that it is important to select the first portion of the scene that can include background areas such that the temperature of the first portion of the scene can be said to be indicative of the ambient temperature, for example, such that the ambient temperature and the common temperature can be the same or approximately the same temperature. One example may be when a first portion of a scene is selected to be part of a scene known or assumed to have a temperature that corresponds to and varies with the measured air temperature in the scene. The ambient temperature is the temperature of the air or other medium and surrounding objects within the geographic location of the scene, as measured by a thermometer or other temperature-indicating device or received from a weather service. One or more background areas of the first portion of the scene may be areas comprising shrubs, trees, fences, facades, or combinations thereof, to name a few.

[0019] In this disclosure, the collected ambient temperatures and the collected thermal image sensor signal values ​​are sometimes collectively referred to as collected calibration data.

[0020] The first period of time can be a long period of time, by which we mean a period spanning one or more days, one or more months, or even one or more years. Thus, by collecting calibration data at multiple time points within the first period of time, the calibration data can be collected over one or more days, one or more months, or even one or more years. Typically, because the thermal camera is located outdoors and the calibration data is collected during the first period of time, the collected calibration data can span a range of ambient temperatures, including daytime and nighttime temperatures and seasonally dependent temperatures, and can also span a range of sensor signal values. It should be understood that the accuracy of the determined calibration function will improve with an increase in the range of collected ambient temperatures and an increase in the range of sensor signal values. However, even if the thermal camera is located indoors and therefore not subject to the same temperature fluctuations, collecting calibration data over a longer period of time will increase the accuracy of the determined calibration function because the number of collected calibration data will increase.

[0021] The calibration function used to determine the temperature of the second portion of the scene within the second time period may be the most recently determined calibration function. However, it should be understood that collection of calibration data may continue after the temperature determination, such that a new most recently determined calibration function may be determined based on calibration data collected before and after the most recent temperature determination. This new most recently determined calibration function may then be used for temperature determination at a new point in time within the second time period.

[0022] Collecting calibration data from a first portion of a scene over an extended period of time while providing a determination of the temperature within a second portion of the scene allows for on-the-fly calibration of the thermal camera. Thus, the thermal camera can be calibrated during its normal operation. This method of updating the calibration function during normal operation also provides the benefit of increasing the number of time points during the first period, thereby increasing the accuracy of the calibration and also accounting for changes within the thermal image sensor over time.

[0023] According to another aspect, the object is achieved by a thermal camera system for temperature calibration and determination of temperature in a scene.

[0024] The collection unit is configured to collect ambient temperatures indicative of a temperature of a first portion of the scene at each of a plurality of time points within the first time period to obtain collected ambient temperatures spanning a range of ambient temperatures.

[0025] The thermal camera is configured to be positioned at the scene and to collect, at each of a plurality of time points within the first time period, one or more thermal image sensor signal values ​​corresponding to the collected ambient temperatures and associated with the first portion of the scene.

[0026] The calibration function determination unit is configured to determine a calibration function based on the collected ambient temperatures and one or more thermal image sensor signal values ​​corresponding to each of the collected ambient temperatures, the calibration function providing a temperature according to the thermal image sensor values.

[0027] The thermal camera is configured to capture a thermal image of the scene within a second time period, the thermal image including thermal image sensor signal values ​​associated with a second portion of the scene.

[0028] Further, the temperature determination unit is configured to determine, within a second period of time, a temperature of a second portion of the scene based on the calibration function and based on one or more thermal image sensor signal values ​​included in the captured thermal image of the scene and associated with the second portion of the scene.

[0029] According to another aspect, the object is achieved by a computer program product comprising a computer readable storage medium having instructions, the instructions being adapted to perform the operations of the methods described herein when executed by a processor.

[0030] Embodiments will now be described in more detail, by way of example only, with reference to the accompanying schematic drawings in which the same reference numerals will be used for similar elements, and in which: [Brief explanation of the drawings]

[0031] [Figure 1A] FIG. 1 shows a schematic diagram of a calibration function for a fully calibrated thermal camera according to the prior art; [Figure 1B] FIG. 1 shows a schematic diagram of a calibration function for a standard calibrated thermal camera according to the prior art; [Figure 2] FIG. 1 is a diagram illustrating a schematic diagram of a thermal camera system according to an embodiment. [Figure 3] 1 is a flowchart of a method for calibration and temperature determination according to an embodiment. [Figure 4] FIG. 2 is a diagram illustrating a schematic of a calibration function for a thermal camera determined according to an embodiment. [Figure 5] FIG. 1 is a diagram illustrating a schematic diagram of a thermal camera system according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0032] Embodiments will now be described more fully hereinafter with reference to the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. The systems and devices disclosed herein are described in operation.

[0033] 2 shows a schematic representation of a thermal camera system 200 including a thermal camera 210 positioned in a scene, e.g., a scene of interest for thermal monitoring. The one-dimensional field of view (FOV) of the thermal camera 210 is shown schematically between two dashed lines. However, it should be understood that the field of view of the thermal camera 210 is two-dimensional.

[0034] Thermal camera 210 generally includes a detector of infrared radiation in the form of a thermal image sensor and can be any type of digital thermal image sensor capable of capturing a thermal image of a scene based on the infrared radiation emitted from the scene. For example, thermal camera 210 can be or include an uncooled thermal camera, which can include an internal thermometer. However, thermal camera 210 can be or include a cooled thermal camera, such as a thermoelectrically cooled thermal camera.

[0035] The thermal camera 210 may be operatively connected to a processing device 220, either wired or wirelessly. The processing device 220 may be physically separate from the thermal camera 210, as shown in FIG. 2, or the processing device 220 may be integrated within the thermal camera 210. In embodiments in which the processing device 220 is physically separate from the thermal camera 210, the processing device 220 is referred to as an external processing device. The thermal camera 210 and the processing device 220 may each be connected to a network 230.

[0036] The processing device 220 and its components may be implemented in software or hardware or a combination thereof. In particular, the processing device 220 may comprise a processor 222 and a memory 224. The memory may serve as a (non-transitory) computer-readable storage medium for storing computer code instructions, which, when executed by the processor, are adapted to perform any of the methods disclosed herein.

[0037] Additionally, processing device 220 may be implemented on a computing platform, such as a personal computer, a custom embedded system, a mobile device, or by one or more of the cameras in a multi-camera system. Some or all of the functionality of processing device 220 in camera system 200 may be incorporated into a software application, such as a Video Management System (VMS), a monitoring tool, or a separate temperature calibration and determination application.

[0038] Network 230 may be any suitable network, e.g., a communications network such as a wired communications network, a wireless communications network, or a combination thereof. The communications network may be a computer network, a telecommunications network, or a combination thereof. For example, network 230 may be a local area network (LAN), a wide area network (WAN), the Internet, cellular, a WiFi network, a cable-based network, an Ethernet-based network, etc., or a combination thereof.

[0039] 3 illustrates a schematic representation of an embodiment of a method for temperature calibration and determining the temperature in a scene. A thermal camera 210 is located in the scene. The thermal camera 210 operates within the thermal camera system 200 to capture a thermal image of the scene. It should be understood that one or more of the operations described below may be optional, and that the operations may be combined or performed in another suitable order. operation 301

[0040] At each of a plurality of time points within a first time period Δt1, an ambient temperature indicative of the temperature of a first portion of the scene is collected.

[0041] Further, at each of a plurality of time points within the first period Δt1, the thermal camera 210 collects one or more thermal image sensor signal values ​​corresponding to the collected ambient temperature and associated with a first portion of the scene.

[0042] As previously mentioned, a thermal image sensor can collect a thermal image of a scene covered by the field of view of the thermal camera, where the thermal image includes thermal image sensor values ​​from the entire field of view of the thermal camera, which are registered by image sensor elements, also commonly referred to as pixels, of the thermal sensor image.

[0043] As previously mentioned, the scene can be a scene in which a thermal camera is positioned to monitor when in operation, and the first portion of the scene can correspond to the field of view of the thermal camera, or the first portion of the scene can correspond to a portion of the field of view. Thus, the first portion of the scene can correspond to a portion of the field of view of the thermal camera 210, or the entire field of view of the thermal camera 210. Thus, the first portion of the scene can correspond to a portion of the captured thermal image or the entire captured thermal image. As previously mentioned, the first portion of the scene can be a background portion of the scene, and the first portion of the scene can be selected to include one or more background areas having a certain temperature, e.g., a normal temperature, such that the ambient temperature is indicative of this temperature. It should be understood that it is important to select the first portion of the scene to include a background area such that the temperature of the first portion of the scene can be said to be indicative of the ambient temperature, e.g., such that the ambient temperature and the normal temperature can be the same or approximately the same temperature. One example may be when a first portion of a scene is selected to be part of a scene known or assumed to have a temperature that corresponds to and varies with the measured air temperature in the scene. The ambient temperature is the temperature of the air or other medium and surrounding objects in the geographic location of the scene, measured by a thermometer or other temperature-indicating device or received from a weather service. One or more background areas of the first portion of the scene may be areas comprising shrubs, trees, fences, facades, or combinations thereof, to name a few. Unrelated to the prior art discussed above, the concepts herein do not use specific objects with known temperatures for calibration. Instead, one or more portions of the image are assumed to have the same temperature as the ambient temperature, which is the temperature measured in the scene, for example, by a thermometer located in or near the thermal image sensor or by a nearby weather station.

[0044] Some examples of how to select the first part of the scene are now described.

[0045] In a first example, a first portion of a scene is selected, e.g., manually selected, based on one or more image sensor elements recording infrared radiation from one or more suitable scene areas, such as vegetation, indirectly sunlit areas, and shaded areas, while avoiding surfaces with a glossy finish and surfaces that reflect the sky or sun. These scene areas are considered suitable because their temperature will be the same or nearly the same as the ambient temperature of the scene. Finding these suitable scene areas can be performed using image analysis tools, such as classification of materials in the scene. Thus, in an outdoor environment and with cloudy weather, most materials in the scene tend to have similar temperatures and will therefore be fairly representative of the collected ambient temperature. Therefore, calibration data collection can be focused on periods with cloudy weather, and that information can be provided from a real-time weather service.

[0046] In a second example, the first portion of the scene is selected, e.g., automatically selected, by the processor 222 of the thermal camera 210 by first calculating an average of the thermal image sensor signal values ​​from all or a portion of the image sensor elements recording the thermal image, second comparing one or more of the thermal image sensor signal values ​​to the average, and third determining the first portion of the scene to correspond to one or more image sensor elements having thermal image sensor signal values ​​within an interval that includes the average. This procedure can be repeated by calculating a new average of the thermal image sensor signal values ​​based on the thermal image sensor signal values ​​that fall within the previous interval, and determining the first portion of the scene to correspond to one or more image sensor elements having thermal image sensor signal values ​​that fall within an interval, e.g., a new interval, that includes the new average.

[0047] The size of the interval within which the average is calculated may be determined based on the standard deviation of the thermal image sensor signal values ​​used to calculate the average.

[0048] When calculating the average thermal image sensor signal value, outliers can be eliminated. Similarly, in this second example, weather information indicating sunny or cloudy weather can be considered, since typically, signals that deviate from the average in sunny weather are unlikely to correspond to ambient temperatures and can be ignored.

[0049] The thermal camera 210 that captures the thermal images has a thermal image sensor 212 that includes a plurality of image sensor elements. Infrared radiation emitted from a first portion of a scene will be recorded by a group of image sensor elements. As such, thermal image sensor signal values ​​from this group of thermal sensor elements are associated with the first portion of the scene. For each of a plurality of time points within a first time period Δt1, the thermal camera 210 can collect thermal image sensor signal values ​​associated with the first portion of the scene from the thermal images captured at each respective time point.

[0050] In some embodiments, the first time period is a long period, such as 24 hours, days, weeks, months, or years. Thus, calibration data is collected both overnight and during the day for 24 hours, days, weeks, months, or years. Furthermore, calibration data can be collected over different seasons, e.g., spring, summer, fall, and / or winter. Depending on the length of the first time period, the range of collected ambient temperatures may vary. For a camera 210 located in an outdoor scene and if collected ambient temperatures can be collected for a full 24-hour period, the collected calibration data may represent the ambient temperature during the warmest part of the day and the coldest part of the night, as well as several collected data points in between. In another example, collected ambient temperatures may be collected for an entire year, including the hottest summer days and the coldest winter days. Similarly, for a thermal camera 210 located in an indoor scene, the first time period can be selected to vary the collected ambient temperatures. The length of the first time period can be based on the desired range of collected calibration data. For example, if the difference between the minimum collected ambient temperature and the maximum collected ambient temperature is small, the length of the first period can be increased until the difference between the minimum collected ambient temperature and the maximum collected ambient temperature becomes large, for example, greater than a threshold such as 5, 10, 20, 30, or 40°C, to name a few.

[0051] Depending on the application, a different temperature range of collected ambient temperatures may be required to obtain acceptable accuracy of the calibration function. For example, if a person is monitored by the thermal camera 210, the range of ambient temperatures used to determine the calibration function may be within the range of 25-40°C. As another example, if the thermal camera 210 is to be calibrated for a warm machine, a larger or different interval of ambient temperatures may be required, or a larger uncertainty in the calibration function may be acceptable. Therefore, the first period should be selected to be long enough to incorporate the desired temperature range.

[0052] In some embodiments, the first period of time is as long as the lifetime of the thermal camera 210. As such, as long as the thermal camera 210 is operating at the operating location, calibration data is collected and the calibration function can be updated and made more accurate, or a new calibration function can be determined. However, it should be understood that a new first period of time can be initiated at any time during the operation of the thermal camera 210, and calibration data collected during this new first period can be used to determine a new calibration function. Additionally, a new first period of time can be initiated after a standby or switch-off period, during which the thermal camera 210 may be relocated. An advantage of using the most recently collected calibration data to determine the calibration function is that the calibration can reflect the most recent operating state of the thermal camera.

[0053] In some embodiments, one or more of the collected ambient temperatures are measured by a thermometer 214, as shown schematically in Figure 2. The thermometer 214 can be an internal thermometer, e.g., a printed circuit board (PCB) thermometer included in the thermal camera 210, or the thermometer 214 can be an external thermometer located in or outside the scene external to the thermal camera 210. The thermometer 214 can also be a remote-sensing thermometer, i.e., a thermometer located at a distance from the scene and in wired or wireless communication with the thermal camera system 200, e.g., over the network 230.

[0054] Alternatively or additionally, one or more of the collected ambient temperatures are collected temperatures from a real-time weather service, so in some embodiments, the thermal camera system 200 can receive or retrieve the ambient temperatures from the real-time weather service via the network 230.

[0055] For example, the collected ambient temperature may represent an ambient temperature within a range of approximately −40° C. to +50° C. Thus, the collected ambient temperature represents the ambient temperature of a normal operating environment for the thermal camera 210. operation 302

[0056] The calibration function is determined based on collected ambient temperatures and one or more thermal image sensor signal values ​​corresponding to each of the collected ambient temperatures. As previously described, the collected ambient temperatures indicate the temperature of a first portion of a scene, and the thermal image sensor values ​​collected during a first time period are associated with the first portion of the scene. The calibration function provides a temperature according to the thermal image sensor values. FIG. 4 shows an example of a calibration function. The illustrated calibration function is determined based on collected ambient temperatures ranging from 10 to 40°C and on a number of collected sensor signal values, the digital signals displayed in the figure. In FIG. 4, the y-axis indicates temperature in °C, and the x-axis indicates image sensor signal values ​​as integers. The calibration function for the thermal camera 210 can be determined in real time after the thermal camera is manufactured. Preferably, the calibration function is determined in situ, i.e., at the installation location for the thermal camera 210 and while the thermal camera 210 is operating. Therefore, calibration does not need to be performed at the manufacturing site. However, it should be understood that the calibration function can be determined at a first location, i.e., a calibration site, and then the determined calibration can be used, and possibly further improved as well, by a thermal camera 210 located at a second location, e.g., an operating site.

[0057] The calibration function may be determined by the thermal camera 210, e.g., by the processing device 220 when included in the thermal camera 210. Alternatively or additionally, the calibration function may be determined by an external processing device, e.g., a processing device 220, e.g., a second instance of the processing device 220, located external to and in communication with the thermal camera 210. The thermal camera 210 may be operably connected to the external processing device via a wire or wirelessly.

[0058] In some embodiments, the calibration function is determined as a function fit of the collected ambient temperatures and the corresponding collected thermal image sensor signal values. As such, the calibration function can be a function that best fits all collected ambient temperatures and all corresponding collected thermal image sensor signal values. The calibration function can be represented, for example, as a mathematical formula or a look-up table. The determination of the calibration function can incorporate one or more temperature thresholds. In this scenario, if the thermal image sensor signal value measured during the second time period exceeds a certain value, this indicates that the determined temperature exceeds a certain predetermined temperature.

[0059] Thus, the calibration function may be determined as a formula, as a look-up table, or as one or more threshold values.

[0060] For at least one of the collected ambient temperatures, an average of one or more thermal image sensor signal values ​​corresponding to the collected ambient temperature can be used when determining the calibration function. For example, an average of multiple, e.g., 25, sensor signal values ​​corresponding to the collected ambient temperatures at the same time within the first time period can be used when determining the calibration function. The 25 sensor signal values ​​can be obtained from 25 thermal image sensor elements arranged in a 5×5 matrix. Reasons for using the average of the sensor signal values ​​instead of all individual sensor signal values ​​include, for example, reducing the number of sensor signal values ​​that need to be stored, reducing potential errors in the individual sensor signal values, or reducing potential noise in the individual sensor signal values, to name a few. For example, by using the average of 25 sensor signal values, only the averaged sensor signal value needs to be stored and used when determining the calibration function, thereby reducing the amount of storage and processing resources required.

[0061] Alternatively or additionally, one or more of the collected thermal image sensor signal values ​​used in determining the calibration function may be at least one of the collected thermal image sensor signal values ​​collected at the same point in time within the first time period.

[0062] Determining the calibration function may further include extrapolating the calibration function to thermal image sensor signal values ​​that provide temperatures outside the range of collected ambient temperatures. The extrapolation may be a linear extrapolation of the collected calibration data. By extrapolating the calibration function, the calibration function may be used to determine temperatures within a scene that are outside the temperature range of the collected calibration data. For example, extrapolating the calibration function may be required when the thermal camera 210 is to be able to indicate or trigger a fire alarm when an object or portion of a scene has a temperature above 250°C because it would be impossible to collect calibration data that includes such high temperatures.

[0063] As previously described, calibration data is collected during a first time period. Furthermore, as previously described, one or more new first time periods can be initiated and new calibration data can be collected. A respective calibration function can be determined for each first time period, for each new first time period, or for a portion thereof. It should be understood that one or more of the determined calibration functions can be stored along with a respective timestamp. The timestamp can indicate one or more points, e.g., a beginning, middle, or end, of the time period during which calibration data used in determining the calibration function was collected. A stored calibration function associated with a timestamp can be used at a later time to determine the temperature of a thermal image captured at a time corresponding to or close to the timestamp. Thus, the stored timestamp can be used when determining which of a plurality of stored calibration functions to use in determining the temperature of a thermal image captured at a certain time. The method of determining the temperature of a captured thermal image is described in more detail below in operation 304.

[0064] It should be understood that the determined calibration function can be used by one or more additional thermal cameras of the same type as the thermal camera 210. Thus, even if not shown in the figures, the thermal camera system 200 can include multiple thermal cameras 210 of the same type. However, the one or more thermal cameras of the same type as the thermal camera 210 included in the thermal camera system 200 can also be thermal cameras included in different camera systems. As used in this disclosure, the expression "thermal cameras of the same type" means that the thermal cameras include one or more image sensors of the same type. operation 303

[0065] The thermal camera 210 captures a thermal image of the scene including thermal image sensor signal values ​​associated with a second portion of the scene within a second time period Δt2.

[0066] In some embodiments, the thermal camera 210 captures a thermal image of the scene at a first time within the second time period Δt2.

[0067] The second time period may not overlap with the first time period. For example, this may be the case when a first time period, during which calibration data, e.g., ambient temperature indicative of a temperature of a first portion of a scene and thermal image sensor signal values ​​associated with the first portion of the scene, is collected, precedes a second time period during which a thermal image of the scene is captured. In such a scenario, the calibration data is collected and the calibration function is determined prior to capturing the thermal image of the scene.

[0068] However, it should be understood that the second time period can at least partially overlap with the first time period. This can be true when the collection of calibration data occurs over a long period of time, such as one or more days, one or more months, or even one or more years, and when the capture of a thermal image of the scene occurs during a first time point within the second time period Δt2 that is also included in the first time period Δt1. The collection of calibration data can continue after the capture of the thermal image, and a new or updated calibration function can be determined based on all of the calibration data collected within the first time period Δt1. It should be understood that the sensor signal values ​​of the image captured at the first time point within the second time period Δt2 can be used in determining the calibration function before the temperature of the scene is determined.

[0069] The first and second portions of the scene may correspond to at least respective first and second portions of the field of view of the thermal camera 210. However, in the case of different zooming of the thermal camera 210, the first and second portions of the scene may correspond to a first portion of the first field of view and a second portion of the second field of view, respectively, of the thermal camera 210. This may be the case, for example, when calibration data is collected using a first field of view of the thermal camera 210 and when the thermal image used to determine the temperature is captured using a different second field of view of the thermal camera. As one example, the first portion of the scene from which the calibration data is collected may be a field of view with the zoom set to a wide angle, and the second portion of the scene from which the temperature determination is performed may be a field of view with the zoom set to a telephoto angle, or the order may be reversed, with the zoom set to a telephoto angle for the first portion of the scene and a wide angle for the second portion.

[0070] In some embodiments, the first and second portions of the scene are non-overlapping portions of the scene. This may be the case when the first portion of the scene used for collecting calibration data does not overlap with the second portion of the scene where temperature determinations are performed. Some examples of when using non-overlapping portions of the scene is advantageous are when the second portion of the scene includes noisy objects, such as moving objects or objects that provide unstable image sensor signals. These noisy objects would obscure the calibration data collected within the first portion of the scene if the first portion of the scene overlapped with the second portion of the scene. In such scenarios, more accurate and stable calibration data would be collected when the first and second portions of the scene do not overlap.

[0071] Alternatively, the first and second portions of the scene are at least partially overlapping portions of the scene. This may be the case when a first portion of the scene used for collecting calibration data at least partially overlaps with a second portion of the scene where temperature determinations are performed. One example of when it may be advantageous to use at least partially overlapping portions of the scene is when a first portion of the scene including trees is used to collect calibration data and determine a calibration function, and when an at least partially overlapping second portion of the scene is used to determine a fire in one of the trees. operation 304

[0072] Within a second time period, a temperature of a second portion of the scene is determined based on the calibration function and based on one or more thermal image sensor signal values ​​included in the thermal image of the scene and associated with the second portion of the scene.

[0073] In some embodiments, the temperature of a second portion of the scene is determined at a second time point within a second time period, which may be different from and subsequent to the first time point. As noted above, a thermal image may be captured by thermal camera 210 at a first time point within the second time period.

[0074] The temperature can be determined by the thermal camera 210, for example, by the processing device 220 when included in the thermal camera 210. The temperature can be determined based on a calibration function and an average of the thermal image sensor signal values ​​included in the captured image and associated with the second portion of the scene.

[0075] Alternatively or additionally, the temperature may be determined by an external processing device, e.g., a processing device 220, e.g., a second instance of processing device 220, located external to and in communication with thermal camera 210. The temperature may be determined based on a calibration function and a received average of thermal image sensor signal values ​​associated with a second portion of the scene.

[0076] As noted above, the second time period can be non-overlapping with the first time period. For example, this can be the case when a first time period, during which calibration data, e.g., ambient temperatures indicative of the ambient temperature of a first portion of a scene, and thermal image sensor signal values ​​are collected, precedes a second time period during which a temperature determination of the scene is determined. In such a scenario, the calibration function is determined before the temperature determination of the scene.

[0077] However, as also noted above, it should be understood that the second time period can at least partially overlap with the first time period. This can be the case when the calibration data collection occurs over a longer period of time, e.g., one or more days, one or more months, or even one or more years, and the scene temperature is determined at a point in time, a second time point within the second time period that is also included in the first time period, using an updated version of the calibration function. The calibration data collection can continue after the temperature is determined, and a new, updated calibration function can be determined based on all calibration data collected within the first time period, i.e., including calibration data collected both before and after the temperature determination. It should be understood that sensor signal values ​​of an image captured at a point in time, e.g., a first time point within the second time period, can be used in determining the calibration function before the scene temperature is determined.

[0078] The calibration function may, as one example, be determined based on selected calibration data, eg, data selected primarily or solely based on later points in time within the first period of time.

[0079] Determining the temperature of the second portion of the scene at a second time within the second time period may be performed in conjunction with capturing the image data at a first time within the second time period.

[0080] In another example, capturing image data at a first time point within the second time period can involve storing the captured image data in suitable image data storage, for example, memory 224. The following step of determining the temperature of a second portion of a scene at a second time point within the second time period can be performed once, when information about the temperature of the second portion of a scene in a captured image is actually needed. At this time, a calibration function determined using any of the methods described above is used. In this manner, the temperature determination of the second portion of a scene can be performed on a previously captured image. As described above, one or more of the determined calibration functions can be stored with respective timestamps. The timestamps can indicate one or more points in time during which the calibration data used in determining the calibration function was collected. As such, the stored calibration function associated with a timestamp can be used at a later time to determine the temperature of a thermal image captured at or near the time indicated by the timestamp. Alternatively, a calibration function created using calibration data collected over a longer period of time, including both points before and after the first and second points in the second time period, can be used. This is because it may provide a more accurate calibration function. operation 305

[0081] In some embodiments, an alarm is indicated when a thermal image sensor signal value in a captured thermal image indicates a temperature that exceeds a temperature threshold. The temperature threshold can be set based on a calibration function. For example, a temperature threshold can be set to indicate an increased temperature, e.g., a temperature that is too high, in an object included in the captured image. For example, a temperature threshold can be set to indicate a fire in the captured image. In such a case, the temperature threshold can be set at, for example, 70°C. However, the temperature threshold can also be set at -40°C to indicate that a chilled item has exceeded a maximum desired temperature, or +40°C for an electronic device, such as a computer server, to indicate that the electronic device has exceeded a maximum desired temperature.

[0082] The alarm may be an audible alarm, a visual alarm, another form of alarm signal, or a combination thereof.

[0083] Additionally, an alarm may be indicated by the thermal camera 210 and / or an alarm may be indicated by the external processing device 220 .

[0084] For example, an alarm may be indicated when thermal image sensor signal values ​​in the captured thermal image indicate a fire in the second portion of the scene.

[0085] An embodiment of a thermal camera system 200 for calibration and temperature determination will now be described with reference to Figure 5. As previously described, a thermal camera 210 and a processing device 220 are configured to operate within the system 200. Furthermore, the thermal camera 210 is configured to capture thermal images of a scene. As previously described, the thermal camera system 200 for temperature calibration and temperature determination within a scene comprises a thermal camera 210.

[0086] The thermal camera system 200 includes a collection unit 216 configured to collect ambient temperatures indicative of the temperature of a first portion of a scene at each of a plurality of time points within a first time period. In FIG. 5 , the collection unit 216 is shown as being part of the thermal camera 210. However, it should be understood that the collection unit 216 can be located external to the thermal camera 210. For example, the collection unit 216 can be included in a processing device 220, which can be included in or in communication with the thermal camera 210.

[0087] In some embodiments, the collection unit 216 is configured to receive the collected ambient temperatures from a thermometer 214 configured to measure one or more of the collected ambient temperatures. Alternatively or additionally, the collection unit 216 may be configured to collect one or more of the collected ambient temperatures from a real-time weather service.

[0088] The thermal camera 210 is configured to be positioned at the scene and to collect, at each of a plurality of time points within a first time period, one or more thermal image sensor signal values ​​corresponding to the collected ambient temperature and relating to a first portion of the scene.

[0089] The thermal camera system 200 includes a calibration determination unit 226-1 configured to determine a calibration function based on the collected ambient temperatures and one or more collected thermal image sensor signal values ​​corresponding to the collected ambient temperatures and associated with a first portion of the scene. As previously mentioned, the calibration function provides a temperature as a function of the thermal image sensor signal values.

[0090] The calibration determination unit 226-1 may be disposed in communication with the collection unit 216 to obtain, eg, receive or retrieve, the collected ambient temperatures used in determining the calibration function.

[0091] The calibration function determination unit 226-1 is included in the processing device 220. As previously mentioned, the processing device 220 may be included in the thermal camera 210. Alternatively or additionally, the processing device 220 or a second instance of the processing device 220 may be an external processing device configured to be located external to and in communication with the thermal camera 210.

[0092] In some embodiments, when processing device 220, including collection unit 216 and calibration function determination unit 226-1, is located external to thermal camera 210, the collection of ambient temperature and determination of the calibration function are performed external to thermal camera 210. In such embodiments, thermal camera 210 obtains, e.g., receives or retrieves, the calibration function from processing device 220. The calibration function may be associated with a timestamp indicating a point in time or a period of time, e.g., a first period of time, when the calibration data was collected.

[0093] The calibration function determination unit 226-1 may be configured to determine as a function fit of the collected temperatures and corresponding collected thermal image sensor signal values.

[0094] One or more of the corresponding collected thermal image sensor signal values ​​used in determining the calibration function can be an average of the thermal image sensor signal values ​​collected at the same point in time within the first time period, meaning that for at least one of the collected ambient temperatures, an average of one or more thermal image sensor signal values ​​corresponding to the collected ambient temperature is used in determining the calibration function.

[0095] Alternatively or additionally, one or more of the corresponding collected thermal image sensor signal values ​​used in determining the calibration function may be at least one of the collected thermal image sensor signal values ​​collected at the same point in time within the first time period.

[0096] In some embodiments, the calibration function determination unit 226-1 is configured to extrapolate the calibration function for thermal image sensor signal values ​​that give temperatures outside the range of collected ambient temperatures.

[0097] Additionally, the thermal camera 210 is configured to capture a thermal image of the scene during a second time period, the thermal image including thermal image sensor signal values ​​associated with a second portion of the scene. In some embodiments, the thermal camera 210 is configured to capture the thermal image at a first time point during the second time period.

[0098] The thermal camera system 200 includes a temperature determination unit 226-2 configured to determine a temperature of a second portion of the scene within a second time period. In some embodiments, the temperature determination unit 226-2 is configured to determine a temperature at a second time point within the second time period.

[0099] Further, the temperature determination unit 226-2 is configured to determine the temperature based on the calibration function and based on one or more thermal image sensor signal values ​​included in a thermal image of the scene and associated with a second portion of the scene. As noted above, the thermal image may be captured by the thermal camera 210 at a first time point within a second time period.

[0100] The first and second portions of the scene may correspond to at least respective first and second portions of the field of view of the thermal camera 210.

[0101] The temperature determination unit 226-2 may be configured to determine the temperature of the second portion of the scene based on the calibration function and based on an average of the thermal image sensor signal values ​​associated with the second portion of the scene.

[0102] In some embodiments, the temperature determination unit 226-2 is configured to indicate an alarm when a thermal image sensor signal value in a captured thermal image indicates a temperature that exceeds a temperature threshold, the temperature threshold being set based on a calibration function.

[0103] It should be understood that the above-mentioned calibration function determination unit 226-1 and temperature determination unit 226-2 can be two separate units included in the processing device 220 or one single determination unit 226 included in the processing device 220.

[0104] The thermal camera system 200 includes a thermometer 214 configured to measure one or more of the collected temperatures. As previously mentioned, the thermometer 214 can be included in the thermal camera 210, or the thermometer 214 can be a thermometer located external to the thermal camera 210.

[0105] In some embodiments, thermal camera system 200 is configured to collect one or more of the collected ambient temperatures from a real-time weather service, for example, by collection unit 216. This may be done, for example, by communicating with the real-time weather service over network 230. The ambient temperatures may thereby be received or retrieved from the real-time weather service over network 230.

[0106] Embodiments herein also relate to a computer program product comprising a computer-readable storage medium having instructions, which when executed by a processor, are adapted to perform a method according to any of the operations described herein.

[0107] It will be appreciated that those skilled in the art can modify the above-described embodiments in many ways and still use the advantages of the present invention as shown in the above embodiments. Therefore, the present invention should not be limited to the embodiments shown, but should be defined only by the appended claims. Furthermore, the embodiments shown can be combined as understood by those skilled in the art.

Claims

1. 1. A method for temperature calibration of a thermal camera configured to be placed in a scene and for determining the temperature within said scene, comprising: collecting ambient temperatures indicative of a temperature in a first portion of the scene at each of a plurality of time points within a first time period to obtain a plurality of ambient temperatures collected in the first portion across a range of ambient temperatures, and collecting, with a thermal camera, one or more thermal image sensor signal values ​​corresponding to each of the collected ambient temperatures and associated with the first portion of the scene to generate a plurality of thermal image sensor signal values, wherein the first portion of the scene includes one or more background areas from which the one or more thermal image sensor signal values ​​are collected; determining a calibration function as a function applied to the plurality of collected environmental temperatures and the plurality of thermal image sensor signal values, the calibration function providing a temperature as a function of the thermal image sensor signal value; and capturing, within a second time period, with the thermal camera, a thermal image of the scene including thermal image sensor signal values ​​associated with a second portion of the scene; and determining a temperature of the second portion of the scene based on the calibration function and based on one or more of the thermal image sensor signal values ​​included in the captured thermal image of the scene and associated with the second portion of the scene. A method comprising:

2. The first period and the second period are - non-overlapping periods, and - At least partially overlapping periods The method of claim 1 , wherein the method is one of:

3. The method of claim 1, wherein one or more of the collected environmental temperatures are measured by a thermometer or collected from a real-time weather service.

4. The method described in claim 1, wherein the collected environmental temperature indicates an environmental temperature within a range of approximately minus 40°C to plus 50°C.

5. The method described in claim 1, wherein for at least one of the collected environmental temperatures, an average of the one or more thermal image sensor signal values ​​corresponding to the collected environmental temperature is used when determining the calibration function.

6. Determining the calibration function comprises: - extrapolating said calibration function for thermal image sensor signal values ​​that give temperatures outside said range of collected environmental temperatures. The method of claim 1 further comprising:

7. Determining the temperature of the second portion of the scene comprises: determining the temperature based on the calibration function and an average of the thermal image sensor signal values ​​associated with the second portion of the scene; The method of claim 1 further comprising:

8. The first and second portions of the scene are - non-overlapping parts of said scenes, and - at least partially overlapping portions of said scenes The method of claim 1 , wherein the method is one of:

9. - indicating an alarm when the thermal image sensor signal value in the captured thermal image indicates a temperature that exceeds a temperature threshold. The method of claim 1 further comprising:

10. 1. A thermal camera system for temperature calibration of a thermal camera configured to be placed in a scene and for determining the temperature within said scene, said thermal camera system comprising: said thermal camera; a collection processor; a calibration function determination processor; - Temperature determination processor and Equipped with the collection processor is configured to collect environmental temperatures indicative of temperatures in a first portion of the scene at each of a plurality of time points within a first time period to obtain a plurality of environmental temperatures collected in the first portion across a range of environmental temperatures; the thermal camera is configured to collect one or more thermal image sensor signal values ​​corresponding to the collected ambient temperature and associated with the first portion of the scene at each of a plurality of time points within the first time period to generate a plurality of thermal image sensor signal values; the first portion of the scene includes one or more background areas from which the one or more thermal image sensor signal values ​​are collected; the calibration function determination processor is configured to determine a calibration function as a function applied to the plurality of collected environmental temperatures and the plurality of thermal image sensor signal values, the calibration function providing a temperature as a function of the thermal image sensor signal values; the thermal camera is configured to capture a thermal image of the scene within a second time period, the thermal image including thermal image sensor signal values ​​associated with a second portion of the scene; the temperature determination processor is configured to determine, during the second time period, a temperature of the second portion of the scene based on the calibration function and based on one or more thermal image sensor signal values ​​included in the captured thermal image of the scene and associated with the second portion of the scene.

11. 11. The thermal camera system of claim 10, wherein the collection processor is configured to receive the collected environmental temperatures from a thermometer configured to measure one or more of the collected environmental temperatures, and / or the collection processor is configured to collect one or more of the collected environmental temperatures from a real-time weather service.

12. the calibration function determination processor and the temperature determination processor; a processing device included in the thermal camera; and an external processing device configured to be located external to and in communication with the thermal camera; 12. The thermal camera system of claim 11, wherein the thermal camera system is included in at least one of:

13. the calibration function determination processor: - extrapolating said calibration function to thermal image sensor signal values ​​that give temperatures outside said range of collected environmental temperatures; The thermal camera system of claim 11 , configured as follows:

14. The temperature determination processor determining the temperature of the second portion of the scene based on the calibration function and an average of the thermal image sensor signal values ​​associated with the second portion of the scene; and / or The temperature determination processor - Indicating an alarm when the thermal image sensor signal value in the captured thermal image indicates a temperature that exceeds the temperature threshold. The thermal camera system of claim 11 , configured as follows:

Images