Methods, apparatuses, and systems for temperature calibration and determining temperature in a scene

Abstract

Methods, devices and systems for temperature calibration and determining temperature in a scene. At each of a plurality of time points in a first time period, an ambient temperature indicative of a temperature at a first portion of the scene is acquired and a thermal image sensor signal value corresponding to the acquired ambient temperature and related to the first portion is acquired. Based on the acquired ambient temperatures and the thermal image sensor signal values corresponding to each of the acquired ambient temperatures, a calibration function is determined. In a second time period, a thermal image of the scene is captured, the thermal image comprising thermal image sensor signal values related to a second portion of the scene, and based on the calibration function and based on the thermal image sensor signal values comprised in the thermal image, a temperature at the second portion of the scene is determined.

Description

Technical Field

[0001] The embodiments described herein relate to the field of thermal cameras. Specifically, the embodiments described herein relate to temperature calibration and determining the temperature in a scene. Background Technology

[0002] Thermal cameras, sometimes called thermal imaging cameras, infrared cameras, or infrared thermal cameras, are imaging devices that use infrared radiation to form images of thermal areas, similar to ordinary cameras that use visible light to form images. Thermal cameras operate in a wavelength range from approximately 3 micrometers to approximately 14 micrometers, rather than the 380-780 nanometer range of visible light cameras.

[0003] Thermal cameras are used in numerous fields such as object detection, anomaly detection, fire monitoring, and trend analysis (to name just a few). In many of these applications, accurately measuring the temperature in the imaging scene is crucial. However, calibrating thermal cameras to accurately measure temperature is currently both difficult and expensive.

[0004] For example, a thermal camera can be calibrated at the manufacturing plant by placing it in a calibration chamber or calibration room with a constant room / room temperature. Once the thermal camera has adapted to the constant temperature in the calibration chamber, various objects with different well-defined absolute temperatures are placed one at a time in front of the thermal camera to calibrate it. Specifically, each object is placed within the scene covered by the thermal camera's field of view. For each object temperature, the thermal camera captures an image of the scene and records the respective image sensor signal value for each sensor element of the image sensor. Once the sensor signal values ​​for all the various objects have been recorded, a calibration function for each sensor element is determined. The calibration function correlates the absolute temperature with the sensor signal value. The calibration process is very time-consuming because multiple objects with different well-defined temperatures must be placed in the calibration chamber and because the sensor signal value for each sensor element of the image sensor must be recorded to obtain the calibration function. Sometimes, a thermal camera is referred to as fully calibrated when a calibration function is determined for each sensor element of the thermal camera.

[0005] US 6,267,501 B1 describes another example of factory calibration using a calibration chamber. In this example, a thermal camera is positioned in a calibration chamber at a controlled ambient temperature. The thermal camera observes an isothermal scene throughout the entire scene through the walls of the calibration chamber, and this temperature is presented to all sensor elements of the thermal camera. During the calibration of the thermal camera, both the chamber temperature and the scene temperature are varied. Specifically, for each chamber temperature, the scene temperature varies within a temperature range, and for each scene temperature, the thermal camera records sensor signal values ​​through a window of the calibration chamber. Subsequently, the chamber temperature is changed, and the thermal camera records sensor signal values ​​for each scene temperature within the temperature range. When sensor signal values ​​have been recorded for all temperatures (i.e., for all chamber temperatures and all scene temperatures), a calibration function is determined for each sensor element and each chamber temperature. The calibration function for each sensor element correlates an absolute temperature with the sensor signal value of the thermal camera operating at the chamber temperature. Because a calibration function is also determined for each sensor element in this example, the thermal camera is referred to as fully calibrated.

[0006] Figure 1A schematically illustrates exemplary calibration functions for different sensor elements of a thermal imaging camera, referred to as fully calibrated. The calibration function for the sensor element labeled "A" is shown as curve A, the calibration function for the sensor element labeled "B" is shown as curve B, and the calibration function for the sensor element labeled "C" is shown as curve C. In Figure 1A, each calibration function is valid for three sensor elements. However, this is merely an example, and it should be understood that each calibration function could be valid for another number of sensor elements. Figure 1A also illustrates calibration performed over a wide temperature range (e.g., between 0 and 300 degrees Celsius).

[0007] Some drawbacks of fully calibrated thermal cameras are that calibration is time-consuming, and they must be recalibrated every few years (sometimes even annually). The thermal camera must then be sent back to the factory, and one of the processes described above must be performed again. This is particularly inconvenient for thermal cameras installed far from the factory.

[0008] Figure 1B schematically illustrates exemplary calibration functions for different sensor elements of a thermal camera, referred to as standard calibration. When used in this disclosure, the expression "standard calibration" means calibrating the same type of thermal camera using a single calibration function. The calibration function is sometimes referred to as the default calibration function. As shown in Figure 1B, the calibration function is the same for all nine sensor elements.

[0009] A drawback of standard-calibrated thermal cameras is that a single calibration function is used to calibrate the same type of thermal camera—that is, thermal cameras with the same type of image sensor. Therefore, performance variations of individual sensor elements are not considered during calibration. Another disadvantage of some standard-calibrated thermal cameras is that they may also need to be recalibrated after a few years (sometimes even annually). Depending on, for example, the lifespan of a standard-calibrated thermal camera, taking into account camera aging, i.e., image sensor aging, a new single calibration function can be used to calibrate the camera.

[0010] US 2018 / 0191967 A1 discloses an infrared (IR) imaging system for determining the concentration of a target species in an object. The imaging system may include an optical system comprising an optical focal plane array (FPA) unit. The optical system may have components defining at least two optical channels that are spatially and spectrally distinct from each other. Each of the at least two optical channels may be positioned to direct IR radiation incident on the optical system toward the optical FPA. The system may include a processing unit containing a processor configured to acquire multispectral optical data representing the target species from the IR radiation received at the optical FPA. Summary of the Invention

[0011] In view of this, the purpose of the embodiments disclosed herein is to overcome or at least mitigate the disadvantages discussed above. Specifically, the purpose is to provide methods, apparatus, and systems that provide simple temperature calibration of thermal cameras and sufficiently accurate determination of the temperature in a scene.

[0012] The embodiments disclosed herein are applicable to the field calibration of thermal cameras. When used in this disclosure, the expression "field temperature calibration" means that the temperature calibration is performed at the same location where the thermal camera is operating or will be operating. Therefore, this location can be the same as the location where the thermal camera is installed during operation. Field calibration enables the immediate calibration of the thermal camera, i.e., calibrating the thermal camera during normal operation.

[0013] According to one aspect, this objective is achieved through methods used for temperature calibration and determining the temperature in a scenario.

[0014] At each of the multiple time points in the first time period, an ambient temperature representing the temperature of the first part of the scene is acquired to obtain an ambient temperature spanning a range, and one or more thermal image sensor signal values ​​corresponding to the acquired ambient temperature and related to the first part of the scene are acquired by means of a thermal camera.

[0015] A calibration function is determined based on the acquired ambient temperature and one or more thermal image sensor signal values ​​corresponding to each of the acquired ambient temperatures. The calibration function gives the temperature as a function of the thermal image sensor values.

[0016] During the second time period, the thermal camera captures thermal images of the scene, which include thermal image sensor values ​​associated with a second part of the scene.

[0017] During the second time period, the temperature at the second part of the scene is determined based on a calibration function and based on one or more thermal image sensor signal values ​​included in the captured thermal image of the scene and related to the second part of the scene.

[0018] Using this method, a calibration function is determined based on the acquired ambient temperature representing the temperature at a first portion of the scene and one or more thermal imaging sensor signal values ​​corresponding to each of the acquired ambient temperatures. The calibration function can be created, for example, through regression analysis, thereby finding a mathematical function that best fits the acquired data. This function can then be created and adjusted to fit the acquired values. The scene can be a scene monitored by a thermal camera during operation, and the first portion of the scene can correspond to the field of view of the thermal camera, or it can correspond to a portion of the field of view. The first portion of the scene can be a background portion of the scene, and it can be selected such that it includes one or more background regions having a temperature (e.g., room temperature), such that the ambient temperature represents that temperature. It should be understood that selecting the first portion of the scene, which may include background regions, is important such that the ambient temperature can be considered, for example, in a way that the ambient temperature and room temperature can be the same or nearly the same temperature in the first portion of the scene. An example could be that the first portion of the scene is selected as a portion of the scene that is known or assumed to have a temperature consistent with and varies with the air temperature measured in the scene.

[0019] Ambient temperature is the temperature of the air or other media and the surrounding environment within the geographic location of the scene, as measured by a thermometer or other temperature indicating device or received from a meteorological service. One or more background areas of the first part of the scene may include areas comprising shrubs, trees, fences, facades, or combinations thereof (to name only).

[0020] In this disclosure, the acquired ambient temperature and the acquired thermal image sensor signal values ​​are sometimes collectively referred to as acquired calibration data.

[0021] The first time period can be a long period, meaning it spans a day, several days, months, or even years. Therefore, by collecting calibration data at multiple points within the first time period, calibration data can be collected over a day, several days, several months, or even several years. Typically, the thermal camera is located outdoors, and because the calibration data is collected during the first time period, the collected calibration data can span an ambient temperature range including daytime and nighttime temperatures as well as seasonally relevant temperatures, and can also span a range of sensor signal values. It should be understood that the accuracy of the determined calibration function will improve as the range of ambient temperatures collected and the range of sensor signal values ​​collected for calibration data increase. However, it should be understood that even if the thermal camera is located indoors and therefore not exposed to the same temperature variations, collecting calibration data over a long time period will still improve the accuracy of the determined calibration function because the amount of calibration data collected will increase.

[0022] The calibration function used to determine the temperature at the second part of the scene in the second time period can be the most recently determined calibration function. However, it should be understood that calibration data acquisition can continue after the temperature is determined, thereby allowing a new, most recently determined calibration function to be determined based on calibration data acquired before and after the latest temperature determination. This new, most recently determined calibration function can then be used to determine the temperature at a new time point in the second time period.

[0023] By acquiring calibration data from the first part of the scene over a long period while simultaneously determining the temperature in the second part of the scene, real-time calibration of the thermal camera is possible. Therefore, the thermal camera can be calibrated during its normal operation. The benefit of this method, which also updates the calibration function during normal operation, is that it increases the number of time points within the first time period, thereby improving calibration accuracy, and also takes into account changes over time within the thermal image sensor.

[0024] According to another approach, this objective is achieved through a thermal imaging camera system used for temperature calibration and determining the temperature in a scene.

[0025] The acquisition unit is configured to acquire ambient temperature representing the temperature of a first part of the scene at each of multiple time points in a first time period to obtain the acquired ambient temperature across a range of ambient temperatures.

[0026] A thermal camera is configured to be located at the scene. Furthermore, the thermal camera is configured to acquire one or more thermal image sensor signal values ​​corresponding to the ambient temperature and related to a first part of the scene at each of multiple time points within a first time period.

[0027] The calibration function determination unit is configured to determine a calibration function based on the acquired ambient temperature and one or more thermal image sensor signal values ​​corresponding to each of the acquired ambient temperatures. The calibration function provides the temperature as a function of the thermal image sensor values.

[0028] The thermal camera is configured to capture thermal images of the scene during a second time period, wherein the thermal images include thermal image sensor values ​​associated with a second part of the scene.

[0029] Furthermore, the temperature determination unit is configured to determine the temperature at the second part of the scene during the second time period based on a calibration function and based on one or more thermal image sensor signal values ​​included in the captured thermal image of the scene and related to the second part of the scene.

[0030] According to another aspect, this objective is achieved by a computer program product comprising a computer-readable storage medium having instructions which, when executed by a processor, are adapted to perform the actions of the methods described herein. Attached Figure Description

[0031] The embodiments will now be described in more detail by way of example and with reference to the accompanying schematic diagrams, in which the same reference numerals will be used for similar elements, wherein:

[0032] Figure 1A schematically illustrates the calibration function of a fully calibrated thermal camera according to the prior art.

[0033] Figure 1B schematically illustrates the calibration function of a thermal camera calibrated according to existing technology standards.

[0034] Figure 2 A thermal imaging camera system according to an embodiment is illustrated schematically.

[0035] Figure 3 This is a flowchart of a method for calibration and temperature determination according to an embodiment.

[0036] Figure 4 The calibration function of the thermal camera, determined according to an embodiment, is illustrated schematically.

[0037] Figure 5 A thermal imaging camera system according to an embodiment is schematically illustrated. Detailed Implementation

[0038] Embodiments will now be described more fully below with reference to the accompanying drawings. However, the invention may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. The systems and apparatuses disclosed herein will be described during the course of work.

[0039] Figure 2 A thermal camera system 200, including a thermal camera 210, is schematically illustrated. The thermal camera 210 is positioned at a scene (e.g., a scene of interest for thermal monitoring). The one-dimensional field of view (FOV) of the thermal camera 210 is schematically illustrated between two dashed lines. However, it should be understood that the field of view of the thermal camera 210 is two-dimensional.

[0040] The thermal camera 210 can typically be any type of digital thermal camera that includes an infrared radiation detector in the form of a thermal image sensor and is capable of capturing a thermal image of the scene based on infrared radiation emitted from the scene. For example, the thermal camera 210 can be or may include an uncooled thermal camera that may include an internal thermometer. However, the thermal camera 210 can be or may include a cooled thermal camera such as a thermoelectrically cooled thermal camera.

[0041] The thermal imaging camera 210 can be operably connected to the processing device 220 via wired or wireless means. The processing device 220 can be physically separated from the thermal imaging camera 210, such as... Figure 2 As illustrated, it can be integrated into the thermal camera 210. In embodiments where the processing device 220 is physically separated from the thermal camera 210, the processing device 220 is referred to as an external processing device. The thermal camera 210 and the processing device 220 can each be connected to the network 230.

[0042] Processing device 220 and its components may be implemented in software or hardware or a combination thereof. Specifically, processing device 220 may include processor 222 and memory 224. Memory may serve as a (non-transitory) computer-readable storage medium or device for storing computer code instructions adapted to perform any of the methods disclosed herein when executed by the processor.

[0043] Furthermore, the processing device 220 can be implemented on a computing platform (such as a personal computer, a custom embedded system, or a mobile device), or by one or more cameras in a multi-camera system. Some or all of the functionality of the processing device 220 within the camera system 200 can be incorporated into software applications such as a video management system (VMS), monitoring tools, or a separate temperature calibration and determination application.

[0044] Network 230 can be any suitable network, such as a wired communication network, a wireless communication network, or a combination thereof. The communication network can be a computer network, a telecommunications network, or a combination thereof. For example, network 230 can be a local area network (LAN), a wide area network (WAN), the Internet, a cellular network, a wireless network, a WiFi network, a cable-based network, an Ethernet-based network, or a combination thereof.

[0045] Figure 3 An embodiment of a method for temperature calibration and determining the temperature in a scene is schematically illustrated. A thermal camera 210 is located at the scene. The thermal camera 210 operates within a thermal camera system 200 and captures thermal images of the scene. It should be understood that one or more of the actions described below may be optional, and the actions may be combined or performed in another suitable order.

[0046] Action 301

[0047] At each of the multiple time points in the first time period Δt1, the ambient temperature representing the temperature of the first part of the scene is collected.

[0048] In addition, at each of the multiple time points in the first time period Δt1, the thermal camera 210 acquires one or more thermal image sensor signal values ​​that correspond to the acquired ambient temperature and are related to the first part of the scene.

[0049] As mentioned earlier, a thermal image sensor can acquire thermal images of a scene covered by the field of view of a thermal camera, and the thermal image includes thermal image sensor values ​​from the entire field of view of the thermal camera. The thermal image sensor signal values ​​are recorded by the image sensor elements (often referred to as pixels of the thermal sensor image).

[0050] As mentioned earlier, the scene can be a scene monitored by a thermal camera during operation, and a first portion of the scene can correspond to the field of view of the thermal camera, or it can correspond to a portion of the field of view. Therefore, the first portion of the scene can correspond to a portion of the field of view of the thermal camera 210 or the entire field of view of the thermal camera 210. Thus, the first portion of the scene can correspond to a portion of the captured thermal image or the entire captured thermal image. As mentioned earlier, the first portion of the scene can be a background portion of the scene, and it can be selected such that it includes one or more background areas having a temperature (e.g., room temperature), such that the ambient temperature represents that temperature. It should be understood that selecting a first portion of the scene that can include background areas is important, such that the ambient temperature can be considered, for example, in a way that the ambient temperature and room temperature can be the same or nearly the same temperature in the first portion of the scene. An example could be that the first portion of the scene is selected as a portion of the scene that is known or assumed to have a temperature consistent with and varies with the air temperature measured in the scene. Ambient temperature is the temperature of the air or other media and the surrounding environment within the geographic location of the scene, measured by a thermometer or other temperature indicating device or received from a meteorological service. One or more background areas of the first part of the scene can be areas including shrubs, trees, fences, facades, or combinations thereof (to name only). Unlike the prior art described previously, the idea here is not to use specific objects with known temperatures for calibration. Instead, it is assumed that one or more parts of the image have the same temperature as the ambient temperature, which is, for example, the temperature measured in the scene by a thermometer located in or near a thermal image sensor or by a nearby weather station.

[0051] Now we will describe some examples of how to select the first part of a scene.

[0052] In the first example, a first portion of the scene is selected (e.g., manually) based on one or more image sensor elements that record infrared radiation from one or more suitable scene areas (such as vegetation, areas not directly exposed to sunlight, parts in shadow, and surfaces with smooth finishes that reflect the sky or sun). These scene areas are considered suitable because their temperatures will be the same as or nearly the same as the ambient temperature of the scene. Finding these suitable scene areas can be performed using image analysis tools, such as the classification of materials within the scene. That is, in outdoor environments and cloudy weather, most materials within a scene tend to have similar temperatures and will therefore represent the ambient temperature collected fairly well. Therefore, the acquisition of calibration data can be focused on cloudy weather time periods whose information can be obtained from real-time weather services.

[0053] In the second example, the first portion of the scene is selected (e.g., automatically) by the processor 222 of the thermal camera 210, as follows: First, the average value of all or part of the thermal image sensor signal values ​​from the image sensor elements recording the thermal image is calculated; second, one or more of the thermal image sensor signal values ​​are compared with the average value; and third, the first portion of the scene is determined to correspond to one or more image sensor elements having thermal image sensor signal values ​​within an interval including the average value. This process can be repeated by calculating a new average value of the thermal image sensor signal values ​​based on the thermal image sensor signal values ​​located within a previous interval and determining the first portion of the scene to correspond to one or more image sensor elements having thermal image sensor signal values ​​within an interval including the new average value (e.g., a new interval).

[0054] The size of the interval including the calculated average value can be determined based on the standard deviation of the thermal image sensor signal values ​​used to calculate the average value.

[0055] Outliers can be excluded when calculating the average thermal imaging sensor signal value. Furthermore, in this second example, meteorological information indicating sunny or cloudy weather can be considered, as signals that deviate significantly from the average in sunny weather are unlikely to correspond to ambient temperature and can be ignored.

[0056] The thermal imaging camera 210, which captures thermal images, has a thermal image sensor 212 comprising multiple image sensor elements. Infrared radiation emitted from a first part of the scene is recorded by a set of image sensor elements. Therefore, the thermal image sensor signal value from this set of thermal sensor elements is correlated with the first part of the scene. For each of a plurality of time points in a first time period Δt1, the thermal imaging camera 210 can acquire thermal image sensor signal values ​​correlated with the first part of the scene from the thermal image captured at each corresponding time point.

[0057] In some embodiments, the first time period is a long period such as 24 hours, days, weeks, months, or even years. Therefore, calibration data is collected over a period of 24 hours, days, weeks, months, or years (including night and day). Furthermore, calibration data can be collected in different seasons, such as spring, summer, autumn, and / or winter. The range of ambient temperatures collected can vary depending on the length of the first time period. For a camera 210 positioned in an outdoor scene where the ambient temperature can be collected for a full 24 hours, the collected calibration data can represent the ambient temperature during the hottest part of the day and the coldest part of the night, as well as several data points in between. In another example, the ambient temperature can be collected year-round, including the hottest summer and the coldest winter. Furthermore, for a thermal camera 210 positioned in an indoor scene, the first time period can be selected such that the ambient temperature collected can vary. The length of the first time period can be based on the desired range of the collected calibration data. For example, if the difference between the lowest and highest ambient temperatures collected is small, the length of the first time period can be increased until the difference between the lowest and highest ambient temperatures collected is large, such as greater than a threshold such as 5, 10, 20, 30 or 40 degrees Celsius (to give some examples).

[0058] Depending on the application, different ranges of ambient temperature may be required to achieve acceptable accuracy in the calibration function. For example, if the thermal camera 210 is monitoring people, the range of ambient temperature used to determine the calibration function could be between 25 and 40 degrees Celsius. As another example, if the thermal camera 210 is being calibrated for warm machinery, a larger or different range of ambient temperatures is needed, or more uncertainty in the calibration function must be acceptable. Therefore, the initial time period should be chosen to be long enough to include the desired temperature range.

[0059] In some embodiments, the first time period is as long as the lifespan of the thermal camera 210. Therefore, as long as the thermal camera 210 is in operation, calibration data is acquired, and the calibration function can be updated and become more accurate, or a new calibration function can be determined. However, it should be understood that a new first time period can begin at any time during the operation of the thermal camera 210, and the calibration data acquired during this new first time period can be used to determine the new calibration function. Furthermore, a new first time period can begin after a standby or off period, during which the thermal camera 210 may have been repositioned. The advantage of using recently acquired calibration data to determine the calibration function is that the calibration can reflect the latest operating state of the thermal camera.

[0060] In some embodiments, one or more of the ambient temperatures collected are measured by thermometer 214, such as Figure 2 The thermometer 214 is schematically illustrated. It can be an internal thermometer, such as a printed circuit board (PCB) thermometer included within the thermal camera 210, or it can be an external thermometer located outside the scene or outside the scene. The thermometer 214 can also be a remote sensing thermometer, i.e., a thermometer located at a distance from the scene and configured to communicate with the thermal camera system 200 via, for example, a network 230, through a wired or wireless means.

[0061] Optionally or additionally, one or more of the ambient temperatures collected are temperatures collected from a real-time weather service. Therefore, in some embodiments, the thermal imaging camera system 200 may receive or retrieve ambient temperatures from a real-time weather service via network 230.

[0062] For example, the collected ambient temperature represents the ambient temperature in the range of approximately -40 degrees Celsius to +50 degrees Celsius. Therefore, the collected ambient temperature represents the ambient temperature of the thermal camera 210 under normal operating conditions.

[0063] Action 302

[0064] A calibration function is determined based on the acquired ambient temperature and one or more thermal image sensor signal values ​​corresponding to each of the acquired ambient temperatures. As mentioned earlier, the acquired ambient temperature represents the temperature at a first part of the scene, and during a first time period, the acquired thermal image sensor values ​​are related to the first part of the scene. The calibration function is given as a function of the thermal image sensor values, which represents the temperature. Figure 4 An example calibration function is shown. The illustrated calibration function is determined based on ambient temperatures collected within the range of 10-40 degrees Celsius and on multiple collected sensor signal values ​​(represented as digital signals in the figure). Figure 4 In the diagram, the y-axis represents temperature in degrees Celsius, and the x-axis represents the image sensor signal values ​​as integers. The calibration function for the thermal camera 210 can be determined in real-time after the thermal camera is manufactured. Preferably, the calibration function is determined in-situ, i.e., at the mounting location of the thermal camera 210 and during its operation. Therefore, calibration at the manufacturing site is unnecessary. However, it should be understood that the calibration function can be determined at a first location (e.g., a calibration site), and the determined calibration can be used and further refined by the thermal camera 210 located at a second location (e.g., a workplace).

[0065] The calibration function can be determined by means of thermal camera 210 (e.g., by means of processing device 220 when processing device 220 is included in thermal camera 210). Optionally or additionally, the calibration function can be determined by means of an external processing device (e.g., processing device 220 disposed outside thermal camera 210 and in communication with thermal camera 210, or a second instance of processing device 220). Thermal camera 210 can be operably connected to the external processing device via wired or wireless means.

[0066] In some embodiments, the calibration function is determined as a function fit between the acquired ambient temperature and the corresponding acquired thermal image sensor signal value. Therefore, the calibration function can be a function that best fits all acquired ambient temperatures and all acquired corresponding thermal image sensor signal values. The calibration function can be expressed, for example, as a mathematical formula or a lookup table. Determining the calibration function may involve finding one or more temperature thresholds. In this case, when the thermal image sensor signal value measured during the second time period is higher than a certain value, this will indicate that the determined temperature is higher than a specific predetermined temperature.

[0067] Therefore, the calibration function can be determined as a formula, a lookup table, or one or more thresholds.

[0068] For at least one of the acquired ambient temperatures, the average of one or more thermal imaging sensor signal values ​​corresponding to the acquired ambient temperature can be used when determining the calibration function. For example, when determining the calibration function, the average of multiple (e.g., 25) sensor signal values ​​corresponding to the ambient temperature acquired at the same time point in a first time period can be used. The 25 sensor signal values ​​can be obtained from 25 thermal imaging sensor elements arranged in a 5×5 matrix. The reason for using the average of the sensor signal values ​​instead of all individual sensor signal values ​​is, for example, to reduce the number of sensor signal values ​​that need to be stored, to reduce possible errors in individual sensor signal values, or to reduce possible noise in individual sensor signal values ​​(to give just a few examples). For example, by using the average of the 25 sensor signal values, only the average sensor signal value needs to be stored and used when determining the calibration function. This reduces the amount of storage and processing resources required.

[0069] Optionally or additionally, one or more of the acquired thermal image sensor signal values ​​used in determining the calibration function may be at least one of the acquired thermal image sensor signal values ​​acquired at the same time point in the first time period.

[0070] Determining the calibration function may further include interpolating the calibration function to the thermal image sensor signal value to a temperature outside the range of the acquired ambient temperature. The interpolation can be a linear interpolation of the acquired calibration data. By interpolating the calibration function, it can be used to determine temperatures in the scene that are outside the temperature range of the acquired calibration data. For example, when the thermal camera 210 should be able to indicate or trigger a fire alarm when an object or part of the scene has a temperature higher than 250 degrees Celsius, the interpolation of the calibration function is necessary because it is impossible to acquire calibration data that includes such high temperatures.

[0071] As mentioned earlier, calibration data is acquired during the first time period. Furthermore, as also mentioned earlier, one or more new first time periods can be started and new calibration data can be acquired. For each first time period, a corresponding calibration function can be determined for each new first time period or for a portion of each new first time period. It should be understood that one or more of the determined calibration functions can be stored along with corresponding timestamps. A timestamp can indicate one or more time points during the time period in which the calibration data used in determining the calibration function was acquired, such as a start time point, intermediate time point, or end time point. The stored calibration function associated with the timestamp can be used at a later time point to determine the temperature in a thermal image captured at a time point corresponding to or close to the time point indicated by the timestamp. Therefore, the stored timestamp can be used when determining which of the multiple stored calibration functions is used to determine the temperature in a thermal image captured at a specific time point. How the temperature in the captured thermal image is determined will be described in more detail in action 304 below.

[0072] It should be understood that the determined calibration function can be used by one or more additional thermal cameras of the same type as thermal camera 210. Therefore, even if not shown in the figures, thermal camera system 200 may include multiple thermal cameras 210 of the same type. However, one or more thermal cameras of the same type as the thermal cameras 210 included in thermal camera system 200 may also be thermal cameras from different thermal camera systems. When used in this disclosure, the expression "the same type of thermal camera" means that the thermal cameras include one or more image sensors of the same type.

[0073] Action 303

[0074] During the second time period Δt2, the thermal camera 210 captures a thermal image of the scene, including thermal image sensor signal values ​​related to the second part of the scene.

[0075] In some embodiments, the thermal camera 210 captures a thermal image of the scene at a first time point in a second time period Δt2.

[0076] The second time period may not overlap with the first time period. For example, this could be a situation where the first time period during which calibration data (e.g., ambient temperature representing the temperature at a first part of the scene and thermal image sensor signal values ​​associated with the first part of the scene) is acquired precedes the second time period during which a thermal image of the scene is captured. In this case, calibration data is acquired, and a calibration function is determined before the thermal image of the scene is captured.

[0077] However, it should be understood that the second time period may at least partially overlap with the first time period. This could be when the acquisition of calibration data occurs over a long period (e.g., over a day or several days, months, or even years) and when a thermal image of the scene is captured at a first time point within the second time period Δt2, which is also included in the first time period Δt1. The acquisition of calibration data can continue after the capture of the thermal image, and a new or updated calibration function can be determined based on all calibration data acquired within the first time period Δt1. It should be understood that sensor signal values ​​from the image captured at the first time point within the second time period Δt2 can be used to determine the calibration function before determining the temperature in the scene.

[0078] The first and second portions of the scene may correspond to at least corresponding first and second portions of the field of view of the thermal camera 210. However, in the case of different zooms of the thermal camera 210, the first and second portions of the scene may correspond to the first portion of the first field of view and the second portion of the second field of view of the thermal camera 210, respectively. For example, this could be the case when acquiring calibration data with the first field of view of the thermal camera 210 and when capturing thermal images for temperature determination with different second fields of view of the thermal camera. As an example, the first portion of the scene for acquiring calibration data may be a field of view where the zoom is set to wide-angle, and the second portion of the scene for performing temperature determination may be a field of view where the zoom is set to telephoto, or conversely, for the first portion of the scene, the zoom is set to telephoto, and for the second portion, the zoom is set to wide-angle.

[0079] In some embodiments, the first and second portions of the scene are non-overlapping parts of the scene. This can be when the first portion of the scene used to acquire calibration data does not overlap with the second portion of the scene in which temperature determination is performed. Some examples of when using non-overlapping portions of the scene is advantageous are when the second portion of the scene includes noisy objects such as moving objects or objects that provide unstable image sensor signals. If the first and second portions of the scene overlap, these noisy objects will obscure the calibration data acquired in the first portion of the scene. In this case, if the first and second portions of the scene do not overlap, more accurate and stable calibration data will be acquired.

[0080] Optionally, the first and second portions of the scene are at least partially overlapping parts of the scene. This could be when the first portion of the scene used to acquire calibration data at least partially overlaps with the second portion of the scene in which temperature determination is performed. An example of when using at least partially overlapping portions of the scene is advantageous is when the first portion of the scene, including the trees, is used to acquire calibration data and determine the calibration function, and when the at least partially overlapping second portion of the scene is used to determine a fire in one of the trees.

[0081] Action 304

[0082] During the second time period, the temperature at the second part of the scene is determined based on a calibration function and based on one or more thermal image sensor signal values ​​included in the thermal image of the scene and related to the second part of the scene.

[0083] In some embodiments, the temperature at a second portion of the scene is determined at a second time point within a second time period. The second time point may differ from and be after the first time point. As mentioned above, the thermal image may be captured by the thermal camera 210 at a first time point within the second time period.

[0084] Temperature can be determined using a thermal camera 210 (e.g., using a processing device 220 if it is included in the thermal camera 210). Temperature can be determined based on a calibration function and on the average value of thermal image sensor signal values ​​included in the captured image and related to a second part of the scene.

[0085] Optionally or additionally, the temperature may be determined by means of an external processing device (e.g., a processing device 220 disposed outside and in communication with the thermal camera 210, or a second instance of the processing device 220). The temperature may be determined based on a calibration function and based on the average value of the received thermal image sensor signal values ​​associated with a second part of the scene.

[0086] As mentioned above, the second time period may not overlap with the first time period. For example, this could be when the first time period during which calibration data (e.g., ambient temperature representing the ambient temperature at a first part of the scene and thermal image sensor signal values) is acquired precedes the second time period during which the temperature in the scene is determined. In this case, calibration data is acquired, and the calibration function is determined before the temperature in the scene is determined.

[0087] However, and as mentioned above, it should be understood that the second time period may at least partially overlap with the first time period. This could be when the acquisition of calibration data occurs over a long period (e.g., over a day or several days, months, or even years) and when the latest version of the calibration function is used to determine the temperature in the scene at a point in time within the second time period (e.g., the second time point), which is also included in the first time period. The acquisition of calibration data can continue after the temperature is determined, and a new, updated calibration function can be determined based on all calibration data acquired during the first time period (i.e., including calibration data acquired before and after the temperature determination). It should be understood that sensor signal values ​​from images captured at a point in time within the second time period (e.g., the first time point) before the temperature in the scene is determined can be used to determine the calibration function.

[0088] As an example, the calibration function can be determined based on selected calibration data (e.g., primarily or solely based on data selected at later points within a first time period).

[0089] The temperature at the second part of the scene can be determined at the second time point in the second time period by combining the image data captured at the first time point in the second time period.

[0090] In another example, after capturing image data at the first time point within the second time period, the captured image data can be stored in an appropriate image data storage device, such as memory 224. The step of determining the temperature at the second part of the scene at the second time point within the second time period can be performed when information about the temperature at the second part of the scene is available in the image data that actually needs to be captured. At this time, any of the determined calibration functions described previously are used. Thus, the temperature determination of the second part of the scene can be performed in the previously captured image data. As mentioned earlier, one or more of the determined calibration functions can be stored along with corresponding timestamps. The timestamps can indicate one or more time points within the time period during which the calibration data used in determining the calibration function was acquired. Therefore, the temperature in the thermal image captured at a time point corresponding to or close to the time point indicated by the timestamp can be determined at a later time point using the stored calibration function associated with the timestamp. Alternatively, a calibration function created using calibration data acquired over a longer time period including the time before and after the first and second time points of the second time period can be used, as this may provide a more accurate calibration function.

[0091] Action 305

[0092] In some embodiments, an alarm is triggered when the thermal imaging sensor signal value in the captured thermal image indicates a temperature above a temperature threshold. The temperature threshold can be set based on a calibration function. For example, the temperature threshold can be set to indicate an elevated temperature, such as an excessively high temperature, in objects included in the captured image. For instance, the temperature threshold can be set to indicate a fire in the captured image. In this case, the temperature threshold could be set to, for example, 70 degrees Celsius. However, the temperature threshold could be set to -40 degrees Celsius to indicate that refrigerated items have exceeded their maximum desired temperature, or for electronic devices such as computer servers, the temperature threshold could be set to +40 degrees Celsius to indicate that the electronic device has exceeded its maximum desired temperature.

[0093] An alarm can be an auditory alarm, a visual alarm, another form of alarm signal, or a combination thereof.

[0094] In addition, the alarm can be indicated by a thermal camera 210 and / or it can be indicated by an external processing device 220.

[0095] For example, an alarm can be issued when the thermal image sensor signal value in the captured thermal image indicates a fire in the second part of the scene.

[0096] Now refer to Figure 5An embodiment of a thermal camera system 200 for calibration and temperature determination is described. As previously described, a thermal camera 210 and a processing device 220 are configured to operate in system 200. Furthermore, the thermal camera 210 is configured to capture thermal images of a scene. Also as previously mentioned, the thermal camera system 200 for temperature calibration and temperature determination in a scene includes the thermal camera 210.

[0097] The thermal imaging camera system 200 includes a data acquisition unit 216, which is configured to acquire an ambient temperature representing the temperature of a first portion of the scene at each of a plurality of time points within a first time period. Figure 5 In the illustration, acquisition unit 216 is shown as part of thermal camera 210. However, it should be understood that acquisition unit 216 may be arranged externally to thermal camera 210. For example, acquisition unit 216 may be included in processing device 220, which may be included in thermal camera 210 or may be arranged to communicate with thermal camera 210.

[0098] In some embodiments, the acquisition unit 216 is configured to receive acquired ambient temperatures from a thermometer 214, which is configured to measure one or more of the acquired ambient temperatures. Optionally or additionally, the acquisition unit 216 may be configured to acquire one or more ambient temperatures from a real-time weather service.

[0099] The thermal camera 210 is configured to be located at the scene and to acquire one or more thermal image sensor signal values ​​corresponding to the acquired ambient temperature and related to a first part of the scene at each of a plurality of time points in a first time period.

[0100] The thermal imaging camera system 200 includes a calibration determination unit 226-1, which is configured to determine a calibration function based on the acquired ambient temperature and one or more acquired thermal image sensor signal values ​​corresponding to the acquired ambient temperature and related to a first part of the scene. As mentioned earlier, the calibration function gives the temperature as a function of the thermal image sensor values.

[0101] The calibration determination unit 226-1 can be arranged to communicate with the acquisition unit 216 in order to acquire (e.g., receive or retrieve) the acquired ambient temperature that will be used when determining the calibration function.

[0102] The calibration function determination unit 226-1 is included in the processing device 220. As mentioned earlier, the processing device 220 may be included in the thermal camera 210. Optionally or additionally, the processing device 220, or a second instance of the processing device 220, may be an external processing device configured to be disposed outside the thermal camera 210 and to communicate with the thermal camera 210.

[0103] In some embodiments, when the acquisition unit 216 and the processing device 220, including the calibration function determination unit 226-1, are arranged outside the thermal camera 210, the acquisition of ambient temperature and the determination of the calibration function are both performed outside the thermal camera 210. In such embodiments, the thermal camera 210 acquires (e.g., receives or retrieves) the calibration function from the processing device 220. The calibration function may be associated with a timestamp indicating the point in time or time period (e.g., a first time period) when calibration data was acquired.

[0104] The calibration function determination unit 226-1 can be configured to determine the calibration function as a function fit between the acquired temperature and the acquired corresponding thermal image sensor signal value.

[0105] One or more of the corresponding thermal image sensor signal values ​​used in determining the calibration function can be the average of the thermal image sensor signal values ​​acquired at the same time point within the first time period. This means that for at least one of the acquired ambient temperatures, the average of one or more thermal image sensor signal values ​​corresponding to the acquired ambient temperature is used when determining the calibration function.

[0106] Optionally or additionally, one or more of the corresponding thermal image sensor signal values ​​acquired when determining the calibration function may be at least one of the acquired thermal image sensor signal values ​​acquired at the same time point in the first time period.

[0107] In some embodiments, the calibration function determination unit 226-1 is configured to interpolate the calibration function to the thermal image sensor signal value that gives a temperature outside the range of the acquired ambient temperature.

[0108] Furthermore, the thermal camera 210 is configured to capture a thermal image of the scene during a second time period, the thermal image including thermal image sensor signal values ​​associated with a second portion of the scene. In some embodiments, the thermal camera 210 is configured to capture a thermal image at a first time point during the second time period.

[0109] The thermal imaging camera system 200 includes a temperature determination unit 226-2, which is configured to determine the temperature at a second portion of the scene during a second time period. In some embodiments, the temperature determination unit 226-2 is configured to determine the temperature at a second time point within the second time period.

[0110] Furthermore, the temperature determination unit 226-2 is configured to determine the temperature based on a calibration function and based on one or more thermal image sensor signal values ​​included in the thermal images of the scene related to the second part of the scene. As mentioned above, the thermal images may have been captured by the thermal camera 210 at a first time point in the second time period.

[0111] The first and second parts of the scene may correspond to at least the corresponding first and second parts of the field of view of the thermal camera 210.

[0112] The temperature determination unit 226-2 can be configured to determine the temperature at the second part of the scene based on a calibration function and based on the average value of the thermal image sensor signal values ​​associated with the second part of the scene.

[0113] In some embodiments, the temperature determination unit 226-2 is configured to issue an alarm when the thermal image sensor signal value in the captured thermal image indicates a temperature higher than a temperature threshold, wherein the temperature threshold is set based on a calibration function.

[0114] It should be understood that the calibration function determination unit 226-1 and temperature determination unit 226-2 described above may be two separate units included in the processing device 220, or a single determination unit 226 included in the processing device 220.

[0115] The thermal camera system 200 includes one or more thermometers 214 configured to measure the acquired temperature. As mentioned earlier, the thermometers 214 may be included in the thermal camera 210, or they may be thermometers disposed outside the thermal camera 210.

[0116] In some embodiments, such as by means of acquisition unit 216, thermal imaging camera system 200 is configured to acquire one or more ambient temperatures from a real-time weather service. This can be performed, for example, by communicating with the real-time weather service via network 230. Thus, ambient temperatures can be received or retrieved from the real-time weather service via network 230.

[0117] The embodiments herein also relate to a computer program product including a computer-readable storage medium having instructions that, when executed by a processor, are adapted to perform any of the actions described herein.

[0118] It will be understood that those skilled in the art can modify the embodiments described above in various ways while still utilizing the advantages of the invention as shown in the above embodiments. Therefore, the invention should not be limited to the embodiments shown, but should be defined only by the appended claims. Furthermore, as those skilled in the art will understand, the embodiments shown can be combined.

Claims

1. A method for calibrating and determining the temperature in a scene using a thermal camera configured to be positioned in the scene, wherein, The method includes: At each of the multiple time points within the first time period, ambient temperatures representing the temperature of a first part of the scene are acquired to obtain multiple ambient temperatures spanning a range. Furthermore, one or more thermal image sensor signal values ​​corresponding to each of the acquired ambient temperatures and related to the first part of the scene are acquired using the thermal camera. Wherein, the first part of the scene includes one or more background regions, and the signal values ​​of the one or more thermal image sensors are collected for the one or more background regions; and The calibration function is determined as a function fit to the acquired ambient temperature and multiple thermal image sensor signal values. The calibration function provides the temperature as a function of the thermal image sensor value; and In the second time period, Using the thermal camera, a thermal image of the scene is captured, the thermal image including thermal image sensor signal values ​​associated with a second part of the scene; and The temperature at the second part of the scene is determined based on the calibration function and based on one or more of the thermal image sensor signal values ​​included in the captured thermal image of the scene and related to the second part of the scene.

2. The method according to claim 1, wherein, The first time period and the second time period are one of the following: Non-overlapping time periods; and The time periods that overlap at least partially.

3. The method according to claim 1, wherein, One or more of the collected ambient temperatures are measured by a thermometer or collected from a real-time weather service.

4. The method according to claim 1, wherein, The collected ambient temperatures represent ambient temperatures within the range of -40 degrees Celsius to +50 degrees Celsius.

5. The method according to claim 1, wherein, For at least one of the acquired ambient temperatures, when determining the calibration function, the average value of the one or more thermal image sensor signal values ​​corresponding to the acquired ambient temperature is used.

6. The method according to claim 1, wherein, Determining the calibration function further includes: The calibration function is interpolated to the thermal image sensor signal value, which is given outside the range of acquired ambient temperatures.

7. The method according to claim 1, wherein, Determining the temperature at the second part of the scene further includes: The temperature is determined based on the calibration function and based on the average value of the thermal image sensor signal values ​​associated with the second part of the scene.

8. The method according to claim 1, wherein, The first part and the second part of the scenario are one of the following: The non-overlapping parts of the scenes; and The scenes at least partially overlap.

9. The method of claim 1, further comprising: An alarm is triggered when the thermal image sensor signal value in the captured thermal image indicates a temperature higher than a temperature threshold.

10. A thermal camera system for calibrating and determining the temperature in a scene using a thermal camera configured to be arranged in a scene, wherein, The thermal imaging camera system includes: The thermal camera; The acquisition unit is configured to acquire ambient temperatures representing the temperature at a first part of the scene at each of multiple time points in a first time period to obtain multiple ambient temperatures across a range of ambient temperatures. The thermal camera is configured to acquire one or more thermal image sensor signal values ​​at each of the plurality of time points within the first time period, corresponding to each of the plurality of acquired ambient temperatures and related to the first part of the scene. The first part of the scene includes one or more background areas, and the signal values ​​of the one or more thermal image sensors are collected for the one or more background areas. The calibration function determination unit is configured to determine the calibration function as a function fit between the acquired ambient temperatures and the acquired thermal image sensor signal values. The calibration function provides the temperature as a function of the thermal image sensor value; and The thermal camera is configured to capture a thermal image of the scene during a second time period, the thermal image including thermal image sensor signal values ​​associated with a second portion of the scene; and A temperature determination unit is configured to determine the temperature at the second part of the scene during the second time period based on the calibration function and based on one or more of the thermal image sensor signal values ​​included in the captured thermal image of the scene and related to the second part of the scene.

11. The thermal imaging camera system according to claim 10, wherein, The acquisition unit is configured to receive multiple ambient temperatures from a thermometer, and the thermometer is configured to measure one or more of the multiple ambient temperatures, and / or The acquisition unit is configured to acquire one or more of the multiple ambient temperatures collected from a real-time weather service.

12. The thermal imaging camera system according to claim 10, wherein, The calibration function determination unit and the temperature determination unit are included in at least one of the following: Processing equipment included in the thermal camera; and It is a processing device for an external processing device, which is configured to be disposed outside the thermal camera and communicate with the thermal camera.

13. The thermal imaging camera system according to claim 10, wherein, The calibration function determination unit is configured as follows: The calibration function is interpolated to the thermal image sensor signal value given at a temperature outside the range of the acquired temperature.

14. The thermal imaging camera system according to claim 10, wherein, The temperature determination unit is configured as follows: The temperature at the second part of the scene is determined based on the calibration function and based on the average value of the thermal image sensor signal values ​​associated with the second part of the scene; and / or The temperature determination unit is configured as follows: An alarm is triggered when the thermal image sensor signal value in the captured thermal image indicates a temperature higher than a temperature threshold.

Images