IMAGE PROCESSING APPARATUS, IMAGE PROCESSING PROGRAM, AND IMAGE PROCESSING METHOD

Abstract

To provide an image processing device which can perform processing in consideration of the influence of the atmosphere without acquiring an actual value of weather data for an image of a heat infrared region.SOLUTION: An image processing device comprises: a radiation temperature conversion unit which converts the radiation luminance in an image of first data of a heat infrared region into the radiation temperature; a low temperature detection unit which detects a low temperature region where the radiation temperature is lower than a threshold in the image of the first data; an atmosphere heat radiation luminance estimation unit which estimates atmosphere heat radiation luminance from a relation between the radiation luminance of the low temperature region and the atmosphere heat radiation luminance; an atmosphere temperature estimation unit which acquires a ground surface temperature obtained from an observation condition and estimates an atmosphere temperature from a relation among the radiation temperature of the entire region of the first data, the ground surface temperature and the atmosphere temperature; an atmosphere transmittance estimation unit which estimates the atmosphere transmittance from a relation among the atmosphere heat radiation luminance, the atmosphere temperature and the atmosphere transmittance; a first atmosphere correction unit which performs atmosphere correction by using the atmosphere heat radiation luminance, the atmosphere temperature and the atmosphere transmittance for the radiation luminance of the entire region; and a post-correction data output unit which converts the radiation luminance after correction into the radiation temperature to output the radiation temperature.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an image processing device that processes images captured using wavelengths in the thermal infrared range to identify subjects. [Background technology]

[0002] Among image processing devices, there are those that process images of subjects such as the earth's surface captured through the atmosphere using wavelengths in the thermal infrared region. This image contains atmospheric effects that must be taken into account when identifying the subject.

[0003] For example, Patent Document 1 discloses taking into account the influence of the atmosphere by using actually observed meteorological data. Specifically, surface meteorological data and upper air meteorological data reflecting the conditions of the observation area at the time of observation are input into a radiative transfer program that performs radiative transfer calculations, and the atmospheric parameters used in the radiative transfer equation to derive the earth's surface brightness temperature in the observation area are obtained from the results of the radiative transfer calculations in the radiative transfer program, thereby taking into account the effects of the atmosphere. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2007-003308 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, the technology disclosed in Patent Document 1 has a problem in that, when considering the influence of the atmosphere on images taken using wavelengths in the thermal infrared region, actual measured values ​​of meteorological data reflecting the state of the observation target area at the time of observation are necessary. It tends to be difficult to obtain all actual measured values ​​of meteorological data reflecting the state of every observation target area at every observation time.

[0006] The present disclosure is intended to solve the above-mentioned problems, and aims to provide an image processing device that can process images captured using wavelengths in the thermal infrared range while taking into account the effects of the atmosphere, without obtaining actual measured values ​​of meteorological data that reflect the state of the observation area at the time of observation. [Means for solving the problem]

[0007] The image processing device of the present disclosure, upon receiving first data including an image captured using wavelengths in the thermal infrared range, includes a radiation temperature conversion unit that converts radiance in the image of the first data into radiation temperature, a low temperature detection unit that detects low temperature areas in the image of the first data where the radiation temperature is lower than a threshold value, an atmospheric thermal radiance estimation unit that uses the radiance in the area detected by the low temperature detection unit to refer to data showing a relationship between the radiance of the low temperature area and atmospheric thermal radiance and estimates the atmospheric thermal radiance, and a radiation temperature of the entire area shown in the image of the first data and a ground surface temperature obtained from observation conditions, and indicates the relationship between the radiation temperature of the entire area, the ground surface temperature, and the atmospheric temperature. the atmospheric temperature estimation unit which estimates the atmospheric temperature by referring to the data; an atmospheric transmittance estimation unit which estimates the atmospheric transmittance by referring to data indicating the relationship between the atmospheric thermal radiance, the atmospheric temperature, and the atmospheric transmittance using the atmospheric thermal radiance estimated by the atmospheric thermal radiance estimation unit and the atmospheric temperature estimated by the atmospheric temperature estimation unit; a first atmospheric correction unit which performs atmospheric correction for the radiance of the entire region in the image using the atmospheric thermal radiance estimated by the atmospheric thermal radiance estimation unit, the atmospheric temperature estimated by the atmospheric temperature estimation unit, and the atmospheric transmittance estimated by the atmospheric transmittance estimation unit; and a corrected data output unit which converts the radiance corrected by the first atmospheric correction unit into a radiation temperature and outputs it. Effect of the Invention

[0008] The present disclosure has the advantage of providing an image processing device that processes images captured using wavelengths in the thermal infrared range while taking into account the influence of the atmosphere, without obtaining actual measured values ​​of meteorological data that reflect the state of the observation area at the time of observation. [Brief description of the drawings]

[0009] [Figure 1] 1 is a diagram illustrating a configuration of an image processing device according to a first embodiment. [Diagram 2] FIG. 2 is a diagram illustrating atmospheric correction in the image processing device. [Diagram 3] FIG. 2 is a diagram illustrating the relationship between atmospheric thermal radiation and the light reflected from the earth's surface. [Figure 4] FIG. 2 is a diagram illustrating the relationship between atmospheric thermal radiation and atmospheric transmittance. [Diagram 5] FIG. 1 is a diagram illustrating the relationship between radiation temperature and radiance. [Figure 6] 4 is a flowchart showing a process performed by the image processing device according to the first embodiment. [Figure 7] FIG. 11 is a diagram illustrating a configuration of an image processing device according to a second embodiment. [Figure 8] 11 is a flowchart showing a process performed by an image processing device according to the second embodiment. [Figure 9] 9 is a flowchart showing the subject estimation process in FIG. 8. [Figure 10] 9 is a flowchart showing a ground illuminance calculation process in FIG. 8. [Figure 11] 9 is a flowchart showing the MWIR data atmospheric correction process in FIG. 8. [Figure 12] 9 is a flowchart showing a temperature calculation process in FIG. 8. [Figure 13] FIG. 2 illustrates an example of a hardware configuration of an image processing apparatus. [Figure 14] FIG. 13 is a diagram illustrating another example of a hardware configuration of the image processing device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] In order to describe the present disclosure in more detail, embodiments of the present disclosure will be described below with reference to the accompanying drawings. Embodiment 1 FIG. 1 is a diagram showing a configuration of an image processing device 10 according to the first embodiment. In FIG. 1, in addition to the image processing device 10, an LWIR sensor 20, a database 31, a database 32, and a database 33 are shown as external components.

[0011] The image processing device 10 performs atmospheric correction on an image captured by the LWIR sensor 20 using data obtained from the image, and outputs the radiation temperature of an object captured in the image using the image after atmospheric correction. The image processing device will be described in detail later.

[0012] The LWIR sensor 20 is a sensor that uses wavelengths in the thermal infrared range (LWIR: Long wavelength infrared). The LWIR sensor 20 is mounted on a flying object such as an artificial satellite or an aircraft, and captures images of targets such as the earth's surface through the atmosphere. The LWIR sensor 20 outputs LWIR data. LWIR data includes images captured using wavelengths in the thermal infrared range, and in addition to image data, includes information that can identify the date and time the image was taken, the location where it was taken, and the area where it was taken. The LWIR data is also referred to as "first data" in the description.

[0013] The databases 31, 32, and 33 are each a database for storing data used in processing by the image processing device 10, and details of each database will be described later in the description of the image processing device 10. The database 31, the database 32, and the database 33 are stored in a storage device not shown in FIG. The database 31, the database 32, and the database 33 may be stored in one storage device, or may be divided and stored in a plurality of storage devices.

[0014] The concept of atmospheric correction in the image processing device according to the present disclosure will be described. FIG. 2 is a diagram for explaining atmospheric correction in the image processing device. When the surface of the earth (earth surface 200) is photographed from the sky by the LWIR sensor 20, the thermal radiation (earth surface thermal radiation 220) of the earth surface 200 is attenuated by the transmittance of the atmosphere 210, and the thermal radiation (atmospheric thermal radiation 221) from the atmosphere 210 and the reflected light of the thermal radiation from the atmosphere 210 to the earth surface 200 (earth surface reflected light 222 of atmospheric thermal radiation) are added and enter the LWIR sensor 20. Therefore, when detecting the earth surface thermal radiation 220, the above is taken into consideration for the image obtained by the LWIR sensor 20. Here, an image captured using wavelengths in the thermal infrared region may contain an object that is very cold, for example, when capturing an image of an urban area, etc. This object has a very low emissivity, such as metal. The thermal radiation from this subject is so small that it can be considered as total atmospheric radiation (= atmospheric thermal radiation between the Earth's surface and the LWIR sensor 20 + reflected light of the atmospheric thermal radiation reflected by the Earth's surface). The present disclosure focuses on this point and, for example, as shown in the following formulas (1) and (2), obtains the original thermal radiation (radiation temperature) of the earth's surface by taking into account all atmospheric radiation. TIFF0007678708000001.tif7150↓ TIFF0007678708000002.tif14150 L sensor : Radiance incident on the sensor L tgt : Thermal radiance from the earth's surface L refl t: The surface reflection brightness of atmospheric thermal radiation L atm : Atmospheric thermal radiance τ: Atmospheric transmittance T: Temperature

[0015] The configuration of the image processing device 10 will be described. The image processing device 10 shown in Figure 1 includes an LWIR data acquisition unit 11, a radiation temperature conversion unit 12, a low temperature detection unit 13, an atmospheric thermal radiance estimation unit 14, an atmospheric temperature estimation unit 15, an atmospheric transmittance estimation unit 16, an atmospheric correction unit 17 (also referred to as a "first atmospheric correction unit" in the description), and a corrected data output unit 18.

[0016] The LWIR data acquisition unit 11 acquires LWIR data (first data) including an image captured using wavelengths in the thermal infrared range. The LWIR data acquisition unit 11 acquires the LWIR data directly or indirectly from the LWIR sensor 20, for example.

[0017] When the radiation temperature conversion unit 12 receives LWIR data including an image captured using wavelengths in the thermal infrared range, it converts the radiance of the image of the first data into a radiation temperature. The radiance and the radiation temperature have a one-to-one correspondence as shown in FIG. 5, for example, and are converted based on information indicating the correspondence.

[0018] The low temperature detection unit 13 detects low temperature regions in an image of LWIR data where the radiation temperature is lower than a threshold. Specifically, the low temperature detection unit 13 compares the radiation temperature with a predetermined threshold, and detects a region showing a radiation temperature lower than the threshold as a low temperature region.

[0019] The atmospheric thermal radiance estimation unit 14 uses the radiance in the area detected by the low temperature detection unit 13 to refer to data (database 31) showing the relationship between the radiance of the low temperature area and the atmospheric thermal radiance to estimate the atmospheric thermal radiance. FIG. 3 is a diagram illustrating the relationship between atmospheric thermal radiation and the light reflected from the earth's surface. Database 31 is a database of atmospheric thermal radiation-ground surface reflected light characteristics of atmospheric thermal radiation (Figure 3) for each combination of atmospheric transmittance (amount of water vapor) and atmospheric temperature when the atmospheric transmittance (amount of water vapor) and atmospheric temperature are changed under typical atmospheric conditions, for example.

[0020] The atmospheric temperature estimation unit 15 acquires the radiation temperature of the entire area shown in the LWIR data image and the ground surface temperature obtained from the observation conditions, and estimates the atmospheric temperature by referring to a database 32 containing data showing the relationship between the radiation temperature of the entire area, the ground surface temperature, and the atmospheric temperature. The observation conditions include, for example, the photographing position (photographing area), the date and time (sun position), and the observation scene (type of object). The photographing position may be information indicating the photographing area, and the date and time may be information indicating the position of the sun. The observation conditions may be information attached to the LWIR data. Specifically, the atmospheric temperature estimation unit 15 calculates the ground surface temperature with respect to the average temperature of the entire image, taking into account an offset according to the observation conditions (time, observation scene (type of object)), for example, using the following equation (3). TIFF0007678708000003.tif7150 T grnd :Surface temperature After estimating the earth's surface temperature, the atmospheric temperature estimation unit 15 refers to the atmospheric temperature measurement data obtained by MODTRAN or the like, and estimates the temperature characteristics of each atmospheric layer.

[0021] The atmospheric transmittance estimation unit 16 uses the atmospheric thermal radiance estimated by the atmospheric thermal radiance estimation unit 14 and the atmospheric temperature estimated by the atmospheric temperature estimation unit 15 to refer to a database 33 containing data showing the relationship between atmospheric thermal radiance, atmospheric temperature, and atmospheric transmittance, and estimates the atmospheric transmittance. FIG. 4 is a diagram illustrating the relationship between atmospheric thermal radiation and atmospheric transmittance. The database 33 is a database in which the relationship between atmospheric thermal radiation and atmospheric transmittance as shown in FIG. 4 is calculated in advance for each atmospheric temperature Tatm.

[0022] The atmospheric correction unit 17 performs atmospheric correction on the radiance of the entire area in the image using the atmospheric thermal radiance estimated by the atmospheric thermal radiance estimation unit 14, the atmospheric temperature estimated by the atmospheric temperature estimation unit 15, and the atmospheric transmittance estimated by the atmospheric transmittance estimation unit 16. FIG. 5 is a diagram illustrating the relationship between radiation temperature and radiance. 5, there is a one-to-one relationship between radiation temperature T and radiance L. In this case, radiation temperature refers to the equivalent blackbody temperature when the emissivity is set to 1. For example, the original thermal radiation (radiant temperature) of the earth's surface can be expressed by the following equation (4). TIFF0007678708000004.tif8150 Lbb: Blackbody radiance ε: Emissivity

[0023] The corrected data output unit 18 converts the radiance corrected by the first atmospheric correction unit 17 into a radiation temperature and outputs it.

[0024] The processing of the image processing device 10 will now be described. FIG. 6 is a flowchart showing the processing of the image processing device 10 according to the first embodiment. The image processing device 10 starts processing when it receives a command signal from a control unit (not shown), for example (START).

[0025] The LWIR data acquisition unit 11 acquires the LWIR data (step ST110). The LWIR data acquisition unit 11 outputs the LWIR data to the radiation temperature conversion unit 12 .

[0026] The radiation temperature conversion unit 12 converts each radiance in the image into a radiation temperature (step ST120). The radiation temperature conversion unit 12 outputs the radiation temperature within the image to the low temperature detection unit 13 .

[0027] The low temperature detection unit 13 detects low temperature areas in the image using the radiation temperature (step ST130). The low temperature detection unit 13 outputs information indicating a low temperature region in the image to the atmospheric heat radiance estimation unit 14.

[0028] The atmospheric thermal radiance estimating unit 14 estimates the atmospheric thermal radiance by using the radiance of the low temperature region (step ST140). Specifically, the atmospheric thermal radiance estimation unit 14 uses the radiance in the area detected by the low temperature detection unit 13 to estimate the atmospheric thermal radiance by referring to data (database 31) showing the relationship between the radiance of the low temperature area and the atmospheric thermal radiance. The atmospheric thermal radiance estimating unit 14 outputs the atmospheric thermal radiance to the atmospheric temperature estimating unit 15 and the atmospheric transmittance estimating unit 16 .

[0029] The atmospheric temperature estimation unit 15 estimates the atmospheric temperature by using the radiation temperature of the entire area (step ST150). Specifically, the atmospheric temperature estimation unit 15 acquires the radiation temperature of the entire area shown in the LWIR data image and the ground surface temperature obtained from the observation conditions, and estimates the atmospheric temperature by referring to a database 32 containing data showing the relationship between the radiation temperature of the entire area, the ground surface temperature, and the atmospheric temperature.

[0030] The atmospheric transmittance estimation unit 16 estimates the atmospheric transmittance using the atmospheric thermal radiance and the atmospheric temperature (step ST160). Specifically, the atmospheric transmittance estimation unit 16 estimates the atmospheric transmittance by referring to the database 33 including data showing the relationship between the atmospheric thermal radiance, the atmospheric temperature, and the atmospheric transmittance.

[0031] The atmospheric correction unit 17 performs atmospheric correction on the image using the estimated atmospheric thermal radiance, atmospheric temperature, and atmospheric transmittance (step ST170). Specifically, the atmospheric correction unit 17 performs atmospheric correction on the radiance of the entire area in the image using the atmospheric thermal radiance estimated by the atmospheric thermal radiance estimation unit 14, the atmospheric temperature estimated by the atmospheric temperature estimation unit 15, and the atmospheric transmittance estimated by the atmospheric transmittance estimation unit 16.

[0032] The corrected data output unit 18 converts the radiance in the image after the atmospheric correction into a radiation temperature and outputs it (step ST180). When the process of step ST180 ends, the process ends (END).

[0033] The image processing device can process images captured using wavelengths in the thermal infrared range, taking into account the effects of the atmosphere, without obtaining actual measured meteorological data that reflects the state of the observation area at the time of observation. The image processing device does not use too much of the resources of the satellite on which the sensor is mounted. The image processing device can perform processing taking into account the effects of the atmosphere without increasing the load on the computer. The image processing device can achieve both efficient and accurate calculations.

[0034] As described above, the image processing device of the present disclosure, upon receiving first data including an image captured using wavelengths in the thermal infrared range, includes a radiation temperature conversion unit that converts radiance in the image of the first data into radiation temperature, a low temperature detection unit that detects low temperature areas in the image of the first data where the radiation temperature is lower than a threshold value, an atmospheric heat radiance estimation unit that uses the radiance in the area detected by the low temperature detection unit to refer to data indicating the relationship between the radiance of the low temperature area and atmospheric heat radiance and estimates the atmospheric heat radiance, and a radiation temperature of the entire area shown in the image of the first data and a ground surface temperature obtained from observation conditions, and indicates the relationship between the radiation temperature of the entire area, the ground surface temperature, and the atmospheric temperature. The image processing apparatus is configured to include an atmospheric temperature estimation unit that estimates atmospheric temperature by referring to data; an atmospheric transmittance estimation unit that estimates atmospheric transmittance by referring to data indicating the relationship between atmospheric thermal radiance, atmospheric temperature, and atmospheric transmittance using the atmospheric thermal radiance estimated by the atmospheric thermal radiance estimation unit and the atmospheric temperature estimated by the atmospheric temperature estimation unit; a first atmospheric correction unit that performs atmospheric correction for the radiance of the entire region in the image using the atmospheric thermal radiance estimated by the atmospheric thermal radiance estimation unit, the atmospheric temperature estimated by the atmospheric temperature estimation unit, and the atmospheric transmittance estimated by the atmospheric transmittance estimation unit; and a corrected data output unit that converts the radiance corrected by the first atmospheric correction unit into radiation temperature and outputs it. This has the effect of providing an image processing device that processes images captured using wavelengths in the thermal infrared range while taking into account the effects of the atmosphere, without obtaining actual measured meteorological data that reflects the state of the observation area at the time of observation.

[0035] As described above, the image processing program according to the present disclosure causes a computer to receive first data including an image captured using wavelengths in the thermal infrared range, and includes a radiation temperature conversion unit that converts radiance in the image of the first data into radiation temperature, a low temperature detection unit that detects low temperature areas in the image of the first data where the radiation temperature is lower than a threshold value, an atmospheric heat radiance estimation unit that uses the radiance in the area detected by the low temperature detection unit to refer to data indicating the relationship between the radiance of the low temperature area and atmospheric heat radiance to estimate the atmospheric heat radiance, and a radiation temperature of the entire area shown in the image of the first data and a ground surface temperature obtained from the observation conditions, and calculates a relationship between the radiation temperature of the entire area, the ground surface temperature, and the atmospheric temperature. the atmospheric temperature estimation unit which estimates the atmospheric temperature by referring to data indicating the relationship; the atmospheric transmittance estimation unit which estimates the atmospheric transmittance by referring to data indicating the relationship between the atmospheric thermal radiance, the atmospheric temperature, and the atmospheric transmittance using the atmospheric thermal radiance estimated by the atmospheric thermal radiance estimation unit and the atmospheric temperature estimated by the atmospheric temperature estimation unit; a first atmospheric correction unit which performs atmospheric correction for the radiance of the entire region in the image using the atmospheric thermal radiance estimated by the atmospheric thermal radiance estimation unit, the atmospheric temperature estimated by the atmospheric temperature estimation unit, and the atmospheric transmittance estimated by the atmospheric transmittance estimation unit; and a corrected data output unit which converts the radiance corrected by the first atmospheric correction unit into radiation temperature and outputs it. This has the effect of providing an image processing program that realizes an image processing device that processes images captured using wavelengths in the thermal infrared range while taking into account the effects of the atmosphere, without obtaining actual measured meteorological data that reflects the state of the observation area at the time of observation.

[0036] As described above, the image processing method according to the present disclosure includes, upon receiving first data including an image captured using wavelengths in the thermal infrared range, a radiation temperature conversion step of converting radiance in the image of the first data into radiation temperature, a low temperature detection step of detecting a low temperature area in the image of the first data where the radiation temperature is lower than a threshold value, an atmospheric heat radiance estimation step of estimating atmospheric heat radiance by referring to data showing a relationship between the radiance of the low temperature area and atmospheric heat radiance using the radiance in the area detected by the low temperature detection step, and an atmospheric heat radiance estimation step of acquiring the radiation temperature of the entire area shown in the image of the first data and the ground surface temperature obtained from the observation conditions, and referring to data showing the relationship between the radiation temperature of the entire area, the ground surface temperature, and the atmospheric temperature, The method is configured to include an atmospheric temperature estimation step of estimating temperature, an atmospheric transmittance estimation step of estimating atmospheric transmittance by referring to data indicating the relationship between atmospheric thermal radiance, atmospheric temperature, and atmospheric transmittance using the atmospheric thermal radiance estimated in the atmospheric thermal radiance estimation step and the atmospheric temperature estimated in the atmospheric temperature estimation step, a first atmospheric correction step of performing atmospheric correction for the radiance of the entire region in the image using the atmospheric thermal radiance estimated in the atmospheric thermal radiance estimation step, the atmospheric temperature estimated in the atmospheric temperature estimation step, and the atmospheric transmittance estimated in the atmospheric transmittance estimation step, and a corrected data output step of converting the radiance corrected by the first atmospheric correction step into radiation temperature and outputting it. This has the effect of providing an image processing method that realizes an image processing device that processes images captured using wavelengths in the thermal infrared range while taking into account the influence of the atmosphere, without obtaining actual measured values ​​of meteorological data that reflect the state of the observation area at the time of observation.

[0037] Embodiment 2 In the first embodiment, an embodiment has been described in which atmospheric correction is performed using an image captured using wavelengths in the thermal infrared region, and the radiation temperature is calculated using the data after atmospheric correction. In the second embodiment, a form is described in which the radiation temperature is calculated using an image taken using wavelengths in the midwave infrared range, and an image taken using wavelengths in the visible range and near infrared range, in addition to an image taken using wavelengths in the thermal infrared range. In the image processing device 100 according to the second embodiment, atmospheric thermal radiance and atmospheric transmittance in an image taken using wavelengths in the midwave infrared range are calculated by using atmospheric characteristic data analyzed from an image taken using wavelengths in the thermal infrared range.

[0038] The configuration of an image processing device 100 according to the second embodiment will be described. FIG. 7 is a diagram showing a configuration of an image processing device 100 according to the second embodiment. FIG. 7 shows, in addition to the image processing device 100, an LWIR sensor 21, a MWIR sensor 22, a VNIR sensor 23, a database 31, a database 32, a database 33, a database 34, a database 35, a database 36, and a database 37 as external components.

[0039] The LWIR sensor 21 is similar to the LWIR sensor 20 in FIG. 1, and therefore a description thereof will be omitted. The MWIR sensor 22 is a sensor that uses a wavelength in the mid-wave infrared range (MWIR). The MWIR sensor 22 is mounted on a flying object such as an artificial satellite or an aircraft, and captures an image of an object such as the earth's surface through the atmosphere. The MWIR sensor 22 outputs MWIR data. MWIR data includes images captured using wavelengths in the mid-wave infrared range at the location where the image of the first data was captured, and in addition to the image data, includes information that can identify, for example, the date and time of capture, the capture location, and the area captured. The MWIR data is also referred to as "second data" in the description.

[0040] The VNIR sensor 23 is a sensor that uses wavelengths in the visible and near infrared ranges (hereinafter also referred to as "visible and near infrared ranges") (VNIR: Visible and Near Infrared). The VNIR sensor 23 is mounted on a flying object such as an artificial satellite or an aircraft, and captures images of targets such as the earth's surface through the atmosphere. The VNIR sensor 23 outputs VNIR data. VNIR data includes images captured using wavelengths in the visible and near-infrared range at the location where the image of the first data was captured, and in addition to the image data, includes information that can identify, for example, the date and time of capture, the capture location, and the area captured. The VNIR data is also referred to as "third data" in the description.

[0041] The LWIR sensor 21, the MWIR sensor 22, and the VNIR sensor 23 may each be mounted on the same flying vehicle and photograph the same area, for example, thereby omitting part of the process of determining whether each piece of data contains images photographed in the corresponding area.

[0042] Database 31, database 32, database 33, database 34 (also referred to as the "fourth database" in the description), database 35 (also referred to as the "fifth database" in the description), database 36 (also referred to as the "sixth database" in the description), and database 37 (also referred to as the "seventh database" in the description) are each databases that store data used for processing by the image processing device, and details of each database will be described later in the description of the image processing device. The database 31, the database 32, the database 33, the database 34, the database 35, the database 36, and the database 37 are stored in a storage device not shown in FIG. Database 31, database 32, database 33, database 34, database 35, database 36, and database 37 may be stored in a single storage device, or each may be stored separately in multiple storage devices.

[0043] The image processing device 100 includes an LWIR data acquisition unit 11, a radiation temperature conversion unit 12, a low temperature detection unit 13, an atmospheric thermal radiance estimation unit 14, an atmospheric temperature estimation unit 15, an atmospheric transmittance estimation unit 16, an atmospheric correction unit 17 (first atmospheric correction unit), a corrected data output unit 18, a VNIR data acquisition unit 121, a subject estimation unit 122, a solar position acquisition unit 131, a ground illuminance calculation unit 132, a MWIR data acquisition unit 141, a MWIR atmospheric information estimation unit 142, a second atmospheric correction unit 143, a reflectance / emissivity estimation unit 151, a temperature calculation unit 152, and an estimated temperature output unit 153.

[0044] The LWIR data acquisition unit 11, radiation temperature conversion unit 12, low temperature detection unit 13, atmospheric heat radiance estimation unit 14, atmospheric temperature estimation unit 15, atmospheric transmittance estimation unit 16, and first atmospheric correction unit 117 shown in Figure 7 are similar to the LWIR data acquisition unit 11, radiation temperature conversion unit 12, low temperature detection unit 13, atmospheric heat radiance estimation unit 14, atmospheric temperature estimation unit 15, atmospheric transmittance estimation unit 16, and atmospheric correction unit 17 (first atmospheric correction unit) in Figure 1, respectively, so their description will be omitted.

[0045] The corrected data output unit 18 converts the radiance corrected by the first atmospheric correction unit 17 into a radiation temperature, and outputs, in addition to the converted radiation temperature, atmospheric characteristic information indicating the atmospheric characteristics used to calculate the radiation temperature. The atmospheric characteristic information output by the corrected data output unit 18 includes, for example, atmospheric thermal radiance, atmospheric temperature, and atmospheric transmittance. The atmospheric transmittance may be information indicating the amount of atmospheric water vapor. The atmospheric transmittance and the amount of atmospheric water vapor can be calculated mutually using one another.

[0046] The VNIR data acquisition unit 121 acquires VNIR data (third data) including an image captured using wavelengths in the visible and near-infrared ranges at the position where the image of the LWIR data (first data) was captured.

[0047] When the subject estimation unit 122 receives VNIR data (third data) including an image captured using wavelengths in the visible and near-infrared ranges at the position where the LWIR data (first data) image was captured, the subject estimation unit 122 refers to database 34 (fourth database) including data indicating the relationship between spectral characteristics, radiation temperature, and type of subject, and estimates the type of subject using the spectral characteristics of the VNIR data image and the radiation temperature output by corrected data output unit 18. The object estimation unit 122 utilizes the fact that objects can be classified by their spectral characteristics from the visible range to the near infrared range. The object estimation unit 122 estimates objects using visible spectral characteristics such as NDVI (Normalized Difference Vegetation Index) that can extract vegetation and NDWI (Normalized Difference Water Index) that can extract water. The object estimation unit 122 also estimates the distinction between metals and artificial objects (such as concrete) using the difference in radiation temperature of LWIR.

[0048] The solar position acquisition unit 131 acquires solar position information indicating the position of the sun. Specifically, the sun position acquisition unit 131 calculates the sun position from information that can identify the shooting date and time, the shooting position, and the shooting area that are attached to the image. The solar position acquisition unit 131 may also acquire solar position information from a device that measures the solar position.

[0049] The ground illuminance calculation unit 132 uses the atmospheric characteristics information output by the corrected data output unit 18 and the solar position information acquired by the solar position acquisition unit 131 to calculate the ground illuminance by referring to a database 35 (fifth database) containing data showing the relationship between atmospheric characteristics, solar position information, and ground illuminance. The database 35 holds the results of calculations of atmospheric characteristics or the characteristics of the solar altitude and the earth's surface illuminance using atmospheric propagation characteristic calculation software such as MODTRAN. Or, if you are only calculating the direct solar irradiance, you can multiply the solar irradiance at the top of the atmosphere by the atmospheric transmittance. However, if you are taking into account the scattering and thermal radiation from the atmosphere to the ground, you will need to create a database.

[0050] The MWIR data acquisition unit 141 acquires MWIR data (second data) including an image captured using wavelengths in the mid-infrared range at the position where the image of the LWIR data (first data) was captured.

[0051] The MWIR atmospheric information estimation unit 142 estimates atmospheric thermal radiation and atmospheric transmittance in the MWIR band using parameters of atmospheric characteristics (atmospheric temperature, water vapor content) obtained from images captured using wavelengths in the thermal infrared range. When the MWIR atmospheric information estimation unit 142 receives MWIR data (second data) including an image taken using wavelengths in the mid-infrared range at the position where the image of the LWIR data (first data) was taken, the MWIR atmospheric information estimation unit 142 uses the atmospheric characteristic information output by the corrected data output unit 18 to refer to a database 36 (sixth database) including data showing the relationship between the atmospheric characteristics when photographed using wavelengths in the thermal infrared range and the atmospheric thermal radiance and atmospheric transmittance when photographed using wavelengths in the mid-infrared range, and estimates the atmospheric thermal radiance and atmospheric transmittance related to the image of the MWIR data. The database 36 holds the results of calculations of atmospheric characteristics and atmospheric thermal radiation / atmospheric transmittance characteristics using atmospheric propagation characteristic calculation software such as MODTRAN.

[0052] The second atmospheric correction unit 143 performs atmospheric correction on the radiance of the entire region in the image of the MWIR data, using the atmospheric thermal radiance and atmospheric transmittance estimated by the MWIR atmospheric information estimation unit 142 .

[0053] The reflectance / emissivity estimation unit 151 refers to database 37 (seventh database) including data indicating the relationship between the type of subject and the reflectance and emissivity, and extracts the reflectance or emissivity according to the type of subject estimated by the subject estimation unit 122. Database 37 stores data showing the relationship between the type of subject, reflectance, and emissivity when wavelengths in the thermal infrared range are used, and data showing the relationship between the type of subject, reflectance, and emissivity when wavelengths in the mid-infrared range are used. The reflectance / emissivity estimation unit 151 extracts the reflectance or emissivity when a wavelength in the thermal infrared range is used, and also extracts the reflectance or emissivity when a wavelength in the mid-infrared range is used.

[0054] The temperature calculation unit 152 calculates the temperature of the subject using the radiance corrected by the second atmospheric correction unit 143, the ground surface illuminance calculated by the ground surface illuminance calculation unit 132, and the reflectance or emissivity extracted by the reflectance / emissivity estimation unit 151. Specifically, temperature calculation unit 152 calculates a first temperature of the subject using the reflectance or emissivity of a wavelength in the thermal infrared range, calculates a second temperature of the subject using the reflectance or emissivity of a wavelength in the mid-infrared range, and changes the reflectance or emissivity extracted by reflectance / emissivity estimation unit 151 so that the first temperature and the second temperature match. The temperature calculation unit 152 calculates the temperature T of the subject from the atmospherically corrected radiance, which is the sum of thermal radiation from the ground surface and solar reflected light (light of sunlight reflected by the atmosphere and light reflected by the ground surface), by removing the influence of the solar reflected light using the emissivity, reflectance, ground surface illuminance, and atmospherically corrected radiance. The temperature calculation unit 152 can perform the calculation using, for example, the following formula (5). TIFF0007678708000005.tif12150 L sensor´ : Radiance after atmospheric correction E: Illuminance on the ground Since ρ=1-ε, there is one variable for each sensor. Furthermore, the temperature calculation unit 152 compares the temperature obtained from the image captured using wavelengths in the thermal infrared range with the temperature obtained from the image captured using wavelengths in the mid-infrared range, and tunes the reflectance or emissivity so that the two match.

[0055] The estimated temperature output unit 153 outputs the temperature calculated by the temperature calculation unit 152.

[0056] The process of the image processing device 100 according to the second embodiment will be described. The process of the image processing device 100 according to the second embodiment differs from that of the flowchart shown in FIG. 6 in that it begins with step ST170. Therefore, in the following description, detailed description of the processes up to step ST170 will be omitted, and the processes after step ST170 will be described.

[0057] FIG. 8 is a flowchart showing the processing of the image processing device 100 according to the second embodiment. When the image processing apparatus 100 finishes the process of step ST170 shown in FIG. 6, the process proceeds to the process of step ST190 shown in FIG.

[0058] In step ST190, the corrected data output unit 18 outputs the atmospheric characteristic information at the radiation temperature in addition to the converted radiation temperature. Specifically, the corrected data output unit 18 converts the radiance corrected by the first atmospheric correction unit 117 into a radiation temperature, and outputs, in addition to the converted radiation temperature, atmospheric characteristic information indicating the atmospheric characteristics used up until the execution of step ST190 to obtain the radiation temperature.

[0059] In the image processing device 100, after executing the processing of step ST190, the process proceeds to subject estimation processing (step ST200), ground surface illuminance calculation processing (step ST300), and MWIR data atmospheric correction processing (step ST400). After the subject estimation processing, ground surface illuminance calculation processing, and MWIR data atmospheric correction processing are completed, the process proceeds to temperature calculation processing (step ST500), and after the temperature calculation processing is completed, the process terminates. The subject estimation process, the ground surface illuminance calculation process, the MWIR data atmospheric correction process, and the temperature calculation process will be described below.

[0060] The subject estimation process will now be described. FIG. 9 is a flowchart showing the subject estimation process in FIG. In the subject estimation process (step ST200), the VNIR data acquisition unit 121 acquires VNIR data (third data) including an image captured using wavelengths in the visible and near-infrared ranges at the position where the image of the first data was captured (step ST210). The VNIR data acquisition unit 121 outputs the VNIR data to the subject estimation unit 122 .

[0061] Upon receiving the VNIR data, subject estimation section 122 estimates the type of subject (step ST220). After estimating the type of the subject, subject estimation section 122 outputs information indicating the type of the subject to reflectance / emissivity estimation section 151.

[0062] The ground surface illuminance calculation process will now be described. FIG. 10 is a flowchart showing the ground surface illuminance calculation process in FIG. In the ground illuminance calculation process, the sun position acquisition unit 131 acquires sun position information indicating the position of the sun (step ST310). The sun position acquisition unit 131 outputs the sun position information to the ground illuminance calculation unit 132.

[0063] The ground illuminance calculation unit 132 calculates the ground illuminance (step ST320). Specifically, the ground illuminance calculation unit 132 calculates the ground illuminance by referring to a database (fifth database) including data showing the relationship between the atmospheric characteristics, the solar position information, and the ground illuminance, using the atmospheric characteristics information output by the corrected data output unit 18 and the solar position information acquired by the solar position acquisition unit 131. The ground illuminance calculation unit 132 outputs the ground illuminance.

[0064] The atmospheric correction process for MWIR data is explained. FIG. 11 is a flowchart showing the MWIR data atmospheric correction process in FIG. In the MWIR data atmospheric correction process, the MWIR data acquisition unit 141 acquires MWIR data (step ST410). The MWIR data acquisition unit 141 acquires MWIR data (second data) including an image captured using wavelengths in the mid-infrared range at the position where the image of the LWIR data (first data) was captured. The MWIR data acquisition unit 141 outputs the MWIR data to the MWIR atmospheric information estimation unit 142 .

[0065] The MWIR atmospheric information estimation unit 142 estimates atmospheric thermal radiance and atmospheric transmittance (step ST420). Specifically, when the MWIR atmospheric information estimation unit 142 receives MWIR data (second data) including an image taken using wavelengths in the mid-infrared range at the location where the image of the LWIR data (first data) was taken, the MWIR atmospheric information estimation unit 142 uses the atmospheric characteristic information output by the corrected data output unit 18 to refer to a database (sixth database) including data showing the relationship between the atmospheric characteristics when photographed using wavelengths in the thermal infrared range and the atmospheric thermal radiance and atmospheric transmittance when photographed using wavelengths in the mid-infrared range, and estimates the atmospheric thermal radiance and atmospheric transmittance related to the image of the second data.

[0066] The second atmospheric correction unit 143 performs atmospheric correction (step ST430). Specifically, the second atmospheric correction unit 143 performs atmospheric correction on the radiance of the entire region in the image of the second data, using the atmospheric thermal radiance and atmospheric transmittance estimated by the MWIR atmospheric information estimation unit 142.

[0067] The temperature calculation process will now be described. FIG. 12 is a flowchart showing the temperature calculation process in FIG. In the temperature calculation process, the reflectance / emissivity estimation unit 151 extracts the reflectance or the emissivity (step ST510). Specifically, reflectance / emissivity estimation section 151 refers to a database (seventh database) that includes data indicating the relationship between the type of subject and the reflectance and emissivity, and extracts the reflectance or emissivity according to the type of subject estimated by subject estimation section 122. The reflectance / emissivity estimation unit 151 extracts the reflectance or emissivity when a wavelength in the thermal infrared range is used, and also extracts the reflectance or emissivity when a wavelength in the mid-infrared range is used. The reflectance / emissivity estimation unit 151 outputs to the temperature calculation unit 152 the reflectance or emissivity when a wavelength in the thermal infrared range is used, and the reflectance or emissivity when a wavelength in the mid-infrared range is used.

[0068] When temperature calculation section 152 receives the reflectance or emissivity from reflectance / emissivity estimation section 151, it calculates the temperature of the object (step ST520). Specifically, the temperature calculation unit 152 calculates the temperature of the subject using the radiance corrected by the second atmospheric correction unit 143, the ground surface illuminance calculated by the ground surface illuminance calculation unit 132, and the reflectance or emissivity extracted by the reflectance / emissivity estimation unit 151. At this time, temperature calculation unit 152 calculates a first temperature of the subject using the reflectance or emissivity of a wavelength in the thermal infrared range, calculates a second temperature of the subject using the reflectance or emissivity of a wavelength in the mid-infrared range, and changes the reflectance or emissivity extracted by reflectance / emissivity estimation unit 151 so that the first temperature and the second temperature match. If the first temperature and the second temperature do not match, the temperature calculation unit 152 may output information indicating that they do not match.

[0069] Estimated temperature output section 153 outputs the temperature calculated by temperature calculation section 152 (step ST530).

[0070] The image processing device estimates the object and estimates the reflectance and emissivity using wavelengths in the thermal infrared range, mid-wave infrared range, visible range, and near-infrared range, so that the temperature of the object can be calculated with greater accuracy. Furthermore, the image processing device calculates the temperature while evaluating and tuning the radiation temperature in the thermal infrared range and the temperature in the mid-wave infrared range, so that the temperature of the subject can be calculated with higher accuracy.

[0071] As described above, in the image processing device according to the present disclosure, the corrected data output unit outputs, in addition to the converted radiation temperature, atmospheric characteristics information indicating the atmospheric characteristics used to calculate the radiation temperature, and further, when second data including an image captured using wavelengths in the mid-infrared range at the position where the image of the first data was captured is received, the corrected data output unit uses the atmospheric characteristics information outputted by the corrected data output unit to refer to data indicating the relationship between the atmospheric characteristics when captured using wavelengths in the thermal infrared range and the atmospheric thermal radiance and atmospheric transmittance when captured using wavelengths in the mid-infrared range, and outputs the atmospheric characteristics information indicating the atmospheric characteristics used to calculate the radiation temperature. a second atmospheric correction unit that performs atmospheric correction on the radiance in the image of the second data by using the atmospheric thermal radiance and atmospheric transmittance estimated by the MWIR atmospheric information estimation unit; a sun position acquisition unit that acquires sun position information indicating the position of the sun; a ground illuminance calculation unit that calculates ground illuminance by referring to data indicating the relationship between the atmospheric characteristics, the sun position information, and ground illuminance using the atmospheric characteristics information output by the corrected data output unit and the sun position information acquired by the sun position acquisition unit; and a subject estimation unit that estimates the type of subject by referring to data indicating the relationship between the spectral characteristics, radiation temperature, and type of subject of the image of the third data and the radiation temperature output by the corrected data output unit, and a reflectance / emissivity estimation unit that estimates the reflectance or emissivity according to the type of subject estimated by the subject estimation unit by referring to data indicating the relationship between the type of subject, reflectance, and emissivity, and a radiance corrected by the second atmospheric correction unit and a ground illuminance calculated by the ground surface illuminance calculation unit. the temperature calculation unit calculates a temperature of the subject using the illuminance and the reflectance or emissivity extracted by the reflectance / emissivity estimation unit, and an estimated temperature output unit outputs the temperature calculated by the temperature calculation unit, wherein the reflectance / emissivity estimation unit extracts the reflectance or emissivity when a wavelength in the thermal infrared range is used, and also extracts the reflectance or emissivity when a wavelength in the mid-infrared range is used, and the temperature calculation unit calculates a first temperature of the subject using the reflectance or emissivity of the wavelength in the thermal infrared range, and calculates a second temperature of the subject using the reflectance or emissivity of the wavelength in the mid-infrared range, and adjusts the first temperature and the second temperature so that they coincide with each other.The reflectance or emissivity extracted by the reflectance / emissivity estimation unit is changed. This makes it possible to provide an image processing device that can calculate the temperature of the subject with higher accuracy.

[0072] Here, a hardware configuration for realizing each of the image processing devices 10 and 100 according to the present disclosure will be described. FIG. 13 is a diagram illustrating an example of a hardware configuration of the image processing apparatuses 10 and 100. As shown in FIG. FIG. 14 is a diagram showing another example of the hardware configuration of the image processing apparatuses 10 and 100. In FIG.

[0073] As shown in FIG. 13 , the functions of the LWIR data acquisition unit 11, 111, the radiation temperature conversion unit 12, 112, the low temperature detection unit 13, 113, the atmospheric thermal radiance estimation unit 14, 114, the atmospheric temperature estimation unit 15, 115, the atmospheric transmittance estimation unit 16, 116, the atmospheric correction unit (first atmospheric correction unit) 17, 117, the corrected data output unit 18, 118, the VNIR data acquisition unit 121, the subject estimation unit 122, the solar position acquisition unit 131, the ground surface illuminance calculation unit 132, the MWIR data acquisition unit 141, the MWIR atmospheric information estimation unit 142, the second atmospheric correction unit 143, the reflectance / emissivity estimation unit 151, the temperature calculation unit 152, or the estimated temperature output unit 153 in the image processing device 10, 100 may be realized by a dedicated processing circuit 1001. The processing circuit 1001 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), an FPGA (Field-Programmable Gate Array), a SoC (System-on-a-Chip), or a system LSI (Large-Scale Integration). In the case of the hardware configuration shown in FIG. 13, the database according to the present disclosure is realized by a storage device 1002. The storage device 1002 may be a non-volatile or volatile semiconductor memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), an EPROM (Erasable Programmable ROM), or a flash memory, or may be a magnetic disk such as a hard disk or a flexible disk, or may be an optical disk such as a CD (Compact Disc) or a DVD (Digital Versatile Disc), or may be a magneto-optical disk. In the case of the hardware configuration shown in FIG. 13, input and output of data related to the image processing device 10, 100 is realized by an input / output interface 1003.

[0074] 14, the image processing device may be configured with a processor 1004 and a memory 1005. The processor 1004 and the memory 1005 are installed in a computer, for example. The memory 1005 stores programs for causing the computer to function as an LWIR data acquisition unit 11, 111, a radiation temperature conversion unit 12, 112, a low temperature detection unit 13, 113, an atmospheric thermal radiance estimation unit 14, 114, an atmospheric temperature estimation unit 15, 115, an atmospheric transmittance estimation unit 16, 116, an atmospheric correction unit (first atmospheric correction unit) 17, 117, a corrected data output unit 18, 118, a VNIR data acquisition unit 121, a subject estimation unit 122, a solar position acquisition unit 131, a ground illuminance calculation unit 132, a MWIR data acquisition unit 141, a MWIR atmospheric information estimation unit 142, a second atmospheric correction unit 143, a reflectance / emissivity estimation unit 151, a temperature calculation unit 152, or an estimated temperature output unit 153. The processor 1004 reads out and executes the programs stored in the memory 1005, thereby realizing the functions of the LWIR data acquisition unit 11, 111, the radiation temperature conversion unit 12, 112, the low temperature detection unit 13, 113, the atmospheric heat radiance estimation unit 14, 114, the atmospheric temperature estimation unit 15, 115, the atmospheric transmittance estimation unit 16, 116, the atmospheric correction unit (first atmospheric correction unit) 17, 117, the corrected data output unit 18, 118, the VNIR data acquisition unit 121, the object estimation unit 122, the solar position acquisition unit 131, the earth surface illuminance calculation unit 132, the MWIR data acquisition unit 141, the MWIR atmospheric information estimation unit 142, the second atmospheric correction unit 143, the reflectance / emissivity estimation unit 151, the temperature calculation unit 152, or the estimated temperature output unit 153. In the case of the hardware configuration shown in FIG. 14, the database according to the present disclosure is realized by the storage device 1002.

[0075] The processor 1004 may be, for example, a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor, a microcontroller, or a digital signal processor (DSP).

[0076] The memory 1005 may be a non-volatile or volatile semiconductor memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), an EPROM (Erasable Programmable ROM), or a flash memory, or a magnetic disk such as a hard disk or a flexible disk, or an optical disk such as a CD (Compact Disc) or a DVD (Digital Versatile Disc), or a magneto-optical disk.

[0077] In the case of the hardware configuration shown in FIG. 14, input and output of data related to the image processing apparatuses 10 and 100 is realized by the input / output interface 1003, similarly to FIG.

[0078] In addition, some of the functions of the LWIR data acquisition unit 11, 111, radiation temperature conversion unit 12, 112, low temperature detection unit 13, 113, atmospheric thermal radiance estimation unit 14, 114, atmospheric temperature estimation unit 15, 115, atmospheric transmittance estimation unit 16, 116, atmospheric correction unit (first atmospheric correction unit) 17, 117, corrected data output unit 18, 118, VNIR data acquisition unit 121, subject estimation unit 122, solar position acquisition unit 131, ground surface illuminance calculation unit 132, MWIR data acquisition unit 141, MWIR atmospheric information estimation unit 142, second atmospheric correction unit 143, reflectance / emissivity estimation unit 151, temperature calculation unit 152, or estimated temperature output unit 153 may be realized by the processor 1004 and memory 1005, and the remaining functions may be realized by the processing circuit 1001.

[0079] The functions of the LWIR data acquisition unit 11, 111, the radiation temperature conversion unit 12, 112, the low temperature detection unit 13, 113, the atmospheric thermal radiance estimation unit 14, 114, the atmospheric temperature estimation unit 15, 115, the atmospheric transmittance estimation unit 16, 116, the atmospheric correction unit (first atmospheric correction unit) 17, 117, the corrected data output unit 18, 118, the VNIR data acquisition unit 121, the object estimation unit 122, the sun position acquisition unit 131, the earth surface illuminance calculation unit 132, the MWIR data acquisition unit 141, the MWIR atmospheric information estimation unit 142, the second atmospheric correction unit 143, the reflectance / emissivity estimation unit 151, the temperature calculation unit 152, or the estimated temperature output unit 153 may be partially realized by dedicated hardware and partially realized by software or firmware. In this way, the processing circuit in the image processing device can realize the above-mentioned functions by hardware, software, firmware, or a combination of these.

[0080] It should be noted that, within the scope of the disclosure, the embodiments may be freely combined, any component of each embodiment may be modified, or any component of each embodiment may be omitted. [Explanation of symbols]

[0081] 10 image processing device, 11,111 LWIR data acquisition unit, 12,112 radiation temperature conversion unit, 13,113 low temperature detection unit, 14,114 atmospheric thermal radiance estimation unit, 15,115 atmospheric temperature estimation unit, 16,116 atmospheric transmittance estimation unit, 17,117 atmospheric correction unit (first atmospheric correction unit), 18,118 corrected data output unit, 20,21 LWIR sensor, 22 MWIR sensor, 23 VNIR sensor, 31 database (first database), 32 database (second database), 33 database (third database), 34 database (fourth database), 35 database (fifth database), 36 database (sixth database), 37 database (seventh database), 121 VNIR data acquisition unit, 122 object estimation unit, 131 solar position acquisition unit, 132 earth surface illuminance calculation unit, 141 MWIR data acquisition unit, 142 MWIR atmospheric information estimation unit, 143 second atmospheric correction unit, 151 reflectance / emissivity estimation unit, 152 temperature calculation unit, 153 estimated temperature output unit, 200 earth's surface, 210 atmosphere, 220 earth's surface thermal radiation, 221 atmospheric thermal radiation, 222 earth's surface reflection of atmospheric thermal radiation, 1001 processing circuit, 1002 storage device, 1003 input / output interface, 1004 processor, 1005 memory.

Claims

1. a radiation temperature conversion unit that converts a radiance of an image of the first data, the image including the image captured using wavelengths in the thermal infrared region, into a radiation temperature when the first data is received; a low temperature detection unit that detects a low temperature region in an image of the first data, the radiation temperature of which is lower than a threshold value; an atmospheric thermal radiance estimation unit that estimates atmospheric thermal radiance by using the radiance in the area detected by the low temperature detection unit and referring to data showing a relationship between the radiance in the low temperature area and the atmospheric thermal radiance; an atmospheric temperature estimation unit that acquires the radiation temperature of the entire area shown in the image of the first data and the ground surface temperature obtained from the observation conditions, and estimates the atmospheric temperature by referring to data showing the relationship between the radiation temperature of the entire area, the ground surface temperature, and the atmospheric temperature; an atmospheric transmittance estimation unit that estimates an atmospheric transmittance by referring to data showing a relationship between the atmospheric thermal radiance, the atmospheric temperature, and the atmospheric transmittance, using the atmospheric thermal radiance estimated by the atmospheric thermal radiance estimation unit and the atmospheric temperature estimated by the atmospheric temperature estimation unit; a first atmospheric correction unit that performs atmospheric correction on the radiance of an entire region in an image using the atmospheric thermal radiance estimated by the atmospheric thermal radiance estimation unit, the atmospheric temperature estimated by the atmospheric temperature estimation unit, and the atmospheric transmittance estimated by the atmospheric transmittance estimation unit; a corrected data output unit that converts the radiance corrected by the first atmospheric correction unit into a radiation temperature and outputs the converted radiation temperature; An image processing device comprising:

2. the corrected data output unit outputs, in addition to the converted radiation temperature, atmospheric characteristic information indicating the atmospheric characteristics used to calculate the radiation temperature; moreover, a MWIR atmospheric information estimation unit that, upon receiving second data including an image captured using wavelengths in the mid-infrared range at the position where the image of the first data was captured, estimates atmospheric thermal radiance and atmospheric transmittance related to the image of the second data by referring to data indicating a relationship between atmospheric characteristics when captured using wavelengths in the thermal infrared range and atmospheric thermal radiance and atmospheric transmittance when captured using wavelengths in the mid-infrared range, using the atmospheric characteristic information output by the corrected data output unit; a second atmospheric correction unit that performs atmospheric correction on the radiance in the image of the second data by using the atmospheric thermal radiance and the atmospheric transmittance estimated by the MWIR atmospheric information estimation unit; a solar position acquisition unit that acquires solar position information indicating the position of the sun; a ground illuminance calculation unit that calculates ground illuminance by referring to data indicating a relationship between atmospheric characteristics, solar position information, and ground illuminance using the atmospheric characteristics information output by the corrected data output unit and the solar position information acquired by the solar position acquisition unit; an object estimation unit that, upon receiving third data including an image captured using wavelengths in the visible and near infrared ranges at the position where the image of the first data was captured, references data indicating a relationship between a spectral characteristic, a radiation temperature, and a type of object, and estimates a type of object using the spectral characteristic of the image of the third data and the radiation temperature output by the corrected data output unit; a reflectance / emissivity estimation unit that refers to data indicating a relationship between a type of object, a reflectance, and an emissivity, and extracts a reflectance or an emissivity according to the type of object estimated by the object estimation unit; a temperature calculation unit that calculates a temperature of a subject using the radiance corrected by the second atmospheric correction unit, the ground surface illuminance calculated by the ground surface illuminance calculation unit, and the reflectance or the emissivity extracted by the reflectance / emissivity estimation unit; an estimated temperature output unit that outputs the temperature calculated by the temperature calculation unit; Equipped with The reflectance / emissivity estimation unit is Extracting reflectance or emissivity when a wavelength in the thermal infrared range is used, and also extracting reflectance or emissivity when a wavelength in the mid-infrared range is used; The temperature calculation unit is calculating a first temperature of the subject using a reflectance or an emissivity of a wavelength in a thermal infrared region, and calculating a second temperature of the subject using a reflectance or an emissivity of a wavelength in a mid-infrared region, and changing the reflectance or the emissivity extracted by the reflectance / emissivity estimation unit so that the first temperature and the second temperature are equal; 2. The image processing device according to claim 1,

3. Computer, a radiation temperature conversion unit that converts a radiance of an image of the first data, the image including the image captured using wavelengths in the thermal infrared region, into a radiation temperature when the first data is received; a low temperature detection unit that detects a low temperature region in an image of the first data, the radiation temperature of which is lower than a threshold value; an atmospheric thermal radiance estimation unit that estimates atmospheric thermal radiance by using the radiance in the area detected by the low temperature detection unit and referring to data showing a relationship between the radiance in the low temperature area and the atmospheric thermal radiance; an atmospheric temperature estimation unit that acquires the radiation temperature of the entire area shown in the image of the first data and the ground surface temperature obtained from the observation conditions, and estimates the atmospheric temperature by referring to data showing the relationship between the radiation temperature of the entire area, the ground surface temperature, and the atmospheric temperature; an atmospheric transmittance estimation unit that estimates an atmospheric transmittance by referring to data showing a relationship between the atmospheric thermal radiance, the atmospheric temperature, and the atmospheric transmittance, using the atmospheric thermal radiance estimated by the atmospheric thermal radiance estimation unit and the atmospheric temperature estimated by the atmospheric temperature estimation unit; a first atmospheric correction unit that performs atmospheric correction on the radiance of an entire region in the image by using the atmospheric thermal radiance estimated by the atmospheric thermal radiance estimation unit, the atmospheric temperature estimated by the atmospheric temperature estimation unit, and the atmospheric transmittance estimated by the atmospheric transmittance estimation unit; a corrected data output unit that converts the radiance corrected by the first atmospheric correction unit into a radiation temperature and outputs the converted radiation temperature; An image processing program that operates in this way.

4. a radiation temperature conversion step of converting a radiance of an image of the first data, the image including the image captured using wavelengths in the thermal infrared region, into a radiation temperature when the first data is received; a low temperature detection step of detecting a low temperature region in an image of the first data, the radiation temperature of which is lower than a threshold value; an atmospheric thermal radiance estimation step of estimating atmospheric thermal radiance by using the radiance in the region detected by the low temperature detection step and referring to data showing a relationship between the radiance in the low temperature region and the atmospheric thermal radiance; an atmospheric temperature estimation step of acquiring the radiation temperature of the entire area shown in the image of the first data and the ground surface temperature obtained from the observation conditions, and estimating the atmospheric temperature by referring to data showing the relationship between the radiation temperature of the entire area, the ground surface temperature, and the atmospheric temperature; an atmospheric transmittance estimation step of estimating an atmospheric transmittance by referring to data showing a relationship between the atmospheric thermal radiance, the atmospheric temperature, and the atmospheric transmittance, using the atmospheric thermal radiance estimated in the atmospheric thermal radiance estimation step and the atmospheric temperature estimated in the atmospheric temperature estimation step; a first atmospheric correction step of performing atmospheric correction on the radiance of an entire region in the image using the atmospheric thermal radiance estimated in the atmospheric thermal radiance estimation step, the atmospheric temperature estimated in the atmospheric temperature estimation step, and the atmospheric transmittance estimated in the atmospheric transmittance estimation step; a corrected data output step of converting the radiance corrected by the first atmospheric correction step into a radiation temperature and outputting the converted radiation temperature; An image processing method comprising:

Images