Image processing apparatus, image processing method, and recording medium

Abstract

The image processing device 100 includes a calculation unit 110 configured to calculate a solar radiation spectrum in a given region on a ground surface based on an observation image of the region, a solar radiation spectral component of the region, and a spectrum of a pure substance estimated in the region, and a conversion unit 120 configured to convert the observation image based on the calculated solar radiation spectrum.

Description

Technical Field

[0001] This disclosure relates to image processing. Background Technology

[0002] In observational images of the Earth's surface, shadows are sometimes included. To observe the Earth's surface with high accuracy, it is necessary to suppress or eliminate the effects of shadows. For example, NPL 1 discloses a technique for classifying high-resolution optical satellite images into shadowed regions (shadow area) and illuminated regions (illuminated area) and correcting the image for each classification region.

[0003] Reference List

[0004] Non-patent literature

[0005] [NPL 1] Wen Liu and Fumio Yamazaki, "Object-Based Shadow Extraction andCorrection of High-Resolution Optical Satellite Images", IEEE JOURNAL OFSELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL.5, NO.4, August 2012, pp.1296-1302.

[0006] [NPL 2] Bird, RE and Riordan, CJ. "Simple Solar Spectral Model for Direct and Diffuse Irradiance on Horizontal and Tilted Planes at the Earth'sSurface for Cloudless Atmospheres", Journal of Climate and Applied Meteorology. Vol. 25(1), January 1986, pp. 87-97. Summary of the Invention

[0007] Technical issues

[0008] In the techniques disclosed in NPL 1, it is difficult to correct the fine grayscale differences of shadows with high precision.

[0009] In addition, this disclosure provides a technique that can suppress the effects of shadows with high precision.

[0010] Solution to the problem

[0011] In one aspect, an image processing apparatus is provided. The image processing apparatus includes: a computing unit for calculating a solar radiation spectrum of a region based on an observed image of a region on the earth's surface, components of the solar radiation spectrum in the region, and an estimated spectrum of pure matter in the region; and a conversion unit for converting the observed image based on the calculated solar radiation spectrum.

[0012] On the other hand, an image processing method is provided. This image processing method includes: calculating the solar radiation spectrum of a region based on an observed image of a region on the Earth's surface, components of the solar radiation spectrum in that region, and an estimated spectrum of pure matter in that region; and converting the observed image based on the calculated solar radiation spectrum.

[0013] In another aspect, a recording medium is provided. This recording medium records a program that causes a computer to perform: processing to calculate the solar radiation spectrum of a region based on an observed image of a region on the Earth's surface, the components of the solar radiation spectrum in that region, and the spectrum of an estimated pure substance in that region; and processing to convert the observed image based on the calculated solar radiation spectrum.

[0014] Advantages of the invention

[0015] According to this disclosure, the effects of shadows are suppressed with high precision. Attached Figure Description

[0016] Figure 1 This is a block diagram illustrating an example configuration of an image processing apparatus according to the present disclosure.

[0017] Figure 2 This is a schematic diagram illustrating an example of the relationship between a pure substance and an observed image in this disclosure.

[0018] Figure 3 This is an illustrative diagram showing an example of the observation model in this disclosure.

[0019] Figure 4 This is a flowchart illustrating an example of a process performed by an image processing apparatus according to the present disclosure.

[0020] Figure 5 This is a block diagram illustrating an example configuration of the image processing system disclosed herein.

[0021] Figure 6 This is a flowchart illustrating another example of a process performed by an image processing apparatus according to the present disclosure.

[0022] Figure 7 This is a block diagram illustrating another example of the configuration of an image processing apparatus according to the present disclosure.

[0023] Figure 8 This is a flowchart illustrating yet another example of a process performed by an image processing apparatus according to the present disclosure.

[0024] Figure 9 This is a block diagram illustrating an example of the hardware configuration of a computer device as disclosed in this disclosure. Detailed Implementation

[0025] First Example Implementation

[0026] Figure 1 This is a block diagram illustrating the configuration of an image processing apparatus 100 according to a first exemplary embodiment. The image processing apparatus 100 includes at least a computing unit 110 and a conversion unit 120. The image processing apparatus 100 may include components other than the computing unit 110 and the conversion unit 120.

[0027] Image processing device 100 is a device configured to convert an observation image containing shadow effects into another form of information in which the effects are suppressed. Here, an observation image refers to an image of the Earth's surface captured from the sky. The Earth's surface mentioned herein is the surface of the Earth and is not necessarily limited to land. The observation image is an image observed using an information-gathering satellite, such as an optical satellite. The number of colors and resolution of the observation image are not limited or specified by any particular value.

[0028] The calculation unit 110 is configured to calculate the solar radiation spectrum. The solar radiation spectrum is the spectrum of sunlight (i.e., solar radiation) illuminating the Earth's surface. The calculation unit 110 is configured to calculate the solar radiation spectrum of at least one region of the Earth's surface. Furthermore, the calculation unit 110 can calculate the solar radiation spectrum for each region. The regions mentioned herein may have a specific shape (e.g., a rectangular shape of predetermined dimensions) or different shapes between regions. In the following description, the region for which the solar radiation spectrum is calculated is also referred to as the "observation region".

[0029] The computing unit 110 is configured to calculate the solar radiation spectrum in the observed region based on the observed image of the region, the components of the solar radiation spectrum of the region, and the spectrum of the pure matter estimated in the region (hereinafter also referred to as the "pure matter spectrum").

[0030] The components of a solar radiation spectrum refer to the corresponding elements in a system that expresses the solar radiation spectrum using several elements. In some models, the components of the solar radiation spectrum are calculated based on a predetermined model. For example, when using the Bird model proposed by Richard E. Bird et al. (see NPL 2), the solar radiation spectrum under clear weather conditions can be expressed using two components: direct light (direct irradiance) and diffuse light (diffuse irradiance). In the case of the Bird model, the components of the solar radiation spectrum can be calculated given predetermined parameters (such as the solar zenith angle). Note that the model used to calculate the components of the solar radiation spectrum is not limited to the Bird model. Furthermore, when the solar radiation spectrum is expressed as a vector, the components of the solar radiation spectrum correspond to basis vectors.

[0031] In the first exemplary embodiment, pure matter refers to the constituent elements of the Earth's surface. Pure matter as referred to herein is also known as an end-member. Pure matter as referred to herein can be a chemical substance (monomer or compound), or can correspond to various scenarios obtained by classifying the Earth's surface into several scenarios (such as water, soil, vegetation, and man-made objects). The types and numbers of pure matter are not particularly limited.

[0032] The reflected light observed when sunlight enters a region of the Earth's surface and is reflected from that region varies depending on the pure substances that make up that region. In cases where the region comprises multiple pure substances, the reflected light from that region varies depending on the proportions of those substances.

[0033] Figure 2 This is a schematic diagram illustrating the relationship between pure matter and observed images. Figure 2 In the example shown, it is assumed that region 210 comprises pure substances R1, R2, and R3. Furthermore, it is assumed that the abundance levels of pure substances R1, R2, and R3 in region 210 are a1, a2, and a3, respectively. The abundance levels mentioned herein are values ​​expressing the proportion of each pure substance in the observed region. Additionally, it is assumed that the spectra of pure substances R1, R2, and R3 are r1, r2, and r3, respectively. The pure substance spectra r1, r2, and r3 are vectors expressing the radiation of reflected light from region 210 in multiple wavelength bands. Note that in this example, it is assumed that shadows do not affect the observed image.

[0034] Sensor 220 includes multiple pixels. Furthermore, typically, the area of ​​a ground surface corresponding to one pixel is large enough for the pixel area. Region 210 can be formed microscopically from a single pure material. However, when observed at a size corresponding to one pixel, region 210 can be formed from a combination of multiple pure materials.

[0035] Therefore, the observed image of region 210 observed by sensor 220 is expressed as a linear sum of the abundance levels a1, a2, and a3 of pure substances R1, R2, and R3 and the spectra r1, r2, and r3 of pure substances. Specifically, the observed image of region 210 is a1r1 + a2r2 + a3r3. However, depending on the pixel, the abundance levels a1, a2, and a3 can be "0".

[0036] The computation unit 110 is configured to compute the norm a for minimizing the model error e based on the observation model expressed, for example, in equation (1) below. m and b p In other words, the computing unit 110 is configured to compute the solar radiation spectrum for each pixel by solving the optimization problem expressed in equation (1).

[0037] Formula 1

[0038]

[0039] Note that the following is defined in equation (1).

[0040] x: The spectrum of each pixel in the observed image.

[0041] a m Weighting coefficients (abundance levels) of pure substance spectra.

[0042] r m Spectrum of pure substances [dimensionless]

[0043] M: The estimable number of pure substances.

[0044] b p The weighting coefficients of the components of the solar radiation spectrum.

[0045] i p Components of the solar radiation spectrum

[0046] P: The number of components in the solar radiation spectrum, and

[0047] e: Model error.

[0048] In addition, the pixel's spectrum x and the pure substance's spectrum r m and the components of the solar radiation spectrum i p It is a vector with the same number of elements as the number of bands. The spectrum x of a pixel is expressed as a vector representing the radiation of multiple bands.

[0049] Figure 3 This is an illustration of the observation model expressed in equation (1). The components i of the solar radiation spectrum p and weighting coefficient b pThe sum of the products corresponds to the solar radiation spectrum in the observation area 300. Here, the weighting coefficient b p Corresponding to the thickness of the shadow. The weighting coefficient b for the direct component. p It approaches "0" when the shadow is thick (i.e., the shadow has a large effect) and approaches "1" when the shadow is thin (i.e., the shadow has a small effect).

[0050] Weighting coefficient a of the spectrum of a pure substance m and pure substance spectrum r m The sum of the products corresponds to the spectrum of reflectance of observation region 300. This spectrum depends only on the characteristics of the pure matter constituting observation region 300 and not on solar radiation. In the observation model expressed in equation (1), the spectrum of the observation image is expressed as the Hadamard product of the spectrum of reflectance of observation region 300 and the solar radiation spectrum (and the model error e).

[0051] The conversion unit 120 is configured to convert the observed image into another form of information. In some modes, the conversion unit 120 corrects the observed image based on the solar radiation spectrum calculated by the calculation unit 110 in order to suppress the effects of shadows. More specifically, from the pixels forming the observed image, the conversion unit 120 corrects pixels with observed brightness values ​​that are smaller (i.e., darker) than the ideal brightness value that would be observed in clear weather, making the pixels closer to the ideal brightness value.

[0052] Alternatively, the conversion unit 120 can convert the observed image with the influence of shadows into another form of information in which the influence of shadows is suppressed. This other form of information may, for example, be the spectrum of reflectance of the observed area, i.e., the weighting coefficient α of the pure material spectrum. m and pure substance spectrum r m The sum of the products of .

[0053] Figure 4 This is a flowchart illustrating the processing performed by the image processing device 100. The image processing device 100 begins the execution of the processing at timings such as the timing when the observed image is input and the timing specified by the user.

[0054] In step S410, the calculation unit 110 is configured to calculate the solar radiation spectrum in the observation area. For example, the calculation unit 110 is configured to calculate the solar radiation spectrum in the observation area for each pixel. Alternatively, the calculation unit 110 can calculate the solar radiation spectrum for a number of adjacent pixels (e.g., four pixels, including two vertically arranged pixels and two horizontally arranged pixels).

[0055] In step S420, the conversion unit 120 is configured to convert the observed image. The conversion unit 120 is configured to convert the observed image based on the solar radiation spectrum calculated in step S410.

[0056] According to the first example embodiment, the image processing device 100 can suppress the effects of shadows with high precision. The image processing device 100 can estimate the effects of shadows for each component of the solar radiation spectrum, thus improving the accuracy of suppression.

[0057] The thickness of the shadow formed in the observation area varies depending on various conditions of the area. For example, if the range corresponding to a pixel in the observation area includes both sunny and shadowed areas, the pixel's brightness value becomes an intermediate value between the brightness value corresponding to the sunny area and the brightness value corresponding to the shadowed area. Furthermore, the thickness of the shadow formed in the observation area varies depending on the tilt of the area and differences in sky factors determined by surrounding trees, buildings, etc. Therefore, the thickness of the shadow formed in the observation area may exhibit subtle variations that cannot be expressed (reproduced) due to the classification of the area.

[0058] The image processing device 100 uses a model that expresses the changes in the solar radiation spectrum by sequentially varying the parameters, estimates the parameters for each pixel, and thus estimates the fine variations in the solar radiation spectrum. In this way, the image processing device 100 can suppress the effects of shadows formed in the observed image.

[0059] Second Example Implementation

[0060] Figure 5 This is a block diagram illustrating the configuration of an image processing system 500 in another example embodiment. The image processing system 500 includes an image supply device 510, an image processing device 520, and an output device 530.

[0061] Image supply device 510 is configured to supply observation data indicating the observed image to image processing device 520. Image supply device 510 is, for example, a database in which observation data is accumulated. Image processing device 520 is configured to correct the observed image based on the observation data supplied from image supply device 510. Output device 530 is configured to output the observation image corrected by image processing device 520. Output device 530 is, for example, a display device configured to display the corrected observation image. Furthermore, output device 530 may be a recording device configured to record the observation data indicating the corrected observation image on a recording medium.

[0062] In the second example embodiment, the observation image is an image observed by an artificial satellite and expressed using electron waves in multiple wavelength bands. Such observation images are typically also referred to as multispectral or hyperspectral images. However, the number of wavelength bands expressing the observation image (hereinafter also referred to as the "band number") is not particularly limited to a specific value. Furthermore, the maximum and minimum wavelength bands expressing the observation image are not limited to specific values. The observation image includes multiple observation regions.

[0063] The observation data includes image information (e.g., brightness values) indicating the observed image and metadata. Meta-information includes, for example, information about the observation area. Meta-information about the observation area includes the observation date and time, location (latitude and longitude), solar zenith angle, water vapor content, temperature, solar azimuth angle, and the zenith and azimuth angle of artificial satellites, etc. The observation data in the second example embodiment includes at least the metadata required to perform the processing described later.

[0064] More specifically, the image processing device 520 includes a region extraction unit 521, a component calculation unit 522, a spectral estimation unit 523, an optimization unit 524, and a shadow correction unit 525. The region extraction unit 521, component calculation unit 522, spectral estimation unit 523, and optimization unit 524 may correspond to examples of the calculation unit 110 described in the first exemplary embodiment. Furthermore, the shadow correction unit 525 corresponds to an example of the conversion unit 120 described in the second exemplary embodiment.

[0065] Region extraction unit 521 is configured to extract regions from the observed image that meet specific conditions. Region extraction unit 521 is configured to extract regions from the observed image where solar radiation is estimated to be less blocked. For example, region extraction unit 521 is configured to extract regions that have the sum of brightness values ​​of bands in the observed image that are equal to or greater than a predetermined threshold. Alternatively, region extraction unit 521 may extract regions based on the brightness values ​​of a predetermined number of bands, which is less than the number of bands in the observed image. The regions extracted by region extraction unit 521 are referred to hereinafter as "specific regions". Specific regions are at least a portion of the observed area.

[0066] The component calculation unit 522 is configured to calculate the components of the solar radiation spectrum. The component calculation unit 522 is configured to calculate the components of the solar radiation spectrum using metadata included in the observation data. The component calculation unit 522 can calculate the direct and diffuse components of the solar radiation spectrum using the solar zenith angle (or metadata capable of calculating the solar zenith angle) and based on the Bird model.

[0067] To calculate the direct and diffuse components of solar radiation using the Bird model, atmospheric conditions are required in addition to the solar zenith angle. These atmospheric conditions are determined by parameters indicating atmospheric conditions (perceptible water vapor (PWV), atmospheric pressure, atmospheric turbidity, etc.). However, atmospheric conditions have a smaller impact on the solar radiation spectrum compared to the solar zenith angle. Therefore, atmospheric conditions can be considered constants in the Bird model.

[0068] In other words, the influence of the solar zenith angle is dominant when calculating the direct and diffuse components. Therefore, the component calculation unit 522 can calculate the direct and diffuse components by acquiring the solar zenith angle in the observation area from the metadata in the observation data.

[0069] When it is assumed that the obstruction is not caused by any covering, the direct and diffuse components calculated by component calculation unit 522 indicate the solar radiation spectrum. Coverings mentioned herein include, for example, clouds in the atmosphere and shadow-forming obstacles. The solar radiation spectrum can also be viewed as the solar radiation spectrum under clear weather conditions or the solar radiation spectrum in a sunlit area (i.e., an area without shadows). The solar radiation spectrum indicated by the direct and diffuse components is a so-called ideal spectrum and therefore does not necessarily match the actual solar radiation spectrum at the Earth's surface.

[0070] The spectral estimation unit 523 is configured to estimate the spectrum of pure matter. The spectral estimation unit 523 is configured to estimate the spectrum of pure matter by using an observation image of a specific region extracted from the observation area by the region extraction unit 521 and components of the solar radiation spectrum calculated by the component calculation unit 522. When estimating the spectrum of pure matter, the spectral estimation unit 523 assumes that the total number of pure substances in the observation area is sufficiently small. Specifically, the spectral estimation unit 523 assumes that the total number of pure substances in the observation area is equal to or less than the number of bands.

[0071] In a second example embodiment, the spectral estimation unit 523 generates a reflectance image by using an observed image of a specific region and solar radiation spectral components, and estimates the pure matter spectrum by using the generated reflectance image.

[0072] The spectral estimation unit 523 assumes that the solar radiation spectrum is the solar radiation spectrum under clear weather or the solar radiation spectrum in a sunny area (i.e., an area without shadows). In this case, in equation (1), b1 = b2 = 1 is satisfied. The spectral estimation unit 523 calculates equation (2) as described below from equation (1) under the condition that b1 = b2 = 1 is satisfied. Then, the spectral estimation unit 523 generates a reflectance image according to equation (2) as described below.

[0073]

[0074] Note that the following is defined in equation (2).

[0075] L n Wavelength (n = 1 to N),

[0076] Y'(L n ): Reflectance image at wavelength L n The spectrum in [dimensionless]

[0077] y(L n ): The observed image at wavelength L n The spectrum in

[0078] i1(L n ): Solar radiation spectrum at wavelength L n The direct component in, and

[0079] i2(L n ): Solar radiation spectrum at wavelength L n The diffuse component in.

[0080] When generating a reflectance image, the spectral estimation unit 523 estimates the spectrum of a pure substance based on the reflectance image using a demixing method. The spectral estimation unit 523 can use known demixing methods such as Vertex Composition Analysis (VCA), N-FINDER, Pixel Purity Index (PPI), and Minimum Volume Transform (MVT).

[0081] The optimization unit 524 is configured to calculate the solar radiation spectrum for each pixel. The optimization unit 524 is configured to calculate the solar radiation spectrum for each pixel by using the observed image, the components of the solar radiation spectrum calculated by the component calculation unit 522, and the pure matter spectrum estimated by the spectral estimation unit 523.

[0082] Similar to the calculation unit 110 in the first example embodiment, the optimization unit 524 can calculate the solar radiation spectrum for each pixel by solving the optimization problem expressed in equation (1). However, in the second example embodiment, the value of P is "2". The optimization unit 524 can specify the effect of shading for each pixel (i.e., the amount reduced from the ideal solar radiation spectrum under clear weather conditions, etc.) by calculating weight coefficients b1 and b2.

[0083] The shadow correction unit 525 is configured to correct the observed image. The shadow correction unit 525 is configured to correct the observed image for each pixel using the solar radiation spectrum calculated for each pixel by the optimization unit 524. The shadow correction unit 525 can correct a portion of the observed image instead of the entire observed image. For example, the shadow correction unit 525 can omit correction for specific regions extracted by the region extraction unit 521 because the effect of shadows is considered small.

[0084] For example, the shadow correction unit 525 suppresses the influence of shadows contained in the spectrum of the observed image according to equation (3) described below.

[0085]

[0086] Note that the following is defined in equation (3).

[0087] L n Wavelength (n = 1 to N),

[0088] x'(L n ): The corrected observation image at wavelength L n The spectrum in

[0089] x(L n ): The observation image before correction at wavelength L n The spectrum in

[0090] i1(L n ): Solar radiation spectrum at wavelength L n The direct component in

[0091] i2(L n ): Solar radiation spectrum at wavelength L n The diffuse component in

[0092] b1: Weighting coefficient of the direct component (equal to or greater than 0 and equal to or less than 1), and

[0093] b2: Weighting coefficient of the diffuse component (equal to or greater than 0 and equal to or less than 1).

[0094] Alternatively, the shadow correction unit 525 can correct the observed image by subtracting the components of the solar radiation spectrum (i.e., i1(L)) from equation (3). n )+i2(L n The value obtained is obtained by )). In other words, the shadow correction unit 525 can correct the observed image according to the equation (4) described below. In equation (4), x'(L n The spectrum indicating the reflectance of the observed area depends only on the characteristics of the observed area.

[0095]

[0096] (Here, x″ is the value of the dimensionless spectrum)

[0097] Figure 6 This is a flowchart illustrating the processing performed by the image processing device 520. The image processing device 520 performs processing for each observation region included in the observed image. Figure 6 The processing is shown in the figure. Note that the order in which the image processing device 520 performs the processing is not limited to... Figure 6 The image processing device 520 can switch the execution order of the steps without causing any technical inconsistencies. Furthermore, the image processing device 520 can execute multiple steps in parallel.

[0098] In step S610, the region extraction unit 521 extracts a specific region from the observation region. In step S620, the component calculation unit 522 calculates the direct and diffuse components of the solar radiation spectrum with reference to the zenith angle of the sun in the observation region. Note that, strictly speaking, the zenith angle of the sun differs between pixels forming the observation region. However, in the second exemplary embodiment, the zenith angle of the sun corresponding to the respective pixels is considered to be approximately equal to each other, and therefore a common value is used.

[0099] In step S630, the spectral estimation unit 523 calculates the pure matter spectrum based on the specific region extracted in step S610 and the components of the solar radiation spectrum extracted in step S620. In step S640, the optimization unit 524 calculates the solar radiation spectrum for each pixel. The optimization unit 524 calculates the solar radiation spectrum for each pixel by applying an optimization problem based on image information about the observation area, the components of the solar radiation spectrum calculated in step S620, and the pure matter spectrum calculated in step S630.

[0100] In step S650, the shadow correction unit 525 corrects the observed image. The shadow correction unit 525 corrects the observed image for each pixel based on the solar radiation spectrum calculated in step S640. The shadow correction unit 525 outputs image information indicating the corrected observed image.

[0101] The image processing system 500 in the second example embodiment has a configuration for extracting a specific region and estimating the pure matter spectrum based on the observed image of the extracted specific region. Using this configuration, the pure matter spectrum can be estimated with high accuracy. Furthermore, the image processing system 500 can calculate the components of the solar radiation spectrum based on the solar zenith angle. The solar zenith angle is included in the overall metadata of the observation data, or can be specified based on the overall metadata.

[0102] Third Example Implementation

[0103] Figure 7 This is a block diagram illustrating the configuration of an image processing apparatus 700 according to yet another exemplary embodiment. The image processing apparatus 700 has a configuration common to the image processing apparatus 520 in the second exemplary embodiment. Specifically, the image processing apparatus 700 and image processing apparatus 520 share the same component calculation unit 522, optimization unit 524, and shadow correction unit 525. Furthermore, the image processing apparatus 700 differs from image processing apparatus 520 in that it includes a spectral storage unit 710 instead of a region extraction unit 521 and a spectral estimation unit 523. The component calculation unit 522, optimization unit 524, and shadow correction unit 525 operate similarly to those described in the second exemplary embodiment.

[0104] The spectral storage unit 710 is configured to store the spectra of pure substances. The spectral storage unit 710 is configured to pre-store the spectra of pure substances estimated to be included in the observation area, such that the pure substance spectra can be supplied to the optimization unit 524. The spectral storage unit 710 can store a set of pure substance spectra corresponding to the classification of the observation area.

[0105] The classification of observation areas mentioned in this article refers to the attributes of the observation areas, characterized by the materials that constitute the earth's surface (hereinafter also referred to as "ground cover objects"), such as: "urban areas," which include a relatively large number of man-made objects such as buildings; "vegetation," which includes a relatively large number of plants; and "soil," in which soil is exposed. When the attributes of the observation areas differ from one another, the combinations of pure materials included in the areas differ. Note that ground cover objects can be classified in more fine terms, for example, classifying "vegetation" into "coniferous forests" and "broadleaf forests," etc.

[0106] When a set of pure substance spectra corresponding to the classification of the observation area is stored, the spectral storage unit 710 can specify the classification of the observation area based on metadata, and can supply the optimization unit 524 with a set of pure substance spectra corresponding to the specified classification. In this case, the spectral storage unit 710 can specify the classification of the observation area based on metadata indicating the location (latitude and longitude) of the observation area. Alternatively, the metadata may include metadata corresponding to the classification of the observation area.

[0107] Figure 8 This is a flowchart illustrating the process performed by the image processing device 700. In step S810, the component calculation unit 522 calculates the direct and diffuse components of the solar radiation spectrum. Step S810 is similar to the process in step S620 of the second example embodiment.

[0108] In step S820, optimization unit 524 calculates the solar radiation spectrum for each pixel. In step S820, optimization unit 524 reads a set of pure matter spectra stored in spectral storage unit 710 and calculates the solar radiation spectrum based on the read pure matter spectra. Step S820 is similar to the processing in step S640 in the second example embodiment, except that the pure matter spectrum replaced in equation (1) is the pure matter spectrum in spectral storage unit 710.

[0109] In step S830, the shadow correction unit 525 corrects the observed image. Step S830 is similar to the processing in step S650 of the second example embodiment. The shadow correction unit 525 outputs image information indicating the corrected observed image. In addition to the image information, the shadow correction unit 525 may also output metadata.

[0110] The image processing device 700 according to the third example embodiment has a configuration that calculates the solar radiation spectrum for each pixel by using pre-stored pure matter spectra. This configuration is similar to the configuration in the second example embodiment (see...). Figure 5 In contrast, no configuration and processing are required for calculating (estimating) the spectrum of a pure substance. The image processing device 700 does not require calculating the spectrum of a pure substance, and therefore, no specific region is needed for calculating the spectrum of a pure substance.

[0111] Modify Example

[0112] This disclosure is not limited to the first to third example embodiments described above. This disclosure may include patterns applied using modifications or applications that are readily available to those skilled in the art. For example, this disclosure includes example modifications described below. Furthermore, this disclosure may include patterns obtained by appropriately combining the content described in the specification as needed. For example, content described in a particular example embodiment may be applied to another example embodiment without causing any contradiction.

[0113] (Modified Example 1)

[0114] The image processing apparatus according to this disclosure may include the configuration of the second example embodiment (see Figure 5 ) and the configuration in the third example embodiment (see Figure 7 Both. In other words, the image processing device is configured to read a portion of the spectrum of the M pure substances used in equation (1) from the spectral storage unit 710, and to estimate the remaining spectrum using the spectral estimation unit 523.

[0115] For example, the image processing apparatus according to Modified Example 1 can read out the spectra of general or representative pure substances that may be included in any observation area from the spectral storage unit 710, and can estimate the spectra of the remaining pure substances using the spectral estimation unit 523. Alternatively, the image processing apparatus according to Modified Example 1 can perform selection based on a portion of the spectra of M pure substances (e.g., the pure substance spectra unique to the classified observation area) read from the spectral storage unit 710, depending on the classification of the ground cover object to the observation area. The classification of the observation area is specified from the location of the area.

[0116] (Modified Example 2)

[0117] The spectral estimation unit 523 can estimate the spectrum of a pure substance based on information indicating the shape of the spectrum of a pure substance. Such information is referred to as "hint information" in the following description.

[0118] The prompt information is, for example, image feature values ​​specified based on the observed image. Image feature values ​​can be features indicating the color of the observed area or features indicating edges or texture. Using such image feature values, ground cover objects can be estimated in certain situations.

[0119] (Modified Example 3)

[0120] The specific hardware configuration of the image processing apparatus according to this disclosure includes various variations and is not limited to a specific configuration. For example, the image processing apparatus according to this disclosure can be implemented using software, or it can be configured such that various types of processing are shared by using multiple types of hardware.

[0121] Figure 9 This is a block diagram illustrating an example of the hardware configuration of a computer device 900 for implementing the image processing apparatus according to the present disclosure. The computer device 900 includes a central processing unit (CPU) 901, a read-only memory (ROM) 902, a random access memory (RAM) 903, a storage device 904, a drive device 905, a communication interface 906, and an input / output interface 907. The image processing apparatus according to the present disclosure can... Figure 9 This is achieved using the configuration (or a portion thereof) shown in the diagram.

[0122] CPU 901 executes program 908 using RAM 903. Program 908 can be stored in ROM 902. Alternatively, program 908 can be recorded on a recording medium 909, such as a memory card, for reading by drive device 905, or transferred from an external device via network 910. Communication interface 906 exchanges data with external devices via network 910. Input / output interface 907 exchanges data with peripheral devices (e.g., input devices and display devices). Communication interface 906 and input / output interface 907 can be used as components for acquiring or outputting data.

[0123] Note that components of the image processing apparatus according to this disclosure may include a single circuit (processor, etc.) or a combination of multiple circuits. The circuits mentioned herein can be either dedicated circuits or general-purpose circuits. For example, a portion of the image processing apparatus according to this disclosure may be implemented using a dedicated processor, and another portion may be implemented using a general-purpose processor.

[0124] The configuration described as a single device in the example embodiments mentioned above can be provided separately as multiple devices. For example, image processing devices 100, 520, or 700 can be implemented by multiple computer devices using cloud computing technology.

[0125] As described above, the present invention has been illustrated using the example embodiments mentioned above as model examples. However, the present invention is not limited to the example embodiments mentioned above. That is, various modes that can be understood by those skilled in the art are applicable within the scope of the present invention.

[0126] The exemplary embodiments of the present invention mentioned above may be described in the following supplementary description, but are not limited to the following supplementary description.

[0127] (Supplementary Note 1)

[0128] An image processing device, comprising:

[0129] A computing device for calculating the solar radiation spectrum of a region based on an observed image of a region on the Earth's surface, components of the solar radiation spectrum in the region, and an estimated spectrum of pure matter in the region; and

[0130] A conversion device for converting observed images based on the calculated solar radiation spectrum.

[0131] (Supplementary Note 2)

[0132] According to Supplementary Note 1, the image processing device also includes:

[0133] Extraction device for extracting specific regions from observed images where solar radiation is less blocked; and

[0134] An estimation device for estimating the spectrum of a pure substance using an extracted image of a specific region, wherein,

[0135] The computing device calculates the solar radiation spectrum based on observed images, components of the solar radiation spectrum, and estimated spectra of pure matter.

[0136] (Supplementary Note 3)

[0137] According to supplementary explanation 1 or 2, the image processing device, wherein,

[0138] The computing device calculates the spectrum of a pure substance by solving an optimization problem that minimizes the error in a predetermined model.

[0139] (Supplementary Note 4)

[0140] The image processing device according to any one of Supplementary Explanations 1 to 3, wherein,

[0141] The computing device calculates the components of the solar radiation spectrum based on the zenith angle of the sun in the region.

[0142] (Supplementary Note 5)

[0143] The image processing device according to any one of Supplementary Explanations 1 to 4, wherein,

[0144] The computing device selects at least a portion of the spectrum of a pure substance based on the location of the region.

[0145] (Supplementary Note 6)

[0146] The image processing device according to any one of Supplementary Explanations 1 to 5, wherein,

[0147] The computing device selects at least a portion of the spectrum of a pure substance based on the classification of the region according to the ground cover objects.

[0148] (Supplementary Note 7)

[0149] The image processing device according to any one of Supplementary Explanations 1 to 6, wherein,

[0150] The computing device reads at least a portion of the spectrum of a pure substance from a storage device.

[0151] (Supplementary Note 8)

[0152] The image processing device according to any one of Supplementary Explanations 1 to 7, wherein,

[0153] The observed image consists of multiple pixels, and

[0154] The computing device calculates the solar radiation spectrum for each of the multiple pixels by using the common value between the components of the solar radiation spectrum and the spectrum of the pure substance for the multiple pixels.

[0155] (Supplementary Note 9)

[0156] An image processing method, comprising:

[0157] The solar radiation spectrum of a region is calculated based on an observed image of a region on the Earth's surface, the components of the solar radiation spectrum in that region, and the estimated spectrum of pure matter in that region; and

[0158] The observed images are converted based on the calculated solar radiation spectrum.

[0159] (Supplementary Note 10)

[0160] According to Supplementary Note 9, the image processing methods also include:

[0161] Extract specific regions from observed images where solar radiation is less blocked;

[0162] The spectrum of a pure substance is estimated by using an observed image of a specific region extracted from the sample; and

[0163] The solar radiation spectrum is calculated based on observed images, components of the solar radiation spectrum, and estimated spectra of pure matter.

[0164] (Supplementary Note 11)

[0165] According to supplementary notes 9 or 10, the image processing methods also include:

[0166] The spectra of pure substances are calculated by solving an optimization problem that minimizes the errors in a predetermined model.

[0167] (Supplementary Note 12)

[0168] The image processing method according to any one of Supplementary Notes 9 to 11 also includes:

[0169] The components of the solar radiation spectrum are calculated based on the zenith angle of the sun in the region.

[0170] (Supplementary Note 13)

[0171] The image processing method according to any one of Supplementary Notes 9 to 12 also includes:

[0172] The selection of at least a portion of the spectrum of a pure substance depends on the location of the region.

[0173] (Supplementary Note 14)

[0174] The image processing method according to any one of Supplementary Notes 9 to 13 also includes:

[0175] The selection of at least a portion of the spectrum of a pure substance is based on the classification of the region by ground cover objects.

[0176] (Supplementary Note 15)

[0177] The image processing method according to any one of Supplementary Notes 9 to 14 also includes:

[0178] At least a portion of the spectrum of a pure substance is read from the storage device.

[0179] (Supplementary Note 16)

[0180] According to any one of the supplementary explanations 9 to 15, the image processing method, wherein,

[0181] The region comprises multiple pixels, and

[0182] The image processing method also includes calculating the solar radiation spectrum for each of the multiple pixels by using the common value between the components of the solar radiation spectrum and the spectrum of the pure substance for the multiple pixels.

[0183] (Supplementary Note 17)

[0184] A recording medium for recording programs that a computer executes:

[0185] The process of calculating the solar radiation spectrum of a region based on an observed image of a region on the Earth's surface, the components of the solar radiation spectrum in that region, and the estimated spectrum of pure matter in that region; and

[0186] The processing of observed images is based on the calculated solar radiation spectrum.

[0187] (Supplementary Note 18)

[0188] According to Supplementary Note 17, the recording medium also enables the computer to perform:

[0189] Processing of extracting specific regions from observed images where solar radiation is less blocked; and

[0190] The process of estimating the spectrum of a pure substance by using an extracted image of a specific region, wherein,

[0191] In the computational process, the computer calculates the solar radiation spectrum based on the observed images, the components of the solar radiation spectrum, and the estimated spectrum of pure matter.

[0192] (Supplementary Note 19)

[0193] According to supplementary notes 17 or 18, the recording medium also enables the computer to:

[0194] In the computational process, the spectrum of a pure substance is calculated by solving an optimization problem that minimizes the error in a predetermined model.

[0195] (Supplementary Note 20)

[0196] According to any of the supplementary notes 17 to 19, the recording medium also enables the computer to:

[0197] In the calculation process, the components of the solar radiation spectrum are calculated based on the zenith angle of the sun in the region.

[0198] (Supplementary Note 21)

[0199] According to any of the supplementary notes 17 to 20, the recording medium also enables the computer to:

[0200] In the computational process, at least a portion of the spectrum of the pure substance is selected based on the location of the region.

[0201] (Supplementary Note 22)

[0202] According to any of the supplementary notes 17 to 21, the recording medium also enables the computer to:

[0203] In the computational process, at least a portion of the spectrum of the pure substance is selected based on the classification of the region according to the ground cover objects.

[0204] (Supplementary Note 23)

[0205] According to any of the supplementary notes 17 to 22, the recording medium also enables the computer to:

[0206] In the computational process, at least a portion of the spectrum of the pure substance is read from the storage device.

[0207] (Supplementary Note 24)

[0208] According to any of the supplementary notes 17 to 23, the recording medium, wherein,

[0209] The region comprises multiple pixels, and

[0210] The recording medium also enables the computer to calculate the solar radiation spectrum for each of the multiple pixels by using the common value between the components of the solar radiation spectrum for multiple pixels and the spectrum of the pure substance.

[0211] This application is based on and claims priority to Japanese Patent Application No. 2016-144486, filed on July 22, 2016, the disclosure of which is incorporated herein by reference in its entirety.

[0212] Figure Labels

[0213] 100, 520, 700 image processing equipment

[0214] 110 Computing Units

[0215] 120 conversion unit

[0216] 500 Image Processing System

[0217] 510 Image Supply Equipment

[0218] 530 Output Device

[0219] 521 Region Extraction Unit

[0220] 522 Component Calculation Unit

[0221] 523 Spectral Estimation Unit

[0222] 524 Optimization Units

[0223] 525 Shadow Correction Unit

[0224] 710 Spectral Storage Unit

[0225] 900 Computer equipment.

Claims

1. An image processing apparatus, comprising: An extraction device for extracting specific regions from observed images where solar radiation is less blocked; An estimation device for estimating the spectrum of a pure substance by using the observed image of the extracted specific region; A computing device is configured to calculate a solar radiation spectrum for each pixel of a plurality of pixels included in a given region, based on an observed image of a given region on the Earth's surface, components of the solar radiation spectrum in the region, and an estimated spectrum of the pure substance. The computing device calculates the components of the solar radiation spectrum based on the zenith angle of the sun in the region, and calculates the solar radiation spectrum for each pixel by solving an optimization problem to minimize errors in a predetermined model, using common values ​​between the spectrum of the pure substance and the components of the solar radiation spectrum for the plurality of pixels. Let r represent the spectrum x of each pixel, where r m The spectrum representing the pure substance, i p The component representing the solar radiation spectrum, and Represents the Hadama product. a m The weighting coefficients representing the spectrum of the pure substance. b p The weighting coefficients representing the components of the solar radiation spectrum. M This represents the estimated number of the pure substance. P The number of components in the solar radiation spectrum is represented by , and e represents the model error; as well as A conversion device for converting the observed image based on the calculated solar radiation spectrum, wherein... The solar radiation spectrum is the spectrum of sunlight used to illuminate the Earth's surface, and The pure substance is a constituent element of the Earth's surface.

2. The image processing apparatus according to claim 1, wherein, The computing device selects at least a portion of the spectrum of the pure substance based on the location of the region.

3. The image processing apparatus according to claim 1, wherein, The computing device selects at least a portion of the spectrum of the pure substance based on the classification of the region according to the ground cover objects.

4. The image processing apparatus according to claim 1, wherein, The computing device reads at least a portion of the spectrum of the pure substance from the storage device.

5. An image processing method, comprising: Extract specific regions from observed images where solar radiation is less blocked; The spectrum of a pure substance is estimated by using the observed image of the extracted specific region; Based on an observed image of a given region on the Earth's surface, the components of the solar radiation spectrum in that region, and the estimated spectrum of the pure substance, a solar radiation spectrum is calculated for each of a plurality of pixels included in the region, wherein the components of the solar radiation spectrum are calculated based on the zenith angle of the sun in the region, and the solar radiation spectrum for each of the plurality of pixels is calculated by solving an optimization problem to minimize errors in a predetermined model using common values ​​between the spectrum of the pure substance and the components of the solar radiation spectrum for the plurality of pixels. Let r represent the spectrum x of each pixel, where r m The spectrum representing the pure substance, i p The component representing the solar radiation spectrum, and Represents the Hadama product. a m The weighting coefficients representing the spectrum of the pure substance. b p The weighting coefficients representing the components of the solar radiation spectrum. M This represents the estimated number of the pure substance. P The number of components of the solar radiation spectrum is represented by , and e represents the model error; and The observed image is converted based on the calculated solar radiation spectrum, wherein The solar radiation spectrum is the spectrum of sunlight used to illuminate the Earth's surface, and The pure substance is a constituent element of the Earth's surface.

6. The image processing method according to claim 5, further comprising: At least a portion of the spectrum of the pure substance is selected based on the location of the region.

7. The image processing method according to claim 5, further comprising: At least a portion of the spectrum of the pure substance is selected based on the classification of the region according to the ground cover objects.

8. The image processing method according to claim 5, further comprising: At least a portion of the spectrum of the pure substance is read from the storage device.

Images