Image processing device, estimation system, image processing method

Abstract

Regardless of whether the overlapping area captured in the first image (taken from the first field) and the second image (taken from the second field) is actually a field, corrections are made to ensure that crop information in the field can be accurately estimated. [Solution] The system includes: a first image taken of a field under first shooting conditions, a second image taken of a field under second shooting conditions, a location information of the land captured in the first image, and a location information of the land captured in the second image, which are used to acquire overlapping areas that are captured in the first and second images; and a conversion model generation unit that generates a conversion model to convert the first vegetation index to the second vegetation index based on learning data that associates the first vegetation index in the overlapping area of ​​the first image with the second vegetation index in the overlapping area of ​​the second image.

Description

Technical Field

[0001] The present invention relates to an image processing apparatus, an estimation system, and an image processing method.

Background Art

[0002] There is a technique for estimating crop information indicating the growth degree of plants grown in a field by using an image obtained by photographing the field from the sky. For example, Patent Document 1 discloses that in remote sensing for photographing a plurality of fields while looking down from the ground, crop information (for example, nitrogen content of rice plants) obtained in a reference field (first field) is used, and a technique of correcting crop information obtained from image data obtained by photographing another field (second field) different from the reference field is disclosed.

Prior Art Literature

Patent Literature

[0003]

Patent Document 1

Summary of Invention

Problem to be Solved by the Invention

[0004] However, in the above-mentioned Patent Document 1, the other field (second field) must be photographed such that a part of the reference field (first field) is included in the photographing range, which has a problem of large constraints. For example, even if the second field is located near the first field, if the two fields are not adjacent to each other, it is difficult to capture an image that includes a part of the first field and the second field in a single image. In addition, when using a satellite image obtained by photographing a field from the sky with a camera mounted on an artificial satellite, it is difficult to control the photographing area by the satellite image, so it is difficult to intentionally capture an image including a part of the first field and the second field. For this reason, regardless of whether the overlapping area photographed in each of the first image in which the first field is captured and the second image in which the second field is captured is a field, it is preferable to be able to perform correction so that crop information in the field can be estimated with high accuracy.

[0005] The present invention has been made in view of these circumstances, and its objective is to provide an image processing device, an estimation system, and an image processing method that can correct overlapping areas captured in a first image of a first field and a second image of a second field, so that crop information in a field can be accurately estimated, regardless of whether or not those overlapping areas are fields. [Means for solving the problem]

[0006] To solve the above-mentioned problems, one aspect of the present invention is an image processing apparatus comprising: a first image taken of a field under first shooting conditions; a second image taken of a field under second shooting conditions; a duplicate region acquisition unit that acquires a duplicate region that is captured in the first and second images using the location information of the land captured in the first image and the location information of the land captured in the second image; and a conversion model generation unit that generates a conversion model that converts the first vegetation index to the second vegetation index based on learning data that associates the first vegetation index in the duplicate region of the first image with the second vegetation index in the duplicate region of the second image.

[0007] Furthermore, one aspect of the present invention includes: an overlapping region acquisition unit that acquires overlapping regions that are captured in the first and second images using a first image taken of a field under first shooting conditions, a second image taken of a field under second shooting conditions, the location information of the land captured in the first image, and the location information of the land captured in the second image; a conversion region acquisition unit that acquires a conversion region different from the overlapping region from the first image; and a vegetation index converter that converts the vegetation index in the conversion region of the first image using a conversion model. The image processing device comprises a conversion unit and a composite image generation unit that generates a composite image by combining a first vegetation index image obtained by converting the vegetation index in the conversion region of the first captured image using a conversion model and a second vegetation index image showing the vegetation index in the region captured in the second captured image, wherein the conversion model is a model that converts the first vegetation index to the second vegetation index, generated based on training data that associates the first vegetation index in the overlapping region of the first captured image with the second vegetation index in the overlapping region of the second captured image.

[0008] Furthermore, one aspect of the present invention is an estimation system comprising the image processing device described above, and a crop condition estimation device that estimates the crop condition of a field captured in the composite image based on the vegetation index in the composite image generated by the image processing device.

[0009] Furthermore, one aspect of the present invention is an image processing method performed by a computer used in an image processing device, wherein an overlapping region acquisition unit acquires overlapping regions that are captured in the first and second images using a first image taken of a field under first shooting conditions, a second image taken of a field under second shooting conditions, the location information of the land captured in the first image, and the location information of the land captured in the second image, and a conversion model generation unit generates a conversion model that converts the first vegetation index to the second vegetation index based on learning data that associates the first vegetation index in the overlapping region of the first image with the second vegetation index in the overlapping region of the second image.

[0010] Furthermore, one aspect of the present invention is an image processing method performed by a computer used in an image processing device, wherein an overlapping region acquisition unit acquires overlapping regions that are captured in the first and second images using a first image taken of a field under first shooting conditions, a second image taken of a field under second shooting conditions, the location information of the land captured in the first image, and the location information of the land captured in the second image; a conversion region acquisition unit acquires a conversion region different from the overlapping region from the first image; and a vegetation index conversion unit uses a conversion model to perform the first shooting The image processing method involves converting the vegetation index in the conversion region of an image, and generating a composite image by combining a first vegetation index image obtained by converting the vegetation index in the conversion region of the first captured image using a conversion model, and a second vegetation index image showing the vegetation index in the region captured in the second captured image, wherein the conversion model is a model that converts the first vegetation index to the second vegetation index, generated based on training data that associates the first vegetation index in the overlapping region of the first captured image with the second vegetation index in the overlapping region of the second captured image. [Effects of the Invention]

[0011] As explained above, according to this invention, regardless of whether the overlapping region captured in the first image, in which the first field was photographed, and the second image, in which the second field was photographed, is a field or not, corrections can be made so that crop information in the field can be estimated with high accuracy. [Brief explanation of the drawing]

[0012] [Figure 1] This is a schematic block diagram showing the configuration of the estimation system 1 according to the embodiment. [Figure 2] This is a diagram illustrating the conditions for capturing satellite images. [Figure 3] This is a diagram illustrating the processing performed by the image processing device 10. [Figure 4] This is a block diagram showing the configuration of the image processing device 10. [Figure 5] This is a diagram illustrating the processing performed by the image processing device 10. [Figure 6] This is a flowchart showing the processing flow performed by the image processing device 10. [Figure 7] This flowchart shows the process flow for generating a composite image performed by the image processing device 10. [Figure 8] This is a diagram illustrating a modified example of the embodiment. [Modes for carrying out the invention]

[0013] Hereinafter, an estimation system and an image processing device according to one embodiment of the present invention will be described with reference to the drawings.

[0014] Figure 1 is a block diagram showing the configuration of the estimation system 1 according to an embodiment. The estimation system 1 comprises an image processing device 10 and a crop condition estimation device 20.

[0015] Image processing device 10 is a computer that performs image processing. Image data of satellite images is input to the image processing device 10. Satellite images are images of fields taken from above using a camera mounted on an artificial satellite, and are an example of "captured images". The satellite image as used herein only needs to be an image obtained by capturing at least an area including farm fields from above the sky. For example, it may be an image captured by a person using a sensor (such as an imaging device or an infrared camera) of the state of the farm field and the surrounding land; or it may be an image captured by a sensor installed or mounted on a structure such as a steel tower in or near the farm field, a drone, an airplane, or a satellite. The farm field captured herein is the same farm field that is the target from which crop information is acquired via a vegetation index. However, the present embodiment performs image processing to absorb differences in imaging conditions between two images (particularly the influence of the atmosphere above the sky). For this reason, compared with using images of the farm field captured from above by a person or a drone as the images handled in the present embodiment, it is more desirable to target images captured from an altitude high enough to be affected by the atmosphere, for example, satellite images or aerial images captured from above by an airplane, Cessna, helicopter, or the like.

[0016] The image processing apparatus 10 generates a composite image based on the input satellite images. The composite image is an image obtained by splicing and combining lands captured in each of a plurality of satellite images, and is an image in which the vegetation index of the land captured in the satellite images is indicated for each pixel. The image processing apparatus 10 outputs the generated composite image to the crop state estimation apparatus 20.

[0017] The vegetation index as used herein is an index indicating the characteristics of light reflection by plants. For example, the vegetation index can be calculated by using RGB values (values of Red, Green, Blue) expressed as pixel values for each pixel of a satellite image and a near-infrared light value (Near InfraRed: NIR value) to calculate the intensity ratio of reflected light having several specific wavelengths. There are various types of vegetation indices such as NDVI, NDRE, NDWI, etc. Any of these vegetation indices (e.g., NDVI) can be used to evaluate whether the growth state of a plant is good or bad, and the vegetation index can be used to grasp the vegetation status.

[0018] The crop state estimation device 20 is a computer that estimates the growth state of an object (crop state) based on a vegetation index. Image data of a synthesized image generated by the image processing device 10 is input to the crop state estimation device 20. The image processing device 10 estimates the crop state using the input synthesized image.

[0019] For example, lodging can be used as the crop state. Lodging refers to a state where plant stems bend and fall over at the harvest time due to reasons such as excessive growth of plants. Estimating lodging makes it possible to implement countermeasures against problems such as deterioration of plant quality, reduction in yield, and failure of combine harvesters during harvesting. Furthermore, as the crop state, the amount of chlorophyll contained in plant leaves, the nitrogen content rate of plants, and the like can also be adopted.

[0020] The crop state estimation device 20 estimates the crop state using, for example, a trained model. The trained model is a model trained by learning training data indicating the correspondence between vegetation indices and crop states, so as to learn the correspondence between vegetation indices and crop states and be capable of estimating crop states based on vegetation indices.

[0021] The crop state estimation device 20 estimates a crop state based on a vegetation index, and outputs an image obtained by mapping the estimation result as an estimation image.

[0022] Figure 2 is a diagram for explaining imaging conditions of satellite images. Figure 2 schematically shows how two satellites, Satellite A and Satellite B, orbit the Earth while imaging the Earth (land) viewed from the satellites.

[0023] In this figure, Satellite A captures satellite image GA in imaging direction DA. Satellite B captures satellite image GB in imaging direction DB. There exists an overlapping region RO that is captured in both satellite image GA and satellite image GB.

[0024] Furthermore, this figure shows that satellite image GA was taken at 11:00 AM on June 21, 2024, and satellite image GB was taken at 11:05 AM on the same day of the same year, indicating that satellite images GA and GB were taken at approximately the same time.

[0025] Furthermore, satellite images are accompanied by attribute information, such as the identification information of the satellite that took the image, the date and time of the image, the area covered, the location information of the land in which the image was taken (latitude and longitude information), and the cloud cover rate at the time of the image.

[0026] Here, even if satellite images are taken of the same land at approximately the same time, differences in pixel values ​​often occur if different satellites take the images. Possible causes of these differences include variations in the angle of sunlight relative to the shooting direction, atmospheric conditions, satellite orbit, shooting angle, the equipment mounted on the satellite, and the aging of that equipment. If vegetation indexes are calculated from satellite images without considering these differences, the results will show that the vegetation conditions of the same land differ depending on the satellite image shooting conditions, even though the land appears to be the same at approximately the same time, thus reducing the accuracy of the vegetation index.

[0027] As a countermeasure, in this embodiment, image processing is performed to absorb differences caused by shooting conditions in satellite images of the same land taken at approximately the same time. By performing such image processing, the vegetation index calculated from satellite images of the same land taken at approximately the same time will show the same value regardless of the shooting conditions, thereby improving the accuracy of the vegetation index.

[0028] More specifically, the image processing device 10 generates a conversion model M and uses the generated conversion model M to convert the vegetation index calculated from satellite images. This corrects the vegetation index calculated from satellite images of the same area taken at approximately the same time so that they show approximately the same value regardless of the satellite image acquisition conditions, thereby suppressing a decrease in the accuracy of the vegetation index.

[0029] Figure 3 is a diagram illustrating the processing performed by the image processing device 10. Figure 3 schematically shows the conversion model M.

[0030] The left side of Figure 3 schematically shows two satellite images, GA and GB. The regions captured in satellite images GA and GB each include overlapping regions (overlapping regions RO). Furthermore, the imaging conditions for satellite images GA and GB are different, and the pixel values ​​corresponding to pixels that capture the same region in the overlapping region RO are different. This section illustrates and explains the case where the vegetation index calculated from satellite image GA is corrected based on the vegetation index calculated from satellite image GB. In this case, satellite image GA is the image (transformed image) on which the vegetation index is transformed using the transformation model M. Satellite image GB is the image (reference image) on which the vegetation index is referenced. In this case, satellite image GA (transformed image) is a satellite image in which the transformation region RT and the overlapping region RO are captured. The transformation region RT is a region in the transformed image that is different from the overlapping region RO, and is the region in which the vegetation index is transformed using the transformation model M. On the other hand, satellite image GB (reference image) is a satellite image in which the reference region RR and the overlapping region RO are captured. The reference region RR is a region in the reference image that is different from the overlapping region RO. The reference region RR is a region that is not subject to transformation using the transformation model M.

[0031] However, this is not the only limiting factor. In this embodiment, it is sufficient to absorb the differences caused by the shooting conditions in two satellite images having an overlapping region RO. For example, satellite image GA may be used as the reference image and satellite image GB as the converted image.

[0032] The image processing device 10 extracts multiple comparison regions RC (here, comparison regions RC1 to RC5) from the overlapping region RO in order to generate the transformation model M. The comparison regions RC are regions included in the overlapping region RO that are identical in the two satellite images (reference image and transformation image) and correspond to land areas with a certain area (for example, about 1 are).

[0033] The center of Figure 3 schematically shows the correspondence between the vegetation indices of the comparison region RC in two satellite images (reference image and transformed image). In the correspondence shown in the center of Figure 3, the horizontal axis (x-axis) represents the vegetation indices of the comparison region RC extracted from satellite image GA, and the vertical axis (y-axis) represents the vegetation indices of the comparison region RC extracted from satellite image GB. In the correspondence shown in the center of Figure 3, the plotted point P1 is a point whose x-coordinate is the vegetation index of the comparison region RC1 in the transformed image (satellite image GA), and whose y-coordinate is the vegetation index of the comparison region RC1 in the reference image (satellite image GB). Similarly to point P1, the coordinate values ​​of the points are combinations of the vegetation indices of the comparison regions RC2 to RC5 in the transformed image (satellite image GA) and the vegetation indices of the comparison regions RC2 to RC5 in the reference image (satellite image GB). In the correspondence shown in the center of Figure 3, the transformation model M is a function F that is fitted to the corresponding points P1 to P5 using a linear function. When the function F corresponding to the transformation model M passes through the point (IA, IB), the function y=F(x) can be expressed as IB=F(IA). Furthermore, since function F is a linear function, it can also be expressed as y=F(x) and y=ax+b, with coefficients a and b being constants. In this case, coefficient a represents the slope and coefficient b represents the intercept. As a method for calculating the linear function F as a fitting curve based on the corresponding points P1 to P5, any method such as the least squares method can be employed. In the following explanation, the transformation model M may sometimes be referred to as the transformation formula.

[0034] The composite image GS is schematically shown on the right side of Figure 3. The composite image GS is an image showing the vegetation index of the regions captured in satellite images GA and GB, respectively, and is an image in which the vegetation index of the transformed region RT of satellite image GA has been transformed (corrected) using the transformation model M.

[0035] Figure 4 is a block diagram showing the configuration of the image processing device 10. As shown in Figure 4, the image processing device 10 includes a satellite image storage unit 100, an overlapping area acquisition unit 101, a conversion area acquisition unit 102, a reference area acquisition unit 103, a latitude and longitude information storage unit 104, a conversion model generation unit 105, a conversion model information storage unit 106, a vegetation index calculation unit 107, a vegetation index information storage unit 108, a vegetation index conversion unit 109, a composite image generation unit 110, and a composite image storage unit 111.

[0036] The image processing device 10 has two functions: a generation function that generates a conversion model M, and a conversion function that converts the vegetation index using the conversion model M. The image processing device 10 includes, as functions for realizing the generation function, a satellite image storage unit 100, an overlapping area acquisition unit 101, a conversion area acquisition unit 102, a reference area acquisition unit 103, a latitude and longitude information storage unit 104, a conversion model generation unit 105, a vegetation index calculation unit 107, and a vegetation index information storage unit 108. Furthermore, the image processing device 10 includes, as functions for realizing the conversion function, a satellite image storage unit 100, an overlapping area acquisition unit 101, a conversion area acquisition unit 102, a reference area acquisition unit 103, a conversion model information storage unit 106, a vegetation index calculation unit 107, a vegetation index information storage unit 108, a vegetation index conversion unit 109, a composite image generation unit 110, and a composite image storage unit 111.

[0037] (Regarding the generation of the transformation model M) First, the image processing device 10 will be described in terms of its functions for realizing generation. The satellite image storage unit 100 stores satellite image data. In this embodiment, satellite images are used that capture all or part of the field so that crop information indicating the growth rate of plants grown in the field can be estimated from the satellite images. The satellite images may capture not only the field but also the surrounding land, such as plains, groves of trees, forests, hills, paths between fields, vacant lots, houses, sheds, and warehouses. Satellite images have pixel-specific values ​​associated with them, such as RGB values, NIR values, or combinations thereof. Satellite images also include attribute information, such as the identification of the satellite that took the image, the date and time of capture, the area captured, the location of the captured land (latitude and longitude), and the cloud cover rate at the time of capture.

[0038] The overlapping area acquisition unit 101 acquires the overlapping area RO in the satellite image. Based on the image information of the satellite image stored in the satellite image storage unit 100, the overlapping area acquisition unit 101 extracts two satellite images in which at least a portion of the shooting range overlaps. The overlapping area acquisition unit 101 defines the overlapping area RO as the overlapping area in the two extracted satellite images. The overlapping area acquisition unit 101 generates information indicating the overlapping area RO, for example, information indicating the latitude and longitude of the location corresponding to the overlapping area RO in the two satellite images, and stores the generated information in the latitude and longitude information storage unit 104.

[0039] The conversion area acquisition unit 102 acquires the conversion area RT in the satellite image. The conversion area acquisition unit 102 selects one of the two satellite images extracted by the overlapping area acquisition unit 101 (two satellite images whose shooting ranges overlap in at least part) as the image to be converted for vegetation index (conversion image). The conversion area acquisition unit 102 defines the area captured in the conversion image, excluding the overlapping area RO, as the conversion area RT. The conversion area acquisition unit 102 generates information indicating the conversion area RT, for example, information indicating the latitude and longitude of the position corresponding to the conversion area RT in the conversion image of the two satellite images, and stores the generated information in the latitude and longitude information storage unit 104.

[0040] The reference area acquisition unit 103 acquires the reference area RR in the satellite image. The reference area acquisition unit 103 selects the satellite image that is not the converted image from the two satellite images (two satellite images whose shooting ranges overlap in at least a portion) extracted by the overlapping area acquisition unit 101 as the image to refer to the vegetation index (reference image). The reference area acquisition unit 103 defines the area excluding the overlapping area RO from the area captured in the reference image as the reference area RR. The reference area acquisition unit 103 generates information indicating the reference area RR, for example, information indicating the latitude and longitude of the position corresponding to the reference area RR in the reference image of the two satellite images, and stores the generated information in the latitude and longitude information storage unit 104.

[0041] The latitude and longitude information storage unit 104 stores information as latitude and longitude information that associates one of the overlapping area RO, the converted area RT, or the reference area RR with each of the latitude and longitudes indicating the location of the area captured in the two satellite images (two satellite images whose shooting ranges overlap in at least a portion) extracted by the overlapping area acquisition unit 101.

[0042] The vegetation index calculation unit 107 calculates a vegetation index based on the pixel value of each pixel in the satellite image. There are various types of vegetation indices, such as NDVI (Normalized Vegetation Index), NDRE (Normalized Red Edge Index), and NDWI (Normalized Water Index), and they are calculated using the following formulas (1) to (4). In equations (1) to (4), NIR represents the amount of near-infrared light at the pixel value, and Red represents the amount of red light at the pixel value. RedEdge indicates the amount of light at the red edge of a pixel value. RedEdge refers to light with wavelengths shorter than near-infrared light, ranging from 680 nm to 750 nm. Terrestrial plants on Earth have a property where their reflectivity increases sharply in the 680 nm to 750 nm band. The reason plants appear green to the human eye is related to this tendency of light reflection in plants. SWIR represents the amount of short-wavelength infrared (SWIR) light in a pixel. SWIR is the wavelength band closest to visible light within the infrared spectrum, ranging from 1000 nm to less than 2500 nm. Light corresponding to SWIR is absorbed by water. Therefore, by calculating a vegetation index using SWIR, an index can be obtained indicating the presence or absence of water and whether or not water is attached to the surface of plants. It is known that water absorbs the wavelength corresponding to SWIR (light with a wavelength of 1450 nm) while transmitting visible light and infrared light (light with a wavelength of approximately 850 nm).

[0043] NDVI=(NIR-Red) / (NIR+Red) …(1) NDRE=(NIR-RedEdge) / (NIR+RedEdge) …(2) NDWI_1=(NIR-SWIR) / (NIR+SWIR) …(3) NDWI_2=(Red-SWIR) / (Red+SWIR) …(4)

[0044] NDVI is a normalized value based on the difference in reflectivity between near-infrared (NIR) light and red light (Red). Plants reflect near-infrared wavelengths but absorb red wavelengths because they are necessary for photosynthesis. Therefore, by obtaining NDVI based on the difference in reflectivity between near-infrared and red light obtained by photographing plants, a larger NDVI value indicates that the plant has a larger area of ​​leaves and stems, and contains more chlorophyll.

[0045] NDRE is a value obtained by emphasizing the relative intensity difference between red edge (RedEdge) wavelength light and near-infrared (NIR) light. Like NDVI, NDRE is a vegetation index that correlates with the amount of chlorophyll in leaves, but compared to NDVI, it is used as an indicator of health in the later stages of growth, or to detect plant growth problems earlier than NDVI.

[0046] There are two types of NDWI: NDWI_1, which defines the amount of water contained in vegetation, and NDWI_2, which defines the amount of water contained in the ground surface. Of these, NDWI_1, which defines the amount of water contained in vegetation, is a normalized value based on the difference in reflectance between near-infrared light (NIR) and short-wavelength infrared light (SWIR), and is an index related to the amount of water contained in plants. In addition, NDWI_1, which defines the amount of water contained in vegetation, may also use a normalized value based on the difference in reflectance between green light and short-wavelength infrared light.

[0047] NDWI_2, which defines the amount of water contained in the ground surface, is a normalized value based on the difference in reflectance between red light and short-wavelength infrared light (SWIR), and is an index related to the amount of water contained in the soil. In addition, NDWI_2, which defines the amount of water contained in the ground surface, may also be a normalized value based on the difference in reflectance between green light and near-infrared light.

[0048] Furthermore, there is the Improved Ratio Vegetation Index (IRVI), which is a vegetation index that improves upon the atmospheric influence of the NDVI. Vegetation indices based on visible light that does not include near-infrared light values, i.e., RGB values ​​of red light intensity, blue light intensity, and green light intensity, include the RGB Vegetation Index (RGBVI) and the Visible Atmospherically Resistant Index (VARI).

[0049] The vegetation index calculation unit 107 calculates the vegetation index by substituting the pixel values ​​of the satellite image into the vegetation index calculation formula. The vegetation index calculation unit 107 stores information that associates the pixel coordinates of the satellite image with the vegetation index calculated based on the pixel values ​​of those pixel coordinates as vegetation index information in the vegetation index information storage unit 108.

[0050] The vegetation index information storage unit 108 stores information as vegetation index information, which associates the vegetation index with the latitude and longitude that indicate the location of the area captured in each pixel of the satellite image.

[0051] The conversion model generation unit 105 generates a conversion model M. The conversion model generation unit 105 refers to the latitude and longitude information storage unit 104 and obtains multiple comparison regions RC randomly selected from the overlapping regions RO captured in the two satellite images (conversion image and reference image) extracted by the overlapping region acquisition unit 101. Furthermore, the conversion model generation unit 105 refers to the vegetation index information storage unit 108 and obtains the vegetation index for each of the two satellite images (converted image and reference image) in multiple comparison regions RC selected from the overlapping region RO. The conversion model generation unit 105 calculates a conversion model M based on the correspondence between the vegetation indices of two satellite images (conversion image and reference image) in multiple comparison regions RC. The conversion model generation unit 105 stores information representing the conversion model M, such as a mathematical formula representing the function F corresponding to the conversion model M, or information representing the slope a and intercept b of a function, in the conversion model information storage unit 106 as conversion model information.

[0052] The conversion model information storage unit 106 stores information representing the conversion model M, such as a mathematical formula representing the function F corresponding to the conversion model M, or information representing the slope a and intercept b of a linear function, as conversion model information.

[0053] (Regarding the conversion of vegetation indices using the conversion model M) Next, the functions of the image processing device 10 that realize the conversion function will be described. The functions of the satellite image storage unit 100, overlapping area acquisition unit 101, conversion area acquisition unit 102, reference area acquisition unit 103, conversion model information storage unit 106, vegetation index calculation unit 107, and vegetation index information storage unit 108 that realize the conversion function are the same as those described in the generation function, so their explanation will be omitted.

[0054] The vegetation index conversion unit 109 converts the vegetation index in the conversion area RT using the conversion model M. First, the vegetation index conversion unit 109 refers to the latitude and longitude information storage unit 104 and obtains the latitude and longitude information of the two satellite images to be combined (the converted image and the reference image). Furthermore, the vegetation index conversion unit 109 refers to the conversion model information storage unit 106 and obtains conversion model information for the two satellite images to be combined (conversion image and reference image). The vegetation index conversion unit 109 identifies a conversion region RT based on the latitude and longitude information acquired from the latitude and longitude information storage unit 104, and extracts the vegetation index calculated from the converted image (satellite image) in the identified conversion region RT from the vegetation index information storage unit 108. The vegetation index conversion unit 109 converts the vegetation index of the conversion region RT extracted from the vegetation index information storage unit 108 using the conversion model M. The vegetation index conversion unit 109 stores information in the vegetation index information storage unit 108 that associates the converted vegetation index with each pixel corresponding to the conversion region RT captured in the conversion image.

[0055] The composite image generation unit 110 generates a composite image. The composite image is an image obtained by combining two satellite images (a transformed image and a reference image), and each pixel is associated with the vegetation index of the land that was photographed. The composite image generation unit 110 refers to the vegetation index information storage unit 108 and obtains the vegetation index calculated from the pixel values ​​of the two satellite images to be combined (transformed image and reference image). The composite image generation unit 110 also obtains the vegetation index converted by the vegetation index conversion unit 109 using the conversion model M for the conversion region RT in the transformed image, one of the two satellite images to be combined, from the vegetation index information storage unit 108. The composite image generation unit 110 generates a composite image in which the vegetation index calculated from the pixel values ​​of the two satellite images to be combined (transformed image and reference image) is mapped with the vegetation index calculated from the reference image for the reference region RR and the overlapping region RO, and the converted vegetation index is mapped for the transformation region RT. The composite image generation unit 110 stores image information of the generated composite image, for example, information that associates the vegetation index of the land photographed with each pixel of the composite image, in the composite image storage unit 111.

[0056] The composite image storage unit 111 stores information indicating a composite image as composite image information.

[0057] Here, the features of this embodiment will be explained using Figure 5. Figure 5 is a diagram illustrating the processing performed by the image processing device 10. Figure 5 schematically shows the relationship between land captured in satellite images GA and GB and the surrounding fields RA and RB.

[0058] In conventional techniques, such as those shown in Patent Document 1, one image (the converted image in this embodiment) was corrected according to the other image (the reference image in this embodiment) so that the crop information of the same field, which was captured in both images, such as the lodging status of plants planted in the field, the amount of chlorophyll in the leaves, and the nitrogen content, would be the same. Therefore, there was a constraint that the same field had to be captured in both images. In contrast, in this embodiment, a transformation model M is generated based on the correspondence between the vegetation indices of the satellite image GA (transformed image) and satellite image GB (reference image) for multiple comparison regions RC in the overlapping region RO. In other words, in this embodiment, instead of using crop information that can be calculated in the field, such as the lodging status of plants, the amount of chlorophyll in the leaves, and the nitrogen content, vegetation indices are used in the generation of the transformation model M.

[0059] In other words, as shown in the example in Figure 5, suppose that a portion of field RA is captured in satellite image GA, a portion of field RB is captured in satellite image GB, and furthermore, the overlapping area RO of satellite images GA and GB contains land that is not a field, such as flat land, groves of trees, forests, hills, paths, vacant lots, houses, sheds, or warehouses. Even in such cases where the overlapping area RO is not a field, the reference area RR can be corrected so that the crop information of the two fields RA and RB is less affected by the differences in the shooting conditions of the two satellite images by using the vegetation index of the overlapping area RO (or multiple comparison areas RC selected from it). In this way, by using a vegetation index instead of crop information, it became possible to calculate crop information for fields captured in each of the two satellite images, even if the fields were not captured in the overlapping region (RO), while absorbing the differences in the shooting conditions of the two satellite images.

[0060] Here, the processing flow performed by the image processing device 10 will be explained using Figures 6 to 8. Figures 6 to 8 are flowcharts showing the processing flow performed by the image processing device 10.

[0061] As shown in Figure 6, the image processing device 10 first acquires image data from multiple satellite images (step S10). The image processing device 10 stores the acquired image data in the satellite image storage unit 100. Next, the vegetation index calculation unit 107 of the image processing device 10 calculates a vegetation index from each of the two satellite images to be combined (the transformed image and the reference image) (step S11). The vegetation index calculation unit 107 stores information in the vegetation index information storage unit 108 that associates the pixel coordinates of the satellite image with the vegetation index calculated based on the pixel value of the pixel corresponding to that pixel coordinate. The image processing device 10 obtains the overlapping region RO, the transformed region RT, and the reference region RR from each of the two satellite images to be combined (step S12).

[0062] As described above, in this embodiment, a linear transformation model M is calculated based on the correspondence between vegetation indices in comparison regions RC extracted from two satellite images (transformed image and reference image). Therefore, the number of comparison regions RC selected from the overlapping region RO should be at least two. Furthermore, considering that noise may be introduced into the image during acquisition, the accuracy of the transformation model M can be improved by increasing the number of comparison regions RC. On the other hand, it is expected that once the number of comparison regions RC reaches a certain level, a so-called saturation will be reached where the accuracy of the transformation model M does not improve further even if the number of comparison regions RC is increased. Therefore, in this embodiment, we search for the minimum number of comparison regions RC that can calculate an accurate transformation model M.

[0063] Specifically, the image processing device 10 obtains multiple comparison regions RC from the overlapping region RO. At this time, the image processing device 10 determines whether or not it is possible to select N or more comparison regions RC from the overlapping region RO (step S13). Here, N is any natural number greater than or equal to 2. For example, the image processing device 10 determines whether or not it can extract N or more comparison areas from the overlapping area RO based on the size (area) of the land captured as the overlapping area RO. For example, when acquiring an image area corresponding to land with a certain area, for example, 1[a(ar)], as a comparison area RC, if the size of the land corresponding to the overlapping area RO is less than N[a(ar)] + α (an area arbitrarily set as a margin), the image processing device 10 determines that it cannot select N or more comparison areas RC. On the other hand, if the size of the land corresponding to the overlapping area RO is N[a(ar)] + α or greater, the image processing device 10 determines that it can select N or more comparison areas RC.

[0064] The conversion model generation unit 105 of the image processing device 10 calculates a conversion model M in step S14 if it can select N or more comparison regions RC in step S13. The conversion model generation unit 105 obtains multiple comparison regions RC randomly selected from overlapping regions RO captured in two satellite images (conversion image and reference image) from the latitude and longitude information storage unit 104. The conversion model generation unit 105 obtains the vegetation index of each of the two satellite images (conversion image and reference image) in the comparison regions RC from the vegetation index information storage unit 108. The conversion model generation unit 105 calculates a conversion model M based on the correspondence between the vegetation indices of each of the two satellite images (conversion image and reference image) in the multiple comparison regions RC.

[0065] The image processing device 10 determines whether the accuracy of the conversion model M is above a threshold (step S15). Any method can be used to calculate the accuracy of the conversion model M, but for example, the determination can be made based on the difference with the conversion model M obtained previously (calculated based on a number of comparison regions RC that is less than N). Specifically, the image processing device 10 compares a conversion model M (previous model) calculated based on fewer than N comparison regions RC with a conversion model (current model) calculated based on N comparison regions RC. If the difference between the slope of the previous model and the slope of the current model is large (greater than or equal to a predetermined value), the image processing device 10 determines that the accuracy of the conversion model M is insufficient and that the accuracy of the conversion model M is below a threshold. Alternatively, if the difference between the intercept of the previous model and the intercept of the current model is large (greater than or equal to a predetermined value), the image processing device 10 may determine that the accuracy of the conversion model M is insufficient and that the accuracy of the conversion model M is below a threshold. On the other hand, if the difference between the slope of the previous model and the slope of the current model is small (less than a predetermined value), the image processing device 10 determines that the accuracy of the conversion model M is sufficient, and that it is unlikely that the accuracy will improve dramatically even if the number of comparison regions RC is increased further, and therefore the accuracy of the conversion model M is above a threshold value. Alternatively, if the difference between the slope of the previous model and the slope of the current model is small, AND the difference between the intercept of the previous model and the intercept of the current model is also small (greater than a predetermined value), the image processing device 10 may determine that the accuracy of the conversion model M is above a threshold value.

[0066] In step S15, if the accuracy of the conversion model M is above a threshold, the composite image generation unit 110 of the image processing device 10 generates a composite image (step S16). The conversion model generation unit 105 obtains multiple comparison regions RC randomly selected from the overlapping regions RO captured in the two satellite images (conversion image and reference image) from the latitude and longitude information storage unit 104. The conversion model generation unit 105 obtains the vegetation index of each of the two satellite images (conversion image and reference image) in the comparison regions RC from the vegetation index information storage unit 108. The conversion model generation unit 105 calculates the conversion model M based on the correspondence between the vegetation indices of each of the two satellite images (conversion image and reference image) in the multiple comparison regions RC.

[0067] On the other hand, if the accuracy of the conversion model M in step S15 is below a threshold, the image processing device 10 increases the number of comparison regions RC selected from the overlapping region RO (step S18), returns to the process shown in step S13, and performs the generation of a more accurate conversion model M. This makes it possible to calculate an accurate conversion model M with the minimum number of comparison regions RC. The image processing device 10 determines whether or not the synthesis process has been performed on all satellite images acquired in step S10 (step S17). The synthesis process here refers to the process of generating a composite image by performing the processes shown in steps S12 to S16 on all satellite images acquired in step S10. If the synthesis process has been performed on all satellite images, that is, if the synthesis process has already been performed, the image processing device 10 terminates the process. If the image processing device 10 has not performed the synthesis process on all satellite images, that is, if there are still satellite images that need to be synthesized, it returns to the process shown in step S12.

[0068] Furthermore, if it is not possible to select N or more comparison areas from the overlapping areas in step S13, a reference image is generated (step S19). The reference image is an image that has not undergone any conversion (correction) of the vegetation index, and can be used as a reference when understanding the condition of the field as seen from above. Since the reference image is used only as a reference, the vegetation index mapped to the reference image is not used in calculating crop information in the field. For example, the reference image is an image in which the pixel values ​​of an image created by combining two satellite images (a transformed image and a reference image) are associated with vegetation indices calculated from each image. In the reference image, the overlapping region RO is associated with the vegetation index calculated from the image of the two satellite images (transformed image and reference image) that has a lower cloud cover rate, that is, the image in which the land is captured more clearly.

[0069] Ideally, the two satellite images used as reference images (the converted image and the reference image) should have been taken at approximately the same time on the same day, but it is sufficient if at least two of the satellite images can be correlated. For example, the date on which the converted image was taken and the date on which the reference image was taken may be different. This is because even if the dates on which the converted image and the reference image were taken are different, the general trend of the land conditions captured in the satellite images is unlikely to change so drastically that it would cause a change in the vegetation index.

[0070] This is also true when generating a composite image by combining two satellite images (a transformed image and a reference image). In other words, as long as the two satellite images are correlated, the date on which the transformed image was taken and the date on which the reference image was taken do not need to be different. Whether two satellite images are correlated can be determined, for example, by the variation in the distance from points plotted according to the vegetation index of the comparison region RC (corresponding points P1-P5 in Figure 3) to a straight line corresponding to the transformation model M. This is because, when the same location under the same conditions is photographed from different satellites, differences in pixel values ​​due to the shooting conditions may occur, but the correspondence between these differences is expected to be a linear function. However, if changes in land conditions over time are added in addition to differences due to shooting conditions, it is unlikely that the differences in pixel values ​​of the two satellite images will be a linear function, and the correspondence may become complex, such as being expressed as a polynomial in some regions. Specifically, the image processing device 10 can determine that two satellite images are not correlated if the variability of distance (the distance from the points plotted according to the vegetation index of the comparison area RC (corresponding points P1 to P5 in Figure 3) to the straight line corresponding to the transformation model M) is large, for example, if the variance of distance is greater than or equal to a threshold. On the other hand, the image processing device 10 can determine that two satellite images are correlated if the variability of distance is small, for example, if the variance of distance is less than a threshold.

[0071] Figure 7 shows a flowchart illustrating the process for generating a reference image, corresponding to step S19 in Figure 6. The image processing device 10 obtains the cloud cover percentage for each pixel in the overlapping region RO for each of the two satellite images to be combined (modified image and reference image) (step S190). The image processing device 10 selects pixels with low cloud cover percentages from among the pixels corresponding to the same location in the overlapping region RO obtained from the two satellite images (modified image and reference image) (step S191). The image processing device 10 generates a reference image by mapping the vegetation index calculated from the transformed image in the transformed region RT, mapping the vegetation index calculated from the reference image in the reference region RR, and mapping the vegetation index calculated from the pixels with low cloud cover percentages among the two satellite images (modified image and reference image) in the overlapping region RO.

[0072] As described above, the image processing apparatus 10 of the embodiment includes an overlapping region acquisition unit 101 and a conversion model generation unit 105. The overlapping region acquisition unit 101 acquires an overlapping region RO (an overlapping region that is captured in the first and second images) using a converted image (a first image of the field captured under first shooting conditions), a reference image (a second image of the field captured under second shooting conditions), the location information of the land captured in the converted image (the first image), and the location information of the land captured in the reference image (the second image). The conversion model generation unit 105 generates a conversion model M that converts the vegetation index calculated from the converted image (the first vegetation index) to the vegetation index calculated from the reference image (the second vegetation index), based on learning data (for example, corresponding points P1 to P5) that associates the first vegetation index in the overlapping region RO of the converted image (the overlapping region of the first captured image) with the second vegetation index in the overlapping region RO of the reference image (the overlapping region of the second captured image). As a result, the image processing device 10 of this embodiment can generate a conversion model that converts the vegetation index calculated from one image (reference image) to the vegetation index calculated from the other image (converted image). The vegetation index can be calculated regardless of whether the land captured in the pixel is a field or not, and is an index that indicates the state of vegetation. Therefore, regardless of whether the overlapping region captured in the first image, in which the first field was captured, and the second image, in which the second field was captured, is a field or not, corrections can be made so that crop information in the field can be estimated with high accuracy.

[0073] Furthermore, the image processing apparatus 10 of the embodiment includes an overlapping region acquisition unit 101, a conversion region acquisition unit 102, a vegetation index conversion unit 109, and a composite image generation unit 110. The overlapping region acquisition unit 101 acquires an overlapping region RO (an overlapping region that is captured in both the first and second images) using a converted image (a first image of the field captured under first shooting conditions), a reference image (a second image of the field captured under second shooting conditions), the location information of the land captured in the converted image (the first image), and the location information of the land captured in the reference image (the second image). The conversion region acquisition unit 102 acquires a conversion region RT (a conversion region different from the overlapping region) from the converted image (the first image). The vegetation index conversion unit 109 converts the vegetation index in the conversion region RT (the conversion region of the first image) using a conversion model M. The composite image generation unit 110 generates a composite image. The composite image is an image obtained by combining a first vegetation index image and a second vegetation index image. The vegetation index image is an image in which the vegetation index calculated from the pixel value of the pixel coordinate is mapped to the pixel coordinate in the satellite image. The first vegetation index image is a converted vegetation index image obtained by converting the vegetation index in the conversion region RT (the conversion region of the first captured image) using the conversion model M. The second vegetation index image is a vegetation index image showing the vegetation index in the region captured in the reference image (the second captured image). As a result, the image processing device 10 of the embodiment can convert the vegetation index calculated from the other image (converted image) based on the vegetation index calculated based on one image (reference image) using the conversion model, and can absorb the difference in pixel values ​​caused by the difference in shooting conditions in the two satellite images.

[0074] (Modified examples of the embodiment) Herein, a modified version of the embodiment will be described. This modified version differs from the embodiment described above in that the comparison region RC selected from the overlapping region RO is determined based on the reflectance of the land photographed in the comparison region RC.

[0075] The reflectance of land, as used here, refers to the intensity of light reflected when sunlight reaches the land. For example, if the land photographed in the comparison area RC is land with flowing water, such as a river, reservoir, or irrigation canal, light will be reflected from the surface of the flowing water. In this case, depending on the relationship between the shooting position and the water surface, the amount of light received by the imaging device (the amount of reflected light reflected from the water surface) is expected to increase or decrease, and the amount of light received by the imaging device may become unstable. When the amount of light received by the imaging device becomes unstable, the pixel values ​​in the satellite image will not fall within a certain range, and the accuracy of the vegetation index will decrease, as will the variability of the vegetation index calculated from the pixel values.

[0076] As a countermeasure, in this modified version, the comparison region RC selected from the overlapping region RO is limited to areas where land other than land with flowing water has been photographed. This stabilizes the amount of light received by the imaging device and prevents a decrease in the accuracy of the vegetation index calculated from the pixel values.

[0077] Specifically, the transformation model generation unit 105 determines whether a corresponding point P that corresponds to a comparison region RC randomly selected from the overlapping region RO is an outlier. Here, an outlier is a point among the multiple corresponding points P for the comparison region RC whose distance to the transformation model M is larger than that of the other corresponding points. The transformation model generation unit 105 generates a transformation model M based on the training data (set of corresponding points P) from which outliers that are corresponding points P are excluded.

[0078] More specifically, the transformation model generation unit 105 obtains a set of corresponding points P corresponding to multiple comparison regions RC randomly selected from the overlapping region RO as training data. The transformation model generation unit 105 extracts a first pair from the training data, which is a single corresponding point P (a pair of a first vegetation index and the second vegetation index). The first vegetation index is a vegetation index calculated from the overlapping region RO captured in the transformation image. The second vegetation index is a vegetation index calculated from the overlapping region RO captured in the reference image. The transformation model generation unit 105 calculates a linear function as the transformation model M generated based on the training data including the first pair. The transformation model generation unit 105 also calculates a linear function as the transformation model generated based on the training data excluding the first pair. If the difference in the slopes of the two calculated linear functions is greater than a threshold, the transformation model generation unit 105 determines the first pair to be an outlier. On the other hand, the transformation model generation unit 105 does not determine the first pair as an outlier if the difference in the slopes of the two calculated linear functions is smaller than a threshold.

[0079] Figure 8 is a diagram illustrating a modified embodiment. Similar to Figure 5, Figure 8 schematically shows land captured in satellite images GA and GB. The overlapping region RO of satellite images GA and GB contains images of the river RV. The comparison region RC3, selected from the overlapping region RO, contains images of a portion of the river RV. In this case, the corresponding point P3 in comparison region RC3 may be an outlier in relation to the other corresponding points P1, P2, P4, and P5. If corresponding point P3 is an outlier, the transformation model M is calculated based on the training data (i.e., corresponding points P1, P2, P4, and P5) excluding the outlier corresponding point P3.

[0080] Furthermore, if the corresponding point P3 is excluded, and it is not possible to select N or more comparison regions RC as shown in step S13 of Figure 6, a new comparison region RC is selected. For example, as a result of excluding the corresponding point P3 as an outlier, the conversion model generation unit 105 selects a new comparison region RC6 from the overlapping region RO. In this case, the conversion model M is calculated based on the corresponding point P6 in addition to the corresponding points P1, P2, P4, and P5.

[0081] As described above, in the image processing apparatus 10 according to the modified embodiment, the conversion model generation unit 105 determines whether one of the corresponding points P (a pair of first vegetation index and second vegetation index) included in the set of corresponding points P (training data) is an outlier. If the training data contains an outlier, the conversion model generation unit 105 generates the conversion model based on the training data from which the outlier has been removed. As a result, the image processing apparatus 10 according to the modified embodiment can calculate a highly accurate conversion model M using the set of corresponding points P (training data) from which the outlier has been removed.

[0082] The image processing apparatus and estimation system in the above-described embodiment may be implemented using a computer. In that case, the program for implementing this function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be loaded into the computer system and executed. Here, "computer system" includes hardware such as the OS and peripheral devices. Furthermore, "computer-readable recording medium" refers to portable media such as flexible disks, magneto-optical disks, ROMs, CD-ROMs, and storage devices such as hard disks built into the computer system. Moreover, "computer-readable recording medium" may also include those that dynamically hold programs for a short period of time, such as communication lines used when transmitting programs via networks such as the Internet or communication lines such as telephone lines, and those that hold programs for a certain period of time, such as volatile memory inside the computer system that acts as a server or client in that case. Furthermore, the above-mentioned program may be for implementing a part of the above-mentioned function, or it may be a program that can implement the above-mentioned function in combination with a program already recorded in the computer system, or it may be implemented using a programmable logic device such as an FPGA (Field Programmable Gate Array).

[0083] Although embodiments of this invention have been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments and includes designs and the like that do not depart from the spirit of this invention. [Explanation of symbols]

[0084] 1... Estimation system, 10... Image processing device, 20... Crop condition estimation device, 100... Satellite image storage unit, 101... Overlap area acquisition unit, 102... Transformation area acquisition unit, 103... Reference area acquisition unit, 104... Latitude and longitude information storage unit, 105... Transformation model generation unit, 106... Transformation model information storage unit, 107... Vegetation index calculation unit, 108... Vegetation index information storage unit, 109... Vegetation index transformation unit, 110... Synthetic image generation unit, 111... Synthetic image storage unit

Claims

1. An overlapping area acquisition unit acquires overlapping areas that are captured in the first and second images, using a first image taken of the field under first shooting conditions, a second image taken of the field under second shooting conditions, the location information of the land captured in the first image, and the location information of the land captured in the second image. A conversion model generation unit generates a conversion model that converts the first vegetation index to the second vegetation index based on training data that associates the first vegetation index in the overlapping region of the first captured image with the second vegetation index in the overlapping region of the second captured image, An image processing device equipped with the following features.

2. The conversion model generation unit extracts multiple comparison regions corresponding to the same point in the overlapping region from each of the first and second captured images, and generates the conversion model based on the training data corresponding to each of the extracted comparison regions. The image processing apparatus according to claim 1.

3. The conversion model generation unit generates a linear function that converts the first vegetation index to the second vegetation index as the conversion model. The image processing apparatus according to claim 2.

4. The conversion model generation unit determines whether the pair of the first vegetation index and the second vegetation index included in the training data is an outlier, and if the training data includes an outlier, it generates the conversion model based on the training data from which the outlier has been removed. The image processing apparatus according to claim 3.

5. The transformation model generation unit extracts a first set, which is a pair of the first vegetation index and the second vegetation index, from the training data, and if the difference in the slopes of the two linear functions, one a linear function as the transformation model generated based on the training data including the first set and the other a linear function as the transformation model generated based on the training data excluding the first set, is greater than a threshold, the first set is treated as an outlier. The image processing apparatus according to claim 4.

6. An overlapping area acquisition unit acquires overlapping areas that are captured in the first and second images, using a first image taken of the field under first shooting conditions, a second image taken of the field under second shooting conditions, the location information of the land captured in the first image, and the location information of the land captured in the second image. A conversion region acquisition unit that acquires a conversion region different from the overlapping region from the first captured image, A vegetation index conversion unit that converts the vegetation index in the conversion region of the first captured image using a conversion model, A composite image generation unit generates a composite image by combining a first vegetation index image obtained by converting the vegetation index in the conversion region of the first captured image using a conversion model, and a second vegetation index image showing the vegetation index in the region captured in the second captured image. Equipped with, The conversion model is a model that converts the first vegetation index to the second vegetation index, generated based on training data that associates the first vegetation index in the overlapping region of the first captured image with the second vegetation index in the overlapping region of the second captured image. Image processing device.

7. The image processing apparatus according to claim 6, A crop condition estimation device that estimates the crop condition of a field captured in a composite image based on the vegetation index in the composite image generated by the image processing device, An estimation system equipped with the following features.

8. An image processing method performed by a computer used in an image processing device, The overlapping area acquisition unit uses the first image taken of the field under first shooting conditions, the second image taken of the field under second shooting conditions, the location information of the land captured in the first image, and the location information of the land captured in the second image to acquire the overlapping areas that are captured in the first and second images. The conversion model generation unit generates a conversion model that converts the first vegetation index to the second vegetation index based on training data that associates the first vegetation index in the overlapping region of the first captured image with the second vegetation index in the overlapping region of the second captured image. Image processing methods.

9. An image processing method performed by a computer used in an image processing device, The overlapping area acquisition unit uses the first image taken of the field under first shooting conditions, the second image taken of the field under second shooting conditions, the location information of the land captured in the first image, and the location information of the land captured in the second image to acquire the overlapping areas that are captured in the first and second images. The conversion region acquisition unit acquires a conversion region different from the overlapping region from the first captured image, The vegetation index conversion unit converts the vegetation index in the conversion region of the first captured image using a conversion model. The composite image generation unit generates a composite image by combining a first vegetation index image obtained by converting the vegetation index in the conversion region of the first captured image using a conversion model, and a second vegetation index image showing the vegetation index in the region captured in the second captured image. The conversion model is a model that converts the first vegetation index to the second vegetation index, generated based on training data that associates the first vegetation index in the overlapping region of the first captured image with the second vegetation index in the overlapping region of the second captured image. Image processing methods.

Images