Detection device, detection method, and control program

Abstract

To provide a detection device, detection method and control program for highly accurately detecting a withered tree region where a withered tree exists.SOLUTION: A detection device 1 comprises: a vegetation index image generation unit which generates a first vegetation index image having a vegetation index of each pixel included in a first optical image as a pixel value of a corresponding pixel and a second vegetation index image having a vegetation index of each pixel included in a second optical image as a pixel value of a corresponding pixel; a difference image generation unit which generates a difference image having a difference value between the vegetation index of each pixel included in the first vegetation index image and the vegetation index of each pixel included in the second vegetation index image as a pixel value of a corresponding pixel; an edge pixel extraction unit which extracts an edge pixel from the difference image; a detection unit which detects a withered tree region where a withered tree exists on the basis of the edge pixel; and an output unit (communication unit, display unit) which outputs information about the withered tree region.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a detection device, a detection method, and a control program. [Background technology]

[0002] In recent years, the number of trees to be inspected has been increasing due to the growing size and age of planted trees on the premises, as well as the increase in unmanaged trees, and there is a need to make inspection work more efficient.

[0003] Patent Document 1 describes a method for determining the health of trees in which the tree species estimated by analyzing satellite images is combined with the Normalized Difference Vegetation Index (NDVI) to determine the health of trees according to their species. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-144607 Summary of the Invention [Problem to be solved by the invention]

[0005] In tree inspection work, it is necessary to detect dead and damaged tree areas with high accuracy.

[0006] The present invention aims to provide a detection device, a detection method, and a control program that can detect dead and damaged tree areas where dead and damaged trees exist with high accuracy. [Means for solving the problem]

[0007] a vegetation index image generation unit that generates a first vegetation index image in which the vegetation index of each pixel included in the first optical image is set as the pixel value of the corresponding pixel, and a second vegetation index image in which the vegetation index of each pixel included in the second optical image is set as the pixel value of the corresponding pixel; a difference image generation unit that generates a difference image in which the pixel value of the corresponding pixel is the difference value between the vegetation index of each pixel included in the first vegetation index image and the vegetation index of each pixel included in the second vegetation index image; an edge pixel extraction unit that extracts edge pixels from the difference image; a detection unit that detects dead tree areas where dead trees exist based on the edge pixels; and an output unit that outputs information about the dead tree areas.

[0008] In the detection device according to one aspect of the present invention, the detection unit preferably detects an area surrounded by edge pixels as a dead or damaged tree area.

[0009] In the detection device according to one aspect of the present invention, the edge pixel extraction unit preferably performs a smoothing process on the difference image and extracts edge pixels from the difference image that has been subjected to the smoothing process.

[0010] In the detection device according to one aspect of the present invention, the edge pixel extraction unit preferably extracts inflection points in the difference image as edge pixels.

[0011] In a detection device according to one aspect of the present invention, it is preferable that the index calculation unit further includes a vegetation area extraction unit that calculates a GSI index for each of multiple pixels included in at least one of the first and second optical images, and extracts a vegetation area from one of the images based on the difference between the vegetation index and the GSI index, and that the vegetation index image generation unit generates the first vegetation index image so that the vegetation index of a pixel included in an area corresponding to the vegetation area in the first optical image is set to the pixel value of the corresponding pixel, and generates the second vegetation index image so that the vegetation index of a pixel included in an area corresponding to the vegetation area in the second optical image is set to the pixel value of the corresponding pixel.

[0012] A detection method according to one aspect of the present invention includes the steps of: acquiring, by a computer, a first optical image obtained by photographing a target area from above at a first time period and a second optical image obtained by photographing the target area from above at a second time period different from the first time period; calculating a vegetation index for each of a plurality of pixels included in the first optical image; calculating a vegetation index for each of a plurality of pixels included in the second optical image; generating a first vegetation index image in which the vegetation index of each pixel included in the first optical image is set as the pixel value of the corresponding pixel; and generating a difference image in which the pixel value of the corresponding pixel is the difference value between the vegetation index of each pixel included in the first vegetation index image and the vegetation index of each pixel included in the second vegetation index image; extracting edge pixels from the difference image; detecting dead tree areas where dead trees exist based on the edge pixels; and outputting information about the dead tree areas.

[0013] a first vegetation index image in which the vegetation index of each pixel included in the first optical image is set as the pixel value of the corresponding pixel, and a second vegetation index image in which the vegetation index of each pixel included in the second optical image is set as the pixel value of the corresponding pixel; a difference image in which the pixel value of the corresponding pixel is the difference value between the vegetation index of each pixel included in the first vegetation index image and the vegetation index of each pixel included in the second vegetation index image; edge pixels from the difference image; [Effects of the Invention]

[0014] According to the present invention, the detection device, detection method, and control program can detect dead and damaged tree areas where dead and damaged trees exist with high accuracy. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 is a diagram illustrating an example of a schematic configuration of a detection device. [Figure 2] 10 is a flowchart illustrating an example of a process for detecting a dead or damaged tree region. [Figure 3A] FIG. 10 is a diagram illustrating an example of an RGB image. [Figure 3B] FIG. 10 is a diagram illustrating an example of a red band image. [Figure 4A] FIG. 10 is a diagram illustrating an example of a green band image. [Figure 4B] FIG. 10 is a diagram illustrating an example of a blue band image. [Figure 5A] FIG. 1 is a diagram illustrating an example of a near-infrared band image. [Figure 5B] FIG. 10 is a diagram illustrating an example of a vegetation index image. [Figure 6A]FIG. 10 is a diagram illustrating an example of a GSI index image. [Figure 6B] FIG. 10 is a schematic diagram for explaining a vegetation area. [Figure 7A] FIG. 10 is a diagram showing an example of an RGB image of a coniferous tree captured at a first time. [Figure 7B] FIG. 10 is a diagram showing an example of an RGB image of a coniferous tree captured at a second time. [Figure 7C] FIG. 10 is a graph showing the relationship between pixel vegetation indices. [Figure 8A] FIG. 10 is a diagram showing an example of an RGB image of a broad-leaved tree captured at a first time point. [Figure 8B] FIG. 10 is a diagram showing an example of an RGB image of a broad-leaved tree captured at a second time point. [Figure 8C] FIG. 10 is a graph showing the relationship between pixel vegetation indices. [Figure 9A] FIG. 10 is a diagram showing an example of an RGB image of an unhealthy tree captured at a first stage. [Figure 9B] FIG. 10 is a diagram showing an example of an RGB image of an unhealthy tree captured at a second stage. [Figure 9C] FIG. 10 is a graph showing the relationship between pixel vegetation indices. [Figure 10A] FIG. 10 is a diagram illustrating an example of a difference image. [Figure 10B] FIG. 10 is a diagram illustrating an example of a difference image to which smoothing processing has been applied. [Figure 11] FIG. 10 is a diagram showing an example of edge pixels extracted from a region in a difference image. DETAILED DESCRIPTION OF THE INVENTION

[0016] Various embodiments of the present invention will be described below with reference to the drawings. Please note that the technical scope of the present invention is not limited to these embodiments, but extends to the inventions described in the claims and their equivalents.

[0017] 1 is a diagram showing an example of the schematic configuration of a detection device 1. The detection device 1 is an example of a computer, and detects dead and damaged tree areas where dead and damaged trees exist. The detection device 1 includes a memory unit 11, a communication unit 12, a display unit 13, an operation unit 14, and a processing unit 15.

[0018] The storage unit 11 stores programs or data. The storage unit 11 includes, for example, a semiconductor memory device. The storage unit 11 stores an operating system program, a driver program, an application program, data, and the like used for processing by the processing unit 15. Programs are installed into the storage unit 11 from a computer-readable, non-transitory, portable storage medium such as a CD (Compact Disc)-ROM (Read Only Memory) or a DVD (Digital Versatile Disc)-ROM using a known setup program or the like.

[0019] The communication unit 12 is an example of an output unit. The communication unit 12 enables the detection device 1 to communicate with other devices. The communication unit 12 includes a communication interface circuit. The communication interface circuit included in the communication unit 12 is a communication interface circuit such as a wired LAN (Local Area Network) or a wireless LAN. The communication unit 12 receives data from other devices and supplies the data to the processing unit 15, and also transmits data supplied from the processing unit 15 to other devices.

[0020] The display unit 13 is an example of an output unit. The display unit 13 displays an image. The display unit 13 includes, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display. The display unit 13 displays an image based on display data supplied from the processing unit 15.

[0021] The operation unit 14 accepts input operations by the user to the detection device 1. The operation unit 14 includes, for example, a keypad, a keyboard, or a mouse. The operation unit 14 may also include a touch panel integrated with the display unit 13. The operation unit 14 generates a signal according to the input operation by the user and supplies the signal to the processing unit 15.

[0022] The processing unit 15 is a device that comprehensively controls the operation of the detection device 1 and includes one or more processors and their peripheral circuits. The processing unit 15 includes, for example, a central processing unit (CPU). The processing unit 15 may also include a graphics processing unit (GPU), a digital signal processor (DSP), a large scale integration (LSI), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), etc. The processing unit 15 controls the operation of each component and executes various processes so that the various processes of the detection device 1 are executed in an appropriate order based on the programs stored in the storage unit 11 and inputs from the communication unit 12 and the operation unit 14.

[0023] The processing unit 15 includes, as functional blocks, an image acquisition unit 151, an index calculation unit 152, a vegetation area extraction unit 153, a vegetation index image generation unit 154, a difference image generation unit 155, an edge pixel extraction unit 156, a detection unit 157, and an output control unit 158. Each of these units is a functional module realized by a program executed by the processing unit 15. Each of these units may be implemented in the detection device 1 as firmware.

[0024] 2 is a flowchart showing an example of the dead and damaged tree region detection process. The dead and damaged tree region detection process is executed mainly by the processing unit 15 in cooperation with each element of the detection device 1 based on a program stored in advance in the storage unit 11.

[0025] First, the image acquisition unit 151 acquires a first optical image of a target area taken from above at a first time period from an external information processing device (for example, an information processing device mounted on an optical satellite) via the communication unit 12. The image acquisition unit 151 also acquires a second optical image of the target area taken from above at a second time period different from the first time period from the external information processing device via the communication unit 12 (step S101). The target area is an area that includes trees, such as a forest. The detection device 1 inspects the trees included in the target area and detects dead and damaged tree areas where dead trees exist from the target area.

[0026] The second time period is set to a time a predetermined period after the first time period. The predetermined period is generally set to the period from when a tree begins to wither until the change can be recognized visually by humans (e.g., one year). The first time period and the second time period may be set at any time, and may be set in different seasons or different time periods. The second time period may be set within one year of the first time period, or may be set two or more years after the first time period.

[0027] As will be described later, the detection device 1 detects dead and damaged tree areas by comparing the first optical image with the second optical image. The first and second periods are preferably set in summer. Generally, healthy trees have more leaves than dead and damaged trees. In summer, trees are more active, and the difference between the leaf amount of healthy trees and the leaf amount of dead and damaged trees is greater compared to other seasons. By setting the first and second periods to summer, the detection device 1 can detect dead and damaged tree areas with higher accuracy.

[0028] The first optical image and the second optical image are images of the Pleiades optical satellite, etc. The first optical image and the second optical image are images in which the intensity of reflected light (electromagnetic waves) of sunlight at each position within a target area and the intensity of emitted light (electromagnetic waves) from each position, sensed by an optical sensor mounted on the optical satellite, are used as pixel values ​​for each pixel corresponding to each position. The first optical image and the second optical image include multiple images generated for each wavelength of a predetermined width (band) of the electromagnetic waves sensed by the optical sensor mounted on the optical satellite.

[0029] The first optical image and the second optical image each include a red band image, a green band image, a blue band image, and a near-infrared band image. The red band image is an image in which the pixel value of each pixel is the light intensity in a band including red light wavelengths. The green band image is an image in which the pixel value of each pixel is the light intensity in a band including green light wavelengths. The blue band image is an image in which the pixel value of each pixel is the light intensity in a band including blue light wavelengths. The near-infrared band image is an image in which the pixel value of each pixel is the light intensity in a band including near-infrared light wavelengths.

[0030] The first optical image and the second optical image each further include an RGB image. The RGB image is a color image in which the pixel value of each pixel is a 48-bit color value consisting of the pixel value (16 bits) of the corresponding pixel in the red band image, the pixel value (16 bits) of the corresponding pixel in the green band image, and the pixel value (16 bits) of the corresponding pixel in the blue band image. Here, the number of bits of the pixel value is just an example, and pixels having pixel values ​​of 8 bits or 12 bits may also be used depending on the specifications of the satellite, etc.

[0031] FIG. 3A shows an example of an RGB image 300 captured from above of a predetermined target area 301.

[0032] 3A, target area 301 included in RGB image 300 includes vegetation area 302, which includes trees, and non-vegetation area 303, which does not include trees. Vegetation area 302 also includes dead and damaged tree area 304, which includes dead and damaged trees, and non-dead and damaged tree area 305, which does not include dead and damaged trees. Non-vegetation area 303 includes soil area 306, where soil is exposed on the ground surface, such as a bare ground area, and non-soil area 307, where concrete or the like is present and soil is not exposed on the ground surface.

[0033] FIG. 3B shows an example of a red band image 310 captured from above the target area 301.

[0034] In the example shown in Figure 3B, pixels corresponding to positions where the intensity of emitted or reflected red light is greater are shown with a higher brightness (closer to white), and pixels corresponding to positions where the intensity of emitted or reflected red light is smaller are shown with a lower brightness (closer to black).

[0035] FIG. 4A shows an example of a green band image 400 captured from above the target area 301.

[0036] In the example shown in FIG. 4A, pixels corresponding to positions where the intensity of emitted or reflected green light is greater are shown with a higher brightness (closer to white), and pixels corresponding to positions where the intensity of emitted or reflected green light is smaller are shown with a lower brightness (closer to black).

[0037] FIG. 4B shows an example of a blue band image 410 captured from above the target area 301.

[0038] In the example shown in Figure 4B, pixels corresponding to positions where the intensity of emitted or reflected blue light is greater are shown with a higher brightness (closer to white), and pixels corresponding to positions where the intensity of emitted or reflected blue light is smaller are shown with a lower brightness (closer to black).

[0039] FIG. 5A shows an example of a near-infrared band image 500 captured from above the target area 301.

[0040] In the example shown in FIG. 5A, pixels corresponding to positions where the intensity of emitted or reflected near-infrared light is greater are shown with higher brightness (closer to white), and pixels corresponding to positions where the intensity of emitted or reflected near-infrared light is smaller are shown with lower brightness (closer to black).

[0041] Next, the index calculation unit 152 calculates a vegetation index for each of multiple pixels included in the first optical image from the first optical image (step S102). The vegetation index is an index for understanding the state of vegetation in the target area, and indicates the amount or vitality of plants. The index calculation unit 152 calculates a Wide Dynamic Range Vegetation Index (WDRVI) as the vegetation index. The WDRVI has a higher value as the amount or vitality of plants increases, and a lower value as the amount or vitality of plants decreases. The index calculation unit 152 calculates the vegetation index for each pixel using the following formula (1) based on the red band image and the near-infrared band image. WDRVI=(αIR-R) / (αIR+R) …(1)

[0042] Here, α is a correction coefficient, and is set to a value within a range of, for example, 0.1 to 0.2. IR is the pixel value of the corresponding pixel in the near-infrared band image, i.e., the intensity of near-infrared light at the position corresponding to that pixel. R is the pixel value of the corresponding pixel in the red band image, i.e., the intensity of red light at the position corresponding to that pixel. WDRVI has a value within a range of -1.0 to +1.0. It is known that plants have a high reflectance of near-infrared light and a low reflectance of red light due to the action of chlorophyll. It is also known that as plants weaken, the activity of chlorophyll slows, resulting in a decrease in the reflectance of near-infrared light. Therefore, the detection device 1 can improve the detection accuracy of vegetation areas by using the WDRVI calculated by the above formula (1) as a vegetation index.

[0043] The index calculation unit 152 may calculate a Normalized Difference Vegetation Index (NDVI) as a vegetation index instead of the WDRVI. In this case, the index calculation unit 152 calculates the vegetation index of each pixel using the following formula (2) based on the red band image and the near-infrared band image. NDVI = (IR-R) / (IR+R) …(2)

[0044] The WDRVI is a vegetation index that was improved to alleviate the problem of NDVI value saturation in areas with a high Leaf Area Index (LAI). The LAI is an index that represents the amount of leaves in a plant community, and the more leaves there are, the higher the LAI value. By using the WDRVI as a vegetation index, the detection device 1 can further improve the accuracy of detecting vegetation areas compared to using the NDVI as a vegetation index.

[0045] Furthermore, the index calculation unit 152 may calculate a Green-red Ratio Vegetation Index (GRVI) as a vegetation index instead of the WDRVI. In this case, the index calculation unit 152 calculates the vegetation index for each pixel using the following formula (3) based on the green band image and the red band image. GRVI=(GR) / (G+R) …(3)

[0046] Here, G is the pixel value of the corresponding pixel in the green band image, i.e., the intensity of green light at the position corresponding to that pixel.

[0047] The index calculation unit 152 generates a vegetation index image in which the pixel value of each pixel is used as a vegetation index.

[0048] FIG. 5B shows an example of a vegetation index image 510 generated from the red band image 310 and the near-infrared band image 500 using WDRVI as the vegetation index.

[0049] In the example shown in Fig. 5B, pixels with a higher vegetation index are shown with a higher brightness (closer to white), and pixels with a lower vegetation index are shown with a lower brightness (closer to black). As shown in Fig. 5B, in an area 511 corresponding to the target area 301, an area 512 corresponding to the vegetation area 302 has a high vegetation index, and an area 513 corresponding to the non-vegetation area 303 has a low vegetation index. Therefore, by using the WDRVI as a vegetation index, the detection device 1 can accurately distinguish between vegetation areas and non-vegetation areas.

[0050] Of the region 512 corresponding to the vegetation region 302, the region 515 corresponding to the non-dead tree region 305 has a high vegetation index overall, while the region 514 corresponding to the dead tree region 304 has a slightly low vegetation index in some areas. Also, of the region 513 corresponding to the non-vegetation region 303, the region 517 corresponding to the non-soil region 307 has a sufficiently low vegetation index, while the region 516 corresponding to the soil region 306 has a slightly high vegetation index.

[0051] Next, the index calculation unit 152 calculates a GSI (Grain Size Index) for each of the plurality of pixels included in the first optical image from the first optical image (step S103). The GSI index, also known as a grain size index, is an index for detecting bare ground areas. The higher the GSI index value, the more likely it is that the position corresponding to that pixel is included in a bare ground area. The index calculation unit 152 calculates the GSI index for each pixel using the following formula (4) based on the red band image, green band image, and blue band image. GSI index=(RB) / (R+G+B) …(4)

[0052] where B is the pixel value of the corresponding pixel in the blue band image, i.e., the intensity of blue light at the position corresponding to that pixel. The GSI index, like the WDRVI, has a value in the range of -1.0 or more and +1.0 or less.

[0053] The index calculation unit 152 generates a GSI index image in which the pixel value of each pixel is used as the GSI index.

[0054] FIG. 6A shows an example of a GSI index image 600 generated from the red band image 310, the green band image 400, and the blue band image 410.

[0055] In the example shown in Fig. 6A, pixels with a higher GSI index are displayed with a higher brightness (closer to white), and pixels with a lower GSI index are displayed with a lower brightness (closer to black). As shown in Fig. 7A, among regions 601 corresponding to the target region 301, a region 606 corresponding to the soil region 306 has a higher GSI index than a region 602 corresponding to the vegetation region 302 and a region 607 corresponding to the non-soil region 307. Therefore, by using the GSI index, the detection device 1 can accurately distinguish between soil regions and other regions.

[0056] Next, the index calculation unit 152 calculates the difference between the vegetation index and the GSI index for each of the plurality of pixels included in the first optical image (step S104). The index calculation unit 152 generates an image as an index image in which the pixel value of each pixel is a difference value obtained by subtracting the pixel value of the corresponding pixel in the GSI index image from the pixel value of the corresponding pixel in the vegetation index image.

[0057] As described above, the vegetation index is an index for understanding the vegetation status in a target area, and has a high value in vegetated areas and a low value in non-vegetated areas. However, among non-vegetated areas, soil areas may include grasslands or farmlands, in which case the vegetation index also has a high value in the soil areas. The GSI index has a higher value in areas corresponding to soil areas than in other areas. Therefore, the difference obtained by subtracting the GSI index from the vegetation index maintains a high value close to the vegetation index in vegetated areas and has a value sufficiently low relative to the vegetation index in soil areas. Note that in non-soil areas, the vegetation index is low, so the difference obtained by subtracting the GSI index from the vegetation index has a low value. By correcting the vegetation index using the GSI index, which indicates the likelihood of a bare ground area, the index calculation unit 152 can reduce the possibility of extracting grasslands or farmlands as vegetated areas.

[0058] Next, the vegetation region extraction unit 153 performs a binarization process on the index image generated by the index calculation unit 152 (step S105). The vegetation region extraction unit 153 determines whether the pixel value of each pixel included in the index image is equal to or greater than a threshold. The threshold is determined, for example, by Otsu's binarization. The threshold may be set to the average value or median value of the pixel values ​​of each pixel included in the index image. The threshold may also be set to a fixed value in advance. The vegetation region extraction unit 153 generates a binary image in which the pixel value of each pixel is set to a valid value when the pixel value of the corresponding pixel in the index image is equal to or greater than the threshold, and is set to an invalid value when the pixel value of the corresponding pixel in the index image is less than the threshold.

[0059] Next, the vegetation area extraction unit 153 extracts a vegetation area from the first optical image based on the difference between the vegetation index calculated by the index calculation unit 152 and the GSI index (step S106). The vegetation area extraction unit 153 performs processing such as labeling on the binary image and extracts an area consisting of adjacent valid pixels as a valid area. The vegetation area extraction unit 153 generates data indicating the coordinates of the extracted valid area as vegetation area polygon data, and extracts an area corresponding to the vegetation area polygon data in the optical image as a vegetation area.

[0060] FIG. 6B is a schematic diagram for explaining the vegetation area.

[0061] An image 610 shown in Fig. 6B is the same as the RGB image 300 shown in Fig. 3A. In the image 610, an area 612 indicates an area corresponding to the vegetation area 302 extracted by the vegetation area extraction unit 153. As shown in Fig. 6B, each area corresponding to the vegetation area 302 is extracted as the vegetation area 612 by the vegetation area extraction unit 153. In this way, the detection device 1 can detect the vegetation area with high accuracy by utilizing the difference between the vegetation index and the GSI index.

[0062] Furthermore, since processing is subsequently performed only on this vegetation area, the detection device 1 can speed up the dead tree area detection process. Note that if the vegetation area extraction unit 153 has previously acquired information on land use status, for example, from a high-resolution land use and land cover map of Japan provided by JAXA, the vegetation area extraction unit 153 may extract the vegetation area based on that information.

[0063] In step S102, the index calculation unit 152 may calculate a vegetation index for each of multiple pixels included in the second optical image from the second optical image. In this case, in step S103, the index calculation unit 152 calculates a GSI index for each of multiple pixels included in the second optical image from the second optical image, and in step S104, calculates the difference between the vegetation index and the GSI index for each of multiple pixels included in the first optical image. Then, in step S106, the vegetation region extraction unit 153 extracts a vegetation region from the second optical image. This allows the vegetation region extraction unit 153 to extract a vegetation region that includes trees that did not exist when the first optical image was captured but that had grown before the second optical image was captured. On the other hand, by using the vegetation index and GSI index calculated from the first optical image, the vegetation region extraction unit 153 can extract a vegetation region that includes trees that had completely died before the second optical image was captured.

[0064] Alternatively, the index calculation unit 152 may calculate a vegetation index for each of a plurality of pixels included in the first optical image from the first optical image, while calculating a vegetation index for each of a plurality of pixels included in the second optical image from the second optical image. In this case, the vegetation region extraction unit 153 extracts, as a vegetation region, only a region of the vegetation region extracted from the first optical image in step S106 that corresponds to (overlaps with) a vegetation region extracted from the second optical image. Alternatively, the vegetation region extraction unit 153 may extract, as a vegetation region, a region that corresponds to at least one of the vegetation region extracted from the first optical image and the vegetation region extracted from the second optical image in step S106.

[0065] Next, the vegetation index image generation unit 154 calculates a vegetation index for each of the plurality of pixels included in the first optical image, and calculates a vegetation index for each of the plurality of pixels included in the second optical image (step S107). The vegetation index image generation unit 154 calculates a vegetation index for each pixel in the same manner as the process in step S102. The vegetation index image generation unit 154 calculates a vegetation index for each pixel included in an area corresponding to a vegetation area in the first optical image. The vegetation index image generation unit 154 also calculates a vegetation index for each pixel included in an area corresponding to a vegetation area in the second optical image. By calculating vegetation indices only for areas corresponding to vegetation areas in each optical image, the detection device 1 can speed up the dead tree area detection process compared to when calculating vegetation indices for all pixels included in each optical image.

[0066] In addition, for the first optical image and the second optical image for which the vegetation index has already been calculated by the index calculation unit 152 in step S102, the vegetation index image generation unit 154 may use the calculated vegetation index and omit calculating the vegetation index.

[0067] Next, the vegetation index image generating unit 154 generates a first vegetation index image in which the vegetation index of each pixel included in the first optical image is set as the pixel value of the corresponding pixel. Also, the vegetation index image generating unit 154 generates a second vegetation index image in which the vegetation index of each pixel included in the second optical image is set as the pixel value of the corresponding pixel (step S108).

[0068] The vegetation index image generating unit 154 generates a first vegetation index image by setting the vegetation index of each pixel included in an area corresponding to the vegetation area in the first optical image to the pixel value of the corresponding pixel. Also, the vegetation index image generating unit 154 generates a second vegetation index image by setting the vegetation index of each pixel included in an area corresponding to the vegetation area in the second optical image to the pixel value of the corresponding pixel. By setting the pixel values ​​of only the pixels in the area corresponding to the vegetation area to the vegetation index, the detection device 1 can speed up the dead tree area detection process compared to when setting pixel values ​​of all pixels to the vegetation index.

[0069] Next, the differential image generating unit 155 generates a differential image in which the pixel value of a corresponding pixel is the difference value between the vegetation index of each pixel included in the first vegetation index image and the vegetation index of each pixel included in the second vegetation index image (step S109). The differential image generating unit 155 generates a differential image in which the pixel value of a corresponding pixel is the difference value obtained by subtracting the vegetation index of the corresponding pixel in the first vegetation index image from the vegetation index of each pixel included in a region corresponding to the vegetation region in the second vegetation index image.

[0070] Corresponding pixels in the first optical image and the second optical image depict the same position. If the activity level of the trees at each position does not change between the first and second periods, the gradation value of the pixel corresponding to that position in the difference image will be approximately 0. On the other hand, if the activity level of the trees at each position decreases between the first and second periods, the gradation value of the pixel corresponding to that position in the difference image will be a negative value with a large absolute value.

[0071] Figure 7A shows an example of an RGB image 700 of a coniferous tree that remained healthy from the first time point to the second time point one year later, captured at a first time point. Figure 7B shows an example of an RGB image 710 of the same coniferous tree captured at a second time point.

[0072] All of the trees shown in the RGB image 700 in Fig. 7A and the RGB image 710 in Fig. 7B are in a healthy state. However, the sunlight conditions change between the first and second periods, and the RGB image 710 is brighter (higher luminance) overall than the RGB image 700.

[0073] FIG. 7C shows a graph 720 illustrating the relationship between the vegetation indices of corresponding pixels in the RGB image 700 and the RGB image 710.

[0074] The horizontal axis of FIG. 7C represents pixel positions on line 701 in RGB image 700 and line 711 corresponding to line 701 in RGB image 710. The vertical axis of FIG. 7C represents the vegetation index (WDRVI) of the pixel located at each pixel position. Point cloud 721 represents the vegetation index of each pixel on line 701, i.e., each point plotting the vegetation index at the first time point. Point cloud 722 represents the vegetation index of each pixel on line 711, i.e., each point plotting the vegetation index at the second time point. As shown in FIG. 7C, for coniferous trees that remained healthy from the first time point to the second time point, the vegetation index at the first time point and the vegetation index at the second time point are similar to each other. However, because the sunlight conditions changed between the first and second time points, the vegetation index at the first time point and the vegetation index at the second time point are generally different from each other. However, the difference between the vegetation index at the first time point and the vegetation index at the second time point is approximately constant at all positions.

[0075] Fig. 8A shows an example of an RGB image 800 of a broadleaf tree that remained healthy from the first time point to the second time point one year later, captured at the first time point. Fig. 8B shows an example of an RGB image 810 of the broadleaf tree captured at the second time point.

[0076] All of the trees shown in the RGB image 800 in Fig. 8A and the RGB image 810 in Fig. 8B are in a healthy state. However, the sunlight conditions change between the first and second periods, and the RGB image 810 is brighter (higher luminance) overall than the RGB image 800.

[0077] FIG. 8C shows a graph 820 illustrating the relationship between the vegetation indices of corresponding pixels in the RGB image 800 and the RGB image 810.

[0078] The horizontal axis of FIG. 8C represents pixel positions on a line 801 in the RGB image 800 and a line 811 corresponding to the line 801 in the RGB image 810. The vertical axis of FIG. 8C represents the vegetation index (GRVI) of the pixel located at each pixel position. A point cloud 821 represents the vegetation index of each pixel on the line 801, i.e., each point obtained by plotting the vegetation index at the first time point. A point cloud 822 represents the vegetation index of each pixel on the line 811, i.e., each point obtained by plotting the vegetation index at the second time point. As shown in FIG. 8C, for broad-leaved trees that remained healthy from the first time point to the second time point, the vegetation index at the first time point and the vegetation index at the second time point are similar to each other. However, because the sunlight conditions changed between the first and second time points, the vegetation index at the first time point and the vegetation index at the second time point are generally different from each other. However, the difference between the vegetation index at the first time point and the vegetation index at the second time point is approximately constant at all positions.

[0079] Figure 9A shows an example of an RGB image 900 of a tree captured at a first time point that was healthy at a first time point but became unhealthy one year later at a second time point. Figure 9B shows an example of an RGB image 910 of the tree captured at a second time point.

[0080] All of the trees shown in the RGB image 900 of Fig. 9A are healthy. However, in the RGB image 910 of Fig. 9B, some of the trees shown in an area 912 corresponding to the area 902 of the RGB image 900 are unhealthy. Furthermore, the sunlight conditions change between the first and second periods, and the RGB image 910 is brighter (higher luminance) overall than the RGB image 900.

[0081] FIG. 9C shows a graph 920 illustrating the relationship between the vegetation indices of corresponding pixels in the RGB image 900 and the RGB image 910 .

[0082] The horizontal axis of FIG. 9C indicates pixel positions on a line 901 in the RGB image 900 and a line 911 corresponding to the line 901 in the RGB image 910. The vertical axis of FIG. 9C indicates the vegetation index (WDRVI) of the pixel located at each pixel position. A point cloud 921 indicates each point on which the vegetation index of each pixel on the line 901, i.e., the vegetation index at the first time point, is plotted. A point cloud 922 indicates each point on which the vegetation index of each pixel on the line 911, i.e., the vegetation index at the second time point, is plotted. As shown in FIG. 9C, in a range 923 corresponding to a region 912 where trees are in an unhealthy state, the vegetation index at the second time point is lower than the vegetation index at the first time point. In particular, the difference between the vegetation index at the second time point in the range 923 and the vegetation index at the first time point is larger than the difference between the vegetation index at the second time point in the ranges other than the range 923 and the vegetation index at the first time point.

[0083] FIG. 10A shows an example of a difference image 1000.

[0084] 10A, pixels with lower difference values ​​are displayed with higher brightness (closer to white), and pixels with higher difference values ​​are displayed with lower brightness (closer to black). Each region 1001 in the difference image 1000 is a region where trees that were healthy in the first period but became unhealthy in the second period exist, and regions 1002 other than each region 1001 are regions where trees that remained healthy from the first period to the second period exist.

[0085] 10A, the amount of change (decrease) in the vegetation index is large in area 1001, where trees exist that were healthy in the first period but became unhealthy in the second period. On the other hand, even in area 1002, where trees exist that remained healthy from the first period to the second period, the amount of change (decrease) in the vegetation index is small, but the vegetation index has changed (decrease) due to the influence of changes in sunlight conditions between the first period and the second period.

[0086] The differential image generating unit 155 may generate a differential image such that the absolute value of the difference value obtained by subtracting the vegetation index of the corresponding pixel in the first vegetation index image from the vegetation index of each pixel included in the area corresponding to the vegetation area in the second vegetation index image is used as the pixel value of the corresponding pixel.

[0087] Next, the edge pixel extraction unit 156 performs a smoothing process on the difference image generated by the difference image generation unit 155 (step S110). As the smoothing process, the edge pixel extraction unit 156 applies a median filter to the difference image, for example. By applying the median filter to the difference image, the edge pixel extraction unit 156 replaces the pixel value of each pixel in the difference image with the median value of the pixel and its surrounding pixels. In this way, the edge pixel extraction unit 156 removes noise such as salt-and-pepper noise or spike noise.

[0088] FIG. 10B shows an example of a difference image 1010 to which a median filter has been applied as a smoothing process.

[0089] A difference image 1010 shown in Fig. 10B is an image generated by applying a median filter to the difference image 1000 shown in Fig. 10A. As shown in Fig. 10B, applying the median filter to the difference image 1000 removes noise while maintaining the contour of an area 1001 where the vegetation index has changed significantly. Therefore, the detection device 1 can detect with high accuracy the contour of an area where the vegetation index has changed significantly from the difference image.

[0090] The edge pixel extraction unit 156 may apply other known smoothing processes, such as a Gaussian filter or an iterative approximation filter, to the difference image.

[0091] Next, the edge pixel extraction unit 156 extracts edge pixels from the difference image (step S111). The edge pixel extraction unit 156 extracts an edge image from the difference image on which the smoothing process has been performed. The edge pixel extraction unit 156 extracts, for example, inflection points in the difference image as edge pixels. An inflection point is, for example, a point where the sign of the curvature of a curve indicating the vegetation index of each pixel changes in a group of pixels that are continuous in the horizontal or vertical direction in the difference image.

[0092] For example, the edge pixel extraction unit 156 performs first differentiation on the pixel values ​​of multiple horizontally consecutive pixels in the difference image using the Sobel method, and extracts pixels where the gradient of the pixel values ​​(first derivative value) becomes a maximum or minimum value as inflection points. Similarly, the edge pixel extraction unit 156 performs first differentiation on the pixel values ​​of multiple vertically consecutive pixels in the difference image using the Sobel method, and extracts pixels where the gradient of the pixel values ​​(first derivative value) becomes a maximum or minimum value as inflection points.

[0093] The edge pixel extraction unit 156 may perform second-order differentiation using the Laplacian method on the pixel values ​​of multiple horizontally consecutive pixels in the difference image, and extract pixels (zero-crossing points) where the gradient of the pixel values ​​(second-order differential value) is 0 as inflection points. In this case, the edge pixel extraction unit 156 further performs second-order differentiation using the Laplacian method on the pixel values ​​of multiple vertically consecutive pixels in the difference image, and extracts pixels (zero-crossing points) where the gradient of the pixel values ​​(second-order differential value) is 0 as inflection points.

[0094] Furthermore, the edge pixel extraction unit 156 may extract as edge pixels only those inflection points where the difference value obtained by subtracting the minimum value from the maximum value of the pixel values ​​of the surrounding pixels is equal to or greater than a predetermined threshold. The predetermined threshold is set, through prior experiments, to a value between the difference value calculated at the boundary between the area showing healthy trees and the area showing unhealthy trees and the difference value calculated within the area showing healthy trees. The surrounding pixels are set to pixels within a predetermined distance (for example, 3 pixels).

[0095] Furthermore, the edge pixel extraction unit 156 may extract, as edge pixels, pixels whose pixel value difference from adjacent pixels or pixels separated by a predetermined distance in the horizontal or vertical direction is equal to or greater than the predetermined threshold value.

[0096] This allows the detection device 1 to extract the boundary between the dead and damaged tree region and the non-dead and damaged tree region with high accuracy.

[0097] FIG. 11 shows an example of edge pixels extracted from the region in the difference image 1010 shown in FIG. 10B.

[0098] An image 1100 shown in Fig. 11 is the same image as the difference image 1010 shown in Fig. 10B. In the image 1100, pixels on the contour 1101 of each area 1001 where the vegetation index has changed significantly are extracted as edge pixels.

[0099] Next, the detection unit 157 detects a dead tree region where a dead tree exists based on the edge pixels extracted by the edge pixel extraction unit 156 (step S112). The detection unit 157 detects a region surrounded by the edge pixels extracted by the edge pixel extraction unit 156 as a dead tree region. The detection unit 157 generates an edge image in which edge pixels are defined as valid pixels and non-edge pixels are defined as invalid pixels. The detection unit 157 may also perform expansion / contraction processing on valid pixels in the edge image so as to connect valid pixels (edge ​​pixels) that are spaced a predetermined distance apart in the edge image. The detection unit 157 detects a region surrounded by valid pixels from the generated edge image by labeling or the like. The detection unit 157 detects a region in the difference image that corresponds to the region detected from the edge image as a dead tree region. Meanwhile, the detection unit 157 detects regions other than dead tree regions in the vegetation region as non-dead tree regions.

[0100] In the image 1100 shown in FIG. 11, an area 1102 surrounded by each contour 1101 is detected as a dead tree area.

[0101] As shown in Figures 7C and 8C, vegetation indices calculated from two optical images of a healthy tree taken at different times may differ due to changes in sunlight conditions. Therefore, when dead tree regions are detected based on the magnitude of the difference between vegetation indices calculated from two optical images taken at different times, changes in sunlight conditions may result in regions that do not contain dead or damaged trees being erroneously detected as dead or damaged tree regions. On the other hand, the difference between the vegetation indices at the first time point and the second time point, which occurs due to changes in sunlight conditions, is approximately constant at all positions within the image. Furthermore, as shown in Figure 9C, the difference between the vegetation indices at the first time point and the second time point changes significantly at the boundary between the region where the tree has become unhealthy and the region where the tree remains healthy.

[0102] The detection device 1 detects dead and damaged tree regions based on edge pixels in a difference image that indicates the difference in vegetation index calculated from two optical images captured at different times, i.e., based on changes in the difference in vegetation index between different times. This allows the detection device 1 to detect dead and damaged tree regions with high accuracy from two optical images generated at different times, without being affected by fluctuations in pixel values ​​that affect the entire image due to changes in sunlight conditions.

[0103] Next, the output control unit 158 ​​outputs information about the dead and damaged tree regions (step S113), completing the series of dead and damaged tree region detection processes. The output control unit 158 ​​outputs information about the dead and damaged tree regions detected by the detection unit 157 by displaying it on the display unit 13. The output control unit 158 ​​may also output information about the dead and damaged tree regions by transmitting it to an external information processing device via the communication unit 12. The output control unit 158 ​​may highlight the dead and damaged tree regions on an RGB image by displaying them in a different color from other regions or by surrounding them with a thick line, thereby generating an image in which the dead and damaged tree regions are displayed so as to be distinguishable from other regions, and output this image as information about the dead and damaged tree regions. The output control unit 158 ​​may also output the latitude or longitude indicating the dead and damaged tree regions as information about the dead and damaged tree regions.

[0104] The process of step S103 may be omitted, and in step S106, the vegetation region extraction unit 153 may extract a vegetation region by using only the vegetation index image without using the GSI index image. Also, in step S106, the vegetation region extraction unit 153 may extract a vegetation region by using only the GSI index image without using the vegetation index image. Also, the processes of steps S102 to S106 may be omitted, and in step S107, the vegetation index image generation unit 154 may calculate a vegetation index for each pixel included in the entire region of the first optical image and a vegetation index for each pixel included in the entire region of the second optical image. Also, the process of step S110 may be omitted, and in step S111, the edge pixel extraction unit 156 may extract an edge image from a difference image on which smoothing processing has not been performed.

[0105] While FIG. 9 uses WDRVI to detect the unhealthy state of trees, this is not limiting, and other vegetation indices such as the above-mentioned NDVI and GRVI, and the Soil Adjusted Vegetation Index (SAVI) may also be used. Two or more vegetation indices may also be used in combination. Generally, it is considered that the accuracy of detecting vegetation areas when using each indices varies depending on the time of shooting and the characteristics of the sensor used for shooting. Therefore, when using a combination of indices, the detection unit 157 can, for example, calculate the logical sum of dead tree areas found from the multiple indices to reduce missed detection of vegetation areas.

[0106] Furthermore, by calculating the logical product of the dead tree areas determined by the detection unit 157 from multiple indices, areas that are more likely to be vegetated can be identified and those areas can be given priority for visual on-site inspection, thereby improving the efficiency of inspection work. Furthermore, it is also possible to combine and use the logical sum and logical product of the dead tree areas determined by the detection unit 157 from multiple indices.

[0107] As described above in detail, the detection device 1 detects dead and damaged tree areas based on changes in the difference between vegetation indices at different times. This allows the detection device 1 to detect dead and damaged tree areas from two optical images generated at different times without being affected by fluctuations in pixel values ​​that occur across the entire image due to changes in sunlight conditions. Therefore, the detection device 1 can detect dead and damaged tree areas where dead and damaged trees exist with high accuracy.

[0108] Furthermore, the detection device 1 can detect dead and damaged tree regions with high accuracy using any two optical images captured at different times. Therefore, the detection device 1 can efficiently detect dead and damaged tree regions without having to select optical images for detecting dead and damaged tree regions according to specific criteria.

[0109] Furthermore, the detection device 1 automatically detects dead and damaged tree regions through image processing, allowing users to efficiently inspect trees. Therefore, the detection device 1 can improve user convenience.

[0110] It should be understood by those skilled in the art that various changes, substitutions, and alterations can be made to the present invention without departing from the spirit and scope of the present invention. For example, the processes of the above-described parts may be executed in a different order as appropriate within the scope of the present invention. Furthermore, the above-described embodiments and modifications may be combined as appropriate within the scope of the present invention. [Explanation of symbols]

[0111] 1. Detection device 151 Image acquisition unit 152 Indicator calculation section 153 Vegetation area extraction unit 154 Vegetation index image generation unit 155 Differential image generation unit 156 Edge pixel extraction unit 157 Detector 158 Output control section

Claims

1. an image acquisition unit that acquires a first optical image of a target area taken from above at a first time period and a second optical image of the target area taken from above at a second time period different from the first time period; an index calculation unit that calculates a vegetation index for each of a plurality of pixels included in the first optical image and calculates a vegetation index for each of a plurality of pixels included in the second optical image; a vegetation index image generating unit that generates a first vegetation index image in which the vegetation index of each pixel included in the first optical image is set as a pixel value of the corresponding pixel, and a second vegetation index image in which the vegetation index of each pixel included in the second optical image is set as a pixel value of the corresponding pixel; a differential image generating unit that generates a differential image in which a difference value between the vegetation index of each pixel included in the first vegetation index image and the vegetation index of each pixel included in the second vegetation index image is used as a pixel value of the corresponding pixel; an edge pixel extraction unit that extracts edge pixels from the difference image; a detection unit that detects a dead tree region where a dead tree exists based on the edge pixels; an output unit that outputs information about the dead tree region; A detection device comprising:

2. The detection device according to claim 1 , wherein the detection unit detects an area surrounded by the edge pixels as the dead tree area.

3. The detection device according to claim 1 , wherein the edge pixel extraction unit performs a smoothing process on the difference image, and extracts the edge pixels from the difference image after the smoothing process.

4. The detection device according to claim 3 , wherein the edge pixel extraction unit extracts inflection points in the difference image as the edge pixels.

5. the index calculation unit calculates a GSI index for each of a plurality of pixels included in at least one of the first optical image and the second optical image; a vegetation area extraction unit that extracts a vegetation area from the one image based on a difference between the vegetation index and the GSI index; The vegetation index image generating unit generating the first vegetation index image so that the vegetation index of a pixel included in a region corresponding to the vegetation region in the first optical image is set to a pixel value of the corresponding pixel; generating the second vegetation index image so that the vegetation index of a pixel included in a region corresponding to the vegetation region in the second optical image is set to the pixel value of the corresponding pixel; The detection device according to any one of claims 1 to 4.

6. The computer acquiring a first optical image of a target area photographed from above at a first time period and a second optical image of the target area photographed from above at a second time period different from the first time period; Calculating a vegetation index for each of a plurality of pixels included in the first optical image, and calculating a vegetation index for each of a plurality of pixels included in the second optical image; generating a first vegetation index image in which the vegetation index of each pixel included in the first optical image is set as a pixel value of the corresponding pixel, and a second vegetation index image in which the vegetation index of each pixel included in the second optical image is set as a pixel value of the corresponding pixel; generating a difference image in which the difference value between the vegetation index of each pixel included in the first vegetation index image and the vegetation index of each pixel included in the second vegetation index image is used as the pixel value of the corresponding pixel; extracting edge pixels from the difference image; detecting a dead tree region where a dead tree exists based on the edge pixels; outputting information about the dead tree region; A detection method characterized by:

7. A control program for a computer having an output unit, acquiring a first optical image of a target area photographed from above at a first time period and a second optical image of the target area photographed from above at a second time period different from the first time period; calculating a vegetation index for each of a plurality of pixels included in the first optical image; calculating a vegetation index for each of a plurality of pixels included in the second optical image; generating a first vegetation index image in which the vegetation index of each pixel included in the first optical image is set as a pixel value of the corresponding pixel, and a second vegetation index image in which the vegetation index of each pixel included in the second optical image is set as a pixel value of the corresponding pixel; generating a difference image in which the difference value between the vegetation index of each pixel included in the first vegetation index image and the vegetation index of each pixel included in the second vegetation index image is used as the pixel value of the corresponding pixel; extracting edge pixels from the difference image; detecting a dead tree region where a dead tree exists based on the edge pixels; outputting information about the dead tree region from the output unit; A control program that causes the computer to execute the above steps.

Images