Image processing apparatus, image processing method, and image processing program

Abstract

To identify a land area in an aerial captured image without using large-volume data, such as numerical altitude data.SOLUTION: A statistic calculation unit 102 calculates, for each of a plurality of partial image regions subdivided from a captured image 110 in a space including a land area and a water area, captured from the above, statistics of luminance values of pixels in each of the partial image regions, A binarization unit 103 binarizes the statistics of the luminance values of pixels in the partial image regions, using a statistics threshold which is a threshold for statistics of luminance values. A binarization mapping unit 104 maps a result of binarization by the binarization unit 103 for the partial image regions, in association with positions of the partial image regions in the captured image 110.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a technique for analyzing a captured image and mapping the analysis results. [Background technology]

[0002] In algorithms that determine redundant elements using the brightness of each pixel in a captured image, it is common to exclude areas where redundant elements do not exist from the target area for processing by, for example, superimposing a mask image on them. For example, in the technology disclosed in Patent Document 1, it is determined on the aircraft whether or not an extra element such as a ship is included in an image captured from the sky by the aircraft. When detecting redundant elements in water areas, as in the technology of Patent Document 1, digital elevation data is often used to mask land areas. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. WO2018 / 155683 Summary of the Invention [Problem to be solved by the invention]

[0004] As described above, the technology in Patent Document 1 detects redundant elements in a water area on an aircraft. The devices mounted on aircraft generally have limited memory capacity. Meanwhile, digital elevation data is a large volume of data. For this reason, it is not desirable to store digital elevation data in equipment mounted on an aircraft, as this would put a strain on memory capacity. In this way, it is desirable to identify land areas to be masked without using large amounts of data such as digital elevation data.

[0005] The present disclosure has been made in consideration of the above circumstances. More specifically, the present disclosure has a main object to enable identification of land areas in images captured from the air without using large amounts of data such as digital elevation data. [Means for solving the problem]

[0006] The image processing device according to the present disclosure includes: a statistical value calculation unit that calculates a statistical value of the brightness values ​​of pixels in each of a plurality of partial image regions divided from an image of a space including land and water regions captured from the sky; a binarization unit that binarizes the statistical value of the luminance values ​​of pixels in each partial image region using a statistical value threshold that is a threshold for the statistical value of the luminance values; and a binarization mapping section that maps the binarization result for each partial image region by the binarization section so that it corresponds to the position of each partial image region within the captured image. [Effects of the Invention]

[0007] According to the present disclosure, it is possible to identify land areas in images captured from the sky without using large amounts of data such as digital elevation data. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of the functional configuration of an image processing device according to a first embodiment. [Figure 2] FIG. 1 is a diagram showing an example of the hardware configuration of an image processing device according to a first embodiment. [Figure 3] 4 is a flowchart showing an example of the operation of the image processing device according to the first embodiment. [Figure 4] 4 is a flowchart showing an example of the operation of the image processing device according to the first embodiment. [Figure 5] 3A and 3B are diagrams showing examples of scanning frames and partial image regions according to the first embodiment. [Figure 6] FIG. 3 is a conceptual diagram illustrating an example of calculating a statistical value according to the first embodiment. [Figure 7]FIG. 2 is a diagram showing an example of a captured image according to the first embodiment. [Figure 8] FIG. 3 is a diagram showing an example of a binarization map according to the first embodiment. [Figure 9] FIG. 4 is a diagram showing an example of a determination result map according to the first embodiment. [Figure 10] FIG. 3 is a diagram showing an example of the shape of an exponential distribution according to the first embodiment. [Figure 11] FIG. 10 is a diagram showing an example of the functional configuration of an image processing device according to a second embodiment. [Figure 12] 10 is a flowchart showing an example of the operation of the image processing device according to the second embodiment. [Figure 13] 10 is a flowchart showing an example of the operation of the image processing device according to the second embodiment. [Figure 14] FIG. 10 is a diagram showing an example of a land area candidate extraction result map according to the second embodiment. [Figure 15] FIG. 10 is a diagram showing an example of a determination result map according to the second embodiment. [Figure 16] FIG. 10 is a diagram showing an example of the functional configuration of an image processing device according to a third embodiment. [Figure 17] FIG. 11 is a diagram showing an example of a morphology determination result map according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments will be described with reference to the drawings. In the following description of the embodiments and the drawings, the same reference numerals denote the same or corresponding parts.

[0010] Embodiment 1 ***Configuration Description*** FIG. 1 shows an example of the functional configuration of an image processing device 100 according to this embodiment. FIG. 2 shows an example of the hardware configuration of the image processing device 100 according to this embodiment. The image processing device 100 may be mounted on a flying object such as an artificial satellite or an airplane, or may be located in a ground station that manages the flying object.

[0011] The image processing device 100 according to this embodiment is a computer. The operation procedure of the image processing device 100 corresponds to an image processing method, and the program that realizes the operation of the image processing device 100 corresponds to an image processing program.

[0012] As shown in FIG. 2, the image processing device 100 includes, as hardware, a processor 911, a main storage device 912, an auxiliary storage device 913, and a communication device 914. 1, the image processing device 100 includes, as functional components, a partial image region dividing unit 101, a statistical value calculation unit 102, a binarization unit 103, a binarization mapping unit 104, and a land area range determination unit 105. The functions of these functional components are realized, for example, by a program. The auxiliary storage device 913 stores programs that realize the functions of these functional components. These programs are loaded from the auxiliary storage device 913 into the main storage device 912. Then, the processor 911 executes these programs to perform the operations of these functional components. FIG. 2 shows a schematic diagram of a state in which the processor 911 is executing a program that implements the functions of the functional components.

[0013] The image processing device 100 acquires, for example, a captured image 110 captured by an imaging device inside the flying object. The captured image 110 is, for example, an image of a space including water and land captured from the sky. The captured image 110 is, for example, a synthetic aperture radar image (hereinafter referred to as a SAR image).

[0014] Next, an example of the functional configuration of the image processing device 100 will be described with reference to FIG. In FIG. 1, the image processing device 100 is independent from the imaging device, but the image processing device 100 and the imaging device may be integrated. In the following description, an example in which the image processing device 100 is independent from the imaging device will be described.

[0015] The partial image region dividing unit 101 acquires a captured image 110 . Furthermore, the partial image region dividing unit 101 divides the captured image 110 into a plurality of partial image regions 111. Fig. 5 shows an example of the partial image regions 111. Details of Fig. 5 will be described later. The partial image region dividing unit 101 outputs a plurality of partial image regions 111 to the statistical value calculation unit 102 .

[0016] The statistical value calculation unit 102 calculates a statistical value 112 of the luminance values ​​of the pixels in each of the plurality of partial image regions 111 . The statistical value calculation unit 102 calculates one or more of the standard deviation, variance, skewness, kurtosis, and moment as the statistical value 112 . Then, the statistical value calculation unit 102 outputs the statistical value 112 for each partial image region 111 to the binarization unit 103 . The statistical value calculation unit 102 may map the calculated statistical values ​​112 in correspondence with the positions of the partial image regions 111 in the captured image 110, and generate a statistical value map indicating the mapping results. In this case, the statistical value calculation unit 102 outputs the statistical value map to the binarization unit 103 instead of or together with the statistical values ​​112. The processing performed by the statistical value calculation unit 102 corresponds to statistical value calculation processing.

[0017] The binarization unit 103 binarizes the statistical value 112 of each partial image region 111 using a statistical value threshold that is a threshold for the statistical value 112 . Then, the binarization unit 103 outputs the binarization result 113 for each partial image region 111 to the binarization mapping unit 104 . The binarization result 113 indicates a value of "1" or a value of "0" for each partial image region 111. The processing performed by the binarization unit 103 corresponds to binarization processing.

[0018] The binarization mapping unit 104 maps the binarization results 113 for each partial image region 111 obtained by the binarization unit 103 so that they correspond to the positions of the partial image regions 111 within the captured image 110. Then, the binarization mapping unit 104 generates a binarization map 120 that indicates the mapping results. The binarization mapping unit 104 outputs the generated binarization map 120 to the land area range determination unit 105 . The processing performed by the binarization mapping unit 104 corresponds to binarization mapping processing.

[0019] The land area determination unit 105 analyzes the binarization map 120 to determine the land area. The land area is the area corresponding to the land area in the captured image 110. Then, the land area determination unit 105 outputs the determination result 130. The determination result 130 indicates, for example, the numbers of pixels that form the contour line of the land area range. For example, the land area determination unit 105 outputs the determination result 130 to a mechanism (not shown in FIG. 1) that superimposes a mask image on the land area.

[0020] ***Explanation of Operation*** FIG. 3 shows an example of the operation of the image processing device 100 according to this embodiment. An example of the operation of the image processing device 100 according to this embodiment will be described below with reference to FIG.

[0021] In step S101 , the partial image region dividing unit 101 acquires the captured image 110 and further divides the captured image 110 into partial image regions 111 . FIG. 5 is a conceptual diagram illustrating an example of division into partial image regions 111 by the partial image region dividing unit 101. In FIG.

[0022] The partial image region dividing unit 101 sets a scanning frame 1110 that is significantly smaller than the size of the captured image 110 . For example, the partial image region dividing unit 101 sets a scanning frame 1110 of several to tens of pixels square. Then, the partial image region dividing unit 101 scans the captured image 110 with the scanning frame 1110, and divides the captured image 110 into a plurality of partial image regions 111. For example, the partial image region dividing unit 101 scans the captured image 110 by moving the scanning frame 1110 by one pixel at a time, and obtains a plurality of partial image regions 111 that are shifted from each other by one pixel. 5 shows partial image areas 111-1, 111-2, and 111-Z obtained in this manner. Partial image area 111-1 is the partial image area located at the top left of the captured image 110. Partial image area 111-2 is the partial image area located immediately to the right of partial image area 111-1. Partial image area 111-Z is the partial image area located at the bottom right of the captured image 110. If there is no need to distinguish between the partial image area 111-1, the partial image area 111-2, and the partial image area 111-Z, they will be collectively referred to as the partial image area 111.

[0023] Next, in step S102, the statistical value calculation unit 102 calculates, for each partial image region 111, a statistical value 112 of the luminance values ​​of the pixels in the partial image region 111. FIG. 6 is a conceptual diagram illustrating an example of calculation of the statistical value 112 by the statistical value calculation unit 102. In FIG.

[0024] The statistical value calculation unit 102 calculates a statistical value 112 of the luminance values ​​of the pixels in the partial image area 111-1. Furthermore, the statistical value calculation unit 102 calculates a statistical value 112 of the luminance values ​​of the pixels in the partial image area 111-2. The statistical value calculation unit 102 performs the same operation up to the partial image region 111-Z. In this way, the statistical value calculation unit 102 calculates, for each partial image region 111 , a statistical value 112 of the luminance values ​​of the pixels in the partial image region 111 . Then, the statistical value calculation unit 102 outputs the statistical value 112 of each partial image region 111 to the binarization unit 103 .

[0025] Next, in step S103, the binarization unit 103 binarizes the statistical value 112 of each partial image region 111 using a statistical value threshold. More specifically, the binarization unit 103 sets a first value (for example, value "1"), which is one of the two values, to a partial image area where a statistical value 112 equal to or greater than the statistical value threshold is calculated. On the other hand, the binarization unit 103 sets a second value (for example, value "0"), which is the other of the two values, to a partial image area where a statistical value less than the statistical value threshold is calculated. In the following description, the first value is set to "1" and the second value is set to "0". The statistical value threshold will be described in detail later. The binarization unit 103 outputs the binarization result 113 for each partial image region 111 to the binarization mapping unit 104 .

[0026] Next, in step S104, the binarization mapping unit 104 maps the binarization results 113 of each partial image region 111 in correspondence with the position of each partial image region 111 in the captured image 110, thereby generating a binarization map 120. That is, the binarization mapping unit 104 maps the binarization result 113 for partial image region 111-1 to the leftmost position in the top row of the binarization map 120. The binarization mapping unit 104 also maps the binarization result 113 for partial image region 111-2 to the immediate right of the binarization result for partial image region 111-1. Finally, the binarization mapping unit 104 maps the binarization result 113 for partial image region 111-Z to the rightmost position in the bottom row of the binarization map 120. The binarization mapping unit 104 outputs the binarization map 120 to the land area range determination unit 105 .

[0027] Finally, in step S105, the land area determination unit 105 analyzes the binarization map 120 to determine the land area.

[0028] FIG. 4 shows the details of step S105.

[0029] In step S1051, the land area range determination unit 105 extracts a range in the binarization map 120 where the value "1" is continuous as a continuous range. Consecutive values ​​of "1" mean that there is another value "1" above, below, to the left, to the right, to the upper left, to the upper right, to the lower left, or to the lower right of the value "1". When multiple continuous regions are extracted, the land area determination unit 105 performs labeling processing for each continuous region. The labeling processing assigns an identifier to each continuous region and treats each continuous region as a separate continuous region. The land area range determination unit 105 may use any labeling method for the labeling process. For example, the land area range determination unit 105 may use a library such as OpenCV.

[0030] Next, in step S1052, the land area determination unit 105 calculates the area of ​​each continuous area. The land area determination unit 105 calculates the area of ​​a continuous region by, for example, accumulating the number of pixels in the continuous region.

[0031] Next, in step S1053, the land area determination unit 105 determines, for each continuous area, whether the area of ​​the continuous area is equal to or greater than an area threshold. The area threshold is a threshold for the area of ​​the continuous area. If the area of ​​the continuous range is equal to or greater than the area threshold, in step S1054, the land area determination unit 105 determines that the continuous range is a land area range.

[0032] After steps S1052 to S1054 have been performed for all continuous regions, the land area range determination unit 105 outputs the determination result 130 in step S1055. As described above, the determination result 130 indicates, for example, the numbers of pixels that form the contour line of the land area.

[0033] Next, an example of the operation of the image processing device 100 according to this embodiment will be described using a specific example.

[0034] Here, it is assumed that the partial image region dividing unit 101 acquires a captured image 110 exemplified in FIG. FIG. 7 shows an example of a captured image 110 of a space where water areas (ocean areas) and land areas exist. In FIG. 7, the darkly hatched areas are water areas, and the lightly hatched areas are land areas. Also, tiny rectangles within the water area are redundant elements such as ships.

[0035] FIG. 8 shows an example of a binarized map 120 generated from the captured image 110 of FIG. In the binarization map 120, partial image regions 111 whose statistical values ​​112 are less than the statistical value threshold, i.e., partial image regions 111 to which a value of "0" is set, are represented in black. On the other hand, partial image regions 111 whose statistical values ​​112 are equal to or greater than the statistical value threshold, i.e., partial image regions 111 to which a value of "1" is set, are represented in white. Since the brightness values ​​of pixels in land areas are high, the statistical value 112 of the partial image area 111 corresponding to the land area is equal to or greater than the statistical value threshold. Therefore, the value "1" is set for the partial image area 111 corresponding to the land area. Furthermore, because the brightness values ​​of the pixels of the surplus elements, such as ships, are also high, the statistical value 112 of the partial image area 111 corresponding to the surplus elements is also equal to or greater than the statistical value threshold. Therefore, the value "1" is also set for the partial image area 111 corresponding to the surplus elements.

[0036] As described above, the land area range determination unit 105 extracts a range in which the value "1" is continuous as a continuous range. At this stage, the continuous ranges corresponding to the three land areas in Figure 7 and the range corresponding to the surplus elements in the water area are extracted. Next, the land area determination unit 105 determines whether the area of ​​each continuous area is equal to or greater than an area threshold. Since the continuous range of each surplus element is less than the area threshold, the land area determination unit 105 does not recognize the continuous range of each surplus element as a land area range. As a result, as shown in FIG. 9, the land area determination unit 105 determines each of the areas 136, 137, and 138 as land area areas. 9 shows an example of the determination result map 135. The determination result map 135 is a map corresponding to the determination result 130. As described above, the determination result 130 indicates, for example, the numbers of pixels forming the contour of the land area. The determination result map 135 is a map that represents the contour of the land area as a boundary line between land and water based on the determination result 130. Here, determination result map 135 is shown as an example to make it easier to understand the operation of land area range determination unit 105. Land area range determination unit 105 may or may not generate determination result map 135. If land area range determination unit 105 generates determination result map 135, land area range determination unit 105 may output determination result map 135 instead of or together with determination result 130.

[0037] In the binarization map 120 of Fig. 8, the boundary line between the water area and the land area (the contour line of the land area) has many discontinuous parts (steps), and is not as smooth as the boundary line in the captured image 110. However, in the binarization map 120 of Fig. 8, the water area and the land area are roughly separated. Therefore, the land area range determination unit 105 can determine the land area range as shown in Fig. 9 using the binarization map 120 of Fig. 8. Therefore, according to this embodiment, it is possible to identify land areas without using digital elevation data.

[0038] Next, the statistical threshold value used by the binarization unit 103 will be described. Here, the statistical value calculation unit 102 calculates, as the statistical value 112, either only the skewness s, only the kurtosis k, or both the skewness s and the kurtosis k. For this reason, statistical thresholds for skewness s and kurtosis k will be explained below.

[0039] The skewness s is defined by the following equation (1): s=E[(x-μ) 3 ] / σ 3 Formula (1)

[0040] Moreover, the kurtosis k is defined by the following equation (2). k=E[(x-μ)4 ] / σ 4 Formula (2)

[0041] In equations (1) and (2), x represents the brightness of each pixel in the partial image region 111 (more precisely, the intensity of radar backscattering). μ represents the average value of x. σ represents the standard deviation of x. E[t] represents the expected value of quantity t. According to the above definition of kurtosis k, the kurtosis of a normal distribution is k = 3. The definition of kurtosis k is often obtained by subtracting 3 from the definition of equation (2) so that the kurtosis of a normal distribution becomes zero, but in this embodiment, kurtosis k is defined as above. Since the brightness of ships is higher than the brightness of the sea surface, the skewness and kurtosis of the partial image area 111 containing ships will be larger than the skewness and kurtosis of the partial image area 111 containing only the sea surface. This is because the more data (x>μ) larger than the average μ is mixed in the partial image area 111, the smaller the numerator E[(x-μ) 3 ] and E[(x-μ) 4 ] becomes larger.

[0042] Here, we will give an overview of the brightness distribution (intensity distribution) of SAR images of the sea surface. The probability density function of speckle intensity on the sea surface, etc., closely matches the exponential distribution shown in the following equation (3).

[0043]

number

[0044] In the rightmost equation of equation (3), λ=1 / 2σ 2 The shape of the exponential distribution when λ=1 is shown in Figure 10. The mean μ of the exponential distribution is 1 / λ. The standard deviation σ is also 1 / λ. The skewness is 2. The kurtosis is 9. Note that the kurtosis of 9 is in the case of the definition in this embodiment, and in the case of the definition in which the kurtosis of a normal distribution is zero, the kurtosis is 6. In this way, the shape of the exponential distribution is determined by a single parameter λ, and the skewness and kurtosis are constants. When ships are mixed in the partial image region 111, the probability density function does not follow an exponential distribution, and the skewness and kurtosis become values ​​greater than 2 and 9, respectively. In reality, the brightness distribution of an image of only the sea surface is not a perfectly exponential distribution, so the skewness and kurtosis are not exactly 2 and 9. However, the theoretical values ​​of skewness: 2 and kurtosis: 9 obtained from the brightness distribution of an image of only the sea surface are effective as thresholds for distinguishing between water and land areas. For this reason, the binarization unit 103 determines a statistical value threshold for the skewness based on the skewness: 2 (theoretical value of the skewness of the exponential distribution). Furthermore, the binarization unit 103 determines a statistical value threshold for kurtosis based on kurtosis: 9 (theoretical value of kurtosis of exponential distribution). Prior to step S103, the binarization unit 103 determines a statistical value threshold in advance.

[0045] ***Explanation of the effect of the embodiment*** The binarized map according to this embodiment can roughly separate water areas from land areas. Therefore, according to this embodiment, land areas can be identified without using digital elevation data. Therefore, a mask image can be superimposed on the image area of ​​land areas without consuming memory capacity on the aircraft. As a result, it is possible to determine on the aircraft whether or not a captured image contains extraneous elements such as ships without consuming memory capacity.

[0046] Embodiment 2, In this embodiment, differences from the first embodiment will be mainly described. The matters not explained below are the same as those in the first embodiment.

[0047] FIG. 11 shows an example of the functional configuration of an image processing device 100 according to this embodiment. In FIG. 11, compared to FIG. 1, a land area candidate extraction unit 106 and a land area candidate average value calculation unit 107 are added. Furthermore, in this embodiment, the operations of the statistical value calculation unit 102, the binarization mapping unit 104, and the land area range determination unit 105 are different from those in the first embodiment.

[0048] In this embodiment, the statistical value calculation unit 102 calculates an average value 114 in addition to the statistical value 112 . That is, the statistical value calculation unit 102 calculates, for each partial image region 111, an average value 114 of the brightness values ​​of the pixels in the partial image region 111. Then, the statistical value calculation unit 102 outputs the average value 114 for each partial image region 111 to the candidate land area average value calculation unit 107. The statistical value calculation unit 102 may map the calculated average value 114 in correspondence with the position of each partial image region 111 in the captured image 110, and generate an average value map indicating the mapping result. In this case, the statistical value calculation unit 102 outputs the average value map to the candidate land area average value calculation unit 107 instead of or together with the average value 114.

[0049] In this embodiment, the binarization mapping unit 104 outputs the binarization map 120 to the land area candidate extraction unit 106 rather than to the land area range determination unit 105 .

[0050] The land area candidate extraction unit 106 acquires the binarization map 120 from the binarization mapping unit 104 . Furthermore, the land area candidate extraction unit 106 extracts land area candidates from the binarization map 120. Land area candidates are candidates for land area ranges. The land area candidate extraction unit 106 extracts land area candidates in the same manner as the land area range determination unit 105 in the first embodiment determines the land area range. The land area candidate extraction unit 106 outputs the land area candidate extraction result 140 to the land area range determination unit 105 and the land area candidate average value calculation unit 107 . The extracted land area candidate is shown in the land area candidate extraction result 140. For example, the land area candidate extraction result 140 shows the numbers of pixels forming the contour line of the land area candidate.

[0051] The land area candidate average value calculation unit 107 acquires the average value 114 for each partial image area 111 from the statistical value calculation unit 102 . Furthermore, the land area candidate average value calculation unit 107 acquires the land area candidate extraction result 140 from the land area candidate extraction unit 106 . Then, the land area candidate average value calculation unit 107 calculates the land area candidate average value 150 from the average value 114 for each partial image region 111 and the land area candidate extraction result 140 . The land area candidate average value 150 is the average value of the brightness values ​​of pixels in the land area candidate shown in the land area candidate extraction result 140. When there are multiple land area candidates, the land area candidate average value calculation unit 107 calculates the land area candidate average value 150 for each land area candidate. The land area candidate average value calculation unit 107 outputs the land area candidate average value 150 to the land area range determination unit 105 .

[0052] In this embodiment, the land area range determination unit 105 acquires the land area candidate extraction result 140 from the land area candidate extraction unit 106. The land area range determination unit 105 also acquires the land area candidate average value 150 from the land area candidate average value calculation unit 107. The land area range determination unit 105 then determines whether the land area candidate average value 150 is equal to or greater than the average value threshold. The average value threshold is a threshold for the land area candidate average value 150. If the land area candidate average value 150 is equal to or greater than the average value threshold, the land area range determination unit 105 determines that the corresponding land area candidate is a land area range. Then, the land area determination unit 105 outputs the determination result 160. In the determination result 160, like the determination result 130, pixels forming the boundary line between the water area and the land area (the contour line of the land area) are shown.

[0053] The other components shown in FIG. 11 operate in the same manner as in the first embodiment.

[0054] The functions of the land area candidate extraction unit 106 and the land area candidate average value calculation unit 107 are also realized by a program, similar to the partial image region division unit 101 etc. The program realizing the functions of the land area candidate extraction unit 106 and the land area candidate average value calculation unit 107 is executed by the processor 901.

[0055] ***Explanation of Operation*** FIG. 12 shows an example of the operation of the image processing device 100 according to this embodiment. An example of the operation of the image processing device 100 according to this embodiment will be described below with reference to FIG.

[0056] Step S101 is the same as that described in the first embodiment, and therefore a description thereof will be omitted.

[0057] Next, in step S201, the statistical value calculation unit 102 calculates the average value 114 of the luminance values ​​of the pixels in each partial image region 111. Then, the statistical value calculation unit 102 outputs the average value 114 for each partial image region 111 to the land area candidate average value calculation unit 107 . In FIG. 12, step S201 is performed before step S102, but the order of these steps may be reversed.

[0058] Steps S102 to S104 are the same as those described in the first embodiment, and therefore a description thereof will be omitted.

[0059] In step S202, the land area candidate extraction unit 106 extracts land area candidates from the binarization map 120. The land area candidate extraction unit 106 extracts land area candidates in accordance with the procedure shown in FIG. The details of FIG. 13 will be described later. The land area candidate extraction unit 106 outputs the land area candidate extraction result 140 to the land area range determination unit 105 and the land area candidate average value calculation unit 107 . The land area candidate extraction result 140 indicates, for example, the numbers of pixels included in the land area candidate.

[0060] Next, in step S203, the land area candidate average value calculation unit 107 calculates the land area candidate average value 150 from the average value 114 for each partial image region 111 and the land area candidate extraction result 140. Specifically, the candidate land area average value calculation unit 107 selects the average value 114 of the partial image region 111 included in the candidate land area from the average values ​​114 obtained from the statistical value calculation unit 102. Then, the candidate land area average value calculation unit 107 calculates the average value of the selected average values ​​114 as the candidate land area average value 150. When there are multiple land area candidates, the land area candidate average value calculation unit 107 calculates the land area candidate average value 150 for each land area candidate. The land area candidate average value calculation unit 107 outputs the land area candidate average value 150 for each land area candidate to the land area range determination unit 105 .

[0061] Next, in step S204, the land area determination unit 105 determines the land area. Specifically, the land area range determination unit 105 determines whether the land area candidate average value 150 is equal to or greater than the average value threshold. If the land area candidate average value 150 is equal to or greater than the average value threshold, the land area range determination unit 105 determines that the land area candidate is a land area range. When there are multiple land area candidates, the land area range determination unit 105 compares the land area candidate average value 150 with the average value threshold for each land area candidate. Then, the land area determination unit 105 outputs the determination result 160.

[0062] FIG. 13 shows the details of step S202.

[0063] In step S2021, similarly to step S1051 in FIG. 4, the land area candidate average value calculation unit 107 extracts a range in the binarization map 120 where the value "1" is continuous as a continuous range.

[0064] Next, in step S2022, the land area candidate average value calculation unit 107 calculates the area of ​​each continuous region, similarly to step S1052.

[0065] Next, in step S2023, the land area candidate average value calculation unit 107 determines, for each continuous region, whether or not the area of ​​the continuous range is equal to or greater than the area threshold, similarly to step S1053. If the area of ​​the continuous range is equal to or greater than the area threshold, in step S2204, the land area determination unit 105 extracts the continuous range as a land area candidate.

[0066] Next, an example of the operation of the image processing device 100 according to this embodiment will be described using a specific example.

[0067] In this embodiment, it is also assumed that the binarized map 120 in FIG. 8 is obtained from the captured image 110 in FIG.

[0068] The land area candidate extraction unit 106 extracts a range in which the value "1" is continuous as a continuous range. At this stage, the continuous ranges corresponding to the three land areas in Figure 7 and the range corresponding to the surplus elements in the water area are extracted. Next, the land area candidate extraction unit 106 determines whether the area of ​​each continuous range is equal to or greater than an area threshold. Since the continuous range of each redundant element is less than the area threshold, the land area candidate extraction unit 106 does not extract the continuous range of each redundant element as a land area candidate. As a result, as shown in FIG. 14, the land area determination unit 105 extracts areas 146, 147, and 148 as land area candidates. FIG. 14 shows an example of a land area candidate extraction result map 145 corresponding to the land area candidate extraction result 140 of the land area candidate extraction unit 106. For example, the numbers of pixels forming the contour of the land area candidate are indicated in the land area candidate extraction result 140. The land area candidate extraction result map 145 is obtained by drawing the contour of the land area candidate using the pixel numbers indicated in the land area candidate extraction result 140. Here, a land area candidate extraction result map 145 is shown to make it easier to understand the operation of the land area candidate extraction unit 106. The land area candidate extraction unit 106 may or may not generate the land area candidate extraction result map 145. If the land area candidate extraction unit 106 generates the land area candidate extraction result map 145, the land area candidate extraction unit 106 may output the land area candidate extraction result map 145 to the land area candidate average value calculation unit 107 instead of the land area candidate extraction result 140 or together with the land area candidate extraction result 140.

[0069] The land area candidate average value calculation unit 107 obtains the average value 114 of each partial image area 111 included in the range 146 in FIG. 14 from the average values ​​114 of the multiple partial image areas 111. Then, the land area candidate average value calculation unit 107 calculates the average value of the obtained average values ​​114 of each partial image area 111 as the land area candidate average value 150. Similarly, the land area candidate average value calculation unit 107 obtains the average value 114 of each partial image area 111 included in the range 147 in FIG. 14 from the average values ​​114 of the multiple partial image areas 111. Then, the land area candidate average value calculation unit 107 calculates the average value of the obtained average values ​​114 of each partial image area 111 as the land area candidate average value 150. Similarly, the land area candidate average value calculation unit 107 obtains the average value 114 of each partial image area 111 included in the range 148 in FIG. 14 from the average values ​​114 of the multiple partial image areas 111. Then, the land area candidate average value calculation unit 107 calculates the average value of the obtained average values ​​114 of each partial image area 111 as the land area candidate average value 150. Then, the land area candidate average value calculation unit 107 outputs the land area candidate average value 150 for each of the ranges 146, 147, and 148 in FIG.

[0070] The land area range determination unit 105 determines whether or not the land area candidate average value of each of the ranges 146, 147, and 148 in FIG. 14 is equal to or greater than the average value threshold. Here, it is assumed that the land area candidate average value 150 of each of the ranges 146, 147, and 148 is equal to or greater than the average value threshold. For this reason, the land area range determination unit 105 determines each of the ranges 146, 147, and 148 as a land area range, as shown in Fig. 15. In Fig. 15, the range 146 in Fig. 14 is determined as a land area range 166. Furthermore, the range 147 in Fig. 14 is determined as a land area range 167. Furthermore, the range 148 in Fig. 14 is determined as a land area range 168. 15 shows an example of the determination result map 165. The determination result map 165 is a map corresponding to the determination result 160 of the land area range determination unit 105. The determination result 160 indicates, for example, the numbers of pixels forming the contour of the land area. The determination result map 165 is obtained by drawing the contour of the land area using the pixel numbers indicated in the determination result 160. Here, determination result map 165 is shown as an example to make it easier to understand the operation of land area range determination unit 105. Land area range determination unit 105 may or may not generate determination result map 165. If land area range determination unit 105 generates determination result map 165, land area range determination unit 105 may output determination result map 165 instead of determination result 160 or together with determination result 160.

[0071] In this embodiment, the land area range determination unit 105 uses the land area candidate average value 150 to obtain a more accurate contour line of the land area range. By using the land area candidate average value 150, which is the average of the land area candidate average values ​​114, it is possible to reduce erroneous determinations of land areas. Specifically, if the excess elements in the water area of ​​Figure 8 are close to each other, the land area range determination unit 105 may extract a continuous range that exceeds the area threshold. Alternatively, noise spreading over a wide area in the water area of ​​Figure 8 may become excess elements, causing the land area range determination unit 105 to extract a continuous range that exceeds the area threshold. The average value 114 and land area candidate average value 150 calculated from such an area are smaller than the average value threshold because they are not land areas. Therefore, the land area range determination unit determines that the area is not land, thereby reducing erroneous determinations.

[0072] ***Explanation of the effect of the embodiment*** This embodiment also makes it possible to identify land areas without using digital elevation data. Furthermore, this embodiment makes it possible to obtain a more accurate contour line of the land area range compared to the first embodiment. Therefore, a mask image can be more accurately superimposed on the image area of ​​the land area without putting a strain on the memory capacity on the aircraft. As a result, it is possible to more accurately determine on the aircraft whether or not a captured image contains an extra element such as a ship, without putting a strain on the memory capacity.

[0073] Embodiment 3, In this embodiment, differences from the second embodiment will be mainly described. The matters not explained below are the same as those in the first embodiment.

[0074] FIG. 16 shows an example of the functional configuration of the image processing device 100 according to this embodiment. In FIG. 16, compared to FIG. 1, a morphology processing unit 108 is added. Furthermore, in this embodiment, the operation of the land area range determination unit 105 differs from that in the second embodiment.

[0075] The land area determination unit 105 outputs the determination result 160 to the morphology processing unit 108. As explained in the second embodiment, the determination result 160 indicates pixels that form the boundary line between the water area and the land area (the contour line of the land area), as shown in FIG.

[0076] The morphology processing unit 108 acquires the determination result 160 from the land area range determination unit 105 . The morphology processing unit 108 performs morphology processing on the land area. In this embodiment, the morphology processing unit 108 performs opening processing and closing processing as the morphology processing. Note that the morphology processing unit 108 may perform only one of the opening processing and closing processing as the morphology processing. The morphology processing unit 108 may use any technique for the morphology processing, and may use a library such as OpenCV, for example. The morphology processing unit 108 outputs the determination result 160 after the morphology processing as a morphology determination result 170 . For example, the morphology processing unit 108 outputs the morphology determination result 170 to a mechanism (not shown in FIG. 16) that superimposes a mask image on the land area. The morphology determination result 170 indicates, for example, the numbers of pixels that form the contour line of the land area after morphology processing.

[0077] The other components shown in FIG. 16 operate in the same manner as in the second embodiment.

[0078] The function of the morphology processing unit 108 is also realized by a program, similar to the partial image region dividing unit 101 etc. The program that realizes the function of the morphology processing unit 108 is executed by the processor 901.

[0079] FIG. 17 shows an example of a morphology judgment result map 175, which is a map obtained after morphology processing is performed on the judgment result map 165 of FIG. In the determination result map 165 of Fig. 15, there are discontinuous portions (steps) in some of the contours of the land area. In contrast to this, in the morphology determination result map 175 of Fig. 17, the discontinuous portions (steps) have been eliminated by morphology processing. Furthermore, even if there is a partial area determined not to be land within the land area range of the determination result map 165, it is possible to create a morphology determination result map in which the partial area is determined to be a land area range by performing morphology processing. Range 176 in Figure 17 corresponds to range 166 in Figure 15. Range 177 in Figure 17 corresponds to range 167 in Figure 15. Range 178 in Figure 17 corresponds to range 168 in Figure 15. 15 shows an example of the determination result map 165. The determination result map 165 is a map corresponding to the determination result 160 of the land area range determination unit 105. The morphology determination result 170 indicates, for example, the numbers of pixels that form the contour of the land area after morphology processing. The morphology determination result map 175 is obtained by drawing the contour of the land area using the pixel numbers indicated in the morphology determination result 170. Here, a morphology determination result map 175 is illustrated to facilitate understanding of the operation of the morphology processing unit 108. The morphology processing unit 108 may or may not generate the morphology determination result map 175. If the morphology processing unit 108 generates the morphology determination result map 175, the morphology processing unit 108 may output the morphology determination result map 175 instead of or together with the morphology determination result 170.

[0080] Next, the evaluation result of the morphology determination result 170 according to this embodiment will be described. Below, the evaluation results obtained by comparing the determination results using digital elevation data (hereinafter referred to as target determination results) with the morphology determination results 170 will be described. The following digital elevation data was used: Geospatial Information Authority of Japan Digital Elevation Model: https: / / fgd.gsi.go.jp / download / menu.php Moreover, the target determination result and the morphology determination result 170 were obtained from the same captured image (11235 pixels).

[0081] Here, the comparison results are shown according to the following evaluation items A) to D). A) The land area in the target judgment result is judged as a land area in the morphology judgment result 170 (TP: True Positive) B) The sea area in the target judgment result is not judged as a land area in the morphology judgment result 170 (it is judged as a sea area) (TN: True Negative) C) The target area is judged as a land area by the morphology result 170 (FP: False Positive). D) The land area in the target judgment result is not judged as a land area in the morphology judgment result 170 (it is judged as a sea area) (FN: False Negative)

[0082] In addition, the evaluation items A) to D) above are evaluated using the following indices. Accuracy: The percentage of morphologically determined land and sea areas that were correctly identified in 170 cases Precision: The percentage of land areas that match the target land area among the 170 morphology judgment results. Recall: The percentage of land areas correctly identified as land by the morphology judgment result 170 among the land areas in the target judgment result.

[0083] The number of pixels that correspond to TP (the number of pixels that were determined to be land) is 1612 out of 11235 pixels. The number of pixels that fall under TN (the number of pixels that are determined to be ocean) is 9293 out of 11235 pixels. The number of pixels that correspond to FP (the number of pixels that are determined to be sea but not land) is 329 out of 11,235 pixels. The number of pixels that correspond to FN (the number of pixels that are determined to be land but not sea) is 1 out of 11,235 pixels.

[0084] Correct answer rate Accuracy=TP+TN / (TP+FN+FP+TN) = (1612 + 9293) / 11235 ≒ 0.971 (97.1%) Precision rate Precision=TP / (TP+FP) =1612 / (1612+329)≒0.831(83.1%) Recall rate Recall=TP / (TP+FN) =1612 / (1612+1)≒0.999(99.9%)

[0085] As such, the accuracy rate was high at approximately 97.1%, and the morphology judgment result 170 was able to accurately distinguish between land and sea areas. Because the recall is greater than the precision, the morphology determination result 170 determines a larger land area than the land area in the target determination result. However, the difference between the land area in the morphology determination result 170 and the land area in the target determination result is small. These evaluation results indicate that the morphology determination result 170 obtained by the image processing device 100 is suitable for practical use.

[0086] Although the first to third embodiments have been described above, these embodiments may be combined and implemented. Alternatively, one of these embodiments may be partially implemented. Alternatively, these embodiments may be partially combined and implemented. Furthermore, the configurations and procedures described in these embodiments may be modified as necessary.

[0087] ***Additional hardware configuration information*** Here, a supplementary explanation of the hardware configuration of the image processing device 100 will be given. The processor 901 shown in FIG. 2 is an integrated circuit (IC) that performs processing. The processor 901 is a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or the like. The main storage device 902 shown in FIG. 2 is a RAM (Random Access Memory). The auxiliary storage device 903 shown in FIG. 2 is a ROM (Read Only Memory), a flash memory, an HDD (Hard Disk Drive), or the like. The communication device 904 shown in FIG. 2 is an electronic circuit that performs data communication processing. The communication device 904 is, for example, a communication chip or a NIC (Network Interface Card).

[0088] The auxiliary storage device 903 also stores an OS (Operating System). At least a part of the OS is executed by the processor 901 . The processor 901 executes at least a part of the OS, while executing a program that realizes the functions of the functional components shown in FIGS. The processor 901 executes the OS, which performs task management, memory management, file management, communication control, and the like. In addition, at least one of information, data, signal values, and variable values ​​indicating the results of processing of the functional components shown in Figures 1, 11, and 16 is stored in at least one of the main memory device 902, the auxiliary memory device 903, and the register and cache memory within the processor 901. 1, 11, and 16 may be stored on a portable recording medium such as a magnetic disk, a flexible disk, an optical disk, a compact disk, a Blu-ray (registered trademark) disk, a DVD, etc. The portable recording medium storing the program for realizing the functions of the functional components shown in FIG. 1, 11, and 16 may be distributed.

[0089] Furthermore, the "part" of at least one of the functional components shown in FIGS. 1, 11 and 16 may be read as a "circuit" or a "step" or a "procedure" or a "process" or a "circuitry". The image processing device 100 may also be realized by a processing circuit, such as a logic integrated circuit (IC), a gate array (GA), an application specific integrated circuit (ASIC), or a field-programmable gate array (FPGA). In this case, the functional components shown in Figures 1, 11 and 16 are each implemented as part of a processing circuit. In this specification, the term "processing circuitry" refers to a generic concept that encompasses a processor and a processing circuit. That is, a processor and a processing circuit are each specific examples of "processing circuitry."

[0090] Various aspects of the present disclosure are summarized below as appendices. (Appendix 1) a statistical value calculation unit that calculates a statistical value of the brightness values ​​of pixels in each of a plurality of partial image regions divided from an image of a space including land and water regions captured from the sky; a binarization unit that binarizes the statistical value of the luminance values ​​of pixels in each partial image region using a statistical value threshold that is a threshold for the statistical value of the luminance values; and a binarization mapping unit that maps the binarization result for each partial image region by the binarization unit so that it corresponds to the position of each partial image region within the captured image. (Appendix 2) The binarization unit a first value, which is one of two values, is set to a partial image region for which a statistical value equal to or greater than the statistical value threshold is calculated, and a second value, which is the other of the two values, is set to a partial image region for which a statistical value less than the statistical value threshold is calculated; The image processing device further comprises: An image processing device as described in Appendix 1, further comprising a land area determination unit that extracts a range in which the first value is continuous in the mapping result by the binary mapping unit as a continuous range, determines whether the area of ​​the continuous range is equal to or greater than an area threshold that is a threshold for the area of ​​the continuous range, and if the area of ​​the continuous range is equal to or greater than the area threshold, determines the continuous range to be a land area range that is a range in the captured image that corresponds to the land area. (Appendix 3) The binarization unit a first value, which is one of two values, is set to a partial image region for which a statistical value equal to or greater than the statistical value threshold is calculated, and a second value, which is the other of the two values, is set to a partial image region for which a statistical value less than the statistical value threshold is calculated; The image processing device further comprises: An image processing device as described in Appendix 1, having a land area candidate extraction unit that extracts a range in which the first value is continuous in the mapping result by the binary mapping unit as a continuous range, determines whether the area of ​​the continuous range is equal to or greater than an area threshold that is a threshold for the area of ​​the continuous range, and if the area of ​​the continuous range is equal to or greater than the area threshold, extracts the continuous range as a land area candidate that is a candidate for a land area range that is a range in the captured image that corresponds to the land area. (Appendix 4) The image processing device further comprises: a land area candidate average value calculation unit that calculates an average value of luminance values ​​of pixels within the land area candidate; and a land area range determination unit that determines whether an average value of the luminance values ​​of pixels in the land area candidate is equal to or greater than an average value threshold that is a threshold for the average value of the luminance values ​​of pixels in the land area candidate, and determines that the land area candidate is the land area range if the average value of the luminance values ​​of pixels in the land area candidate is equal to or greater than the average value threshold. (Appendix 5) The image processing device further comprises: 5. The image processing device according to claim 4, further comprising a morphology processing unit that performs morphology processing on the land area determined by the land area determination unit. (Appendix 6) The statistical value calculation unit 2. The image processing device according to claim 1, wherein the statistical value is calculated to be one or more of standard deviation, variance, skewness, kurtosis, and moment. (Appendix 7) a statistical value calculation process in which a computer calculates a statistical value of the brightness values ​​of pixels in each of a plurality of partial image regions divided from an image of a space including a water area and a land area captured from the sky; a binarization process in which the computer binarizes the statistical value of the luminance values ​​of pixels in each partial image region using a statistical value threshold that is a threshold for the statistical value of the luminance values; and a binarization mapping process in which the computer maps the binarization results for each partial image area obtained by the binarization process so that the binarization results correspond to the position of each partial image area within the captured image. (Appendix 8) a statistical value calculation process for calculating a statistical value of the brightness values ​​of pixels in each of a plurality of partial image regions divided from an image of a space including water and land areas captured from the sky; a binarization process for binarizing the statistical value of the luminance values ​​of pixels in each partial image region using a statistical value threshold, which is a threshold for the statistical value of the luminance values; and a binarization mapping process for mapping the binarization results for each partial image area obtained by the binarization process in correspondence with the position of each partial image area within the captured image. [Explanation of symbols]

[0091] 100 Image processing device, 101 Partial image area division unit, 102 Statistical value calculation unit, 103 Binarization unit, 104 Binarization mapping unit, 105 Land area range determination unit, 106 Land area candidate extraction unit, 107 Land area candidate average value calculation unit, 108 Morphology processing unit, 110 Captured image, 111 Partial image area, 112 Statistical value, 113 Binarization result, 114 Average value, 120 Binarization map, 130 Determination result, 135 Determination result map, 140 Land area candidate extraction result, 145 Land area candidate extraction result map, 150 Land area candidate average value, 160 Determination result, 165 Determination result map, 170 Morphology determination result, Morphology determination result map, 175 Morphology determination result map, 901 Processor, 902 Main memory device, 903 Auxiliary memory device, 904 Communication device.

Claims

1. a statistical value calculation unit that calculates a statistical value of the brightness values ​​of pixels in each of a plurality of partial image regions divided from an image of a space including land and water regions captured from the sky; a binarization unit that binarizes the statistical value of the luminance values ​​of pixels in each partial image region using a statistical value threshold that is a threshold for the statistical value of the luminance values; an image processing device having a binarization mapping section that maps the binarization result for each partial image area by the binarization section so that it corresponds to the position of each partial image area within the captured image;

2. The binarization unit a first value, which is one of two values, is set to a partial image area for which a statistical value equal to or greater than the statistical value threshold is calculated, and a second value, which is the other of the two values, is set to a partial image area for which a statistical value less than the statistical value threshold is calculated; The image processing device further comprises:

2. The image processing device according to claim 1, further comprising a land area determination unit that extracts a range in which the first value is continuous in the mapping result by the binary mapping unit as a continuous range, determines whether the area of ​​the continuous range is equal to or greater than an area threshold that is a threshold for the area of ​​the continuous range, and, if the area of ​​the continuous range is equal to or greater than the area threshold, determines the continuous range to be a land area range that is a range in the captured image that corresponds to the land area.

3. The binarization unit a first value, which is one of two values, is set to a partial image area for which a statistical value equal to or greater than the statistical value threshold is calculated, and a second value, which is the other of the two values, is set to a partial image area for which a statistical value less than the statistical value threshold is calculated; The image processing device further comprises:

2. The image processing device according to claim 1, further comprising a land area candidate extraction unit that extracts a range in which the first value is continuous in the mapping result by the binarization mapping unit as a continuous range, determines whether the area of ​​the continuous range is equal to or greater than an area threshold that is a threshold for the area of ​​the continuous range, and, if the area of ​​the continuous range is equal to or greater than the area threshold, extracts the continuous range as a land area candidate that is a candidate for a land area range that is a range in the captured image that corresponds to the land area.

4. The image processing device further comprises: a land area candidate average value calculation unit that calculates an average value of luminance values ​​of pixels within the land area candidate; 4. The image processing device according to claim 3, further comprising a land area range determination unit that determines whether an average value of the luminance values ​​of pixels in the land area candidate is equal to or greater than an average value threshold that is a threshold for the average value of the luminance values ​​of pixels in the land area candidate, and determines the land area candidate to be the land area range if the average value of the luminance values ​​of pixels in the land area candidate is equal to or greater than the average value threshold.

5. The image processing device further comprises:

5. The image processing device according to claim 4, further comprising a morphology processing unit that performs morphology processing on the land area determined by the land area determination unit.

6. The statistical value calculation unit The image processing device according to claim 1 , wherein the statistical values ​​are calculated as one or more of standard deviation, variance, skewness, kurtosis, and moment.

7. a statistical value calculation process in which a computer calculates a statistical value of the brightness values ​​of pixels in each of a plurality of partial image regions divided from an image of a space including a water area and a land area captured from the sky; a binarization process in which the computer binarizes the statistical value of the luminance values ​​of pixels in each partial image region using a statistical value threshold that is a threshold for the statistical value of the luminance values; and a binarization mapping process in which the computer maps the binarization results for each partial image area obtained by the binarization process so that the binarization results correspond to the position of each partial image area within the captured image.

8. a statistical value calculation process for calculating a statistical value of the brightness values ​​of pixels in each of a plurality of partial image regions divided from an image of a space including water and land areas captured from the sky; a binarization process for binarizing the statistical value of the luminance values ​​of pixels in each partial image region using a statistical value threshold, which is a threshold for the statistical value of the luminance values; and a binarization mapping process for mapping the binarization results for each partial image area obtained by the binarization process in correspondence with the position of each partial image area within the captured image.

Images