Pixel selection device of captured image and its program

Abstract

To efficiently select an interesting pixel from a captured image with high precision.SOLUTION: In a pixel selection device of a captured image having a processor, the captured image has spectrum data including strength values of electromagnetic waves in multiple bands that are larger than three primary colors for each pixel, and the processor acquires teacher spectrum data having spectrum data of first multiple preselected pixels, converts the teacher spectrum data to distribution data for each band, calculates similarity to teacher spectrum data of spectrum data of a test pixel on the basis of a similarity function with distribution information as a parameter for each band for second multiple test pixels of a search candidate of a captured image, sorts the second multiple test pixels by a similarity score that is an average value or a total value of the similarity of the entire band of the test pixels, and executes first processing for selecting k high-order test pixels (k: plural) of the similarity score.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a pixel selection device for a captured image and a program therefor. [Background technology]

[0002] Images of the surfaces of planets and the Earth are often used to search for various minerals and plants. For example, multispectral images of the surface of the moon taken with a multispectral camera or hyperspectral camera (hereafter abbreviated as multispectral camera) can be analyzed to mine minerals of interest. In the agricultural field, multispectral images of farmland can be analyzed to observe the growth status of crops, irrigation conditions, fertilizer distribution, etc. In the meteorological field, images of the ocean surface can be analyzed to analyze the El Niño phenomenon. In this way, multispectral images are used in the space, agriculture, meteorological, and other fields.

[0003] A multispectral image contains electromagnetic wave intensity data for each pixel in multiple wavelength bands (hereinafter referred to as bands) that are greater than the three primary colors of RGB. Here, such electromagnetic wave intensity data for multiple bands is referred to as a spectral signature or spectral data. The surface of a planet or the Earth has different electromagnetic wave reflection or transmission characteristics for each band depending on its condition. Therefore, to detect the location of a surface where a certain mineral is buried, the known spectral data of that mineral is used as training data. In other words, by finding pixels in the captured image data that have spectral data similar to the training data, the location of the mineral of interest can be detected.

[0004] Spectral data for multiple pixels is acquired from an image of a planet's surface, and a human visually determines whether the spectral data for a given pixel matches or is similar to the spectral data for the mineral of interest. Specifically, a pixel is randomly selected from the image, and a determination is made as to whether that pixel matches or is similar to the spectral data for the mineral of interest, and the same determination is then repeated for the pixels adjacent to the matching or similar pixel.

[0005] The following patent document discloses a method for classifying and determining time-series data, but does not disclose a method for selecting interesting pixels from a captured image. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Publication No. 2023-97910 Summary of the Invention [Problem to be solved by the invention]

[0007] The above selection method is performed manually by experts on the spectral data of a huge number of pixels, which consumes a huge amount of time and human energy. Furthermore, because it is a manual process, there is uncertainty in the judgment of whether the test pixel matches the spectral data of the mineral of interest, making it impossible to make an objective and accurate judgment.

[0008] Therefore, an object of a first aspect of the present embodiment is to provide a pixel selection device and a program therefor that can efficiently and accurately select pixels of interest from a captured image. [Means for solving the problem]

[0009] A first aspect of the present embodiment is a pixel selection device for a captured image, the pixel selection device having a processor and a memory accessed by the processor, the captured image having spectral data including intensity values ​​of electromagnetic waves in a plurality of bands that are greater than three primary colors for each pixel, the processor: acquiring teacher spectral data having spectral data of a first plurality of preselected pixels; converting the teacher spectral data into distribution data for each band, the distribution data having distribution information of a plurality of intensity values ​​for each band included in the spectral data of the plurality of pixels of the teacher spectral data; calculating, for each band, a similarity of the spectral data of the test pixels to the teacher spectral data for a second plurality of test pixels that are search candidates in the captured image, based on a similarity function that uses the distribution information as a parameter; The pixel selection device for a captured image performs either a first process of sorting the second plurality of test pixels by a similarity score, which is the average or total value of the similarity of all bands of the test pixels, and selecting the top k (k is plural) test pixels in terms of the similarity score, or a second process of selecting test pixels of the second plurality of test pixels whose similarity score is lower than a predetermined threshold. [Effects of the Invention]

[0010] According to the first aspect, it is possible to efficiently select pixels of interest from a captured image with high accuracy. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram showing an example of a pseudo image of a captured image of, for example, the surface of a planet or the Earth. [Figure 2] 10 is a diagram showing an example of a pseudo image of a captured image in which circled pixels selected by a pixel selection device are indicated; FIG. [Figure 3] 1 is a diagram illustrating an example of the configuration of a pixel selection device for a captured image according to an embodiment of the present invention. [Figure 4] FIG. 10 is a flowchart showing the steps of a pixel selection program (pixel selection method) for a first captured image in the present embodiment. [Figure 5] 1A and 1B are diagrams illustrating examples of teacher spectrum data, its distribution data, and test spectrum data. [Figure 6] FIG. 6 is a diagram showing an example of a graph of the distribution data of FIG. 5. [Figure 7] FIG. 10 is a diagram showing a graph of the spectral data of test pixels TPX_1 to TPX_3 superimposed on a graph of distribution data (broken line) of teacher spectral data. [Figure 8]FIG. 10 is a diagram showing a graph of the spectral data of test pixels TPX_1 to TPX_3 superimposed on a graph of distribution data (broken line) of teacher spectral data. [Figure 9] FIG. 10 is a diagram showing a graph of the spectral data of test pixels TPX_1 to TPX_3 superimposed on a graph of distribution data (broken line) of teacher spectral data. [Figure 10] FIG. 10 is a flowchart illustrating an example of a process S3 for converting teacher spectrum data into distribution data. [Figure 11] FIG. 10 is a flowchart showing specific processing of the process S4 for calculating the fuzzy similarity. [Figure 12] FIG. 10 is a graph showing a similarity function for calculating a fuzzy similarity FS_V. [Figure 13] FIG. 10 is a flowchart of a process for calculating a similarity function. [Figure 14] 5 is a flowchart showing a specific example of step S5 of the first pixel selection method shown in FIG. 4. FIG. [Figure 15] FIG. 10 is a flowchart showing the steps of a pixel selection program (pixel selection method) for a second captured image in the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] [First embodiment] FIG. 1 is a diagram showing an example of a pseudo image of a captured image of, for example, the surface of a planet or the Earth. The captured image IMG in FIG. 1 has, as an example, pixels PX0,0 to PXn-1,n-1 arranged in n rows and n columns. In the example of FIG. 1, n=12. The captured image data is raster data and has multiple pixel data arranged in a matrix. Each pixel data has intensity data of electromagnetic waves in multiple bands that are greater than the three primary colors. If the number of bands is 10, then each pixel data is spectral data having intensity data for 10 bands. For simplicity, the captured image will be referred to as an image below.

[0013] In the image IMG, a first area AREA_1, a second area AREA_2, and a third area AREA_3, each of which is a different color from the background, are shown by solid and dashed lines in a pseudo manner. The second and third areas AREA_2 and AREA_3 are assumed to be of the same color.

[0014] In this embodiment, an expert in the technical field to be analyzed visually inspects the image IMG and selects a predetermined number of pixels that he or she is interested in. It is assumed that the five pixels selected by the expert are those indicated by stars in the figure.

[0015] The pixel selection device in this embodiment then selects pixels having spectral data similar to the spectral data of the selected five pixels from the 139 pixels remaining after excluding the five pre-selected pixels from the 12 x 12 = 144 pixels in the image IMG.

[0016] 2 is a diagram showing an example of a pseudo image of a captured image in which circled pixels selected by a pixel selection device are shown, in the example of FIG. 2, 10 circled pixels are selected.

[0017] In the first selection method, the top k pixels in terms of similarity are selected, where k is a number determined appropriately by an expert. Alternatively, in the second method, pixels in terms of similarity equal to or greater than a predetermined threshold are selected, where the threshold is a value determined appropriately by an expert. In the second method, it is unpredictable or difficult to predict how many pixels will be selected. k or the threshold may be selected by an expert, for example, by trial and error. The detailed selection method will be described later.

[0018] 3 is a diagram showing an example of the configuration of a pixel selection device for a captured image according to this embodiment. The pixel selection device 1 has a processor 10, a main memory 12, an interface 14, storages 20 to 26, and a bus 28 connecting them. The storage stores a pixel selection program 20 for the captured image, teacher spectral data 22, distribution data 24 of the teacher spectral data, and spectral data of test pixels (test spectral data) 26. The pixel selection program 20 for the captured image is loaded into the main memory 12, and the processor 10 executes the loaded program to perform pixel selection processing for the captured image.

[0019] The pixel selection device 1 is communicatively connected to a plurality of client terminal devices 30, 32 via a network NW such as the Internet. The client terminal devices 30, 32 connect to the pixel selection device 1 and request pixel selection within an image to be analyzed. In response to this request, the processor 10 executes a pixel selection program 20 for the captured image and receives image data to be analyzed and data for the selected plurality of pixels from the client terminal devices 30, 32. The processor 10 then executes the pixel selection program 20 to select pixels of interest to the client from the image data to be analyzed.

[0020] 4 is a flowchart showing the steps of a pixel selection program (pixel selection method) for the first captured image in this embodiment. A processor of the pixel selection device executes pixel selection program 20 to perform the pixel selection steps shown in FIG.

[0021] Assume that an image to be analyzed has already been acquired. As mentioned above, image data has pixel data for multiple pixels in the image. The pixel data is then spectral data having intensity values ​​SV of electromagnetic waves in multiple bands from the location of the pixel. The multiple bands are, for example, bands greater than the three primary colors red, green, and blue. If the number of bands is 10, the spectral data for the pixel will be as follows: Spectral data: SV_0, SV_1, SV_2, SV_3, SV_4, SV_5, SV_6, SV_7, SV_8, SV_9 Here, SV_0 to SV_9 are the intensity values ​​of the electromagnetic waves in each band.

[0022] In the pixel selection method, a processor inputs a plurality of (p) pixels of interest selected by an expert to be analyzed from an image by visual inspection or the like (S1). The processor then acquires the spectral data TC_SPDB of the selected p pixels and stores the spectral data of the selected p pixels in storage as teacher spectral data 22 (S2).

[0023] FIG. 5 shows an example of teacher spectral data, its distribution data, and test spectral data. The teacher spectral data TC_SPDB is the spectral data for each of p selected pixels. As described above, the spectral data is intensity value data SV_0 to SV_9 for multiple bands of electromagnetic waves from the pixel locations. In the example of FIG. 5, the number of selected pixels is p=5, and the number of bands is 10. The intensity values ​​SV_0 to SV_9 shown in FIG. 5 are just an example. FIG. 5 shows the intensity values ​​of 10 bands for each of pixels PX_1 to PX_5.

[0024] Next, in the pixel selection method shown in FIG. 4, the processor converts the p pieces of teacher spectral data into distribution data DST_SPDB for each of the bands BND_0 to BND_9 (S3).

[0025] 10 is a flowchart showing an example of the process S3 for converting teacher spectral data into distribution data. After acquiring p teacher spectral data TC_SPDB (S2), the processor detects or calculates the minimum value min, maximum value max, and mean value mean of the intensity values ​​SV_# (#=0 to 9) for each band and stores them as distribution data (S31 to S33). Since there are p teacher spectral data TC_SPDBs, each band contains p electromagnetic wave intensity values.

[0026] The processor detects the minimum value (min) and maximum value (max) of each band from the p intensity values ​​and calculates the mean value (mean) from the p intensity values. The mean value (mean) may be a representative value such as the median ((minimum value + maximum value) / 2) or the median (50th percentile value) of the p spectral data.

[0027] 5 has a minimum value min, a maximum value max, and a mean value mean for each of the bands BND_0 to BND_9. The distribution data may be any value that represents the distribution of intensity values ​​in each band, and the minimum, maximum, and mean values ​​(or a representative value including the mean or median) represent the simplest distribution.

[0028] The distribution data may include the 25th percentile, 50th percentile, and 75th percentile in addition to the minimum, maximum, and representative value. Alternatively, instead of the maximum value, a first value less than or equal to the maximum value and greater than the representative value (e.g., the 75th percentile) may be used, and instead of the minimum value, a second value greater than or equal to the minimum value and less than the representative value (e.g., the 25th percentile). In the following example, the distribution data will be described using the minimum, maximum, and average values.

[0029] Figure 6 is a diagram showing an example graph of the distribution data of Figure 5. The horizontal axis represents bands BND_0 to BND_9, and the vertical axis represents intensity values. The mean value (mean) is shown with a solid line, and the minimum value (min) and maximum value (max) are shown with dashed lines. This graph of distribution data makes it possible to visually grasp the distribution of spectral data for a pixel of interest.

[0030] Next, in the pixel selection method shown in Fig. 4, the processor calculates, for each band of multiple test pixels in the image, the similarity of the test spectral data to p teacher spectral data based on a similarity function that uses distribution data (including at least the minimum value, maximum value, and average value (or representative values ​​including the mean value and median)) as parameters (S4). As will be described later, the similarity function uses distribution data as parameters and can calculate fuzzy similarity that is not based on the distance from a single representative value such as the mean value, median, or median of the distribution data. In this sense, the similarity function of this embodiment can calculate fuzzy similarity.

[0031] Furthermore, the processor calculates the average value of the fuzzy similarities across all bands for each test pixel (S5). This average value of the fuzzy similarities across all bands is the fuzzy similarity score. Note that instead of the average value, the sum of the fuzzy similarities across all bands may be calculated as the fuzzy similarity score. The processor then sorts the fuzzy similarity scores of all test pixels and extracts the test pixels with the top k fuzzy similarity scores (S5).

[0032] The test spectrum data TST_SPDB shown in Figure 5 is an example of the spectrum data for three test pixels TPX_1 to TPX_3. The spectrum data for each test pixel has electromagnetic wave intensity values ​​SV_0 to SV_9 for each band. Below, we will explain an example of a pixel selection method in which the test spectrum data for these three test pixels is calculated to have similarity scores relative to the teacher spectrum data, and the top k pixels (k=1 in the following example) are extracted by sorting by similarity score.

[0033] 11 is a flowchart showing the specific steps of step S4 for calculating fuzzy similarities. In step S4, the processor acquires a set of test spectrum data TST_SPDB (S41). Then, for each test spectrum data TST_SPDB (S42, S46), the processor calculates a fuzzy similarity FS_V for each band intensity value based on a similarity function using distribution data as a parameter (S44) for the intensity value of each band of the test spectrum data (S43, S45). The average or total value of the fuzzy similarities FS_V for the nine bands is the similarity score.

[0034] FIG. 12 is a graph showing a similarity function for calculating the fuzzy similarity FS_V. FIG. 13 is a flowchart showing the calculation process of the similarity function. The fuzzy similarity FS_V is calculated based on a similarity function that uses distribution data (minimum value, maximum value, and average value (or a representative value including a median or intermediate value)) of the teacher spectral data as parameters. The fuzzy similarity FS_V is normalized to a value between 0 and 1. As shown in FIG. 13, the fuzzy similarity FS_V becomes FS_V_a, FS_V_b, FS_V_c, FS_V_d, or FS_V_e (S62, S64, S66, S68, S70) depending on the range of the intensity value SV_# (# is the band) (S61, S63, S65, S67, S69).

[0035] Specifically, the fuzzy similarity FS_V depends on the intensity value SV_# of the test spectrum data and takes the following values: Strength value SV_# <min: FS_V_a= 1 min≦Intensity value SV_# <mean:FS_V_b = 0.5*A Intensity value SV_# = mean: FS_V_c = 0 0< Strength value SV_# ≦ max: FS_V_d = 0.5*B max < intensity value SV_#: FS_V_e = 1 Here, A and B are interpolated values ​​calculated by the following interpolation calculation based on the graph of dashed lines A and B shown in FIG. A = (mean - SV_#) / (mean - min) B = (SV_# -mean) / (max - mean) According to the similarity function for calculating the fuzzy similarity FS_V described above, when the intensity value SP_V of the spectral data of the test pixel is equal to the mean value mean of the distribution data, the fuzzy similarity FS_V will be the minimum value (most similar), 0. When it is less than the minimum value min or greater than the maximum value max of the distribution data, the fuzzy similarity FS_V will be the maximum value (least similar), 1. When the intensity value SP_V of the spectral data of the test is equal to or greater than the minimum value min and less than the mean value mean, or greater than the mean value mean and less than the maximum value max, the fuzzy similarity FS_V will be half the interpolated value based on the minimum value min, the mean value mean, the maximum value max, and the intensity value SP_V of the spectral data.

[0036] In this way, the fuzzy similarity FS_V is a similarity that corresponds to the distribution of the intensity values ​​SP_V for each band, so that the similarity of the test pixel can be made a fuzzy similarity, which is a more desirable similarity.

[0037] Furthermore, in FIG. 4, the processor sorts the test pixels by the average similarity (similarity score FS_SC) across all bands, and extracts the top k test pixels (S5).

[0038] Fig. 14 is a flowchart showing a specific example of step S5 of the first pixel selection method shown in Fig. 4. First, the processor sorts a plurality of test pixels by fuzzy similarity score FS_SC (S50).

[0039] Next, the processor inputs k, which has been arbitrarily set by an expert (S51). In other words, process S51 is a process performed by a human expert. An expert in the technical field who searches for an interesting pixel in an image may arbitrarily set k without studying the similarity score of the spectral data of the interesting pixel.

[0040] Next, the processor extracts the top k test images based on the fuzzy similarity score, i.e., the test images with the top k fuzzy similarities (S52). These test images with the top k fuzzy similarities are considered to contain candidates for so-called champion data.

[0041] Next, the processor inputs the result of determining whether the test pixel with the worst fuzzy similarity score (specifically, the lowest one) among the top k extracted test pixels is appropriate (S53). In other words, process S53 is also essentially a process performed by a human expert. This determination process only needs to be performed for a relatively small number of test pixels, so it is a relatively light process that can be performed by a human.

[0042] If the determination result is appropriate (YES in S54), the processor increases k (S55). On the other hand, if the determination result is inappropriate (NO in S54), the processor decreases k (S56). The processor then repeats steps S52 and S53 again. When the number of repetitions reaches a predetermined number or the expert determines that k is an appropriate value (not shown in FIG. 14), the process S5 by the processor ends.

[0043] The test spectrum data table TST_SPDB in Figure 5 shows examples of the spectrum data of three test pixels, the fuzzy similarity score FS_SC, and an example of a ranking rank in descending order of fuzzy similarity score. According to this, the test pixel TPX_3 has the best (lowest) fuzzy similarity score and is ranked first.

[0044] 7, 8, and 9 are graphs showing the spectral data of the test pixels TPX_1 to TPX_3 superimposed on a graph of the distribution data (dashed line) of the teacher spectral data. Comparing the three figures, it can be seen that the intensity values ​​of the spectral data in each band of the test pixel TPX_3 shown in FIG. 7 are closer to the mean value of the distribution data than the other test pixels. It can also be seen that the intensity values ​​of the spectral data in each band of the test pixel TPX_3 are between the maximum and minimum values ​​of the distribution data across all bands.

[0045] [Second embodiment] 15 is a flowchart showing the steps of a pixel selection program (pixel selection method) for a second captured image in the second embodiment. The processor of the pixel selection device executes the pixel selection program 20 to perform the pixel selection steps shown in FIG.

[0046] In the second pixel selection method, the prerequisites and steps S1 to S4 are the same as those of the first pixel selection method of Fig. 4. In the second pixel selection method, the processor calculates the average value of the fuzzy similarities across all bands for each test pixel (S5_1). This average value of the fuzzy similarities across all bands is the fuzzy similarity score. As described above, instead of the average value, the sum of the fuzzy similarities across all bands may be calculated as the fuzzy similarity score.

[0047] Then, the processor extracts pixels whose fuzzy similarity scores are lower than a predetermined threshold (pixels whose fuzzy similarity scores are better than the predetermined threshold) from all test pixels (S5_1). Before executing this process S5_1, an expert who is trying to extract interesting pixels from an image needs to research and set the predetermined threshold.

[0048] An expert may then visually determine whether the selected test pixels are suitable, and if they are not suitable, the threshold may be changed to a smaller value, and the processor may again select test pixels having fuzzy similarity scores smaller than the changed threshold.

[0049] In the first pixel selection method of the first embodiment described above, fuzzy similarity scores are calculated for all test pixels and the top k test pixels are selected. k may be set to any number. Meanwhile, the reflectance of planetary surfaces varies due to various factors, including sunlight. Therefore, the intensity of the spectral data in the captured image fluctuates depending on the timing of image capture by the multispectral camera. For these reasons, the second pixel selection method, which involves setting a predetermined threshold to select pixels desired by experts from the test pixels, may not be desirable. Furthermore, finding the optimal threshold may be a challenging research topic, which may result in delays in pixel selection for a given image.

[0050] In contrast, the first pixel selection method sets k to an arbitrary number and selects the pixels with the top k fuzzy similarity scores. The top k pixels always contain champion data. Therefore, by checking the test pixels selected by the first pixel selection method and then searching for k through trial and error, it may be possible to select the optimal test pixel in a shorter time. Furthermore, since the first pixel selection method can be performed without searching for a predetermined threshold, it can be started immediately. Furthermore, the process of sorting multiple test pixels by their fuzzy similarity scores and selecting the top k pixels requires a relatively small amount of computation. The process of comparing all test pixels with a predetermined threshold in the second pixel selection method generally requires a large amount of computation.

[0051] As described above, according to this embodiment, it is possible to efficiently select pixels that are deemed desirable from an image of the planet's surface. In addition to images of the planet's surface, this embodiment can be applied to, for example, analyzing multispectral images of cultivated land in the agricultural field to observe the growth state of crops, irrigation status, fertilizer distribution, etc., and analyzing images of the ocean surface in the meteorological field to analyze the El Niño phenomenon. [Explanation of symbols]

[0052] IMG: Captured image, image PX: pixel AREA:Area 10: Processor 12: Main memory 20: Pixel selection program for captured images 22. TC_SPDB: Supervised spectral data 24. DST_SPDB: Distribution data of teacher spectral data 26. TST_SPDB: Spectral data of test pixels

Claims

1. A pixel selection device for a captured image, comprising a processor and a memory accessed by the processor, the captured image has, for each pixel, spectral data including intensity values ​​of electromagnetic waves in a plurality of bands that are greater than the three primary colors; The processor: acquiring teacher spectral data having spectral data of a first plurality of preselected pixels; converting the teacher spectral data into distribution data for each band, the distribution data having distribution information of a plurality of intensity values ​​for each band included in the spectral data of the plurality of pixels of the teacher spectral data; calculating, for each band, a similarity of the spectral data of the test pixels to the teacher spectral data for a second plurality of test pixels that are search candidates in the captured image, based on a similarity function that uses the distribution information as a parameter; a pixel selection device for a captured image that performs either a first process of sorting the second plurality of test pixels by a similarity score, which is the average or total value of the similarity of all bands of the test pixels, and selecting the top k (k is plural) test pixels in terms of the similarity score, or a second process of selecting test pixels of the second plurality of test pixels whose similarity score is lower than a predetermined threshold value.

2. the distribution information has a representative value including at least an average value, a median value, or a median value of a plurality of intensity values ​​for each band included in the spectral data of the first plurality of pixels of the teacher spectral data, a first value that is greater than the representative value and is equal to or less than a maximum value, and a second value that is smaller than the representative value and is equal to or greater than a minimum value, The pixel selection device for a captured image according to claim 1 , wherein the similarity function is a function that uses the first value, the second value, and the representative value as parameters.

3. 3. The pixel selection device for a captured image according to claim 2, wherein the similarity function outputs a first similarity if the intensity value of each band of the spectral data of the test pixel is greater than a first value or less than a second value of the distribution data, outputs a minimum similarity smaller than the first similarity if the intensity value matches the representative value, and outputs an interpolated value between the minimum similarity and the first similarity if the intensity value is between the representative value and the first value or between the representative value and the second value.

4. 2. The pixel selection device for a captured image according to claim 1, wherein when the first process is performed, if the top k selected test pixels are appropriate, k is increased, and if the top k selected test pixels are not appropriate, k is decreased, and then the first process is repeated.

5. The pixel selection device for a captured image according to claim 1 , wherein, when the second process is performed, if the selected test pixel is not appropriate, the threshold is changed to a smaller value, and then the second process is repeated.

6. 2. The pixel selection device for a captured image according to claim 1, wherein the captured image is an image of a planet, the surface of the Earth, the surface of cultivated land, or the surface of the ocean captured by a multispectral camera.

7. A pixel selection program that causes a computer to execute pixel selection processing for a captured image, the captured image has, for each pixel, spectral data including intensity values ​​of electromagnetic waves in a plurality of bands that are greater than the three primary colors; The pixel selection process includes: acquiring teacher spectral data having spectral data of a first plurality of preselected pixels; converting the teacher spectral data into distribution data for each band, the distribution data having distribution information of a plurality of intensity values ​​for each band included in the spectral data of the plurality of pixels of the teacher spectral data; calculating, for each band, a similarity of the spectral data of the test pixels to the teacher spectral data for a second plurality of test pixels that are search candidates in the captured image, based on a similarity function that uses the distribution information as a parameter; a pixel selection program that executes either a first process of sorting the second plurality of test pixels by a similarity score, which is the average or total value of the similarity of all bands of the test pixels, and selecting the top k test pixels (k is plural) in terms of the similarity score, or a second process of selecting test pixels of the second plurality of test pixels whose similarity score is lower than a predetermined threshold value.

Images