Image processing method and image processing device

Abstract

An image processing method according to the present disclosure is an image processing method for separating and identifying a detection object (201) and a background (202) in an image (200) including position information and optical spectrum information about the detection object and the background, wherein an image is acquired, the standard deviation of the reflectance is calculated for each optical spectrum included in the image, the standard deviation is histogrammed, the threshold value of the standard deviation is determined from the obtained histogram, a region having a standard deviation equal to or larger than the threshold value is determined as the detection object, and a region having a standard deviation equal to or less than the threshold value is separated as the background.

Description

Image processing method and image processing device

[0001] The present disclosure relates to an image processing method and an image processing device, and more particularly to an image processing method and an image processing device for hyperspectral images.

[0002] Hyperspectral images store one line of spectral data per pixel, and by selecting the wavelength to use for rendering, it is possible to visualize the distribution of chemical components in water, oil, etc. In addition, analytical techniques that combine hyperspectral images with multivariate analysis and machine learning are being actively researched as methods for identifying chemical components and quantifying their concentrations.

[0003] Patent Document 1 discloses a search method in which the presence or absence of a foreign object is confirmed by performing principal component analysis of a captured image including position information and spectral information, and spectral information of the foreign object and background is identified, and a search target is identified by using a Partial Least Squares (PLS) regression analysis.

[0004] Japanese Patent Application Laid-Open No. 2018-132309

[0005] When capturing a hyperspectral image, if the entire object is captured, the background will be reflected in the image. Because the background is unrelated to the chemical composition of the object, it is necessary to remove the background pixels before performing multivariate analysis or machine learning.

[0006] However, hyperspectral images can have image file sizes exceeding 1 GB, and background removal using principal component analysis as disclosed in Patent Document 1 takes an enormous amount of time. Therefore, there is a need for an efficient method for removing background pixels.

[0007] The present disclosure is intended to solve the above-mentioned problems, and aims to provide an image processing method and an image processing device that efficiently remove background pixels.

[0008] The image processing method according to the present disclosure is an image processing method for separating and identifying a detection object and a background in an image containing positional information and spectral information of the detection object and the background, and includes the steps of: acquiring the image; calculating a standard deviation of reflectance for each spectral spectrum contained in the image; creating a histogram of the standard deviation; determining a threshold value for the standard deviation from the obtained histogram; and dividing the image into areas where the threshold value is equal to or greater than the detection object and areas where the threshold value is equal to or less than the background.

[0009] The spectrum used to calculate the standard deviation may include information on reflectance in the range of 950 to 1650 nm. After determining the threshold value of the standard deviation, the image may be subjected to binarization processing. The image may be a hyperspectral image including a plurality of pixels each storing a spectrum.

[0010] The image processing device according to the present disclosure is an image processing device for separating and identifying a detection object and a background in an image containing positional information and spectral information of the detection object and the background, and includes an acquisition unit that acquires the image, a calculation unit that calculates a standard deviation of reflectance for each spectral spectrum included in the image and creates a histogram of the standard deviation, a determination unit that determines a threshold value for the standard deviation from the histogram obtained by the calculation unit, and a processing unit that divides an area equal to or greater than the threshold value into the detection object and an area equal to or less than the threshold value into the background.

[0011] The present disclosure can provide an image processing method and an image processing device that efficiently remove background pixels.

[0012] 1 is a flowchart illustrating an image processing method according to the present disclosure; FIG. 2 is a diagram illustrating an image processing method according to the present disclosure; FIG. 3 is a histogram of standard deviation of reflectance calculated for each spectral spectrum included in an image in the image processing method according to the present disclosure; FIG. 4 is an image recognized as a background in the image processing method according to the present disclosure; FIG. 5 is an image detected as a detection target in the image processing method according to the present disclosure; FIG. 6 is a histogram of variance of reflectance calculated for each spectral spectrum included in an image in the image processing method according to the present disclosure; FIG. 7 is an image recognized as a background in the image processing method according to the present disclosure; FIG. 8 is an image detected as a detection target in the image processing method according to the present disclosure; FIG. 9 is a histogram of first principal component scores of reflectance calculated for each spectral spectrum included in an image in the image processing method according to the present disclosure; FIG. 10 is an image recognized as a background in the image processing method according to the present disclosure; FIG. 11 is an image detected as a detection target in the image processing method according to the present disclosure;

[0013] The image processing method and image processing device according to the present disclosure will be described below with reference to the drawings. Fig. 1 is a flowchart illustrating the image processing method according to the present disclosure, Figs. 2, 3A to 3C, 4A to 4C, and 5A to 5C are diagrams explaining the image processing method according to the present disclosure, and Fig. 6 is a block diagram illustrating the image processing device according to the present disclosure.

[0014] As shown in FIG. 6 , the image processing device 100 according to the present disclosure includes an acquisition unit 101 , a calculation unit 102 , a determination unit 103 , and a processing unit 104 .

[0015] In the image processing method according to the present disclosure, first, the acquisition unit 101 acquires an image 200 (see S11 in FIG. 1). The image 200 shown in the left diagram A of FIG. 2 is, for example, an image of a leaf pie placed on a piece of white paper, measuring 139 pixels wide by 224 pixels high (=31,136 pixels), and is a hyperspectral image including multiple pixels, each storing a spectral spectrum. The image 200 includes positional information and spectral information of the detection object 201 (leaf pie) and the background 202 (white paper), and is drawn using a reflectance of 1,300 nm. Therefore, the image 200 includes 31,136 pieces of spectral data.

[0016] The acquisition unit 101 transmits the spectral data of the acquired image 200 to the calculation unit 102. Thereafter, the calculation unit 102 calculates the standard deviation of the reflectance for each spectral component included in the image 200 (see S12 in FIG. 1 ) and creates a histogram of the standard deviation (see S13 in FIG. 1 ).

[0017] 2 shows the spectra of the detection object 201 and the background 202 at specific positions in the image 200, with the horizontal axis representing wavelength (nm) and the vertical axis representing reflectance. By calculating the standard deviation of the reflectance of the spectra, i.e., the degree of spread (variation) of each spectrum in the vertical axis direction, the detection object 201 and the background 202 can be accurately separated.

[0018] The spectrum shown in the right panel B of Figure 2 was acquired at 5 nm intervals from 950 to 1650 nm and therefore includes reflectance information for 141 points, but the wavelength range and intervals are not limited to this. Furthermore, the standard deviation can be calculated with reflectance information for three points, so it can also be applied to RGB images and RGB videos, for example.

[0019] The means and method for creating a histogram are not particularly limited, and for example, the number of histogram bars may be determined as the square root of the number of data (k = √n, where k is the number of histogram bars and n is the number of data, i.e., the number of pixels). In the above example, since 31,136 pieces of spectral data are included, it is recommended to create about 176 histograms.

[0020] The calculation unit 102 transmits the obtained histogram to the determination unit 103. Thereafter, the determination unit 103 determines a threshold value for the standard deviation (see S14 in FIG. 1) and performs binarization processing (see S15 in FIG. 1).

[0021] After the determination unit 103 determines the threshold and performs binarization processing, the processing unit 104 divides the image 200 into a detection object 201 and a background 202 (see S16 in FIG. 1). The background 202 has less irregularities in the shape of its spectral spectrum than the detection object 201, and therefore tends to have a smaller standard deviation. Therefore, as will be described later, by regarding an area with a relatively small standard deviation as the background 202, the detection object 201 and the background 202 in the image 200 can be divided.

[0022] The determination of the threshold, the binarization process, and the division into the detection object 201 and the background 202 will be described with reference to Figures 3A to 3C. Figure 3A is a histogram of the standard deviation of the reflectance calculated for each spectral spectrum included in the image 200 shown in Figure 2A on the left.

[0023] In the image processing method according to the present disclosure, it is preferable to determine the threshold value using Otsu's binarization method for the histogram. Otsu's binarization method assumes that the histogram forms two peaks, one corresponding to the detected object and the other corresponding to the background. The collection of the detected object and the background in the image is treated as a class, and the threshold value is set so as to maximize the degree of separation between the classes. Note that the determination of the threshold value is not limited to this, and other binarization methods or manual threshold setting may also be used. The Otsu's binarization method is described in detail in Otsu, N., "A Threshold Selection Method from Gray-Level Histograms," IEEE Transactions on Systems, Man, and Cybernetics, Vol. 9, No. 1, 1979, pp. 62-66.

[0024] In this embodiment, a threshold is set by Otsu's binarization method, and an area equal to or greater than the determined threshold is determined as the detection object 201, and an area equal to or less than the threshold is determined as the background 202. The binarization process is performed by replacing the reflectances of the background 202 and the detection object 201 with 0 and 1, but is not limited to this, and the reflectances may be replaced with NaN (Not a Number) and 1, or only the reflectance of the background 202 may be replaced with NaN.

[0025] Figure 3B shows the background 202, which is the area below the threshold (dotted line in Figure 3A) set by Otsu's binarization method, and Figure 3C shows the detected object 201, which is the area above the threshold. The degree of separation after binarization was 0.898. In this way, it is possible to efficiently and accurately remove background pixels.

[0026] 4A to 4C show binarization processing using variance instead of standard deviation, and division into the detection object 201 and the background 202. FIG. 4A is a histogram of the variance of reflectance calculated for each spectrum, FIG. 4B shows the background 202, which is an area where the variance is below a threshold (dotted line in FIG. 4A), and FIG. 4C shows the detection object 201, which is an area where the variance is above the threshold. The degree of separation after binarization processing was 0.878, and a portion of the leaf pie (the lower right part of FIG. 4B) was identified as part of the background 202. This shows that using standard deviation rather than variance allows for more accurate separation and identification of the background 202 and the detection object 201.

[0027] 5A to 5C show binarization processing and segmentation of the detection object 201 and the background 202 using the first principal component score instead of the standard deviation. FIG. 5A is a histogram of the first principal component scores of the reflectance calculated for each spectral spectrum. FIG. 5B shows the background 202, which is the region where the first principal component score is below the threshold (the dotted line in FIG. 5A ). FIG. 5C shows the detection object 201, which is the region where the first principal component score is above the threshold. The degree of separation after binarization processing was 0.792, but the entire leaf pie and the central hole were identified as the background 202, and the white paper was mistakenly identified as the detection object 201. This demonstrates that even when principal component analysis cannot correctly distinguish between the background and the detection object, the image processing method disclosed herein can accurately separate and identify the background 202 and the detection object 201, and efficiently remove background pixels.

[0028] Next, examples of applying the image processing method according to the present disclosure to various images will be described. As an example of an image, a single image can be obtained by combining multiple monochrome microscopic images captured at multiple different fluorescent wavelengths. For example, an image captured at a red fluorescent wavelength of approximately 610 nm, an image captured at a green fluorescent wavelength of approximately 550 nm, and an image captured at a blue fluorescent wavelength of approximately 465 nm (each of which is a monochrome image) can be combined to create a single color display image (RGB image). The image processing method according to the present disclosure can then be applied to the combined RGB image to create two images separated into a background and a detected object.

[0029] It is also possible to use multiband illumination to synthesize, in the same manner as above, eight monochrome images taken at eight different wavelengths to create a single multiband image, and then apply the image processing method disclosed herein to create two images separated into background and object.

[0030] In addition to the above, it is also possible to convert monochrome images such as X-ray CT images or thermographic thermal images into RGB images by applying pseudo-color display, and then apply the image processing method according to the present disclosure to create two images divided into a background and a detected object. Details will be explained using Figure 7, which shows a series of processes performed using a thermal image of a hand.

[0031] First, monochrome thermal image A is processed into image B of three wavelengths of RGB. By converting image A into an RGB image, it can be divided into image C of the red component, image D of the green component, and image E of the blue component. Next, a histogram F of the standard deviation of the RGB brightness values ​​contained in image B is created for image B, which is the combined RGB image. A threshold is determined from histogram F, and image G recognized as the background and image H detected as the detection target are obtained.

[0032] In this way, it can be seen that by applying the image processing method according to the present disclosure as described above to monochrome images as well, it is possible to detect the detection target with high accuracy.

[0033] The present disclosure is not limited to the above-described embodiments and may be modified as appropriate without departing from the spirit and scope of the present disclosure. In the image processing method according to the present disclosure, the standard deviation of the reflectance of the spectral spectrum is calculated, but the present disclosure is not limited to this. For example, instead of the standard deviation or variance of the reflectance of the spectral spectrum or the first principal component score, the interquartile range, the difference between the maximum and minimum values, or the like may be used. Furthermore, instead of the reflectance of the spectral spectrum, the absorbance or the luminance value of the RGB image may be used.

[0034] This application claims priority based on Japanese Patent Application No. 2023-139575, filed on August 30, 2023, the disclosure of which is incorporated herein in its entirety by reference.

[0035] REFERENCE SIGNS LIST 100 Image processing device 101 Acquisition unit 102 Calculation unit 103 Determination unit 104 Processing unit 200 Image 201 Detection object 202 Background

Claims

1. An image processing method for separating and identifying a detection object and a background in an image containing positional information and spectral information of the detection object and the background, comprising the steps of: acquiring the image; calculating a standard deviation of reflectance for each spectral spectrum contained in the image; creating a histogram of the standard deviation; determining a threshold for the standard deviation from the obtained histogram; and dividing the image into areas where the threshold is equal to or greater than the detection object and areas where the threshold is equal to or less than the background.

2. The image processing method according to claim 1, wherein the spectrum used to calculate the standard deviation includes information on reflectance in the range of 950 to 1650 nm.

3. The image processing method according to claim 1, further comprising the step of: performing a binarization process on the image after determining the standard deviation threshold.

4. The image processing method according to any one of claims 1 to 3, wherein the image is a hyperspectral image including a plurality of pixels each storing a spectroscopic spectrum.

5. The image processing method according to claim 4, wherein the image is a color display image obtained by synthesizing a plurality of monochrome images taken at a plurality of wavelengths, a multiband image obtained by synthesizing a plurality of monochrome images taken at a plurality of wavelengths, or a color display image obtained by pseudocolor displaying a single monochrome image.

6. An image processing device for separating and identifying a detection object and a background in an image containing positional information and spectral information of the detection object and the background, comprising: an acquisition unit that acquires the image; a calculation unit that calculates a standard deviation of reflectance for each spectral spectrum contained in the image and creates a histogram of the standard deviation; a determination unit that determines a threshold value of the standard deviation from the histogram obtained by the calculation unit; and a processing unit that divides an area equal to or greater than the threshold value into the detection object and an area equal to or less than the threshold value into the background.

Images