Image processing method and image processing device

Abstract

The image processing method according to the present disclosure is an image processing method for separating and identifying a detection object (201) and a background (202) in an image (200) including position information and spectral information of the detection object and the background, which acquires an image, calculates a standard deviation of reflectance for each spectral spectrum included in the image, creates a histogram of the standard deviation, determines a threshold value for the standard deviation from the obtained histogram, and divides the image into areas equal to or greater than the threshold value as the detection object and areas equal to or less than the threshold value as the background.

Description

[Technical field]

[0001] The present disclosure relates to an image processing method and an image processing device, and more particularly to an image processing method and an image processing device for hyperspectral images. [Background technology]

[0002] In a hyperspectral image, one line of spectral data is stored per pixel, and by selecting the wavelength to be used for drawing, it is possible to visualize the distribution of chemical components in water, oil, etc. In addition, analytical techniques that combine hyperspectral images with multivariate analysis and machine learning are being actively researched as a method for identifying chemical components and quantifying their concentrations.

[0003] Patent Document 1 discloses a search method in which the presence or absence of a foreign object is confirmed by performing a principal component analysis of a captured image including position information and spectral information, and the spectral information of the foreign object and the background is identified, and a PLS (Partial Least Squares) regression analysis is used to identify the object to be searched for. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2018-132309 A Summary of the Invention [Problem to be solved by the invention]

[0005] When capturing a hyperspectral image, if the entire object is captured, the background is reflected in the image. Because the background is unrelated to the chemical composition of the object, it is necessary to remove the background pixels before performing multivariate analysis or machine learning.

[0006] However, the image file size of hyperspectral images can exceed 1 GB, and the background removal using principal component analysis disclosed in Patent Document 1 takes an enormous amount of time, so there is a demand for an efficient method of removing background pixels.

[0007] The present disclosure is devised to solve the above-mentioned problems, and has an object to provide an image processing method and an image processing device that efficiently remove background pixels. [Means for solving the problem]

[0008] The image processing method according to the present disclosure is an image processing method for separating and identifying a detection object and a background in an image including positional information and spectral information of the detection object and the background, the method comprising: acquiring the image; calculating a standard deviation of reflectance for each spectral spectrum included in the image; converting the standard deviation into a histogram; determining a threshold value for the standard deviation from the obtained histogram; and dividing the image into areas where the threshold value is equal to or greater than the detection object and areas where the threshold value is equal to or less than the background.

[0009] The spectrum used to calculate the standard deviation may include information on reflectance at 950 to 1650 nm. After determining a threshold value for the standard deviation, a binarization process may be performed on the image. The image may be a hyperspectral image including a plurality of pixels each storing a spectrum.

[0010] The image processing device according to the present disclosure is an image processing device for separating and identifying a detection object and a background in an image including positional information and spectral information of the detection object and the background, and includes an acquisition unit that acquires the image, a calculation unit that calculates a standard deviation of reflectance for each spectral spectrum included in the image and creates a histogram of the standard deviation, a determination unit that determines a threshold value of the standard deviation from the histogram obtained by the calculation unit, and a processing unit that divides an area equal to or greater than the threshold value into the detection object and an area equal to or less than the threshold value into the background. Effect of the Invention

[0011] The present disclosure makes it possible to provide an image processing method and an image processing device that efficiently remove background pixels. [Brief description of the drawings]

[0012] [Figure 1] 4 is a flowchart illustrating an image processing method according to the present disclosure. [Diagram 2] FIG. 1 is a diagram illustrating an image processing method according to the present disclosure. [Figure 3A] 13 is a histogram of standard deviations of reflectances calculated for each spectrum included in an image in the image processing method according to the present disclosure. [Figure 3B] 1 is an image that is recognized as a background in the image processing method according to the present disclosure. [Figure 3C] 1 is an image in which an object is detected as a detection target in an image processing method according to the present disclosure. [Figure 4A] 13 is a histogram of the variance of reflectance calculated for each spectrum included in an image in the image processing method according to the present disclosure. [Figure 4B] 1 is an image that is recognized as a background in the image processing method according to the present disclosure. [Figure 4C] 1 is an image in which an object is detected as a detection target in an image processing method according to the present disclosure. [Figure 5A] 13 is a histogram of first principal component scores of reflectance calculated for each spectral spectrum included in an image in the image processing method according to the present disclosure. [Figure 5B] 1 is an image that is recognized as a background in the image processing method according to the present disclosure. [Figure 5C] 1 is an image in which an object is detected as a detection target in an image processing method according to the present disclosure. [Figure 6] FIG. 1 is a block diagram illustrating an image processing device according to the present disclosure. [Figure 7] FIG. 1 is a diagram showing an example in which the image processing method according to the present disclosure is applied to a thermal image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] The image processing method and image processing device according to the present disclosure will be described below with reference to the drawings. Fig. 1 is a flowchart showing the image processing method according to the present disclosure, Fig. 2, Fig. 3A to Fig. 3C, Fig. 4A to Fig. 4C, Fig. 5A to Fig. 5C are diagrams explaining the image processing method according to the present disclosure, and Fig. 6 is a block diagram showing the image processing device according to the present disclosure.

[0014] As shown in FIG. 6, an image processing device 100 according to the present disclosure includes an acquisition unit 101, a calculation unit 102, a determination unit 103, and a processing unit 104.

[0015] In the image processing method according to the present disclosure, first, the acquisition unit 101 acquires an image 200 (see S11 in FIG. 1). The image 200 shown in the left diagram A of FIG. 2 is, for example, an image of a leaf pie placed on a white paper, with a horizontal size of 139 pixels and a vertical size of 224 pixels, or 31136 pixels, and is a hyperspectral image including a plurality of pixels each storing a spectral spectrum. The image 200 includes position information and spectral information of the detection object 201 (leaf pie) and the background 202 (white paper), and is drawn with a reflectance of 1300 nm. Therefore, the image 200 includes 31136 pieces of spectral data.

[0016] The acquisition unit 101 transmits the spectral data of the acquired image 200 to the calculation unit 102. Thereafter, the calculation unit 102 calculates the standard deviation of the reflectance for each spectral component included in the image 200 (see S12 in FIG. 1), and creates a histogram of the standard deviation (see S13 in FIG. 1).

[0017] 2B shows the spectra at specific positions of the detection object 201 and the background 202 in the image 200, with the horizontal axis representing wavelength (nm) and the vertical axis representing reflectance. By calculating the standard deviation of the reflectance of the spectra, that is, the degree of spread (variation) of each spectrum in the vertical axis direction, the detection object 201 and the background 202 can be accurately divided.

[0018] The spectrum shown in the right diagram B of Fig. 2 was acquired at 5 nm intervals from 950 to 1650 nm, and therefore includes reflectance information at 141 points, but the wavelength range and intervals are not limited to this. Also, the standard deviation can be calculated with reflectance information at three points, so it is applicable to, for example, RGB images and RGB videos.

[0019] The means and method for making a histogram are not particularly limited, and for example, the number of histogram bars may be determined as the square root of the number of data (k=√n, k: number of histogram bars, n: number of data, i.e., number of pixels). In the above example, since 31136 pieces of spectral data are included, it is recommended to create about 176 histograms.

[0020] The calculation unit 102 transmits the obtained histogram to the determination unit 103. Thereafter, the determination unit 103 determines a threshold value for the standard deviation (see S14 in FIG. 1), and performs binarization processing (see S15 in FIG. 1).

[0021] After the determination unit 103 determines the threshold and performs binarization processing, the processing unit 104 divides the image 200 into a detection object 201 and a background 202 (see S16 in FIG. 1). The background 202 has less irregularities in the shape of the spectral spectrum compared to the detection object 201, and therefore tends to have a smaller standard deviation. Therefore, as described below, by regarding an area with a relatively small standard deviation as the background 202, it is possible to divide the detection object 201 and the background 202 in the image 200.

[0022] The determination of the threshold, the binarization process, and the division of the detection object 201 and the background 202 will be described with reference to Fig. 3A to C. Fig. 3A is a histogram of the standard deviation of the reflectance calculated for each spectrum included in the image 200 shown in the left diagram A of Fig. 2.

[0023] In the image processing method according to the present disclosure, it is preferable to determine the threshold value using Otsu's binarization method for the histogram. Otsu's binarization method assumes that the histogram creates two peaks corresponding to the detected object and the background, respectively, and sets the threshold value so that the degree of separation between the classes is maximized by treating the collection of the detected object and the background in the image as classes. Note that the determination of the threshold value is not limited to this, and other binarization methods or the threshold value may be set manually. Otsu's binarization method is described in detail in Otsu, N., “A Threshold Selection Method from Gray-Level Histograms.” IEEE Transactions on Systems, Man, and Cybernetics. Vol.9, No.1, 1979, pp.62-66.

[0024] In the embodiment, a threshold is set by Otsu's binarization method, and an area equal to or greater than the determined threshold is determined as the detection object 201, and an area equal to or less than the threshold is determined as the background 202. The binarization process is performed by replacing the reflectances of the background 202 and the detection object 201 with 0 and 1, but is not limited to this, and may be replaced with NaN (Not a Number) and 1, or only the reflectance of the background 202 may be replaced with NaN.

[0025] Figure 3B shows the background 202, which is the area below the threshold (dotted line in Figure 3A) set by Otsu's binarization method, and Figure 3C shows the detected object 201, which is the area above the threshold. The degree of separation after binarization was 0.898. In this way, it is possible to efficiently and accurately remove background pixels.

[0026] 4A to 4C show binarization processing and division of the detection object 201 and the background 202 using variance instead of standard deviation. FIG. 4A is a histogram of the variance of reflectance obtained for each spectrum, FIG. 4B shows the background 202, which is an area where the variance is below a threshold (dotted line in FIG. 4A), and FIG. 4C shows the detection object 201, which is an area where the variance is above the threshold. The degree of separation after binarization processing was 0.878, and a part of the leaf pie (the lower right part in FIG. 4B) was identified as the background 202. This shows that the use of standard deviation rather than variance allows for more accurate separation and identification of the detection object 201 from the background 202.

[0027] 5A to 5C show binarization processing and division of the detection object 201 and the background 202 using the first principal component score instead of the standard deviation. FIG. 5A is a histogram of the first principal component score of the reflectance obtained for each spectral spectrum, FIG. 5B shows the background 202 in the region where the first principal component score is equal to or less than the threshold (dotted line in FIG. 5A), and FIG. 5C shows the detection object 201 in the region where the first principal component score is equal to or more than the threshold. The degree of separation after the binarization processing was 0.792, but the entire leaf pie and the central hole were identified as the background 202, and the white paper was erroneously identified as the detection object 201. From this, it can be seen that even when the background and the detection object cannot be correctly identified using principal component analysis, the image processing method according to the present disclosure can accurately separate and identify the background 202 and the detection object 201, and can efficiently remove background pixels.

[0028] Next, examples of applying the image processing method according to the present disclosure to various images will be described. As an example of an image, a single image composed of a plurality of monochrome microscopic images captured at a plurality of different fluorescent wavelengths can be used. For example, an image captured at a red fluorescent wavelength of 610 nm, an image captured at a green fluorescent wavelength of 550 nm, and an image captured at a blue fluorescent wavelength of 465 nm (each of which is a monochrome image) are composed to create a single color display image (RGB image). After that, the image processing method according to the present disclosure is applied to the composed RGB image, and two images divided into a background and a detection target can be created.

[0029] It is also possible to use multiband illumination to synthesize, in the same manner as described above, eight monochrome images taken at eight different wavelengths to create a single multiband image, and then apply the image processing method disclosed herein to create two images separated into background and object.

[0030] In addition to the above, it is also possible to convert monochrome images such as X-ray CT images and thermal images from thermography into RGB images by performing pseudo-color display, and then apply the image processing method according to the present disclosure to create two images divided into a background and a detection target. Details will be described with reference to Figure 7, which shows a series of processes performed using a thermal image of a hand.

[0031] First, monochrome thermal image A is processed into image B with three wavelengths of RGB. By converting image A into an RGB image, it can be divided into image C with red components, image D with green components, and image E with blue components. Next, a histogram F of the standard deviation of the RGB brightness values ​​contained in image B is created for image B, which is the composite RGB image. A threshold is determined from histogram F, and image G recognized as the background and image H detected as the object to be detected are obtained.

[0032] In this way, it can be seen that by applying the image processing method according to the present disclosure as described above to monochrome images as well, it is possible to detect the detection target object with high accuracy.

[0033] The present disclosure is not limited to the above embodiment, and can be modified as appropriate without departing from the spirit of the present disclosure. In the image processing method according to the present disclosure, the standard deviation of the reflectance of the spectral spectrum is calculated, but the present disclosure is not limited to this. For example, the interquartile range, the difference between the maximum value and the minimum value, etc. may be used instead of the standard deviation or variance of the reflectance of the spectral spectrum, or the first principal component score. Also, the absorbance or the luminance value of the RGB image may be used instead of the reflectance of the spectral spectrum.

[0034] This application claims priority to Japanese Patent Application No. 2023-139575, filed on August 30, 2023, the disclosure of which is incorporated herein in its entirety. [Explanation of symbols]

[0035] 100 Image processing device 101 Acquisition Department 102 Calculation section 103 Decision Section 104 Processing section 200 images 201 Detection target 202 Background

Claims

1. An image processing method for separating and identifying a detection object and a background in an image including position information and spectral information of the detection object and the background, comprising: Acquiring the image, Calculating a standard deviation of reflectance for each spectral spectrum included in the image; The standard deviation is histogrammed; determining the standard deviation threshold from the obtained histogram; The area where the threshold value is equal to or greater is determined as the detection object, and the area where the threshold value is equal to or less is determined as the background. Image processing methods.

2. The spectrum used to calculate the standard deviation includes information on the reflectance in the range of 950 to 1650 nm. The image processing method according to claim 1 .

3. Furthermore, after determining the standard deviation threshold, a binarization process is performed on the image. The image processing method according to claim 1 .

4. The image is a hyperspectral image that includes a number of pixels each storing a spectral spectrum. The image processing method according to any one of claims 1 to 3.

5. The image is a color display image obtained by combining a plurality of monochrome images taken at a plurality of wavelengths; A multiband image obtained by combining multiple monochrome images taken at multiple wavelengths, or A color display image obtained by pseudocolor displaying one monochrome image. The image processing method according to claim 4.

6. An image processing device for separating and identifying a detection object and a background in an image including position information and spectral information of the detection object and the background, An acquisition unit that acquires the image; a calculation unit that calculates a standard deviation of a reflectance for each spectrum included in the image and creates a histogram of the standard deviation; a determination unit that determines the standard deviation threshold value from the histogram obtained by the calculation unit; a processing unit that divides an area equal to or greater than the threshold into the detection object and an area equal to or less than the threshold into the background. Image processing device.

Images