Processing apparatus, processing method, and processing program

Abstract

To provide a segmentation technique for determining a target region with high accuracy.SOLUTION: A processing apparatus 1 includes: a segmentation preprocessing unit (preprocessing unit) 11 which compresses a high-dimensional image to a low-dimensional image; and a segmentation processing unit (processing unit) 12 which extracts a target region from the low-dimensional image.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a processing device, a processing method, and a processing program. [Background technology]

[0002] There is a technology that uses hyperspectral images to analyze objects such as plants. In order to improve the accuracy of the analysis, cropping is performed to extract the object in the image from the background. In cropping, segmentation is performed to classify the area in the image into object area and background area.

[0003] As a segmentation method for hyperspectral images, there is the K-means clustering method (Non-Patent Document 1). However, since it is necessary to specify the number of clusters in advance, it cannot be used effectively when the number of regions differs for each image. Therefore, there is the Level-set method as a segmentation method that does not require specifying the number of clusters in advance (Non-Patent Document 2). [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Katherine Meacham-Hensold, 10 others, “Plot-level rapid screening for photosynthetic parameters using proximal hyperspectral imaging”, Journal of Experimental Botany, Vol.71, No.7, 2020, p.2312-p.2328, Available at: https: / / doi.org / 10.1093 / jxb / eraa068 [Non-Patent Document 2] Ryo Kurazume, "Level Set Method and Its Implementation", Information Processing Society of Japan Research Report, 2006-CVIM-156(17), p.133-p.145 Summary of the Invention [Problem to be solved by the invention]

[0005] The level set method is an effective method for identifying the contours of clustered regions. However, because hyperspectral images are high-dimensional data, applying the level set method to each dimension individually requires a huge amount of computation.

[0006] The present disclosure has been made in consideration of the above circumstances, and aims to provide a segmentation technique capable of identifying an object region with high accuracy. [Means for solving the problem]

[0007] A processing device according to one aspect of the present disclosure includes a preprocessing unit that compresses a high-dimensional image into a low-dimensional image, and a processing unit that extracts a target region from the low-dimensional image.

[0008] A processing method according to one aspect of the present disclosure is a processing method performed by a processing device, in which a high-dimensional image is compressed into a low-dimensional image, and a target region is extracted from the low-dimensional image.

[0009] A processing program according to one embodiment of the present disclosure causes a computer to function as the processing device. Effect of the Invention

[0010] According to the present disclosure, it is possible to provide a segmentation technique capable of identifying an object region with high accuracy. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a processing device. [Diagram 2] FIG. 2 is a diagram illustrating the image transformation and segmentation method. [Diagram 3] FIG. 3 is a diagram showing an example of conversion of a hyperspectral image. [Figure 4]FIG. 4 is a diagram showing the segmentation result (this embodiment). [Diagram 5] FIG. 5 is a diagram showing a segmentation result (conventional). [Figure 6] FIG. 6 is a diagram illustrating a hardware configuration of the processing device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0013] [overview] In this disclosure, a high-dimensional image is compressed into a low-dimensional image, and a region of interest is extracted from the low-dimensional image.

[0014] In this way, a high-dimensional image is compressed into a low-dimensional image, and a target region is extracted from the low-dimensional image, thereby providing a segmentation technique that can distinguish a target region with high accuracy.

[0015] For example, a hyperspectral image, which is high-dimensional data, is compressed to three dimensions using PCA (Principal Component Analysis). The first principal component extracted by PCA is the principal component that is affected by the intensity of the entire original image, and this first principal component is regarded as a luminance signal (Y) and converted into a YUV format image. Since image conversion makes it possible to apply image filters related to sharpening, a specified filter is applied to the converted image, and segmentation is performed using the level set method or the like.

[0016] The hyperspectral image is an example of a high-dimensional image. For example, a high-dimensional image expressed based on an arbitrary feature space, such as an image divided by wavelength or a color space captured by a metalens, a multispectral camera, a visible light camera, or the like, may be extracted.

[0017] The number of dimensions, 3, is the number of dimensions selected to take advantage of existing image formats (e.g., YUV image formats). Dimension compression methods other than 3 may be used. Image formats other than YUV may be used. For example, image formats such as RGB, HSV, and HLS color spaces may be used.

[0018] The YUV format is one of the formats for expressing color information, and is a method for expressing color using a combination of three components: a luminance signal (Y), the difference between the luminance signal and the blue component (U), and the difference between the luminance signal and the red component (V). The YUV format only requires a luminance signal and any two color differences, and the format can be YCbCr or YPbPr.

[0019] Hereinafter, a hyperspectral image will be used as an example of a high-dimensional image.

[0020] [Processing device configuration] 1 is a diagram showing an example of the configuration of a processing device 1 according to this embodiment. The processing device 1 includes a segmentation pre-processing unit 11 and a segmentation processing unit 12.

[0021] The segmentation pre-processing unit (pre-processing unit) 11 has a function of compressing a hyperspectral image captured by a hyperspectral camera into a low-dimensional image.

[0022] In addition, the segmentation pre-processing unit 11 has a function of converting the above-mentioned low-dimensional image using an image format (e.g., a YUV image format) that can represent information of one or more channels and in which the amount of information in at least one channel is greater than the amount of information in the other channels.

[0023] During the image conversion, the segmentation pre-processing unit 11 has a function of allocating one or more components in a low-dimensional image that have a high degree of contribution to the image configuration to the one or more channels having a large amount of information in the image format.

[0024] Specifically, in this embodiment, the segmentation pre-processing unit 11 compresses the hyperspectral image, which is high-dimensional data, into a three-dimensional image (dimensional compression) by PCA. Since the "principal components" are extracted by PCA in order of the degree of contribution to the image configuration, starting from the first principal component, the principal component (dimension) with the highest degree of contribution is assigned to Ych, which has a large amount of information in the YUV image format. Finally, the segmentation pre-processing unit 11 applies a predetermined filter to the converted YUV format image.

[0025] The segmentation processing unit (processing unit) 12 has a function of extracting a target region from a hyperspectral image that has been subjected to low-dimensional compression, image conversion, and filtering.

[0026] Specifically, in this embodiment, the segmentation processing unit 12 segments (discriminates, classifies) the areas contained in the YUV format image after the filter is applied into an object area and a background area using a level set method or the like, and performs trimming to extract the object area by comparing it with the original image of the hyperspectral image.

[0027] [Hyperspectral image segmentation method] 2 is a diagram showing a segmentation method of a hyperspectral image according to the present embodiment. In the present embodiment, a hyperspectral camera image of a lettuce leaf taken by a hyperspectral camera with 151 wavelengths is used.

[0028] First, the segmentation pre-processing unit 11 acquires a 151-channel hyperspectral image of a lettuce leaf captured by the hyperspectral camera (step S1).

[0029] Hyperspectral image data is extremely high-dimensional; for a few hundred images, the number of dimensions in the image data is hundreds of thousands of times greater than the number of image data items, making it impossible to use general statistical methods or multivariate analysis methods.

[0030] Therefore, the segmentation pre-processing unit 11 reduces the dimension (dimensionality compression) of the hyperspectral image from 151 ch to 3 ch by PCA (principal component analysis) (step S2). Since the hyperspectral image is reduced in dimension by PCA, the number of dimensions can be reduced while maintaining the characteristics of the original image information.

[0031] Next, the “principal components” are extracted in order of their contribution to the image structure, starting with the first principal component, by the PCA in step S2, and the segmentation pre-processing unit 11 assigns the principal components (dimensions) with the highest contribution to the channels with the largest amount of information in the image format.

[0032] For example, when using a YUV image format, the segmentation pre-processing unit 11 regards the first principal component with the highest contribution as the luminance in the YUV image format. In other words, the first principal component is assigned to Ych, which has a larger amount of information than Uch or Vch. Then, the YUV format image is converted to an RGB format image (RGB image) (step S3). Since the first principal component with a high contribution to the image configuration is assigned to Ych, which has a larger amount of information, the original image information can be compressed while maintaining its characteristics.

[0033] When using an image format in which the amount of information between channels is equal rather than an image format such as YUV in which the amount of information between channels differs, the segmentation pre-processing unit 11 may assign the first principal component to any channel.

[0034] Next, the segmentation pre-processing unit 11 applies a predetermined filter to the RGB image (step S4).

[0035] For example, first, apply a filter with a broken line tone curve to enhance the contrast of an RGB image. Next, apply a filter with a posterization to simplify the tones. Next, apply a filter with a grayscale to convert the RGB image to grayscale. Finally, apply a filter with an S-shaped tone curve to sharpen the RGB image.

[0036] Examples of conversion of four types of hyperspectral images using each process described above are shown in Figure 3. From the RGB image in the second row from the left, it can be seen that by converting the 151ch hyperspectral image into 3ch, the area of ​​the lettuce leaf, which is the target object, is clearly classified from the background area.

[0037] Moreover, it can be seen that the RGB images after application of the filter in the third to fifth columns from the left show that the shadows on the three-dimensional contours of the lettuce leaves have disappeared and the contours have become clearer. This makes it possible to improve the accuracy of segmentation in the subsequent segmentation processing unit 12, and to distinguish the object region with high accuracy.

[0038] Next, the segmentation pre-processing unit 11 reduces the filtered RGB image (step S5).

[0039] Next, the segmentation processing unit 12 segments the area included in the reduced RGB image into lettuce leaf areas and areas other than the lettuce leaf areas (background areas) by the level set method (step S6).

[0040] FIG. 4 shows the segmentation result of the background region according to this embodiment. It can be seen from FIG. 4 that all regions other than the lettuce leaf region are classified as background regions. Meanwhile, FIG. 5 shows the segmentation result of the background region using only the level set method without performing the image processing of this embodiment. Only some of the regions other than the lettuce leaf region are classified as background regions. Also, some of the lettuce leaf regions are classified as background regions.

[0041] Next, the segmentation processing unit 12 enlarges the segmented RGB image (step S7).

[0042] Finally, the segmentation processing unit 12 compares the segmented and enlarged RGB image with the original hyperspectral image, and removes background regions from the RGB image (step S8).

[0043] In the case of this embodiment shown in Fig. 4, only the lettuce leaves are extracted. On the other hand, in the conventional case shown in Fig. 5, in addition to the lettuce leaves, part of the background is also extracted, and part of the lettuce leaves is cut out. Therefore, this embodiment makes it possible to distinguish the object region with high accuracy.

[0044] (Variation 1) In this embodiment, the process from step S4 onwards is performed using an RGB image, but the process from step S4 onwards may also be performed using a YUV image.

[0045] (Variation 2) In this embodiment, the RGB image is reduced in step S5, but this is not necessarily required. If step S5 is not performed, step S7 may also not be performed.

[0046] (Variation 3) In this embodiment, a dimensionality reduction method using a simple PCA (principal component analysis) has been described, but a dimensionality reduction method such as kernel principal component analysis or probabilistic principal component analysis may also be used.

[0047] [Effects of the embodiment] According to this embodiment, a hyperspectral image is compressed into a low-dimensional image, and a target region is extracted from the low-dimensional image, so that a segmentation technique capable of identifying a target region with high accuracy can be provided.

[0048] [others] The present disclosure is not limited to the above-described embodiment, and various modifications are possible within the scope of the present disclosure.

[0049] The processing device 1 of this embodiment described above can be realized, for example, using a general-purpose computer system including a CPU 901, a memory 902, a storage 903, a communication device 904, an input device 905, and an output device 906, as shown in FIG. 6.

[0050] The memory 902 and the storage 903 are storage devices. In the computer system, the CPU 901 executes a predetermined program loaded onto the memory 902, whereby each function of the processing device 1 is realized.

[0051] The processing device 1 may be implemented in one computer. The processing device 1 may be implemented in multiple computers. The processing device 1 may be a virtual machine implemented in a computer.

[0052] The program for the processing device 1 can be stored in a computer-readable recording medium such as a HDD, SSD, USB memory, CD, or DVD. The computer-readable recording medium is, for example, a non-transitory recording medium. The program for the processing device 1 can also be distributed via a communication network. [Explanation of symbols]

[0053] 1 Processing equipment 11 Segmentation preprocessing section 12 Segmentation processing section 901 CPU 902 Memory 903 Storage 904 Communication equipment 905 Input Device 906 Output Device

Claims

1. a pre-processing unit that compresses a high-dimensional image into a low-dimensional image; a processor for extracting a region of interest from the low dimensional image; A processing device comprising:

2. The processing device according to claim 1 , wherein the preprocessing unit converts the low-dimensional image using an image format capable of expressing information of one or more channels.

3. The processing device according to claim 2 , wherein the image format is an image format in which an amount of information in at least one channel is greater than an amount of information in the other channels.

4. The processing device according to claim 3 , wherein the preprocessing unit allocates one or more components in the low-dimensional image that have a high contribution to an image configuration to the one or more channels.

5. In a processing method performed by a processing device, Compress high-dimensional images into low-dimensional images, Extracting a region of interest from the reduced dimensional image; Processing methods.

6. A processing program that causes a computer to function as the processing device according to any one of claims 1 to 4.

Images