Image processing method and image processing device

Abstract

This image processing method comprises: a step in which a pixel value of each pixel of a cell image (31) is converted to a relative value; a step in which a frequency component image (50) that results from extracting a prescribed frequency component is acquired from a converted cell image (40); a step in which a first image (61a) that results from binarizing the converted cell image (40) and a second image (62) that results from binarizing the frequency component image (50) are acquired; and a step in which a binarized image (70) of the cell image (31) is generated by combining the first image (61) and the second image (62).

Description

[Technical field]

[0001] The present invention relates to an image processing method and an image processing device. [Background technology]

[0002] Conventionally, a technique for performing binarization processing on a cell image has been disclosed, for example, in Japanese Patent Application Laid-Open No. 2014-18184.

[0003] The above-mentioned Japanese Patent Application Laid-Open No. 2014-18184 discloses a technique for evaluating the quality of a colony of pluripotent stem cells based on a differential filter processed image of a microscopic image obtained by using an optical microscope. It also discloses that a binarization process is performed to improve the accuracy of the image analysis. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2014-18184 A Summary of the Invention [Problem to be solved by the invention]

[0005] The binarization process of cell images is used to classify the cell regions from the background regions in the image and to analyze the morphological characteristics of the cells, such as their size. However, the binarization process of cell images has the following problems that make it difficult to extract the cell regions with high accuracy.

[0006] First, when brightness unevenness exists in a cell image due to illumination variations in a microscope image, it becomes difficult to set a threshold value for distinguishing between cell regions and background regions.

[0007] Secondly, in a cell image, the brightness value of the inner region of a cell or the fine structure of a cell tends to be low. When such a low brightness region of a cell is superimposed on the brightness unevenness described above, it becomes difficult to accurately extract the low brightness region of a cell by binarization processing. For example, the low brightness part of the inner region of a cell will be binarized as if a hole had been formed inside the cell. In addition, as a fine structure of a cell, there is a pseudopodia that protrudes from the cell body. When a cell having a pseudopodia is binarized, the low brightness part of the pseudopodia cannot be distinguished from the background, and the pseudopodia and the cell body may be separated in the binarized image. For example, when calculating the size of a cell, the size of the cell region after binarization processing will be smaller than the actual size of the entire cell including the pseudopodia (by the amount of the separated pseudopodia excluded from the size calculation).

[0008] For this reason, in the binarization process of cell images, it is desirable to reduce the effects of brightness unevenness that is specific to microscope images and to be able to accurately extract low-brightness areas of cells that contain their fine structures.

[0009] The present invention has been made to solve the above-mentioned problems, and one object of the present invention is to provide an image processing method and an image processing device that can reduce the effect of brightness unevenness in a cell image in the binarization processing of the cell image and accurately extract low-brightness areas of the cell, including the cell's fine structure. [Means for solving the problem]

[0010] In order to achieve the above-mentioned object, an image processing method in a first aspect of the present invention is an image processing method for performing binarization processing on a cell image, which is a multi-value image, and includes the steps of extracting a background luminance distribution in the cell image, converting the pixel values ​​of each pixel of the cell image into relative values ​​with respect to the background luminance distribution, obtaining a frequency component image in which predetermined frequency components corresponding to the detailed structure of the cell are extracted from the converted cell image, obtaining a first image obtained by binarizing the converted cell image and a second image obtained by binarizing the frequency component image, and generating a binary image of the cell image by combining the first image and the second image.

[0011] An image processing device in a second aspect of the present invention includes an image acquisition unit that acquires a cell image which is a multi-valued image, a background extraction unit that extracts a background luminance distribution in the cell image, a relativization processing unit that converts the pixel value of each pixel of the cell image into a relative value with respect to the background luminance distribution, a frequency component extraction unit that acquires a frequency component image by extracting predetermined frequency components corresponding to the detailed structure of the cell from the converted cell image, a binarization processing unit that acquires a first image obtained by binarizing the converted cell image through a binarization process, and a second image obtained by binarizing the frequency component image, and a synthesis processing unit that generates a binary image of the cell image by synthesizing the first image and the second image. Effect of the Invention

[0012] In the image processing method according to the first aspect and the image processing device according to the second aspect, the background luminance distribution in the cell image is extracted, and the pixel value of each pixel of the cell image is converted into a relative value with respect to the background luminance distribution, so that the pixel value of the cell image after conversion indicates the degree of deviation from the background luminance level. Therefore, even if there is luminance unevenness in the cell image, the binarization process can be performed based on the degree of deviation from the background luminance level for each pixel, so that the influence of the luminance unevenness can be reduced. Since the binarization process is performed on the cell image converted into a relative value, a first image that accurately extracts even low luminance areas in the cell close to the background luminance level can be obtained. Furthermore, since the binarization process is performed on a frequency component image that extracts a predetermined frequency component corresponding to the detailed structure of the cell, a second image that accurately extracts the detailed structure of the cell can be obtained. Then, by synthesizing the first image and the second image, the first image that extracts the main morphology of the cell can be complemented with the second image that extracts the detailed structure, so that a binarized image that accurately extracts the entire morphology of the cell including the low luminance areas in the cell can be obtained. As a result, in the binarization process of a cell image, the influence of brightness unevenness in the cell image can be reduced, and low brightness regions of the cell that include the cell's fine structure can be accurately extracted. [Brief description of the drawings]

[0013] [Figure 1] 1 is a block diagram showing an image processing system including an image processing device according to an embodiment of the present invention. [Diagram 2] FIG. 1 shows an example of a cell image (A), a binarized image of the cell image (B), an enlarged view of the cell image (C), and an enlarged view of the binarized image (D). [Diagram 3] FIG. 2 is a functional block diagram for explaining functions of a processor of the image processing device. [Figure 4] FIG. 4 is a flow chart for explaining a processing operation of the image processing device according to the present embodiment. [Diagram 5] FIG. 11 is a diagram for explaining details of the pre-processing. [Figure 6] 11A and 11B are diagrams for explaining details of the process of extracting a background luminance distribution and the process of converting it into a relative value. [Figure 7]1A and 1B are diagrams for explaining the luminance distribution of a cell image and a background image. [Figure 8] FIG. 11 is a diagram for explaining the luminance distribution of a normalized image. [Figure 9] 11A to 11C are diagrams for explaining details of a process for generating a frequency component image. [Figure 10] FIG. 2 is a diagram for explaining frequency bands included in a frequency component image. [Figure 11] FIG. 13 is a diagram for explaining selection of a parameter set for smoothing processing. [Figure 12] FIG. 4 is a diagram for explaining details of a binarization process. [Figure 13] 5A to 5C are diagrams for explaining details of a synthesis process of a first image and a second image. [Figure 14] FIG. 11 is a diagram for explaining details of post-processing. [Figure 15] 1A is a diagram showing a binarized image after post-processing according to the present embodiment, and FIG. 1B is a diagram showing a binarized image according to a comparative example. [Figure 16] FIG. 13 is a block diagram showing an image processing device according to a modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.

[0015] The configuration of an image processing system 200 including an image processing device 100 according to this embodiment and an image processing method will be described with reference to FIGS.

[0016] (Image Processing System) The image processing system 200 shown in FIG. 1 is an image processing system that enables a user performing cell culture or the like to capture a cell image 30, perform image processing on the cell image 30, and view the processed image, all in one integrated system.

[0017] (Image Processing System Overview) The image processing system 200 includes an image processing device 100 , a computer 110 , and an imaging device 120 .

[0018] FIG. 1 shows an example of an image processing system 200 constructed in a client-server model. The computer 110 functions as a client terminal in the image processing system 200. The image processing device 100 functions as a server in the image processing system 200. The image processing device 100, the computer 110, and the imaging device 120 are connected to each other via a network 130 so that they can communicate with each other. The image processing device 100 performs various information processing in response to a request (processing request) from the computer 110 operated by a user. The image processing device 100 performs image processing on a cell image 30 in response to the request, and transmits the processed image to the computer 110. The reception of operations on the image processing device 100 and the display of images processed by the image processing device 100 are performed on a GUI (Graphical User Interface) displayed on a display unit 111 of the computer 110.

[0019] The network 130 connects the image processing device 100, the computer 110, and the imaging device 120 so that they can communicate with each other. The network 130 may be, for example, a local area network (LAN) built in a facility. The network 130 may be, for example, the Internet. When the network 130 is the Internet, the image processing system 200 may be a system built in the form of cloud computing.

[0020] The computer 110 is a so-called personal computer, and includes a processor and a storage unit. A display unit 111 and an input unit 112 are connected to the computer 110. The display unit 111 is, for example, a liquid crystal display device. The display unit 111 may be an electroluminescence display device, a projector, or a head-mounted display. The input unit 112 is, for example, an input device including a mouse and a keyboard. The input unit 112 may be a touch panel. One or more computers 110 are provided in the image processing system 200.

[0021] The imaging device 120 captures an image of a cell to generate a cell image 30. The imaging device 120 can transmit the generated cell image 30 to the computer 110 and / or the image processing device 100 via the network 130. The imaging device 120 captures a microscopic image of the cell. The imaging device 120 performs imaging using an imaging method such as bright-field observation, dark-field observation, phase contrast observation, or differential interference observation. Depending on the imaging method, one or more types of imaging devices 120 are used. The image processing system 200 can be provided with one or more imaging devices 120.

[0022] The image processing device 100 includes a processor 10 such as a central processing unit (CPU), a field-programmable gate array (FPGA), or an application specific integrated circuit (ASIC). The processor 10 executes a predetermined program 21, whereby the image processing device 100 performs arithmetic processing.

[0023] The image processing device 100 includes a storage unit 20. The storage unit 20 includes a non-volatile storage device. The non-volatile storage device is, for example, a hard disk drive, a solid state drive, or the like. The storage unit 20 stores various programs 21 executed by the processor 10. The storage unit 20 stores image data 22. The image data 22 includes a cell image 30 captured by the imaging device 120, and various processed images (binarized images 80) generated by image processing of the cell image 30. In this embodiment, of the image processing functions that the image processing device 100 can execute, the binarization process of the cell image 30 in particular will be described.

[0024] The image processing device 100 performs binarization processing on the cell image 30 in response to a request from the computer 110. As a result of the image processing, the image processing device 100 generates a binarized image 80 of the cell image 30. The image processing device 100 transmits the generated binarized image 80 to the computer 110. The computer 110, which has received the information, causes the display unit 111 to display the binarized image 80.

[0025] <Cell images> 2, the cell image 30 is, for example, a microscope image of a cultured cell cultured using a cell culture instrument. The type of the cell image 30 is not particularly limited, but as an example, in this embodiment, the cell image 30 is a fluorescent stained image of a cell. The cell image 30 is a multi-value image (multi-tone image) and a color image.

[0026] The cell image 30 shows an image of a cell 90 (cell image) and a background 93. The cell 90 shown in the cell image 30 shown in Fig. 2(A) includes a cell body 91 and a detailed structure 92. The detailed structure 92 shown in the example of Fig. 2 is a filopodia in which cytoplasm protrudes from the cell body 91, and is a filopodia protruding in a thread-like (linear) shape from the cell body 91. In Figs. 2(A) and 2(C), the region of the detailed structure 92 and the inner region of the cell body 91 tend to include low-luminance regions that have relatively low luminance (low pixel value) within the cell 90 (tend to be dark within the cell image 30).

[0027] As shown in Figures 2(B) and 2(D), the binarization process is a process in which, for an image to be processed, the pixel values ​​of pixels having pixel values ​​equal to or greater than the binarization threshold are set to "1 (white)" and the pixel values ​​of pixels having pixel values ​​less than the binarization threshold are set to "0 (black)." In the binarization process of a cell image 30, the cell image 30 is binarized so that the image of the cell 90 (cell image) is extracted as a white region and the background 93 other than the cell 90 becomes a black region. The binarization threshold is set to a value that distinguishes between the range of pixel values ​​to which the cell image belongs and the range of pixel values ​​to which the background 93 belongs.

[0028] Therefore, when the brightness distribution of the background 93 varies and relatively high brightness and low brightness areas are formed in the background 93, the difference between the pixel values ​​of the low brightness areas in the cell 90 and the pixel values ​​of the high brightness areas in the background 93 becomes small, making it difficult to extract the low brightness areas (into white areas) by binarization processing.

[0029] The image processing device 100 according to this embodiment can generate a binarized image 80 in which low-luminance regions in a cell 90 are extracted with high accuracy, as shown in Figures 2(B) and 2(D), even if the luminance distribution of the background 93 in the cell image 30 varies. The image processing device 100 will be described in detail below.

[0030] (Detailed configuration of image processing device) FIG. 3 is a block diagram showing a configuration related to binarization processing of the image processing device 100 and an overview of image processing.

[0031] The processor 10 of the image processing device 100 includes, as functional blocks, an image acquisition unit 11, a pre-processing unit 12, a background extraction unit 13, a relativization processing unit 14, a frequency component extraction unit 15, a binarization processing unit 16, a synthesis processing unit 17, and a post-processing unit 18. In other words, the processor 10 executes a program 21 stored in a storage unit 20 to function as the image acquisition unit 11, the pre-processing unit 12, the background extraction unit 13, the relativization processing unit 14, the frequency component extraction unit 15, the binarization processing unit 16, the synthesis processing unit 17, and the post-processing unit 18.

[0032] The image acquiring unit 11 has a function of acquiring a cell image 30. The image acquiring unit 11 acquires the cell image 30 to be subjected to binarization processing by reading the cell image 30 stored in the storage unit 20 (see FIG. 1). The image acquiring unit 11 may acquire the cell image 30 transmitted from the imaging device 120 or the computer 110 via the network 130 (see FIG. 1). The image acquiring unit 11 outputs the acquired cell image 30 to the pre-processing unit 12.

[0033] The pre-processing unit 12 executes pre-processing for binarization. Specifically, the pre-processing unit 12 converts a color cell image 30 into a grayscale cell image 31. A grayscale image is a monochromatic (non-color information) multi-valued image. The pixel value of each pixel in the cell image 31 indicates the luminance (image brightness) of that pixel. The pre-processing unit 12 outputs the grayscale cell image 31 to the background extraction unit 13 and the relativization processing unit 14, respectively.

[0034] The background extraction unit 13 has a function of extracting the background luminance distribution in the cell image 31. The background luminance distribution refers to the luminance distribution of the ambient light (illumination light) in the cell image 31. Ideally, the background luminance distribution is constant across the entire image, but in reality, it varies within the cell image 31 due to variations in the intensity of the illumination light, etc. Furthermore, the background luminance distribution differs for each cell image 31 due to variations in exposure time, etc. The background extraction unit 13 generates a background image 32 that represents the background luminance distribution of the cell image 31. The background image 32 represents the background luminance distribution of the cell image 31 in units of one pixel of the cell image 31.

[0035] Here, the pixel value of each pixel in the cell image 31 can be considered to be the sum of a cell image component and a background luminance component. The cell image component is an image of an optical signal including image information of the cell 90 (see FIG. 2) that is the observation target. The background luminance component is an image of background light that is unavoidably observed in the imaging environment of the cell 90. Therefore, by removing the cell image component from each pixel in the cell image 31, the background luminance component for each pixel, i.e., the background luminance distribution, can be obtained.

[0036] In this embodiment, the background extraction unit 13 generates a background image 32 indicating a background luminance distribution by performing a filter process on the cell image 31 to remove the cells 90 in the cell image 31. The filter process to remove the cells 90 is a median filter process with a kernel size corresponding to the size of the cells 90 shown in the cell image 31. The median filter process is a process to replace the pixel value of a central pixel of interest in the kernel with the median value of the pixel values ​​of the surrounding pixels other than the pixel of interest in the kernel. The background extraction unit 13 outputs the generated background image 32 to the relativization processing unit 14. The background image 32 is an example of a "background luminance distribution" in the claims.

[0037] The relativization processing unit 14 converts the pixel value of each pixel of the cell image 31 into a relative value with respect to the background luminance distribution. The relativization processing unit 14 generates a normalized image 40 in which the pixel value of each pixel of the cell image 31 is converted into a relative value based on the background image 32. As will be described later, the pixel value of each pixel of the normalized image 40 indicates the ratio of the luminance of that pixel to the background luminance. The relativization processing unit 14 outputs the generated normalized image 40 to the frequency component extraction unit 15 and the binarization processing unit 16, respectively.

[0038] The frequency component extraction unit 15 obtains a frequency component image 50 by extracting a predetermined frequency component corresponding to a detailed structure 92 (see FIG. 2) of a cell 90 from the normalized image 40. When focusing on the spatial frequency of the image, the detailed structure 92 of the cell 90 corresponds to a high frequency component having a higher frequency than the background 93, which is a low frequency component, and can be clearly distinguished from the background 93. Therefore, the frequency component extraction unit 15 extracts a predetermined frequency component corresponding to the detailed structure 92 from the normalized image 40, thereby generating a frequency component image 50 in which the detailed structure 92 has been extracted while excluding the background 93. The frequency component extraction unit 15 outputs the generated frequency component image 50 to the binarization processing unit 16.

[0039] The binarization processing unit 16 binarizes the normalized image 40 and the frequency component image 50. The binarization processing unit 16 generates a first image 61 by binarizing the normalized image 40 and a second image 62 by binarizing the frequency component image 50. In this embodiment, the binarization processing unit 16 binarizes the normalized image 40 with different binarization thresholds to generate a plurality of first images 61, including a first threshold image 61a and a second threshold image 61b. The binarization processing unit 16 outputs the generated first image 61 (the first threshold image 61a and the second threshold image 61b) and the second image 62 to the synthesis processing unit 17.

[0040] The synthesis processing unit 17 generates a binary image 70 of the cell image 31 by synthesizing a first image 61 obtained by binarizing the normalized image 40 and a second image 62 obtained by binarizing the frequency component image 50. The main part of the cell 90, excluding the detailed structure 92, is included in the first image 61. The detailed structure 92 of the cell 90 is included in the second image 62 obtained by binarizing the frequency component image 50. By synthesizing these images, a binary image 70 is obtained in which both the main part of the cell 90 and the detailed structure 92 of the cell 90 are extracted. As will be described later, the first threshold image 61a is used for synthesis with the second image 62, and the second threshold image 61b is used for noise removal processing of the second image 62. The synthesis processing unit 17 outputs the binary image 70 of the cell image 31 to the post-processing unit 18.

[0041] The post-processing unit 18 performs processes such as image shaping and noise removal on the binarized image 70 to generate a post-processed binarized image 80. The post-processed binarized image 80 is stored in the storage unit 20 as the final binarization processing result for the input cell image 30. The binarized image 80 is also transmitted to the computer 110 in response to a request and displayed on the display unit 111.

[0042] (Image processing method) Next, an image processing method of this embodiment will be described. The image processing method of this embodiment is an image processing method for performing binarization processing on a cell image 31, which is a multi-valued image. The image processing method can be executed by an image processing device 100 (processor 10).

[0043] The image processing method of this embodiment includes at least the following steps. (1) Step of extracting background brightness distribution (background image 32) in cell image 30 (2) converting the pixel value of each pixel in the cell image 30 into a relative value (normalized image 40) with respect to the background luminance distribution; (3) A step of obtaining a frequency component image 50 by extracting predetermined frequency components corresponding to the detailed structure 92 of the cell 90 from the transformed cell image 31 (normalized image 40) (4) A step of obtaining a first image 61 by binarizing the converted cell image 30 and a second image 62 by binarizing the frequency component image 50. (5) generating a binary image 70 of the cell image 31 by combining the first image 61 and the second image 62;

[0044] A step (1) of extracting a background luminance distribution in the cell image 31 is executed by the background extraction unit 13. A step (2) of converting the pixel value of each pixel of the cell image 31 into a relative value with respect to the background luminance distribution is executed by the relativization processing unit 14. A step (3) of acquiring a frequency component image 50 by extracting a predetermined frequency component corresponding to a detailed structure 92 of a cell 90 from the converted cell image 31 is executed by the frequency component extraction unit 15. A step (4) of acquiring a first image 61 obtained by binarizing the converted cell image 30 and a second image 62 obtained by binarizing the frequency component image 50 is executed by the binarization processing unit 16. A step (5) of generating a binarized image 70 of the cell image 31 by synthesizing the first image 61 and the second image 62 is executed by the synthesis processing unit 17. The image processing method of this embodiment further includes a process by the pre-processing unit 12 and a process by the post-processing unit 18.

[0045] Next, the flow of processing by the image processing device 100 will be described in detail with reference to FIGS.

[0046] Image Acquisition In step S1, the image acquisition unit 11 (see FIG. 3) acquires the cell image 30 from the storage unit 20, the imaging device 120, or the computer 110. As described above, the acquired cell image 30 is a multi-value image and a color image.

[0047] <Pretreatment> In step S2, the preprocessing unit 12 (see FIG. 3) performs preprocessing for binarization on the cell image 30 acquired in step S1. The details of the preprocessing will be described with reference to FIG.

[0048] In step S2a, the preprocessing unit 12 separates the cell image 30, which is a color image, into a plurality of color component images. The number of color component images to be separated is the number of color channels included in the cell image 30. When the color image is expressed in the RGB format, the cell image 30 is separated into three color component images, a red image 30r, a green image 30g, and a blue image 30b. Each color component image is a grayscale image.

[0049] In step S2b, the pre-processing unit 12 obtains the distribution of pixel values ​​of the separated color component images (red image 30r, green image 30g, and blue image 30b). The pre-processing unit 12 creates, for example, histograms of pixel values ​​of the color component images (histograms Hr, Hg, and Hb). The histograms are calculated by calculating the number of pixels having each pixel value.

[0050] In step S2c, the pre-processing unit 12 compares the distribution of pixel values ​​(histograms Hr, Hg, Hb) of each color component image, and selects the color component image with the highest pixel value. For example, the pre-processing unit 12 selects the color component image with the highest average pixel value from among multiple color component images. The example in FIG. 5 shows an example in which a green image 30g is selected as the color component image with the highest average pixel value. The pre-processing unit 12 outputs the selected color component image (green image 30g) as a grayscale cell image 31. In this way, the pre-processing unit 12 separates the color cell image 30 into multiple color component images, and generates a grayscale cell image 31 from the color component image with the highest pixel value.

[0051] Usually, when a color image is converted into a grayscale image, the pixel value of each pixel after conversion is the average value of the pixel values ​​of the same pixel in each color component image. Therefore, the brightness of a normal grayscale image is the average brightness of each color component image. However, in a fluorescent stained cell image, the brightness (pixel value) of a specific color component image to which the fluorescent wavelength belongs is significantly high, and the brightness (pixel value) of other color component images that do not include the fluorescent wavelength is low. Therefore, when a normal grayscale conversion is performed on a fluorescent stained cell image 30, the average brightness of the converted image decreases. Therefore, instead of averaging the pixel values, the color component image with the highest pixel value is used as a grayscale cell image 31, thereby suppressing the decrease in brightness of the grayscale image.

[0052] <Extraction of background luminance distribution> 4, the background extraction unit 13 executes a process (step (1) above) of extracting a background luminance distribution in the cell image 31. The process of extracting the background luminance distribution will be described in detail with reference to FIG.

[0053] Step S3 of extracting the background luminance distribution includes step S3a of reducing the cell image 31, step S3b of performing a filter process on the reduced cell image 31 to remove cells 90, and step S3c of enlarging the filtered background image 32 back to the image size before reduction.

[0054] In step S3a, the background extraction unit 13 reduces the cell image 31 at a preset predetermined ratio. The reduction ratio is stored in advance as setting information in the storage unit 20. The reduction ratio is not particularly limited, but may be in the range of 1 / 2 to 1 / 10. As an example, the background extraction unit 13 reduces the cell image 31 to 1 / 8. The reduced cell image 31 is referred to as a reduced image 31a.

[0055] In step S3b, the background extraction unit 13 executes a first median filter process for the reduced image 31a to remove the cells 90. The kernel size of the first median filter process is set to a value that sufficiently removes the cells in the image and is sufficiently smaller than the period of the brightness fluctuation of the background. The kernel size can vary depending on the imaging magnification of the cell image 30.

[0056] In one example, the kernel size of the first median filter process on the reduced image 31a is about 30 (pixels). For example, the kernel is a square pixel range of 30 x 30 pixels. Since the first median filter process is performed on the reduced image 31a of 1 / 8 size, this process is substantially equivalent to performing a median filter process with a kernel size (240 x 240 (pixels)) that is 8 times larger than the original cell image 31 before reduction.

[0057] The pixels forming the image (cell image) of the cell 90 have pixel values ​​higher than the pixel values ​​of the pixels belonging to the background 93, but when median filter processing is performed with a sufficiently large kernel, the pixel values ​​belonging to the background 93 around the cell 90 are adopted as the median values, and the pixel values ​​of the pixels forming the cell image are replaced with the pixel values ​​of the background 93. As a result of the first median filter processing, the cell image is removed from the reduced image 31a, and a reduced background image 32a indicating the background luminance distribution is generated.

[0058] In step S3c, the background extraction unit 13 enlarges the reduced background image 32a to return it to the image size before reduction. As described above, when the cell image 31 is reduced to 1 / 8 in step S3a, the background extraction unit 13 enlarges the reduced background image 32a by 8 times. As a result, the background image 32 of the same size as the cell image 31 is obtained.

[0059] 7 shows a graph 36 (brightness profile) of pixel values ​​for each pixel along the same line 35 of an original cell image 31 and an acquired background image 32. In the graph 36, the horizontal axis indicates position (pixels along the line 35) and the vertical axis indicates pixel value.

[0060] In the pixel value distribution of cell image 31 shown by the solid line in graph 36, the areas with high pixel values ​​that appear locally indicate cell images, and the remaining baseline indicates the background. From graph 36, it can be seen that the level of pixel values ​​(luminance) of the background is not constant, but varies depending on the position (the baseline is wavy). This variation in background luminance is a factor that prevents the extraction of low luminance areas in the binarization process. Looking at the pixel value distribution of background image 32 shown by the dashed line in graph 36, only the distribution of background luminance that varies depending on the position has been accurately extracted. In this way, in step S3, background image 32 is generated by background extraction unit 13.

[0061] Note that in Figures 5 to 15, the sizes of the images shown are different, but this is only done for the sake of convenience; in this embodiment, the actual image size is changed only in the reduction process of step S3a and the enlargement process of step S3c.

[0062] Relative Value Processing In step S4 of FIG. 4, the relativization processing unit 14 (see FIG. 3) executes the process of converting the pixel value of each pixel of the cell image 31 into a relative value with respect to the background luminance distribution (step (2) above).

[0063] Specifically, as shown in FIG. 6, first, in step S4a, the relativization processing unit 14 performs a filter process (second median filter process) on the cell image 31 to remove noise in the cell image 31.

[0064] The kernel size of the second median filter process is set to a size corresponding to minute noise contained in the cell image 31, and is smaller than the effective size of the kernel of the first median filter process (the kernel size converted into the size of the cell image 31 before reduction). In one example, the kernel size of the second median filter process is several pixels. For example, the kernel of the second median filter process is a square pixel range of 3 x 3 pixels. The cell image 31 after the second median filter process is referred to as cell image 31b.

[0065] In step S4b, the relativization processor 14 converts the pixel value of each pixel in the cell image 31b into a relative value by dividing the pixel value of each pixel in the background image 32. Specifically, the relativization processor 14 generates a normalized image 40 by the following formula. Normalized image 40 = {(cell image 31b / background image 32)-1} x 100 (%) That is, the pixel value of each pixel in the normalized image 40 is the value obtained by dividing the pixel value of the cell image 31b at the same pixel by the pixel value of the background image 32, and then subtracting "1" from the result. The pixel value of each pixel in the normalized image 40 is expressed as a percentage.

[0066] By dividing the pixel value of the cell image 31b by the pixel value of the background image 32, the pixel value of each pixel of the normalized image 40 becomes a dimensionless quantity that represents the "ratio of the luminance of the image signal to the luminance of the background image 32". In other words, the pixel value of each pixel of the normalized image 40 indicates the degree of deviation of that pixel from the luminance level of the background 93. The reason for subtracting "1" after the division is to offset the pixel value so that the pixel value becomes equal to the background luminance when the pixel value is 0%. The pixel value of each pixel of the normalized image 40 indicates that the luminance is equal to the background luminance of that pixel when the pixel value = 0%, and that the luminance is 1.5 times the background luminance of that pixel when the pixel value = 50%. In order to convert the pixel value of each pixel of the cell image 31b into a dimensionless quantity that is a ratio to the background luminance, the process of converting to a relative value in step S4b may be rephrased as a "normalization process".

[0067] 8 shows a graph 46 (profile) of pixel values ​​for each pixel along a line 45 of normalized image 40. In graph 46, the horizontal axis indicates position (pixels along line 45) and the vertical axis indicates pixel value (percentage). The position of line 45 is the same as the position of line 35 shown in cell image 31 and background image 32 of FIG.

[0068] In graph 46 of normalized image 40, the pixel value level of the background (baseline) is approximately constant around 0% (in the range of -5% to 5%), and the variation in background brightness that existed in graph 36 of FIG. 7 is eliminated. Therefore, even if a constant binarization threshold is applied to the entire image in the binarization process, the cell region and the background region can be accurately distinguished. In other words, since the brightness distribution of the background 93 (see FIG. 2) is constant in normalized image 40, low brightness regions such as the detailed structure 92 (see FIG. 2) of the cell region and the inner part of the cell body 91 (see FIG. 2) can be easily distinguished from the background 93.

[0069] Extraction of frequency components In step S5 of Fig. 4, the frequency component extraction unit 15 (see Fig. 3) executes a process (step (3) above) of acquiring a frequency component image 50 by extracting predetermined frequency components corresponding to the detailed structure 92 of the cell 90 from the normalized image 40 (cell image 31 converted into relative values). The process of acquiring the frequency component image 50 will be described in detail with reference to Fig. 9.

[0070] Step S5 of acquiring frequency component image 50 includes step S5a of acquiring smoothing parameters (first parameter, second parameter), step S5b of generating a first smoothed image 41 and a second smoothed image 42 having different frequency characteristics by smoothing processing on normalized image 40, and step S5c of generating frequency component image 50 by the difference between the first smoothed image 41 and the second smoothed image 42.

[0071] In step S5a, the frequency component extraction unit 15 acquires smoothing process parameters (first parameter, second parameter) for generating the first smoothed image 41 and the second smoothed image 42. In this embodiment, the smoothing process is a Gaussian filter process, and the parameter for the smoothing process is the standard deviation σ of the Gaussian filter. The larger the value of the parameter (standard deviation σ), the more high-frequency components are removed from the image (the stronger the blurring). The frequency component extraction unit 15 acquires the first parameter and the second parameter based on image processing conditions preset in the storage unit 20 (see FIG. 1). The second parameter has a larger value than the first parameter.

[0072] In step S5b, the frequency component extraction unit 15 performs Gaussian filtering to which the first parameter is applied on the normalized image 40 to obtain a first smoothed image 41. The frequency component extraction unit 15 performs Gaussian filtering to which the second parameter is applied on the normalized image 40 to obtain a second smoothed image 42.

[0073] In step S5c, the frequency component extraction unit 15 obtains a frequency component image 50 by subtracting the second smoothed image 42 from the first smoothed image 41. Image difference refers to subtracting pixel values ​​between identical pixels.

[0074] The frequency component image 50 will be described with reference to the schematic diagram shown in FIG. 10. Comparing the spatial frequencies of the images, the first smoothed image 41 includes the remaining frequency components excluding the high frequency components from the high frequency side to the first frequency 55 among the frequency components included in the normalized image 40. The second smoothed image 42 includes the remaining frequency components excluding the high frequency components from the high frequency side to the second frequency 56 among the frequency components included in the normalized image 40. Due to the difference in the smoothing parameters, the second frequency 56 is lower than the first frequency 55. Therefore, by subtracting the second smoothed image 42 from the first smoothed image 41, an image (frequency component image 50) including frequency components corresponding to the frequency band between the first frequency 55 and the second frequency 56 is obtained.

[0075] For convenience of explanation, a first frequency 55 and a second frequency 56 are assumed here, but in reality, in smoothing processing using a Gaussian filter, the strength of removal of high-frequency components follows a Gaussian distribution, so that frequency components higher than a specific spatial frequency (first frequency 55, second frequency 56) are not completely removed as shown in Fig. 10. Each smoothed image is an image in which the proportion of frequency components gradually decreases (image becomes blurred) as the spatial frequency increases.

[0076] 10, the frequency component image 50 is obtained by extracting image elements in a specific frequency band. The frequency band to be extracted is determined by a combination of smoothing parameters (first parameter, second parameter) for generating the first smoothed image 41 and the second smoothed image 42. Therefore, by matching the frequency band to be extracted to a frequency band including a detailed structure 92 of a cell (see FIG. 2), it is possible to obtain a frequency component image 50 in which the detailed structure 92 included in the cell image 31 is selectively extracted.

[0077] Therefore, smoothing parameters (first parameter, second parameter) capable of extracting a frequency band including the detailed structure 92 (filopodium) are selected for the cell image 31.

[0078] Here, as shown in Fig. 11, a plurality of parameter sets 57 of a first parameter and a second parameter are stored in advance in the storage unit 20. Fig. 11 shows an example in which the first parameter is expressed as σ1=ck and the second parameter is expressed as σ2=c(k+1). c is a coefficient that defines a numerical interval for each parameter set 57. k is a variable that specifies the parameter set 57. When c=0.5, in the set 57 of k=1, σ1=0.5 and σ2=1.0. In the set 57 of k=2, σ1=1.0 and σ2=1.5. Fig. 11 shows four parameter sets 57.

[0079] The user selects one of the settings A, B, C, and D corresponding to the variable k=1 to 4 by operating the input unit 112 in the GUI 58 displayed on the display unit 111 (see FIG. 1), for example. The input selection result is transmitted from the computer 110 to the image processing device 100 via the network 130. As a result, in the above-mentioned step S5a (see FIG. 9), the processor 10 (frequency component extraction unit 15) selects a parameter set 57 including a first parameter for generating the first smoothed image 41 and a second parameter for generating the second smoothed image 42 from among a plurality of parameter sets 57 preset in the storage unit 20 according to the user's selection result. As a result, the parameter set 57 according to the frequency band to which the detailed structure 92 depicted in the cell image 30 belongs is acquired according to the user's intention.

[0080] <Binarization processing> In step S6 of FIG. 4, the binarization processing unit 16 (see FIG. 3) executes a process (step (4) above) to obtain a first image 61 obtained by binarizing the normalized image 40 (cell image 31 converted to a relative value) and a second image 62 obtained by binarizing the frequency component image 50.

[0081] The binarization process will be described in detail with reference to Fig. 12. The binarization processor 16 performs binarization process on the normalized image 40 generated in step S4 and the frequency component image 50 generated in step S5. The binarization process in this embodiment is a simple binarization process that binarizes the entire image using one binarization threshold value.

[0082] The binarization processing unit 16 binarizes the normalized image 40 using a first threshold 65 and a second threshold 66 that is lower than the first threshold 65. Therefore, the first image 61 includes a first threshold image 61a obtained by binarizing the cell image 31 using the first threshold 65, and a second threshold image 61b obtained by binarizing the cell image 31 using the second threshold 66 that is lower than the first threshold 65. In addition, the binarization processing unit 16 generates a second image 62 by binarizing the frequency component image 50 using a third threshold 67.

[0083] Since the second threshold 66 is set to a value lower than the first threshold 65, in the second threshold image 61b, pixels with lower brightness (low pixel value) in the normalized image 40 are extracted as white regions. In the graph 46 showing the distribution of pixel values ​​of the normalized image 40 shown in FIG. 8, the baseline corresponding to the background 93 is approximately constant around 0% (in the range of -5% to 5%). Therefore, as an example of the binarization threshold, for example, the second threshold = 10% and the first threshold = 20%. In this case, in the first threshold image 61a (see FIG. 12) binarized with the first threshold 65 set to a value sufficiently higher than the upper limit (5%) of the baseline, the possibility that noise other than cells is extracted as a white region (cell image) can be sufficiently reduced. Since the second threshold 66 is set to a value closer to the upper limit (5%) of the baseline than the first threshold 65, a relatively low brightness (low pixel value) region of the cell 90 such as the detailed structure 92 can be accurately extracted. Therefore, the second threshold image 61b (see FIG. 12) is more likely to extract noise and background as white regions (cell images) than the first threshold image 61a.

[0084] Since the frequency component image 50 is an image in which image elements belonging to a frequency band including the detailed structure 92 of the cell 90 are extracted, it does not include image elements of a low frequency band to which the background 93 belongs, and even if the third threshold 67 is set low, there is no risk of extracting the background 93 as a white region. Therefore, an example of the binarization threshold for the frequency component image 50 is, for example, the third threshold = 1%. That is, in the frequency component image 50, pixels having pixel values ​​greater than zero are extracted as cell regions (white regions). As a result, as shown in FIG. 12, the second image 62 is an image in which image elements corresponding to the contours of the cells including the detailed structure 92 of the cells 90 are extracted as white regions (cell images). Thus, in this embodiment, the third threshold 67 is set to a value lower than the binarization thresholds for the normalized image 40 (the first threshold 65 and the second threshold 66).

[0085] <Composition Processing> In step S7 in Fig. 4, the synthesis processing unit 17 executes synthesis processing (step (5) above) for generating a binary image 70 of the cell image 31 by synthesizing the first image 61 (the binarized cell image 31) and the second image 62 (the binarized frequency component image 50). The details of the synthesis processing will be described with reference to Fig. 13.

[0086] In this embodiment, the step of generating a binary image 70 of the cell image 31 includes a step S7a of removing from the second image 62 the mismatched portions with the second threshold image 61b, and a step S7b of combining the first threshold image 61a with the second image 62a from which the mismatched portions with the second threshold image 61b have been removed.

[0087] In step S7a, the synthesis processing unit 17 performs a logical AND operation between the second threshold image 61b and the second image 62. That is, the synthesis processing unit 17 compares the same pixel in the second threshold image 61b and the second image 62, and when the pixel values ​​are a combination of "1:1", the pixel value of the pixel is set to "1 (white)", and when the pixel values ​​are a combination of "0:0", the pixel value of the pixel is set to "0 (black)". On the other hand, when the pixel values ​​are a combination (1:0, 0:1) that do not match, the pixel value of the pixel is set to "0 (black)". Note that (1:1) means "pixel value of the second threshold image 61b: pixel value of the second image 62". As a result, the pixel value of the non-matching portion of the second image 62 with the second threshold image 61b is converted to "0 (black)", and the non-matching portion between the second threshold image 61b and the second image 62 is removed.

[0088] By calculating the logical product of the second threshold image 61b and the second image 62, image elements such as noise appearing only in either the second image 62 or the second threshold image 61b are removed while image elements of the contours of cells appearing in common with the second threshold image 61b in the second image 62 are left in the white region. Thus, step S7a is a noise removal process.

[0089] For example, when the original cell image 30 is not of high image quality and contains a relatively large number of noise factors, noise can be effectively removed from the second image 62 by step S7a. On the other hand, when the original cell image 30 is of high image quality and contains almost no noise factors, the second image 62 hardly changes even when the logical product of the second threshold image 61b and the second image 62 is calculated, so that the process of step S7a does not need to be performed. Hereinafter, the second image 62 from which the non-matching portion with the second threshold image 61b has been removed by step S7a is referred to as the "second image 62a."

[0090] Next, in step S7b, the synthesis processing unit 17 synthesizes the first threshold image 61a and the second image 62a by calculating a logical sum of the first threshold image 61a and the second image 62a.

[0091] That is, the synthesis processor 17 compares the same pixel in the first threshold image 61a and the second image 62a, and if either pixel value is "1 (white)" (1:1, 1:0, or 0:1), the pixel value of that pixel is set to "1 (white)", and if both pixel values ​​are "0 (black)" (0:0), the pixel value of that pixel is set to "0 (black)." By synthesis (logical sum operation), the synthesis processor 17 generates a binary image 70.

[0092] By calculating the logical sum of the first threshold image 61a and the second image 62a, a binarized image 70 is obtained in which the cell image extracted in the first threshold image 61a is supplemented with a detailed structure 92 that is difficult to extract.

[0093] Post-processing 4, the post-processing unit 18 performs processes such as image shaping and noise removal on the binarized image 70. The post-processing will be described in detail with reference to FIG.

[0094] In step S8a, the post-processing unit 18 performs image shaping processing on the binarized image 70 (before post-processing) output from the synthesis processing unit 17. The image shaping processing is processing for shaping the cell image so as to interpolate localized defects in the cell image.

[0095] Specifically, the post-processing unit 18 performs a closing process on the binarized image 70. The closing process is a process in which a contraction process of a white region in an image is performed after an expansion process on the white region. The closing process can connect linear parts that are disconnected at a short distance and fill holes (black regions) that exist locally inside a cell image without changing the size of the cell image (white region).

[0096] The post-processing unit 18 performs a closing process including one expansion process and one contraction process, for example, by using a kernel 85 shown in Fig. 14. The kernel 85 has a shape in which the four corners of a rectangular region are removed. This makes it possible to prevent the cell image after shaping by the closing process from becoming unnaturally angular.

[0097] As shown in enlarged views 86a and 86b before and after the closing process in FIG. 14, thin notches and holes in the cell image that existed in enlarged view 86a before the process have been filled in enlarged view 86b after the process.

[0098] Next, in step S8b, the post-processing unit 18 performs a noise removal process on the binarized image 70 after the image shaping process.

[0099] The noise removal process in step S8b is a process for removing minute dot-like noise present in the image. Specifically, the post-processing unit 18 removes white areas present in the image that have an area (number of pixels) equal to or less than a predetermined value and an aspect ratio equal to or less than a predetermined value (i.e., sets the pixel value to "0 (black)"). The aspect ratio is the value of (long side / short side) in the minimum circumscribing rectangle of the target white area. As an example, a white area with an area of ​​15 pixels or less and an aspect ratio of 3.0 or less is replaced with a black area. As a result, minute dot-like noise present in the binarized image 70 is removed.

[0100] As a result of the noise removal process, the post-processing unit 18 generates a post-processed binary image 80. This post-processed binary image 80 is output as the final result (processed image) of the binary process on the cell image 30 input to the image processing device 100.

[0101] Storage and output of processed images (binarized images) 4, the processor 10 stores the post-processed binary image 80 output from the post-processing unit 18 in the storage unit 20. The processor 10 also transmits the post-processed binary image 80 to the computer 110 via the network 130. The computer 110 displays the binary image 80 on the display unit 111 as a processing result.

[0102] In this embodiment, in addition to the binarized image 80 after post-processing, each image generated in each of steps S2 to S8 of the image processing is stored in the storage unit 20 as image data 22 (see FIG. 1). The processor 10 can transmit each image stored in the storage unit 20 to the computer 110 in response to a request from the computer 110.

[0103] Therefore, if the user wants to check the binary image 80 after post-processing and change the image processing conditions, the user can visually check the first smoothed image 41, the second smoothed image 42, and the frequency component image 50 to determine whether the setting of the frequency band for extracting the detailed structure 92 (selection of the smoothing parameters) is appropriate, or whether the binary thresholds of the first threshold image 61a, the second threshold image 61b, and the second image 62 are appropriate, thereby visually confirming the appropriateness of the image processing conditions from each image in the processing process.

[0104] This completes the image processing method performed by the image processing device 100 of this embodiment.

[0105] (Action of this embodiment) Next, the operation of the image processing method according to the present embodiment will be described with reference to Fig. 15. Fig. 15(A) shows a binarized image 80 after post-processing obtained by the image processing method according to the present embodiment for a cell image 30. Fig. 15(B) shows a binarized image 500 of a comparative example obtained by performing a known binarization process on the same cell image 30.

[0106] The binarized image 500 of the comparative example is an image obtained by adaptive binarization processing, which calculates a binarization threshold for each small region in an image and applies the obtained binarization threshold to each small region separately, and is known as a process that can suppress the influence of brightness unevenness in an image.

[0107] Comparing the binarized image 80 of the present embodiment with the binarized image 500 of the comparative example, particularly in the region P1, the linear fine structure (cell filopodia) that is interrupted and separated in the binarized image 500 of the comparative example is extracted as a continuous linear region in the binarized image 80 of the present embodiment, improving the extraction accuracy of the fine structure. Also, particularly in the region P2, in the binarized image 500 of the comparative example, the relatively low brightness part in the inner region of the cell body 91 cannot be extracted due to the overlap of the brightness unevenness and the brightness change of the cell itself, resulting in a rough image. In contrast, in the binarized image 80 of the present embodiment, the inner region of the cell body 91 shown in the region P2 is extracted as a uniform white region.

[0108] As described above, according to this embodiment, it has been confirmed that the accuracy of extracting fine structures is improved and the effects of brightness unevenness can be further reduced, even when compared with the binarization method (adaptive binarization processing) conventionally used to suppress the effects of brightness unevenness.

[0109] (Effects of this embodiment) In this embodiment, the following effects can be obtained.

[0110] That is, in the image processing method and image processing device 100 of this embodiment, the background luminance distribution (background image 32) in the cell image 31 is extracted, and the pixel value of each pixel of the cell image 31 is converted into a relative value with respect to the background luminance distribution, so that the pixel value of the normalized image 40 after conversion indicates the degree of deviation from the luminance level of the background 93. Therefore, even if there is luminance unevenness in the cell image 31, binarization processing can be performed based on the degree of deviation from the luminance level of the background 93, so that the influence of the luminance unevenness in the cell image 31 can be reduced. Since the binarization processing is performed on the normalized image 40, even in the low luminance area in the cell 90 close to the background luminance level, a first image 61 that is accurately extracted can be obtained. In addition, since the binarization processing is performed on the frequency component image 50 from which a predetermined frequency component corresponding to the detailed structure 92 of the cell 90 is extracted, a second image 62 in which the detailed structure 92 of the cell 90 is accurately extracted can be obtained. Then, by combining the first image 61 and the second image 62, the first image 61 from which the main morphology of the cell is extracted can be complemented by the second image 62 from which the detailed structure 92 is extracted, so that it is possible to obtain a binarized image 70 from which the entire morphology of the cell 90, including the low brightness areas in the cell 90, is accurately extracted. As described above, in the binarization process of the cell image, the influence of brightness unevenness in the cell image can be reduced, and the low brightness areas of the cell, including the fine structure of the cell, can be accurately extracted.

[0111] In addition, in this embodiment, further effects can be obtained by configuring as follows.

[0112] That is, in this embodiment, in step S3 of extracting the background luminance distribution, a filter process is performed on the cell image 31 to remove the cells 90 from the cell image 31, thereby generating a background image 32 indicating the background luminance distribution. With this configuration, by generating the background image 32 from which the cells 90 have been removed from the cell image 31, the background luminance distribution of the cell image 31 (background pixel value for each pixel) can be easily obtained without complex processing such as image analysis.

[0113] Moreover, in this embodiment, the filter process for removing the cells 90 is a median filter process with a kernel size corresponding to the size of the cells 90 depicted in the cell image 31. With this configuration, a background image 32 showing the background luminance distribution can be easily obtained by a simple process of performing a median filter process on the cell image 31. Here, the median filter process can remove the image element (cell 90) while leaving low-frequency components (background luminance distribution) larger than the kernel by setting an appropriate kernel size corresponding to the image element (cell 90 in this case) to be removed. As a result, the background luminance distribution of the cell image 31 can be extracted with high accuracy.

[0114] In this embodiment, step S3 of extracting the background luminance distribution includes step S3a of reducing the cell image 31, step S3b of performing a filter process for removing the cells 90 from the reduced image 31a, and step S3c of enlarging the background image (reduced background image 32a) after the filter process for removing the cells 90 so as to return it to the image size before reduction. With this configuration, the filter process is performed on the reduced cell image (reduced image 31a), so that the effective kernel size when the image size is returned to the size before reduction can be increased while the kernel size in the filter process is reduced. As a result, the kernel size in the filter process is reduced, so that the amount of calculation in the filter process can be effectively reduced.

[0115] In addition, in this embodiment, in step S4 of converting pixel values ​​into relative values, the pixel value of each pixel of the cell image 31 is converted into a relative value by dividing it by the pixel value of each pixel of the background image 32. With this configuration, the pixel value of each pixel of the cell image 31 can be easily converted into a relative value. Since the pixel value of each pixel after conversion indicates the ratio of the luminance of the image signal to the background luminance, a cell image (normalized image 40) that is not dependent on the relative brightness of each pixel caused by luminance unevenness can be obtained. As a result, the influence of the variation in pixel values ​​caused by luminance unevenness can be reduced even when performing binarization processing.

[0116] In this embodiment, step S5 of acquiring the frequency component image 50 includes step S5b of generating a first smoothed image 41 and a second smoothed image 42 having different frequency characteristics by smoothing the normalized image 40, and step S5c of generating a frequency component image 50 by the difference between the first smoothed image 41 and the second smoothed image 42. With this configuration, a predetermined frequency component can be easily extracted from the normalized image 40 by a simple process of performing a smoothing process with different parameters on the cell image 31 and performing a difference process on the obtained first smoothed image 41 and second smoothed image 42. In addition, in the frequency component extraction process, it is important to appropriately set the frequency range (frequency band) to be extracted according to the detailed structure 92 to be extracted. According to the above configuration, the first smoothed image 41 and the second smoothed image 42 reflecting the difference in the parameters of the smoothing process and the frequency component image 50 which is the difference between them are obtained, so that it is possible to visually confirm whether the detailed structure 92 to be extracted has been appropriately extracted. Therefore, the smoothing parameters can be easily optimized.

[0117] Moreover, in this embodiment, step S5 of acquiring the frequency component image 50 further includes step S5a of selecting a parameter set 57 of a first parameter for generating the first smoothed image 41 and a second parameter for generating the second smoothed image 42 from a plurality of preset parameter sets 57. With this configuration, the user can select a parameter set 57 suitable for a frequency band to which a detailed structure 92 to be extracted belongs from among the plurality of parameter sets 57 options, thereby determining the parameter sets 57 for generating the first smoothed image 41 and the second smoothed image 42. As a result, even a user without specialized knowledge can easily determine appropriate smoothing parameters suited to the detailed structure 92 to be extracted.

[0118] In this embodiment, the smoothing process is a Gaussian filter process, and the parameter of the smoothing process is the standard deviation of the Gaussian filter. With this configuration, the frequency components removed by the Gaussian filter process can be easily adjusted with only one smoothing parameter. Furthermore, since the magnitude of the standard deviation value corresponds to the high and low of the frequency components, the parameter of the smoothing process can be easily determined.

[0119] In this embodiment, the detailed structure 92 of the cell 90 is the filopodia of the cell 90. In a microscopic image of the cell 90, the filopodia have a linear structure extending elongatedly from the cell body 91 and tend to have low pixel values ​​(become dark), making it difficult to distinguish and extract the filopodia from the background 93 by a simple binarization process. For this reason, the binarization method according to this embodiment, in which the filopodia are extracted using the frequency component image 50 and then synthesized, is particularly effective for generating a binarized image 70 in which the filopodia are accurately extracted. In other words, the binarization method according to this embodiment is particularly suitable for generating a binarized image 70 of a cell image 30 in which a fine structure (image elements of high frequency components localized in a small area) of the cell 90 and a structure that tends to have low pixel values ​​is depicted.

[0120] In the present embodiment, the first image 61 includes a first threshold image 61a obtained by binarizing the normalized image 40 with a first threshold 65, and a second threshold image 61b obtained by binarizing the normalized image 40 with a second threshold 66 smaller than the first threshold 65. Step S7 for generating the binarized image 70 of the cell image 31 includes step S7a for removing a mismatched portion with the second threshold image 61b from the second image 62, and step S7b for synthesizing the first threshold image 61a and the second image 62 from which the mismatched portion with the second threshold image 61b has been removed. With this configuration, noise other than the cells 90 in the second image 62 can be removed by removing the mismatched portion with the second threshold image 61b from the second image 62. Furthermore, since the first threshold image 61a is binarized with the first threshold 65 higher than the second threshold 66, it is possible to suppress the background 93 and noise from being mixed into the cell image (white region) extracted by binarization. Then, by combining the second image 62a from which noise has been removed with the first threshold image 61a, it is possible to obtain a binarized image 70 in which the detailed structure 92 of the cell 90 is accurately extracted while suppressing the introduction of noise. Therefore, even if the image quality of the original cell image 30 (cell image 31) is not high, for example, it is possible to generate a binarized image 70 with less noise.

[0121] [Variations] The embodiments disclosed herein should be considered to be illustrative and not restrictive in all respects. The scope of the present invention is indicated by the claims, not by the description of the embodiments above, and further includes all modifications (variations) within the meaning and scope of the claims.

[0122] For example, in the above embodiment, the image processing device 100 functions as a server of the image processing system 200 constructed in a client-server model, but the present invention is not limited to this. In the present invention, for example, as shown in FIG. 16, the image processing device 100 may be configured by an independent computer. In the example of FIG. 16, the image processing device 100 is configured by a computer 300 including a processor 210 and a storage unit 220. A display unit 230 and an input unit 240 are connected to the computer 300. The computer 300 is connected to the imaging device 120 so as to be able to communicate with the image capturing device 120. The processor 210 of the computer 300 includes, as functional blocks, the image acquisition unit 11, the pre-processing unit 12, the background extraction unit 13, the relativization processing unit 14, the frequency component extraction unit 15, the binarization processing unit 16, the synthesis processing unit 17, and the post-processing unit 18 shown in the above embodiment (see FIG. 3).

[0123] In addition, in the above embodiment and the modified example shown in FIG. 16, an example is shown in which all image processing (processing as the image acquisition unit 11, pre-processing unit 12, background extraction unit 13, relativization processing unit 14, frequency component extraction unit 15, binarization processing unit 16, synthesis processing unit 17, and post-processing unit 18) is performed by a single processor 10 (210), but the present invention is not limited to this. Each image processing for the cell image 30 may be shared and performed by multiple processors. Each process may be performed by a separate processor. The multiple processors may be provided in separate computers. In other words, the image processing device 100 may be composed of multiple computers that perform image processing.

[0124] In the above embodiment, an example was shown in which the background image 32 is generated by performing a filter process on the cell image 31 to remove the cells 90 from the cell image 31, but the present invention is not limited to this. In the present invention, the background image 32 may be generated by a method other than the filter process. For example, the background image may be generated by performing a Fourier transform on the cell image 31, extracting a low-frequency band corresponding to the background luminance in the spatial frequency domain, and performing an inverse Fourier transform on the extracted low-frequency band.

[0125] In the above embodiment, an example is shown in which the background image 32 indicating the background luminance distribution is generated, but the present invention is not limited to this. In the present invention, the background luminance distribution does not have to be extracted as an image (background image 32). The background luminance distribution may be acquired, for example, as a function representing the value (pixel value) of the background luminance at each position coordinate in the image. In other words, the background luminance distribution may be expressed by a function that uses the x and y coordinates of the image as variables and outputs a pixel value corresponding to the background luminance.

[0126] In the above embodiment, the filtering process for removing the cells 90 from the cell image 31 is a median filtering process, but the present invention is not limited to this. In the present invention, the filtering process for removing the cells 90 from the cell image 31 may be a filtering process other than the median filtering process.

[0127] In the above embodiment, in step S3 of extracting the background luminance distribution, a filter process for removing the cells 90 is performed on the reduced image 31a, but the present invention is not limited to this. In the present invention, a filter process for removing the cells 90 may be performed on the cell image 31 of the original size without reducing the cell image 31. In this case, it is of course not necessary to enlarge the reduced background image 32a after the filter process.

[0128] In the above embodiment, in step S4 of converting the pixel value of each pixel of cell image 31 into a relative value with respect to background image 32, an example was shown in which the pixel value of normalized image 40 was determined by the formula {(cell image 31b / background image 32)-1}×100(%), but the present invention is not limited to this. In the present invention, the pixel value of normalized image 40 may be determined by an arithmetic formula different from that in the above embodiment. In the above embodiment, it is not necessary to subtract "1" from the value obtained by dividing the pixel value of cell image 31b by the pixel value of background image 32.

[0129] In the above embodiment, an example has been shown in which the frequency component image 50 is generated by subtracting the first smoothed image 41 and the second smoothed image 42, which have different frequency characteristics, but the present invention is not limited to this. In the present invention, the frequency component image 50 may be generated by a method other than image subtraction. For example, the normalized image 40 may be Fourier transformed, a predetermined frequency band corresponding to the detailed structure 92 may be extracted in the spatial frequency domain, and the other frequency components may be removed and then the frequency component image may be generated by inverse Fourier transform.

[0130] In the above embodiment, an example is shown in which, when generating the frequency component image 50, the parameter set 57 of the first parameter and the second parameter is selected from a plurality of preset parameter sets 57 (see FIG. 11 ), but the present invention is not limited to this. In the present invention, the parameter set 57 for generating the frequency component image 50 does not have to be selected from a plurality of parameter sets 57. The input of the value of the first parameter and the input of the value of the second parameter may be received separately.

[0131] In the above embodiment, the smoothing process for generating the first smoothed image 41 and the second smoothed image 42 is Gaussian filter processing, but the present invention is not limited to this. The smoothing process for generating the first smoothed image 41 and the second smoothed image 42 may be performed by a filter processing other than Gaussian filter processing. The other filter processing may be, for example, a moving average filter processing. Since the parameters of the smoothing process differ depending on the filter processing, the parameters of the smoothing process are not limited to the standard deviation σ.

[0132] In addition, in the above embodiment, an example was shown in which the detailed structure 92 of the cell 90 is a filopodia of the cell 90, but the present invention is not limited to this. The detailed structure may be something other than a filopodia. The detailed structure of a cell is a structure that is relatively finer than the main structure of the cell in a cell image, and is a concept that depends on the imaging magnification of the cell image. For example, in a cell image in which only the filopodia of a cell are captured at an imaging magnification higher than that of the cell image 30 (see FIG. 2) described in the above embodiment, the detailed structure may be a part of the filopodia captured in the image.

[0133] In the above embodiment, the first threshold image 61a and the second threshold image 61b having different binarization thresholds are generated, but the present invention is not limited to this. In the present invention, only one first image 61 may be generated using one binarization threshold. In this case, the first image 61 and the second image 62 are simply synthesized without performing step S7a.

[0134] In the above embodiment, an example was shown in which pre-processing was performed to convert the cell image 30, which is a color image, into a grayscale cell image 31, but the present invention is not limited to this. If the original cell image is a grayscale image, pre-processing is not necessary. In addition, when performing pre-processing, an example was shown in which the color component image with the highest brightness value was selected as the grayscale cell image 31, but the grayscale cell image 31 may also be generated by averaging the pixel values ​​of the color component images of each color.

[0135] In the above embodiment, an example was shown in which post-processing was performed on the binarized image 70 obtained by the synthesis process, and the binarized image 80 after post-processing was generated as the final processed image, but the present invention is not limited to this. In the present invention, post-processing does not have to be performed. The binarized image 70 obtained by the synthesis process may be generated as the final processed image.

[0136] Furthermore, the specific values ​​of the kernel size, smoothing parameter, binarization threshold, and the like shown in the above embodiment are merely examples, and are not limited to the above values.

[0137] [Aspects] It will be appreciated by those skilled in the art that the exemplary embodiments described above are examples of the following aspects.

[0138] (Item 1) An image processing method for performing binarization processing on a cell image which is a multi-valued image, comprising the steps of: extracting a background luminance distribution in the cell image; converting a pixel value of each pixel of the cell image into a relative value with respect to the background luminance distribution; obtaining a frequency component image by extracting predetermined frequency components corresponding to detailed structures of cells from the converted cell image; obtaining a first image obtained by binarizing the converted cell image and a second image obtained by binarizing the frequency component image; generating a binary image of the cell image by combining the first image and the second image.

[0139] (Item 2) 2. The image processing method according to item 1, wherein in the step of extracting the background luminance distribution, a background image showing the background luminance distribution is generated by performing a filter process on the cell image to remove cells from the cell image.

[0140] (Item 3) 3. The image processing method according to item 2, wherein the filter process for removing the cells is a median filter process with a kernel size corresponding to the size of the cells appearing in the cell image.

[0141] (Item 4) The step of extracting a background luminance distribution includes: reducing the cell image; performing a filter process on the reduced cell image to remove the cells; and enlarging the background image after the filter process for removing the cells so as to return the image size to the size before reduction.

[0142] (Item 5) 5. The image processing method according to any one of items 2 to 4, wherein in the step of converting into relative values, the pixel value of each pixel of the cell image is converted into the relative value by dividing the pixel value of each pixel of the background image.

[0143] (Item 6) The step of acquiring a frequency component image includes: generating a first smoothed image and a second smoothed image having different frequency characteristics by performing a smoothing process on the cell image; 6. The image processing method according to any one of items 1 to 5, further comprising: generating the frequency component image from a difference between the first smoothed image and the second smoothed image.

[0144] (Item 7) The step of acquiring a frequency component image includes: 7. The image processing method according to item 6, further comprising a step of selecting a set of first parameters for generating the first smoothed image and a set of second parameters for generating the second smoothed image from among a plurality of preset sets of the parameters.

[0145] (Item 8) the smoothing process is a Gaussian filter process, 8. The image processing method according to item 7, wherein the parameter of the smoothing process is the standard deviation of a Gaussian filter.

[0146] (Item 9) 9. The image processing method according to any one of items 1 to 8, wherein the detailed structure of a cell is a filopodia of a cell.

[0147] (Item 10) the first image includes a first threshold image obtained by binarizing the cell image with a first threshold, and a second threshold image obtained by binarizing the cell image with a second threshold smaller than the first threshold, The step of generating a binarized image of the cell image includes: removing from the second image portions that do not match the second threshold image; 10. The image processing method according to any one of items 1 to 9, further comprising: combining the first threshold image with the second image from which a mismatched portion with the second threshold image has been removed.

[0148] (Item 11) an image acquisition unit for acquiring a cell image which is a multi-valued image; a background extraction unit that extracts a background luminance distribution in the cell image; a relativization processing unit that converts the pixel value of each pixel of the cell image into a relative value with respect to the background luminance distribution; a frequency component extraction unit that acquires a frequency component image by extracting predetermined frequency components corresponding to detailed structures of the cells from the converted cell image; a binarization processing unit that obtains a first image obtained by binarizing the converted cell image and a second image obtained by binarizing the frequency component image by a binarization process; a synthesis processing unit that generates a binary image of the cell image by synthesizing the first image and the second image. [Explanation of symbols]

[0149] 11 Image acquisition section 13 Background extraction part 14 Relativization processing section 15 Frequency component extraction section 16 Binarization processing section 17 Synthesis Processing Section 30, 31, 31b Cell images 31a Reduced image (reduced cell image) 32 Background Images 32a Reduced background image (background image after filtering) 40 Normalized images (cell images converted to relative values) 41 First smoothed image 42 Second smoothed image 50 Frequency Component Images 57 Parameter Set 61 1st image 61a First threshold image 61b Second threshold image 62, 62a 2nd image 65 First Threshold 66 Second Threshold 70, 80 Binarized images 90 cells 92 Detailed structure 93 Background 100 Image processing device

Claims

1. An image processing method for performing binarization processing on a cell image which is a multi-valued image, comprising the steps of: extracting a background luminance distribution in the cell image; converting a pixel value of each pixel of the cell image into a relative value with respect to the background luminance distribution; obtaining a frequency component image by extracting predetermined frequency components corresponding to detailed structures of cells from the converted cell image; obtaining a first image obtained by binarizing the converted cell image and a second image obtained by binarizing the frequency component image; generating a binary image of the cell image by combining the first image and the second image.

2. 2. The image processing method according to claim 1, wherein in the step of extracting the background luminance distribution, a background image showing the background luminance distribution is generated by performing a filter process on the cell image to remove cells from the cell image.

3. The image processing method according to claim 2 , wherein the filter process for removing the cells is a median filter process having a kernel size corresponding to the size of the cells appearing in the cell image.

4. The step of extracting a background luminance distribution includes: reducing the cell image; performing a filter process on the reduced cell image to remove the cells; The image processing method according to claim 2 , further comprising a step of enlarging the background image after the filter process for removing the cells so as to return the background image to an image size before reduction.

5. 3. The image processing method according to claim 2, wherein in the step of converting into relative values, the pixel value of each pixel of the cell image is converted into the relative value by dividing the pixel value of each pixel of the background image.

6. The step of acquiring a frequency component image includes: generating a first smoothed image and a second smoothed image having different frequency characteristics by performing a smoothing process on the cell image; The image processing method according to claim 1 , further comprising: generating the frequency component image by using a difference between the first smoothed image and the second smoothed image.

7. The step of acquiring a frequency component image includes:

7. The image processing method according to claim 6, further comprising a step of selecting a set of first parameters for generating the first smoothed image and a set of second parameters for generating the second smoothed image from among a plurality of preset sets of the parameters.

8. the smoothing process is a Gaussian filter process, 8. The image processing method according to claim 7, wherein the parameter of the smoothing process is a standard deviation of a Gaussian filter.

9. The image processing method according to claim 1 , wherein the cellular microstructure is a filopodia of a cell.

10. the first image includes a first threshold image obtained by binarizing the cell image with a first threshold, and a second threshold image obtained by binarizing the cell image with a second threshold smaller than the first threshold, The step of generating a binarized image of the cell image includes: removing from the second image portions that do not match the second threshold image; The image processing method according to claim 1 , further comprising the step of: combining the first threshold image with the second image from which a mismatched portion between the first threshold image and the second threshold image has been removed.

11. an image acquisition unit for acquiring a cell image which is a multi-valued image; a background extraction unit that extracts a background luminance distribution in the cell image; a relativization processing unit that converts the pixel value of each pixel of the cell image into a relative value with respect to the background luminance distribution; a frequency component extraction unit that acquires a frequency component image by extracting predetermined frequency components corresponding to detailed structures of the cells from the converted cell image; a binarization processing unit that obtains a first image obtained by binarizing the converted cell image and a second image obtained by binarizing the frequency component image by a binarization process; a synthesis processing unit that generates a binary image of the cell image by synthesizing the first image and the second image.

Images