Image processing method and image processing apparatus

By extracting the background brightness distribution from cell images and converting it to relative values, obtaining frequency component images, and synthesizing binarized images, the influence of uneven brightness is resolved, achieving high-precision extraction of cell microstructures and improving the accuracy of cell region recognition.

CN117581100BActive Publication Date: 2026-11-10SHIMADZU SEISAKUSHO LTD
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202280046224.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2021-07-29
Filing Date
2022-05-20
Publication Date
2026-11-10
Estimated Expiration
2042-05-20

AI Technical Summary

Technical Problem

In the binarization of cell images, uneven brightness makes it difficult to accurately extract cell regions and fine structures, especially the distinction between pseudopodia and the main cell body, which affects the accuracy of cell size calculation.

Method used

By extracting the background brightness distribution of cell images, converting pixel values ​​to relative values, obtaining frequency component images, and synthesizing binarized images, the influence of uneven brightness is reduced, and low-brightness areas of cells are accurately extracted.

Benefits of technology

Even in the presence of uneven brightness, it can extract the fine structure and main morphology of cells with high precision, generate accurate binary images, reduce the separation of pseudopodia from the main cell body, and improve the accuracy of cell size calculation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117581100B_ABST
    Figure CN117581100B_ABST
Patent Text Reader

Abstract

This image processing method includes the following steps: converting the pixel values ​​of each pixel in the cell image (31) into relative values; obtaining a frequency component image (50) obtained by extracting a specified frequency component from the converted cell image (40); obtaining a first image (61a) obtained by binarizing the converted cell image (40) and a second image (62) obtained by binarizing the frequency component image (50); and generating a binarized image (70) of the cell image (31) by combining the first image (61) and the second image (62).
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to an image processing method and an image processing apparatus. Background Technology

[0002] Previously, techniques for binarizing cell images have been disclosed. Such techniques are disclosed, for example, in Japanese Patent Application Publication No. 2014-18184.

[0003] Japanese Patent Application Publication No. 2014-18184 discloses a technique for evaluating the colony quality of pluripotent stem cells based on differential filtering of microscopic images obtained using an optical microscope. Furthermore, it discloses a method of performing binarization processing to improve the accuracy of image resolution.

[0004] Existing technical documents

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Application Publication No. 2014-18184 Summary of the Invention

[0007] The problem the invention aims to solve

[0008] Binarization of cell images is used to classify cell regions from background regions in an image to resolve morphological features such as cell size. However, binarization of cell images suffers from problems such as difficulty in extracting cell regions with high precision, as described below.

[0009] First, it is difficult to set a threshold to distinguish cell regions from background regions when there is uneven brightness in cell images due to illumination deviations in microscope images.

[0010] Second, in cell images, the brightness values ​​of the inner regions of cells and the cell's fine structures tend to be low. When such low-brightness areas of cells overlap with the aforementioned uneven brightness, it is difficult to accurately extract these low-brightness areas through binarization. For example, the low-brightness portion of the inner region of a cell may appear as a pore in the binarized image. Furthermore, as a fine structure of the cell, there are pseudopodia protruding from the cell body. When binarizing cells with pseudopodia, it is sometimes impossible to distinguish the low-brightness portion of the pseudopodia from the background, resulting in the pseudopodia separating from the cell body in the binarized image. For example, when calculating the cell size, the size of the binarized cell region is smaller than the actual size of the cell as a whole, including the pseudopodia (the amount excluded from the size calculation).

[0011] Therefore, it is desirable to mitigate the effects of brightness unevenness specific to microscope images during the binarization process of cell images in order to accurately extract low-brightness regions of cells containing their fine structures.

[0012] The present invention was made to solve the problems mentioned above. One object of the present invention is to provide an image processing method and image processing apparatus that can reduce the influence of brightness unevenness in cell images during binarization processing of cell images to accurately extract low-brightness regions of cells containing the fine structure of cells.

[0013] Solution for solving the problem

[0014] To achieve the above objectives, the image processing method of the first aspect of the present invention is used to binarize a cell image as a multi-valued image. The image processing method includes the following steps: extracting the background brightness distribution in the cell image; converting the pixel values ​​of each pixel in the cell image into relative values ​​with respect to the background brightness distribution; obtaining a frequency component image obtained by extracting a predetermined frequency component corresponding to the fine structure of the cell from the converted cell image; obtaining a first image obtained by binarizing the converted cell image and a second image obtained by binarizing the frequency component image; and generating a binarized image of the cell image by combining the first image and the second image.

[0015] The image processing apparatus of a second aspect of the present invention comprises: an image acquisition unit that acquires a cell image as a multi-valued image; a background extraction unit that extracts a background brightness distribution in the cell image; a relativization processing unit that converts the pixel values ​​of each pixel in the cell image into relative values ​​relative to the background brightness distribution; a frequency component extraction unit that acquires a frequency component image obtained by extracting predetermined frequency components corresponding to the fine structure of the cell from the converted cell image; a binarization processing unit that acquires a first image obtained by binarizing the converted cell image through binarization processing and a second image obtained by binarizing the frequency component image through binarization processing; and a synthesis processing unit that generates a binarized image of the cell image by synthesizing the first image and the second image.

[0016] The effects of the invention

[0017] In the image processing method of the first aspect and the image processing apparatus of the second aspect described above, the background brightness distribution in the cell image is extracted, and the pixel value of each pixel in the cell image is converted into a relative value relative to the background brightness distribution. Therefore, the converted pixel value of the cell image represents the degree of deviation from the brightness level of the background. Thus, even if there is brightness unevenness in the cell image, the influence of brightness unevenness can be mitigated because binarization processing can be performed based on the degree of deviation of each pixel from the brightness level of the background. Since the cell image after being converted into relative values ​​is binarized, even low-brightness areas within the cell that are close to the background brightness level can be extracted with high precision into a first image. In addition, since the frequency component image obtained by extracting the prescribed frequency components corresponding to the fine structure of the cell is binarized, a second image obtained by accurately extracting the fine structure of the cell can be obtained. Then, by combining the first image and the second image, the information of the first image obtained by extracting the fine structure can be supplemented by the second image obtained by extracting the main morphology of the cell. Therefore, a binarized image obtained by accurately extracting the overall morphology of the cell, including the low-brightness areas within the cell, can be obtained. Based on the above, it is possible to reduce the impact of uneven brightness in cell images during binarization processing, thereby accurately extracting low-brightness regions of cells containing their fine structures. Attached Figure Description

[0018] Figure 1 This is a block diagram showing an image processing system equipped with the image processing apparatus of this embodiment.

[0019] Figure 2 The diagram shows an example of a cell image (A), a binarized image of the cell image (B), a magnified view of the cell image (C), and a magnified view of the binarized image (D).

[0020] Figure 3 It is a function block diagram used to illustrate the function of the processor in an image processing device.

[0021] Figure 4 This is a flowchart illustrating the processing operations of the image processing apparatus in this embodiment.

[0022] Figure 5 This is a diagram used to illustrate the details of the preprocessing.

[0023] Figure 6 This is a diagram illustrating the details of the background brightness distribution extraction process and the conversion process to relative values.

[0024] Figure 7 It is a graph used to illustrate the brightness distribution of cell images and background images.

[0025] Figure 8 It is a graph used to illustrate the brightness distribution of a normalized image.

[0026] Figure 9 This is a diagram illustrating the details of the generation process for the frequency component image.

[0027] Figure 10 It is a graph used to illustrate the frequency bands included in a frequency component image.

[0028] Figure 11 This is a diagram used to illustrate the selection of groups of parameters for smoothing processing.

[0029] Figure 12 This is a diagram used to illustrate the details of binarization processing.

[0030] Figure 13 This is a diagram used to illustrate the details of the composite processing of the first and second images.

[0031] Figure 14 This is a diagram used to illustrate the details of post-processing.

[0032] Figure 15 This is a diagram showing the post-processed binarized image (A) of this embodiment and the binarized image (B) of the comparative example.

[0033] Figure 16 This is a block diagram showing a modified example of an image processing apparatus. Detailed Implementation

[0034] Hereinafter, embodiments embodying the present invention will be described with reference to the accompanying drawings.

[0035] Reference Figures 1-15 This will be used to explain the structure and image processing method of the image processing system 200 equipped with the image processing apparatus 100 of this embodiment.

[0036] (Image processing system)

[0037] Figure 1 The image processing system 200 shown is an image processing system that allows users performing cell culture and other activities to comprehensively perform image capture of cell image 30, image processing of cell image 30, and browsing of processed images using a single system.

[0038] (Overview of an image processing system)

[0039] The image processing system 200 includes an image processing device 100, a computer 110, and a camera device 120.

[0040] exist Figure 1The image processing system 200, constructed as a client-server model, is illustrated in the diagram. Computer 110 functions as a client terminal within the image processing system 200. Image processing device 100 functions as a server within the image processing system 200. Image processing device 100, computer 110, and camera device 120 are connected via network 130 in a manner enabling mutual communication. Image processing device 100 performs various information processing tasks based on requests (processing requests) from the user-operated computer 110. Image processing device 100 performs image processing on cell image 30 according to the request and sends the processed image to computer 110. The acceptance of operations by image processing device 100 and the display of the image processed by image processing device 100 are performed on a GUI (Graphical User Interface) displayed on display unit 111 of computer 110.

[0041] Network 130 connects image processing device 100, computer 110, and camera device 120 in a manner that enables them to communicate with each other. Network 130 may be, for example, a LAN (Local Area Network) built within a facility. Network 130 may be, for example, the Internet. If network 130 is the Internet, image processing system 200 may be a system built in a cloud computing manner.

[0042] Computer 110 is a so-called personal computer, equipped with a processor and storage unit. Computer 110 is connected to a display unit 111 and an input unit 112. Display unit 111 is, for example, a liquid crystal display device. Display unit 111 can also be an electroluminescent display device, a projector, or a head-mounted display. Input unit 112 is, for example, an input device including a mouse and keyboard. Input unit 112 can also be a touch panel. Computer 110 is provided with one or more of these components in the image processing system 200.

[0043] The imaging device 120 generates a cell image 30 obtained by photographing the cells. The imaging device 120 can send the generated cell image 30 to a computer 110 and / or an image processing device 100 via a network 130. The imaging device 120 captures microscopic images of the cells. The imaging device 120 performs image processing using methods such as bright-field observation, dark-field observation, phase-difference observation, and differential interferometry. One or more imaging devices 120 are used corresponding to the imaging method. One or more imaging devices 120 can be installed in the image processing system 200.

[0044] The image processing device 100 includes processors 10 such as a CPU (Central Processing Unit), FPGA (Field-Programmable Gate Array), and ASIC (Application Specific Integrated Circuit). The processor 10 performs computational processing as the image processing device 100 by executing a predetermined program 21.

[0045] The image processing apparatus 100 includes a storage unit 20. The storage unit 20 includes a non-volatile storage device, such as a hard disk drive or a solid-state drive. Various programs 21 executed by the processor 10 are stored in the storage unit 20. Image data 22 is also stored in the storage unit 20. The image data 22 includes a cell image 30 captured by the imaging device 120 and various processed images (binarized images 80) generated by image processing of the cell image 30. In this embodiment, the binarization processing of the cell image 30, in particular, among the image processing functions that the image processing apparatus 100 can perform, will be described.

[0046] Image processing apparatus 100 performs binarization processing on cell image 30 according to a request from computer 110. As a result of image processing, image processing apparatus 100 generates a binarized image 80 of cell image 30. Image processing apparatus 100 sends the generated binarized image 80 to computer 110. Computer 110, upon receiving the information, causes display unit 111 to display binarized image 80.

[0047] <Cell Images>

[0048] like Figure 2 As shown, cell image 30 is, for example, a microscope image of cultured cells grown using cell culture equipment. The type of cell image 30 is not particularly limited, but as an example, in this embodiment, cell image 30 is a fluorescently stained image of cells. Cell image 30 is a multi-valued image (multi-grayscale image) and a color image.

[0049] An image of cell 90 (cell image) and background 93 is captured in cell image 30. Figure 2 The cell 90 in cell image 30 shown in (A) includes the cell body 91 and fine structures 92. Figure 2 The example shown illustrates a fine structure 92, which is a pseudopodia protruding from the cell body 91, specifically a filamentous pseudopodia protruding from the cell body 91 in a filamentous (linear) manner. Figure 2 (A) Figure 2In (C), the region of fine structure 92 and the inner region of cell body 91 are easily included in the low brightness region of cell 90 with relatively low brightness (low pixel value) (easily darkened within cell image 30).

[0050] like Figure 2 (B) and Figure 2 As shown in (D), binarization is a process that sets the pixel value of pixels with a binarization threshold or higher to "1 (white)" and the pixel value of pixels with a binarization threshold or lower to "0 (black)" for the image being processed. In the binarization of cell image 30, cell image 30 is binarized by extracting the image of cell 90 (cell image) as a white area and making the background 93 other than cell 90 a black area. The binarization threshold is set to a value that distinguishes the range of pixel values ​​belonging to the cell image from the range of pixel values ​​belonging to the background 93.

[0051] Therefore, when the brightness distribution of the background 93 is deviated, resulting in relatively bright and relatively low-brightness parts in the background 93, the difference between the pixel value of the low-brightness area in the cell 90 and the pixel value of the high-brightness part in the background 93 becomes smaller. As a result, it is difficult to extract the low-brightness area (towards the white area) using binarization processing.

[0052] In the image processing apparatus 100 of this embodiment, even if the brightness distribution of the background 93 in the cell image 30 deviates, such as Figure 2 (B) and Figure 2 As shown in (D), it is also possible to generate a binarized image 80 that extracts low-brightness regions from cells 90 with high precision. Details of the image processing apparatus 100 will be described below.

[0053] (Detailed structure of the image processing device)

[0054] Figure 3 This is a block diagram showing the structure related to the binarization processing of the image processing apparatus 100 and an overview of the image processing.

[0055] The processor 10 of the image processing apparatus 100 includes an image acquisition unit 11, a preprocessing unit 12, a background extraction unit 13, a relativization processing unit 14, a frequency component extraction unit 15, a binarization processing unit 16, a compositing processing unit 17, and a post-processing unit 18 as functional blocks. In other words, the processor 10 performs the functions of the image acquisition unit 11, the preprocessing unit 12, the background extraction unit 13, the relativization processing unit 14, the frequency component extraction unit 15, the binarization processing unit 16, the compositing processing unit 17, and the post-processing unit 18 by executing the program 21 stored in the storage unit 20.

[0056] The image acquisition unit 11 has the function of acquiring cell images 30. The image acquisition unit 11 reads data from the storage unit 20 (see reference 20). Figure 1 The image acquisition unit 11 can acquire the cell image 30 stored in the image acquisition unit 130 as the object of binarization processing. The image acquisition unit 11 can also acquire the cell image 30 via the network 130 (see reference 130). Figure 1 The image acquisition unit 11 acquires cell images 30 sent from the camera device 120 or the computer 110. The image acquisition unit 11 outputs the acquired cell images 30 to the preprocessing unit 12.

[0057] The preprocessing unit 12 performs preprocessing for binarization. Specifically, the preprocessing unit 12 converts the color cell image 30 into a grayscale cell image 31. A grayscale image is a monochrome (without color information) multi-valued image. The pixel value of each pixel in the cell image 31 represents the brightness of that pixel (the brightness of the image). The preprocessing unit 12 outputs the grayscale cell image 31 to the background extraction unit 13 and the relativization processing unit 14, respectively.

[0058] Background extraction unit 13 has the function of extracting the background brightness distribution in cell image 31. Background brightness distribution refers to the brightness distribution of ambient light (illumination light) in cell image 31. The background brightness distribution is ideal across the entire image, but in reality, the background brightness distribution in cell image 31 deviates due to variations in illumination light intensity, etc. Furthermore, the background brightness distribution differs for each cell image 31 due to variations in exposure time, etc. Background extraction unit 13 generates a background image 32 representing the background brightness distribution of cell image 31. Background image 32 is an image that represents the background brightness distribution of cell image 31 in one pixel unit.

[0059] Here, the pixel value of each pixel in cell image 31 can be considered as the sum of cell image components and background brightness components. Cell image components include the cells 90 (refer to) that are the objects of observation. Figure 2 The light signal of the image information is obtained by imagerization. The background brightness component is obtained by imagerization of the background light that is inevitably observed in the imaging environment of cell 90. Therefore, by removing the cell image component from each pixel of cell image 31, the background brightness component, i.e., the background brightness distribution, of each pixel can be obtained.

[0060] In this embodiment, the background extraction unit 13 generates a background image 32 showing the background brightness distribution by performing a filtering process on the cell image 31 to remove cells 90 from the cell image 31. The filtering process to remove cells 90 is a median filtering process based on the kernel size corresponding to the size of the cells 90 captured in the cell image 31. The median filtering process is a process that replaces the pixel value of the central pixel of interest within the kernel with the median value of the pixel values ​​of the surrounding pixels within the kernel, excluding the pixel of interest. The background extraction unit 13 outputs the generated background image 32 to the relativization processing unit 14. The background image 32 is an example of the "background brightness distribution" of this disclosure.

[0061] The relativization processing unit 14 converts the pixel values ​​of each pixel in the cell image 31 into relative values ​​relative to the background brightness distribution. Based on the background image 32, the relativization processing unit 14 generates a normalized image 40 obtained by converting the pixel values ​​of each pixel in the cell image 31 into relative values. Although described later, the pixel values ​​of each pixel in the normalized image 40 represent the ratio of the brightness of that pixel to the brightness of the background. The relativization processing unit 14 outputs the generated normalized image 40 to the frequency component extraction unit 15 and the binarization processing unit 16, respectively.

[0062] Frequency component extraction unit 15 acquires fine structures 92 of cells 90 extracted from normalized image 40 (refer to...). Figure 2 The frequency component image 50 is obtained by extracting the specified frequency components corresponding to the background 93. When considering the spatial frequency of the image, the fine structure 92 of the cell 90 corresponds to a high-frequency component with a higher frequency than the background 93, which is a low-frequency component, and can thus be clearly distinguished from the background 93. Therefore, the frequency component extraction unit 15 extracts the specified frequency components corresponding to the fine structure 92 from the normalized image 40 to generate the frequency component image 50 obtained by excluding the background 93 and extracting the fine structure 92. The frequency component extraction unit 15 outputs the generated frequency component image 50 to the binarization processing unit 16.

[0063] The binarization processing unit 16 binarizes the normalized image 40 and the frequency component image 50. The binarization processing unit 16 generates a first image 61 obtained by binarizing the normalized image 40 and a second image 62 obtained by binarizing the frequency component image 50. Furthermore, in this embodiment, the binarization processing unit 16 generates multiple first images 61, such as a first threshold image 61a and a second threshold image 61b, by binarizing the normalized image 40 using different binarization thresholds. The binarization processing unit 16 outputs the generated first images 61 (first threshold image 61a and second threshold image 61b) and the second image 62 to the synthesis processing unit 17.

[0064] The synthesis processing unit 17 generates a binarized image 70 of the cell image 31 by synthesizing a first image 61 obtained by binarizing the normalized image 40 and a second image 62 obtained by binarizing the frequency component image 50. The main portion of the cell 90, excluding the fine structure 92, is contained in the first image 61. The fine structure 92 of the cell 90 is contained in the second image 62 obtained by binarizing the frequency component image 50. By synthesizing these images, a binarized image 70 is obtained by extracting both the main portion of the cell 90 and the fine structure 92 of the cell 90. Furthermore, although described later, a first threshold image 61a is used for synthesis with the second image 62, and a second threshold image 61b is used for noise removal processing of the second image 62. The synthesis processing unit 17 outputs the binarized image 70 of the cell image 31 to the post-processing unit 18.

[0065] The post-processing unit 18 performs image shaping, noise removal, and other processing on the binarized image 70 to generate a post-processed binarized image 80. The post-processed binarized image 80 is stored in the storage unit 20 as the final binarization result of the input cell image 30. In addition, the binarized image 80 is sent to the computer 110 upon request and displayed on the display unit 111.

[0066] (Image processing methods)

[0067] Next, the image processing method of this embodiment will be described. The image processing method of this embodiment is an image processing method for binarizing a cell image 31, which is a multi-valued image. The image processing method can be executed by the image processing apparatus 100 (processor 10).

[0068] The image processing method of this embodiment includes at least the following steps.

[0069] (1) The step of extracting the background brightness distribution (background image 32) from cell image 30.

[0070] (2) The step of converting the pixel values ​​of each pixel in cell image 30 into relative values ​​relative to the background brightness distribution (normalized image 40),

[0071] (3) The step of obtaining a frequency component image 50 obtained by extracting the specified frequency components corresponding to the fine structure 92 of cell 90 from the converted cell image 31 (normalized image 40).

[0072] (4) The steps of obtaining the first image 61 obtained by binarizing the converted cell image 30 and the second image 62 obtained by binarizing the frequency component image 50.

[0073] (5) The step of generating a binarized image 70 of cell image 31 by synthesizing the first image 61 and the second image 62.

[0074] Step (1) of extracting the background brightness distribution in the cell image 31 is performed by the background extraction unit 13. Step (2) of converting the pixel values ​​of each pixel in the cell image 31 into relative values ​​relative to the background brightness distribution is performed by the relativization processing unit 14. Step (3) of acquiring a frequency component image 50 obtained by extracting a predetermined frequency component corresponding to the fine structure 92 of the cell 90 from the converted cell image 31 is performed by the frequency component extraction unit 15. Step (4) of acquiring a first image 61 obtained by binarizing the converted cell image 30 and a second image 62 obtained by binarizing the frequency component image 50 is performed by the binarization processing unit 16. Step (5) of generating a binarized image 70 of the cell image 31 by merging the first image 61 and the second image 62 is performed by the merging processing unit 17. The image processing method of this embodiment also includes processing performed by the preprocessing unit 12 and processing performed by the postprocessing unit 18.

[0075] Next, refer to Figures 4 to 15 The process of processing performed by the image processing device 100 will be explained in detail.

[0076] <Get Image>

[0077] In step S1, the image acquisition unit 11 (refer to...) Figure 3 Cell images 30 are acquired from storage unit 20, camera device 120, or computer 110. As described above, the acquired cell images 30 are multi-valued images and are color images.

[0078] <Preprocessing>

[0079] In step S2, the preprocessing unit 12 (refer to...) Figure 3 The cell image 30 obtained in step S1 undergoes preprocessing for binarization. (Refer to...) Figure 5 This will explain the details of the preprocessing.

[0080] In step S2a, the preprocessing unit 12 separates the cell image 30, which is a color image, into multiple color component images. The number of separated color component images is the number of color channels contained in the cell image 30. When the color image is represented in RGB mode, the cell image 30 is separated into three color component images: a red image 30r, a green image 30g, and a blue image 30b. Each color component image is a grayscale image.

[0081] In step S2b, the preprocessing unit 12 acquires the distribution of pixel values ​​of the separated multiple color component images (red image 30r, green image 30g, and blue image 30b). For example, the preprocessing unit 12 creates a histogram (histogram Hr, Hg, Hb) of the pixel values ​​of the color component images. A histogram is a graph obtained by calculating the number of pixels with each pixel value.

[0082] In step S2c, the preprocessing unit 12 compares the distribution of pixel values ​​(histograms Hr, Hg, Hb) of each color component image to select the color component image with the highest pixel values. For example, the preprocessing unit 12 selects the color component image with the highest average pixel value from multiple color component images. Figure 5 In the example shown, a green image 30g is selected as the color component image with the highest average pixel value. The preprocessing unit 12 outputs the selected color component image (green image 30g) as a grayscale cell image 31. In this way, the preprocessing unit 12 separates the colored cell image 30 into multiple color component images and uses the color component image with the highest pixel value to generate the grayscale cell image 31.

[0083] Typically, when converting a color image to a grayscale image, the pixel value of each pixel after conversion is the average of the pixel values ​​of the same pixel in each color component image. Therefore, the brightness of a typical grayscale image is the brightness after averaging the brightness of each color component image. However, in a cell fluorescence staining image, the brightness (pixel value) of the specific color component image to which the fluorescence wavelength belongs is significantly high, while the brightness (pixel value) of other color component images that do not contain the fluorescence wavelength is low. Therefore, when the cell image 30, which is a fluorescence staining image, is converted to grayscale using the usual grayscale method, the average brightness of the converted image decreases. Therefore, instead of averaging the pixel values, the cell image 31 using the color component image with the highest pixel value is used as grayscale, thereby suppressing the decrease in the brightness of the grayscale image.

[0084] <Extraction of Background Brightness Distribution>

[0085] exist Figure 4 In step S3, the background extraction unit 13 performs the process of extracting the background brightness distribution in the cell image 31 (step (1) above). (Refer to...) Figure 6 This will explain the details of the process for extracting the background brightness distribution.

[0086] Step S3, which extracts the background brightness distribution, includes: step S3a, which reduces the size of the cell image 31; step S3b, which performs a filtering process to remove cells 90 on the reduced cell image 31; and step S3c, which enlarges the filtered background image 32 in a manner that restores it to the size of the image before reduction.

[0087] In step S3a, the background extraction unit 13 reduces the cell image 31 by a predetermined ratio. The reduction ratio is stored in the storage unit 20 as setting information. The reduction ratio is not particularly limited, but can be in the range of 1 / 2 to 1 / 10. As an example, the background extraction unit 13 reduces the cell image 31 to 1 / 8. The reduced cell image 31 is referred to as the reduced image 31a.

[0088] In step S3b, the background extraction unit 13 performs a first median filtering process to remove cells 90 on the reduced image 31a. The kernel size for the first median filtering process is set to a value that sufficiently removes cells from the image and is sufficiently small compared to the period of brightness variation in the background. The kernel size can vary depending on the magnification of the cell image 30.

[0089] In one example, the kernel size of the first median filtering process on the reduced image 31a is approximately 30 (pixels). For example, the kernel is a square pixel range of 30×30 pixels. The first median filtering process is performed on the 1 / 8 size reduced image 31a, therefore, this process is equivalent to performing median filtering on the unreduced cell image 31 with a kernel size of 8 times (240×240 (pixels)).

[0090] The pixels forming the image of cell 90 (cell image) have higher pixel values ​​than the pixels belonging to the background 93. However, when median filtering is performed with a kernel of sufficient size, the pixel values ​​of the pixels forming the cell image are replaced with the pixel values ​​of the background 93 because the pixel values ​​of the background 93 surrounding cell 90 are taken as the median values. As a result of the first median filtering, the cell image is removed from the reduced image 31a, thereby generating a reduced background image 32a representing the background brightness distribution.

[0091] In step S3c, the background extraction unit 13 enlarges the reduced background image 32a to restore it to its original size. As described above, when the cell image 31 is reduced to 1 / 8 in step S3a, the background extraction unit 13 enlarges the reduced background image 32a by a factor of 8. Thus, a background image 32 of the same size as the cell image 31 is obtained.

[0092] Figure 7 A graph 36 (brightness distribution) showing the pixel values ​​of each pixel along the same line 35 for the original cell image 31 and the acquired background image 32 is provided. The horizontal axis of graph 36 represents the position (pixels along line 35), and the vertical axis of graph 36 represents the pixel value.

[0093] In the pixel value distribution of cell image 31, shown by the solid line in graph 36, locally high pixel value areas represent cell images, while other baselines represent the background. According to graph 36, the level of background pixel values ​​(brightness) is not constant but varies depending on location (baseline fluctuations). This variation in background brightness becomes a factor hindering the extraction of low-brightness areas in the binarization process. In graph 36, when observing the pixel value distribution of background image 32, represented by the dashed line, the distribution of background brightness alone, which varies depending on location, is accurately extracted. Thus, in step S3, background image 32 is generated by background extraction unit 13.

[0094] In addition, Figures 5-15 Although the images shown are of different sizes, this is only for the purpose of illustration. In this embodiment, the actual image size is changed only in the shrinking process of step S3a and the enlargement process of step S3c.

[0095] <Relative Value Processing>

[0096] exist Figure 4 In step S4, the relativization processing unit 14 (refer to) Figure 3 Perform the process of converting the pixel values ​​of each pixel in the cell image 31 into relative values ​​relative to the background brightness distribution (step (2) above).

[0097] Specifically, such as Figure 6 As shown, firstly, in step S4a, the relativization processing unit 14 performs a filtering process (second median filtering process) on the cell image 31 to remove noise from the cell image 31.

[0098] The kernel size for the second median filtering is set to correspond to the fine noise included in the cell image 31, and is smaller than the effective kernel size (the kernel size converted to the size of the cell image 31 before reduction) of the first median filtering. In one example, the kernel size for the second median filtering is a few pixels. For example, the kernel for the second median filtering is a 3×3 pixel square pixel range. The cell image 31 after the second median filtering is referred to as cell image 31b.

[0099] In step S4b, the relativization processing unit 14 converts the pixel values ​​of each pixel in the cell image 31b to relative values ​​by dividing the pixel values ​​of each pixel in the background image 32. Specifically, the relativization processing unit 14 generates the normalized image 40 using the following formula.

[0100] Normalized image 40 = {(cell image 31b / background image 32) - 1} × 100 (%)

[0101] In other words, the pixel value of each pixel in the normalized image 40 is the value obtained by dividing the pixel value of the cell image 31b under the same pixel by the pixel value of the background image 32, and then subtracting "1". The pixel values ​​of each pixel in the normalized image 40 are expressed as a percentage.

[0102] By dividing the pixel values ​​of cell image 31b by the pixel values ​​of background image 32, the pixel values ​​of each pixel in the normalized image 40 are made into a dimensionless quantity representing the ratio of the brightness of the image signal to the brightness of the background image 32. In other words, the pixel values ​​of each pixel in the normalized image 40 represent the degree of deviation of that pixel from the brightness level of the background 93. Subtracting "1" after the division operation is to offset the pixel values ​​so that when the pixel value is 0%, it is the same brightness as the background brightness. Regarding the pixel values ​​of each pixel in the normalized image 40, if the pixel value = 0%, it means that the brightness is the same as the background brightness of that pixel, and if the pixel value = 50%, it means that the brightness is equivalent to 1.5 times the background brightness of that pixel. Since the pixel values ​​of each pixel in cell image 31b are converted into a dimensionless quantity such as a ratio to the background brightness, the process of converting to relative values ​​in step S4b can also be called "normalization process".

[0103] Figure 8 A graph 46 (distribution) shows the pixel values ​​of each pixel along line 45 in the normalized image 40. The horizontal axis of graph 46 represents the position (pixel along line 45), and the vertical axis of graph 46 represents the pixel value (percentage). The position of line 45 is related to the position of the pixel along line 45. Figure 7 The position of line 35 shown in cell image 31 and background image 32 is the same.

[0104] In the curve 46 of the normalized image 40, the pixel value of the background (baseline) is around 0% (in the range of -5% to 5%) and is approximately constant. Figure 7 The background brightness deviation present in curve 36 has disappeared. Therefore, in binarization processing, even if a certain binarization threshold is applied across the entire image, it is possible to distinguish cell regions and background regions with high precision. In other words, since the background 93 (refer to) in the normalized image 40 is... Figure 2 The brightness distribution of ) is constant, therefore, it is easy to detect the fine structures in the cellular region 92 (refer to Figure 2 ), cell body 91 (reference) Figure 2 The inner part of the low-brightness area is distinguished from the background 93.

[0105] Extraction of frequency components

[0106] exist Figure 4 In step S5, the frequency component extraction unit 15 (refer to...) Figure 3Perform the following processing (step (3) above): Obtain a frequency component image 50 obtained by extracting the specified frequency components corresponding to the fine structures 92 of cell 90 from the normalized image 40 (cell image 31 after being converted to relative values). Refer to Figure 9 This will explain the details of the process for acquiring the frequency component image 50.

[0107] Step S5 of acquiring the frequency component image 50 includes: step S5a of acquiring smoothing parameters (first parameter, second parameter), step S5b of generating a first smoothed image 41 and a second smoothed image 42 with different frequency characteristics by smoothing the normalized image 40, and step S5c of generating the frequency component image 50 by the difference between the first smoothed image 41 and the second smoothed image 42.

[0108] In step S5a, the frequency component extraction unit 15 acquires parameters (first parameter, second parameter) for each smoothing process used to generate the first smoothed image 41 and the second smoothed image 42. In this embodiment, the smoothing process is a Gaussian filtering process, and the parameter for the smoothing process is the standard deviation σ of the Gaussian filter. The larger the value of the parameter (standard deviation σ), the more high-frequency components in the image are removed (the stronger the blurring). The frequency component extraction unit 15 is based on the storage unit 20 (refer to...). Figure 1 The image processing conditions are preset in the image to obtain the first parameter and the second parameter. The second parameter is a value that is larger than the first parameter.

[0109] In step S5b, the frequency component extraction unit 15 performs Gaussian filtering on the normalized image 40 with the first parameter applied to obtain a first smoothed image 41. The frequency component extraction unit 15 performs Gaussian filtering on the normalized image 40 with the second parameter applied to obtain a second smoothed image 42.

[0110] In step S5c, the frequency component extraction unit 15 obtains the frequency component image 50 by performing difference between the first smoothed image 41 and the second smoothed image 42. The image difference is achieved by subtracting the pixel values ​​of the same pixel from each other.

[0111] use Figure 10The schematic diagram illustrates the frequency component image 50. When comparing the spatial frequencies of the images, the first smoothed image 41 includes the frequency components contained in the normalized image 40, excluding the high-frequency components from the high-frequency side to the first frequency 55. The second smoothed image 42 includes the frequency components contained in the normalized image 40, excluding the high-frequency components from the high-frequency side to the second frequency 56. Due to the difference in smoothing parameters, the second frequency 56 is lower than the first frequency 55. Therefore, by differentiating the first smoothed image 41 and the second smoothed image 42, an image (frequency component image 50) including the frequency components corresponding to the frequency band between the first frequency 55 and the second frequency 56 is obtained.

[0112] Furthermore, for ease of explanation, a first frequency of 55 and a second frequency of 56 are assumed here. However, in reality, during the smoothing process using Gaussian filtering, the removal intensity of high-frequency components follows a Gaussian distribution. Therefore, it is not possible to simply assume a first frequency of 55 and a second frequency of 56. Figure 10 As shown, frequency components higher than a specific spatial frequency (first frequency 55, second frequency 56) are completely removed. Each smoothed image becomes an image where the proportion of frequency components gradually decreases as the spatial frequency increases (image blurring).

[0113] like Figure 10 Thus, the frequency component image 50 is an image obtained by extracting image features from specific frequency bands. The extracted frequency bands are determined based on a combination of smoothing parameters (first parameter, second parameter) used to generate the first smoothed image 41 and the second smoothed image 42. Therefore, by associating the frequency bands to be extracted with the fine structures 92 including cells (see reference...), the extracted frequency bands are further refined. Figure 2 The frequency bands of the cells are matched, thereby enabling the selective extraction of the frequency component image 50 obtained from the fine structures 92 contained in the cell image 31.

[0114] Therefore, for cell image 31, smoothing parameters (first parameter, second parameter) that can extract frequency bands including fine structures 92 (filopodia) are selected.

[0115] Here, as Figure 11 As shown, multiple groups 57 of parameters for the first and second parameters are pre-stored in the storage unit 20. Figure 11 The example shown is expressed as the first parameter σ1 = ck and the second parameter σ2 = c(k+1). c is a coefficient used to define the numerical interval of group 57 for each parameter. k is a variable used to determine group 57 of parameters. When c = 0.5, in group 57 with k = 1, σ1 = 0.5 and σ2 = 1.0. In group 57 with k = 2, σ1 = 1.0 and σ2 = 1.5. Figure 11Group 57, which contains four parameters, is shown in the figure.

[0116] For example, the user operates the input unit 112 to display on the display unit 111 (see reference). Figure 1 In the GUI 58, the user selects any one of the settings A, B, C, and D corresponding to the variable k = 1 to 4. The input selection result is sent from the computer 110 to the image processing device 100 via the network 130. Therefore, the processor 10 (frequency component extraction unit 15) performs the above-described step S5a (refer to...) Figure 9 In the process, based on the user's selection, a set of parameters 57 is selected from a set of multiple parameters preset in the storage unit 20. This set includes a first parameter for generating the first smoothed image 41 and a second parameter for generating the second smoothed image 42. As a result, the set of parameters 57 corresponding to the frequency band to which the fine structure 92 in the cell image 30 belongs can be obtained according to the user's intention.

[0117] <Binarization Processing>

[0118] exist Figure 4 In step S6, the binarization processing unit 16 (refer to...) Figure 3 Perform the following processing (step (4) above): obtain the first image 61 obtained by binarizing the normalized image 40 (cell image 31 after being converted to relative values) and the second image 62 obtained by binarizing the frequency component image 50.

[0119] Reference Figure 12 The details of the binarization process will now be explained. The binarization processing unit 16 performs binarization processing on the normalized image 40 generated in step S4 and the frequency component image 50 generated in step S5, respectively. The binarization processing in this embodiment is a simple binarization process that binarizes the entire image using a binarization threshold.

[0120] The binarization processing unit 16 binarizes the normalized image 40 using a first threshold 65 and a second threshold 66 lower than the first threshold 65. Therefore, the first image 61 includes a first threshold image 61a obtained by binarizing the cell image 31 using the first threshold 65, and a second threshold image 61b obtained by binarizing the cell image 31 using the second threshold 66, which is smaller than the first threshold 65. Additionally, the binarization processing unit 16 generates a second image 62 obtained by binarizing the frequency component image 50 using a third threshold 67.

[0121] The second threshold 66 is set to a value lower than the first threshold 65; therefore, in the second threshold image 61b, pixels with lower brightness (lower pixel values) in the normalized image 40 are extracted as white areas. Figure 8In the graph 46 showing the distribution of pixel values ​​in the normalized image 40, the baseline corresponding to the background 93 is approximately constant, around 0% (in the range of -5% to 5%). Therefore, as an example of a binarization threshold, a second threshold of 10% and a first threshold of 20% are used. In this case, the first threshold image 61a (see reference) is obtained by binarizing using a first threshold 65 set to a value sufficiently high relative to the upper limit (5%) of the baseline. Figure 12 In this process, the possibility of noise other than cells being extracted as white areas (cell images) can be sufficiently reduced. The second threshold 66 is set to a value that is closer to the upper limit (5%) of the baseline compared to the first threshold 65, thus enabling accurate extraction of relatively low-brightness (low pixel value) areas such as fine structures 92 within cells 90. Accordingly, the second threshold image 61b (refer to...) Figure 12 Compared to the first threshold image 61a, the possibility of extracting noise and background as white areas (cell images) is relatively high.

[0122] Frequency component image 50 is an image obtained by extracting image features belonging to the frequency band of the fine structure 92 including cell 90. Therefore, it does not contain image features of the low-frequency band to which the background 93 belongs, so even if the third threshold 67 is set low, there is no concern about extracting the background 93 as a white region. Therefore, as an example of a binarization threshold for frequency component image 50, the third threshold is, for example, 1%. That is, in frequency component image 50, pixels with pixel values ​​greater than zero are extracted as cell regions (white regions). As a result, as Figure 12 As shown, the second image 62 is an image obtained by extracting image elements corresponding to the outline of a cell, including the fine structure 92 of the cell 90, as white areas (cell image). In this embodiment, the third threshold 67 is set to a value lower than the binarization thresholds (first threshold 65, second threshold 66) for the normalized image 40.

[0123] <Synthetic Processing>

[0124] exist Figure 4 In step S7, the synthesis processing unit 17 performs the following synthesis processing (step (5) above): by synthesizing the first image 61 (the cell image 31 obtained after binarization) and the second image 62 (the frequency component image 50 obtained after binarization), a binarized image 70 of the cell image 31 is generated. (Refer to...) Figure 13 To explain the details of the synthesis process.

[0125] In this embodiment, the step of generating the binarized image 70 of the cell image 31 includes: step S7a of removing the part of the second image 62 that is inconsistent with the second threshold image 61b, and step S7b of merging the second image 62a after the part of the second threshold image 61b that is inconsistent with the second threshold image 61b is removed and the first threshold image 61a.

[0126] In step S7a, the compositing processing unit 17 performs a logical AND operation on the second threshold image 61b and the second image 62. That is, the compositing processing unit 17 compares the same pixel in the second threshold image 61b and the second image 62. If the pixel values ​​are in a 1:1 ratio, the pixel value is set to 1 (white); if the pixel values ​​are in a 0:0 ratio, the pixel value is set to 0 (black). Conversely, if the pixel values ​​are inconsistent (1:0, 0:1), the pixel value is set to 0 (black). Furthermore, (1:1) refers to "pixel value of the second threshold image 61b: pixel value of the second image 62". Therefore, since the pixel values ​​of the parts of the second image 62 that are inconsistent with the second threshold image 61b are converted to 0 (black), the inconsistent parts of the second threshold image 61b and the second image 62 are removed from the second image 62.

[0127] By performing a logical AND operation between the second threshold image 61b and the second image 62, image elements such as the outlines of cells captured in both the second threshold image 62 and the second threshold image 61b are retained in the white area, while image elements such as noise captured in only one of the second image 62 and the second threshold image 61b are removed. Thus, step S7a is a noise removal process.

[0128] For example, if the original cell image 30 is not high quality and contains a lot of noise, noise can be effectively removed from the second image 62 through step S7a. On the other hand, if the original cell image 30 is high quality and contains almost no noise, even if the logical AND operation of the second threshold image 61b and the second image 62 is performed, there will be almost no change relative to the second image 62. Therefore, the processing of step S7a can be omitted. Hereinafter, the second image 62 after the part that is inconsistent with the second threshold image 61b is removed by step S7a will be referred to as "second image 62a".

[0129] Next, in step S7b, the compositing processing unit 17 combines the first threshold image 61a and the second image 62a by performing a logical OR operation on the first threshold image 61a and the second image 62a.

[0130] In other words, the compositing processing unit 17 compares the same pixel of the first threshold image 61a and the second image 62a. If any pixel value is "1 (white)" (in any of the ratios 1:1, 1:0, and 0:1), the pixel value of that pixel is set to "1 (white)". If both pixel values ​​are "0 (black)" (in the ratio 0:0), the pixel value of that pixel is set to "0 (black)". The compositing processing unit 17 generates the binarized image 70 by compositing (logical OR operation).

[0131] By performing a logical OR operation between the first threshold image 61a and the second image 62a, a binarized image 70 is obtained, which supplements the cell image extracted from the first threshold image 61a with information on the fine structures 92 that are difficult to extract.

[0132] Post-processing

[0133] exist Figure 4 In step S8, the post-processing unit 18 performs image shaping, noise removal, and other processing on the binarized image 70. (Refer to...) Figure 14 To explain the details of post-processing.

[0134] In step S8a, the post-processing unit 18 performs image shaping processing on the binarized image 70 (before post-processing) output from the synthesis processing unit 17. The image shaping processing is a process of shaping the cell image by interpolating local defects in the cell image.

[0135] Specifically, the post-processing unit 18 performs a closing operation on the binarized image 70. The closing operation is a process of shrinking the white regions in the image after dilution. The closing operation can connect short-distance interrupted linear portions or fill in holes (black regions) that exist locally inside the cell image without changing the size of the cell image (white regions).

[0136] Post-processing unit 18, for example, utilizes Figure 14 The nucleus 85 shown undergoes a closing operation that includes one expansion and one contraction process. The nucleus 85 has the shape obtained after removing the four corners of the rectangular region. This prevents the cell image shaped through the closing operation from becoming unnaturally angular.

[0137] like Figure 14 The magnified images 86a and 86b before and after the closing operation are shown. The fine gaps and holes in the cell image that exist in the magnified image 86a before the operation are filled in the magnified image 86b after the operation.

[0138] Next, in step S8b, the post-processing unit 18 performs noise removal processing on the binarized image 70 after image shaping.

[0139] The noise removal process in step S8b removes minute point-like noise present in the image. Specifically, the post-processing unit 18 performs the following process: removing areas in the white region of the image whose area (number of pixels) is less than a specified value and whose aspect ratio is less than a specified value (i.e., setting the pixel value to "0 (black)"). The aspect ratio is the value of the (long side / short side) of the smallest bounding rectangle of the white region being targeted. As an example, white regions with an area of ​​15 pixels or less and an aspect ratio of 3.0 or less are replaced with black regions. As a result, the minute point-like noise present in the binarized image 70 is removed.

[0140] As a result of the noise removal process, the post-processing unit 18 generates a post-processed binarized image 80. This post-processed binarized image 80 is output as the final result (processed image) of the binarization processing of the cell image 30 input to the image processing device 100.

[0141] <Storing and Outputting Processed Images (Binarized Images)>

[0142] exist Figure 4 In step S9, the processor 10 stores the post-processed binarized image 80 output from the post-processing unit 18 in the storage unit 20. Furthermore, the processor 10 sends the post-processed binarized image 80 to the computer 110 via the network 130. The computer 110 then displays the binarized image 80 as the processing result on the display unit 111.

[0143] Furthermore, in this embodiment, in addition to the post-processed binarized image 80, each image generated in each step S2 to S8 of the image processing is also used as image data 22 (see reference). Figure 1 The images are stored in the storage unit 20. The processor 10 can send the images stored in the storage unit 20 to the computer 110 according to the request from the computer 110.

[0144] Therefore, when a user wants to change the image processing conditions after confirming the post-processed binarized image 80, they can confirm the first smoothed image 41, the second smoothed image 42, and the frequency component image 50. Based on each image in the processing process, they can visually confirm the appropriateness of the image processing conditions. For example, they can determine whether the setting of the frequency band used to extract the fine structure 92 (the selection of smoothing parameters) is appropriate, or whether the binarization thresholds of the first threshold image 61a, the second threshold image 61b, and the second image 62 are appropriate.

[0145] Based on the above, the image processing method implemented by the image processing apparatus 100 of this embodiment is completed.

[0146] (The function of this implementation method)

[0147] Next, refer to Figure 15 This will explain the function of the image processing method in this embodiment. Figure 15 (A) shows a post-processed binarized image 80 obtained by performing the image processing method of this embodiment on the cell image 30. Figure 15 (B) shows a comparative example binarized image 500 obtained by performing a known binarization process on the same cell image 30.

[0148] The comparative example binarized image 500 is an image obtained through adaptive binarization processing. Adaptive binarization processing is a process that calculates a binarization threshold for each small region in the image and applies the obtained binarization threshold to each small region separately. It is known as a process that can suppress the effects of brightness unevenness in the image.

[0149] When the binarized image 80 of this embodiment is compared with the binarized image 500 of the comparative example, particularly in region P1, the linear fine structures (filopodia of cells) that were interrupted and separated in the binarized image 500 of the comparative example are extracted as continuous linear regions in the binarized image 80 of this embodiment, improving the extraction accuracy of fine structures. Furthermore, particularly in region P2, in the binarized image 500 of the comparative example, due to the overlap between uneven brightness and the brightness variations of the cells themselves, the relatively low-brightness portions in the inner region of the cell body 91 cannot be extracted, resulting in a coarse image. In contrast, in the binarized image 80 of this embodiment, the inner region of the cell body 91 shown in region P2 is extracted as a uniform white region.

[0150] As described above, it has been confirmed that, according to this embodiment, even compared with conventional binarization methods (adaptive binarization processing) used to suppress the effects of brightness unevenness, the extraction accuracy of fine structures can be improved, and the effects of brightness unevenness can be further reduced.

[0151] (Effects of this implementation method)

[0152] In this embodiment, the following effects can be achieved.

[0153] That is, in the image processing method and image processing apparatus 100 of this embodiment, the background brightness distribution (background image 32) in the cell image 31 is extracted, and the pixel value of each pixel in the cell image 31 is converted into a relative value relative to the background brightness distribution. Therefore, the pixel value of the converted normalized image 40 represents the degree of deviation from the brightness level of the background 93. Therefore, even if there is brightness unevenness in the cell image 31, since binarization processing can be performed based on the degree of deviation from the brightness level of the background 93, the influence of brightness unevenness in the cell image 31 can be mitigated. Since the normalized image 40 is binarized, even low-brightness areas in the cell 90 that are close to the background brightness level can be extracted with high precision to obtain the first image 61. In addition, since the frequency component image 50 obtained by extracting the predetermined frequency components corresponding to the fine structure 92 of the cell 90 is binarized, the second image 62 obtained by accurately extracting the fine structure 92 of the cell 90 can be obtained. Then, by combining the first image 61 and the second image 62, the second image 62, obtained by extracting the fine structure 92, can supplement the information of the first image 61, obtained by extracting the main morphology of the cell. Therefore, a binarized image 70, which accurately extracts the overall morphology of the cell 90 containing the low-brightness region within the cell 90, can be obtained. Based on the above, the influence of brightness unevenness in the cell image can be reduced during the binarization processing of the cell image, thereby accurately extracting the low-brightness region of the cell containing its fine structure.

[0154] Furthermore, in this embodiment, further effects are achieved by configuring it as follows.

[0155] That is, in this embodiment, in step S3 of extracting the background brightness distribution, a background image 32 showing the background brightness distribution is generated by performing a filtering process on the cell image 31 to remove cells 90 from the cell image 31. By configuring it in this way, the background image 32 obtained after removing cells 90 from the cell image 31 can be generated, thereby making it easy to obtain the background brightness distribution (the pixel value of the background of each pixel) of the cell image 31 without complicated processing such as image analysis.

[0156] Furthermore, in this embodiment, the filtering process for removing cells 90 is a median filtering process based on the kernel size corresponding to the size of the cells 90 captured in the cell image 31. With this configuration, performing such a simple process as median filtering on the cell image 31 makes it easy to obtain a background image 32 showing the background brightness distribution. Here, in the median filtering process, the kernel size is set to an appropriate size corresponding to the image element to be removed (in this case, cell 90), thereby retaining low-frequency components (background brightness distribution) larger than the kernel size and removing the image element (cell 90). As a result, the background brightness distribution of the cell image 31 can be extracted with high precision.

[0157] Furthermore, in this embodiment, step S3, which extracts the background brightness distribution, includes: step S3a, reducing the cell image 31; step S3b, performing a filtering process to remove cells 90 from the reduced image 31a; and step S3c, enlarging the filtered background image (reduced background image 32a) to restore its original image size. With this configuration, performing filtering on the reduced cell image (reduced image 31a) reduces the kernel size during filtering and increases the effective kernel size when restoring the image to its original size. Therefore, since the kernel size is reduced during filtering, the computational load of the filtering process can be effectively reduced.

[0158] Furthermore, in this embodiment, in step S4 of converting pixel values ​​to relative values, the conversion is performed by dividing the pixel value of each pixel in the cell image 31 by the pixel value of each pixel in the background image 32. This configuration allows for easy conversion of the pixel values ​​of each pixel in the cell image 31 into relative values. The converted pixel value represents the ratio of the brightness of the image signal to the brightness of the background; therefore, a cell image (normalized image 40) that is independent of the relative brightness of each pixel caused by brightness unevenness can be obtained. As a result, even when binarization processing is performed, the influence of pixel value deviations caused by brightness unevenness can be mitigated.

[0159] Furthermore, in this embodiment, step S5 of acquiring the frequency component image 50 includes: step S5b of generating a first smoothed image 41 and a second smoothed image 42 with different frequency characteristics by smoothing the normalized image 40, and step S5c of generating the frequency component image 50 by the difference between the first smoothed image 41 and the second smoothed image 42. With this configuration, a predetermined frequency component can be easily extracted from the normalized image 40 by performing a smoothing process with different parameters on the cell image 31 and performing a difference process between the obtained first smoothed image 41 and the second smoothed image 42. In addition, in the frequency component extraction process, it is important to appropriately set the range (band) of the frequency to be extracted in accordance with the fine structure 92 to be extracted. According to the above structure, a first smoothed image 41 and a second smoothed image 42 reflecting the difference in the smoothing process parameters, and a frequency component image 50 as their difference, can be obtained. Therefore, it is possible to visually confirm whether the fine structure 92 to be extracted can be appropriately extracted. Thus, the smoothing parameters can be easily optimized.

[0160] Furthermore, in this embodiment, step S5 of acquiring the frequency component image 50 further includes the following step S5a: selecting a group 57 of parameters from a pre-set group 57 of multiple parameters for generating a first parameter for generating a first smoothed image 41 and a second parameter for generating a second smoothed image 42. This configuration allows the user to select a group 57 of parameters suitable for the frequency band to which the desired fine structure 92 belongs from the selection options of the group 57 of multiple parameters, thereby determining the group 57 of parameters used to generate the first smoothed image 41 and the second smoothed image 42. As a result, even users without specialized knowledge can easily determine appropriate smoothing parameters that match the desired fine structure 92.

[0161] Furthermore, in this embodiment, the smoothing process is a Gaussian filtering process, and the parameter for the smoothing process is the standard deviation of the Gaussian filter. With this configuration, the frequency components removed by the Gaussian filtering process can be easily adjusted using only a single smoothing parameter. Moreover, since the magnitude of the standard deviation corresponds to the intensity of the frequency components, the smoothing parameter can be easily determined.

[0162] Furthermore, in this embodiment, the fine structure 92 of cell 90 is the filamentous pseudopodia of cell 90. In the microscopic image of cell 90, the filamentous pseudopodia have elongated linear structures extending from the cell body 91, and their pixel values ​​tend to decrease (darken), making them difficult to distinguish and extract from the background 93 through simple binarization processing. Therefore, the binarization method of this embodiment, which synthesizes the portion of the filamentous pseudopodia extracted from the frequency component image 50, is particularly effective for generating a binarized image 70 that accurately extracts the filamentous pseudopodia. In other words, the binarization method of this embodiment is particularly suitable for generating a binarized image 70 of cell image 30 that captures fine structures (image elements of high-frequency components locally present in small areas) within cell 90 and whose pixel values ​​tend to decrease.

[0163] Furthermore, in this embodiment, the first image 61 includes a first threshold image 61a obtained by binarizing the normalized image 40 using a first threshold 65, and a second threshold image 61b obtained by binarizing the normalized image 40 using a second threshold 66 smaller than the first threshold 65. Step S7, which generates a binarized image 70 of the cell image 31, includes a step S7a of removing portions from the second image 62 that are inconsistent with the second threshold image 61b, and a step S7b of merging the second image 62 after removing the portions inconsistent with the second threshold image 61b with the first threshold image 61a. With this configuration, by removing portions from the second image 62 that are inconsistent with the second threshold image 61b, noise other than the cells 90 in the second image 62 can be removed. Additionally, since the first threshold image 61a is binarized using a first threshold 65 higher than the second threshold 66, it is possible to suppress the inclusion of background 93 and noise in the cell image (white area) extracted through binarization. Then, by combining the noise-removed second image 62a and the first threshold image 61a, the fine structure 92 of the cell 90 can be accurately extracted, and a binarized image 70 with noise interference suppressed can be obtained. Therefore, even if the image quality of the original cell image 30 (cell image 31) is not high, a binarized image 70 with less noise can be generated.

[0164] [Variation Example]

[0165] Furthermore, it should be considered that the embodiments disclosed herein are illustrative and not restrictive in all respects. The scope of the invention is defined not by the description of the embodiments above but by the claims, and includes all modifications (variations) within the meaning and scope equivalent to the claims.

[0166] For example, in the above embodiment, an example is shown where the image processing apparatus 100 functions as a server in an image processing system 200 constructed as a client-server model, but the present invention is not limited thereto. In the present invention, for example... Figure 16 As shown, it can also be constructed from independent computers. Figure 16 In this example, the image processing apparatus 100 is configured as a computer 300 including a processor 210 and a storage unit 220. The computer 300 is connected to a display unit 230 and an input unit 240. The computer 300 is connected to the camera device 120 in a communicative manner. The processor 210 of the computer 300 includes components described in the above embodiment (see [reference]). Figure 3 The image acquisition unit 11, preprocessing unit 12, background extraction unit 13, relativization processing unit 14, frequency component extraction unit 15, binarization processing unit 16, synthesis processing unit 17, and postprocessing unit 18 shown are used as functional blocks.

[0167] In addition, in the above-described implementation methods and Figure 16 In the illustrated variation, an example is shown where all image processing (as image acquisition unit 11, preprocessing unit 12, background extraction unit 13, relativization processing unit 14, frequency component extraction unit 15, binarization processing unit 16, synthesis processing unit 17, and post-processing unit 18) is performed by a single processor 10 (210), but the present invention is not limited thereto. Each image processing step on the cell image 30 can also be performed by multiple processors. Each process can also be performed by different processors. Multiple processors can also be located on different computers. That is, the image processing apparatus 100 can also be configured with multiple computers performing image processing.

[0168] Furthermore, in the above embodiment, an example was shown of generating a background image 32 by performing a filtering process on the cell image 31 to remove cells 90 from the cell image 31, but the present invention is not limited thereto. In the present invention, the background image 32 can also be generated by methods other than filtering. For example, the background image can also be generated by performing a Fourier transform on the cell image 31, extracting the low-frequency band corresponding to the background brightness in the spatial frequency region, and then performing an inverse Fourier transform.

[0169] Furthermore, while the above embodiment illustrates an example of generating a background image 32 representing the background brightness distribution, the present invention is not limited thereto. In the present invention, the background brightness distribution may not be extracted as an image (background image 32). The background brightness distribution may also be obtained, for example, as a function representing the background brightness value (pixel value) at each position coordinate in the image. That is, the background brightness distribution can also be represented by setting the x and y coordinates of the image as variables and outputting a function corresponding to the pixel value of the background brightness.

[0170] Furthermore, in the above embodiment, an example of median filtering for removing cells 90 from cell image 31 was shown, but the present invention is not limited thereto. In the present invention, the filtering for removing cells 90 from cell image 31 can also be other filtering processes besides median filtering.

[0171] Furthermore, in the above embodiment, an example was shown where cell removal 90 filtering was performed on the reduced image 31a in step S3 of extracting the background brightness distribution; however, the present invention is not limited thereto. In the present invention, cell removal 90 filtering may be performed on the original size cell image 31 without reducing its size. In this case, of course, there is no need to enlarge the filtered reduced background image 32a.

[0172] Furthermore, in the above embodiment, an example was shown where the pixel values ​​of the normalized image 40 were determined using the formula {(cell image 31b / background image 32)-1}×100 (%) in step S4, which converts the pixel values ​​of each pixel of the cell image 31 to relative values ​​with respect to the background image 32. However, the present invention is not limited to this. In the present invention, the pixel values ​​of the normalized image 40 can also be determined using a different formula than those in the above embodiment. Additionally, in the above embodiment, it is not necessary to subtract "1" from the value obtained by dividing the pixel values ​​of the cell image 31b by the pixel values ​​of the background image 32.

[0173] Furthermore, in the above embodiment, an example was shown of generating a frequency component image 50 by differing between a first smoothed image 41 and a second smoothed image 42 with different frequency characteristics, but the present invention is not limited thereto. In the present invention, the frequency component image 50 can also be generated by methods other than image differencing. For example, the frequency component image can also be generated by performing a Fourier transform on the normalized image 40, extracting a predetermined frequency band corresponding to the fine structure 92 in the spatial frequency region, removing other frequency components, and performing an inverse Fourier transform.

[0174] Furthermore, in the above embodiment, an example is shown of selecting a group 57 of parameters, namely the first parameter and the second parameter, from a group 57 of a plurality of preset parameters when generating the frequency component image 50 (see reference). Figure 11 However, the present invention is not limited thereto. In the present invention, the group 57 of parameters for generating the frequency component image 50 may not be selected from the group 57 of multiple parameters. The input of the value of the first parameter and the input of the value of the second parameter may also be accepted separately.

[0175] Furthermore, in the above embodiment, an example of Gaussian filtering for generating the first smoothed image 41 and the second smoothed image 42 is shown, but the present invention is not limited thereto. The smoothing process for generating the first smoothed image 41 and the second smoothed image 42 can also be performed using other filtering processes besides Gaussian filtering. For example, moving average filtering could also be used. The parameters of the smoothing process vary depending on the filtering process; therefore, the parameters of the smoothing process are not limited to the standard deviation σ.

[0176] Furthermore, in the above embodiment, the fine structure 92 of cell 90 is shown as an example of filopodia of cell 90, but the present invention is not limited thereto. The fine structure can also be a structure other than filopodia. Moreover, the fine structure of a cell is a structure that is relatively fine relative to the main structures of the cell in a cell image, and is a concept dependent on the magnification of the cell image. For example, in the cell image 30 described in the above embodiment (see...) Figure 2 High-magnification cameras capture images of cells that only show the pseudopodia within the cells. Fine structures in these images can be part of the filamentous pseudopodia captured in the image.

[0177] Furthermore, the above embodiments illustrate examples of generating a first threshold image 61a and a second threshold image 61b with different binarization thresholds, but the present invention is not limited thereto. In the present invention, only one first image 61 can be generated using a single binarization threshold. In this case, step S7a is omitted, and the first image 61 and the second image 62 are simply synthesized.

[0178] Furthermore, in the above embodiment, an example of preprocessing to convert the cell image 30, which is a color image, into a grayscale cell image 31 was shown, but the present invention is not limited thereto. If the original cell image is a grayscale image, preprocessing is not required. Additionally, an example was shown where the color component image with the highest brightness value was selected as the grayscale cell image 31 after preprocessing, but the grayscale cell image 31 can also be generated by averaging the pixel values ​​of the color component images of each color.

[0179] Furthermore, in the above embodiment, an example was shown where the binarized image 70 obtained through synthesis processing was post-processed to generate the post-processed binarized image 80 as the final processed image; however, the present invention is not limited thereto. In the present invention, post-processing may not be performed. The binarized image 70 obtained through synthesis processing may also be used to generate the final processed image.

[0180] Furthermore, the specific values ​​such as kernel size, smoothing parameter, and binarization threshold shown in the above embodiments are merely illustrative and are not limited to the values ​​described above.

[0181] [Way]

[0182] Those skilled in the art will understand that the above-described embodiments are specific examples of the following methods.

[0183] (Project 1)

[0184] An image processing method for binarizing a cell image as a multi-valued image, the image processing method comprising the following steps:

[0185] Extract the background brightness distribution from the cell image;

[0186] The pixel values ​​of each pixel in the cell image are converted into relative values ​​relative to the background brightness distribution;

[0187] A frequency component image is obtained by extracting the specified frequency components corresponding to the fine structure of the cell from the converted cell image;

[0188] Obtain a first image obtained by binarizing the converted cell image and a second image obtained by binarizing the frequency component image; and

[0189] A binarized image of the cell image is generated by combining the first image and the second image.

[0190] (Project 2)

[0191] According to the image processing method described in Project 1, wherein,

[0192] In the step of extracting the background brightness distribution, a background image showing the background brightness distribution is generated by performing a filtering process on the cell image to remove cells from the cell image.

[0193] (Project 3)

[0194] According to the image processing method described in Project 2, wherein,

[0195] The filtering process for removing the cells is a median filtering process based on the nuclear size, which corresponds to the size of the cells captured in the cell image.

[0196] (Project 4)

[0197] According to the image processing method described in Project 2 or 3, wherein,

[0198] The steps for extracting the background brightness distribution include the following:

[0199] Reduce the size of the cell image;

[0200] Perform a filter process to remove the cells on the scaled-down cell image; and

[0201] The background image, after the cells have been removed and filtered, is enlarged to restore its original size to its original dimensions.

[0202] (Project 5)

[0203] According to the image processing method described in any one of items 2 to 4, wherein,

[0204] In the step of converting to the relative value, the relative value is converted by dividing the pixel value of each pixel in the cell image by the pixel value of each pixel in the background image.

[0205] (Project 6)

[0206] The image processing method according to any one of items 1 to 5, wherein,

[0207] The steps for obtaining the frequency component image include the following:

[0208] By smoothing the cell image, a first smoothed image and a second smoothed image with different frequency characteristics are generated; and

[0209] The frequency component image is generated by the difference between the first smoothed image and the second smoothed image.

[0210] (Project 7)

[0211] According to the image processing method described in Project 6, wherein,

[0212] The step of obtaining the frequency component image further includes the following steps:

[0213] Select from a group of pre-defined parameters a first parameter for generating the first smoothed image and a second parameter for generating the second smoothed image.

[0214] (Project 8)

[0215] According to the image processing method described in Project 7, wherein,

[0216] The smoothing process is a Gaussian filtering process.

[0217] The parameter for the smoothing process is the standard deviation of the Gaussian filter.

[0218] (Project 9)

[0219] The image processing method according to any one of items 1 to 8, wherein,

[0220] The fine structure of the cell is its filopodia.

[0221] (Project 10)

[0222] According to any one of items 1 to 9, the image processing method wherein,

[0223] The first image includes a first threshold image obtained by binarizing the cell image using a first threshold, and a second threshold image obtained by binarizing the cell image using a second threshold smaller than the first threshold.

[0224] The steps for generating a binarized image of the cell image include the following:

[0225] Remove portions of the second image that are inconsistent with the second threshold image; and

[0226] The second image, after removing the parts that are inconsistent with the second threshold image, is combined with the first threshold image.

[0227] (Project 11)

[0228] An image processing apparatus comprising:

[0229] The image acquisition unit acquires cell images as multi-valued images;

[0230] The background extraction unit extracts the background brightness distribution in the cell image;

[0231] The relativization processing unit converts the pixel values ​​of each pixel in the cell image into relative values ​​relative to the background brightness distribution;

[0232] The frequency component extraction unit acquires a frequency component image obtained by extracting specified frequency components corresponding to the fine structure of the cell from the converted cell image;

[0233] The binarization processing unit acquires a first image obtained by binarizing the converted cell image through binarization processing and a second image obtained by binarizing the frequency component image through binarization processing; and

[0234] The synthesis processing unit generates a binarized image of the cell image by compositing the first image and the second image.

[0235] Explanation of reference numerals in the attached figures

[0236] 11: Image acquisition unit; 13: Background extraction unit; 14: Relativization processing unit; 15: Frequency component extraction unit; 16: Binarization processing unit; 17: Synthesis processing unit; 30, 31, 31b: Cell image; 31a: Reduced image (reduced cell image); 32: Background image; 32a: Reduced background image (filtered background image); 40: Normalized image (cell image converted to relative value); 41: First smoothed image; 42: Second smoothed image; 50: Frequency component image; 57: Group of parameters; 61: First image; 61a: First threshold image; 61b: Second threshold image; 62, 62a: Second image; 65: First threshold; 66: Second threshold; 70, 80: Binarized image; 90: Cell; 92: Fine structure; 93: Background; 100: Image processing device.

Claims

1. An image processing method for binarizing a cell image as a multi-valued image, the image processing method comprising the following steps: Extract the background brightness distribution from the cell image; The pixel values ​​of each pixel in the cell image are converted into relative values ​​relative to the background brightness distribution; A frequency component image is obtained by extracting a defined high-frequency component that corresponds to the fine structure of the cell and has a high frequency compared to the background from the converted cell image; Obtain a first image obtained by binarizing the converted cell image and a second image obtained by binarizing the frequency component image; as well as A binarized image of the cell image is generated by combining the first image and the second image.

2. The image processing method according to claim 1, wherein, In the step of extracting the background brightness distribution, a background image showing the background brightness distribution is generated by performing a filtering process on the cell image to remove cells from the cell image.

3. The image processing method according to claim 2, wherein, The filtering process for removing the cells is a median filtering process based on the nuclear size, which corresponds to the size of the cells captured in the cell image.

4. The image processing method according to claim 2, wherein, The steps for extracting the background brightness distribution include the following: Reduce the size of the cell image; Perform a filter process to remove the cells on the scaled-down cell image; and The background image, after the cells have been removed and filtered, is enlarged to restore its original size to its original dimensions.

5. The image processing method according to claim 2, wherein, In the step of converting to the relative value, the relative value is converted by dividing the pixel value of each pixel in the cell image by the pixel value of each pixel in the background image.

6. The image processing method according to claim 1, wherein, The steps for obtaining the frequency component image include the following: By smoothing the cell image, a first smoothed image and a second smoothed image with different frequency characteristics are generated; and The frequency component image is generated by the difference between the first smoothed image and the second smoothed image.

7. The image processing method according to claim 6, wherein, The step of obtaining the frequency component image further includes the following steps: Select from a group of pre-defined parameters a first parameter for generating the first smoothed image and a second parameter for generating the second smoothed image.

8. The image processing method according to claim 7, wherein, The smoothing process is a Gaussian filtering process. The parameter for the smoothing process is the standard deviation of the Gaussian filter.

9. The image processing method according to claim 1, wherein, The fine structure of the cell is its filopodia.

10. An image processing method for binarizing a cell image as a multi-valued image, the image processing method comprising the following steps: Extract the background brightness distribution from the cell image; The pixel values ​​of each pixel in the cell image are converted into relative values ​​relative to the background brightness distribution; A frequency component image is obtained by extracting the specified frequency components corresponding to the fine structure of the cell from the converted cell image; Obtain a first image obtained by binarizing the converted cell image and a second image obtained by binarizing the frequency component image; as well as A binarized image of the cell image is generated by combining the first image and the second image. The first image includes a first threshold image obtained by binarizing the cell image using a first threshold, and a second threshold image obtained by binarizing the cell image using a second threshold smaller than the first threshold. The steps for generating a binarized image of the cell image include the following: Remove portions of the second image that are inconsistent with the second threshold image; and The second image, after removing the parts that are inconsistent with the second threshold image, is combined with the first threshold image.

11. An image processing apparatus comprising: The image acquisition unit acquires cell images as multi-valued images; The background extraction unit extracts the background brightness distribution in the cell image; The relativization processing unit converts the pixel values ​​of each pixel in the cell image into relative values ​​relative to the background brightness distribution; The frequency component extraction unit acquires a frequency component image obtained by extracting a predetermined high-frequency component that corresponds to the fine structure of the cell and has a higher frequency than the background from the converted cell image. The binarization processing unit acquires a first image obtained by binarizing the converted cell image through binarization processing and a second image obtained by binarizing the frequency component image through binarization processing. as well as The synthesis processing unit generates a binarized image of the cell image by compositing the first image and the second image.

Citation Information

Patent Citations

  • Method for evaluating pluripotent stem cells by image analysis

    JP2014018184A

  • Dynamic range compression

    US20020159648A1

  • System and methods for rapid and automated screening of cells

    US20060257013A1

  • Line-based image registration and cross-image annotation devices, systems and methods

    US20160321809A1

  • Converting low-dose to higher dose 3D tomosynthesis images through machine-learning processes

    US20170071562A1