IMAGE PROCESSING METHOD, IMAGE PROCESSING APPARATUS, AND IMAGE PROCESSING SYSTEM

Abstract

In an image processing method according to the present application, a medical image captured by an image capture device is acquired and the strength of a filter applied to the medical image is determined in accordance with the focusing degree of the medical image.

Description

[Technical field]

[0001] The present invention relates to an image processing method, an image processing device, and an image processing system. [Background technology]

[0002] There are digital microscopes and image display devices that use microscopes to observe cell tissue, capture images of the cell tissue, save them as medical images, and use the image data of the medical images to perform pathological diagnoses, etc. In digital microscopes, in order to observe the entire specimen, images of small areas that define the area containing the specimen on a glass slide are captured by a magnifying imaging system, and multiple images of each small area are stitched together to create a single large medical image.

[0003] Autofocus (AF) is used as a focusing method for adjusting the focus of the objective lens of the magnification imaging system to the cell tissue to be imaged. For example, a focusing method has been proposed in which the focal position of the objective lens of the magnification imaging system is moved at predetermined intervals along the optical axis, images are captured at each of the moved positions, and the position at which the image with the highest contrast among the captured images is captured is detected as the in-focus position (see, for example, Patent Document 1). This type of focusing method is called "contrast AF."

[0004] Although the cell tissue image captured in this manner can achieve a relatively high degree of focusing accuracy, it looks different from an optical microscope image observed by an observer such as a doctor through an optical microscope. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2011-197283 A Summary of the Invention [Problem to be solved by the invention]

[0006] Thus, although there is a demand for digital microscope devices to acquire high-quality images of cell tissue, this demand has not yet been adequately resolved.

[0007] In view of the above circumstances, an object of the present technology is to provide a digital microscope device capable of acquiring high-quality images of cell tissue, and an imaging method and program therefor. [Means for solving the problem]

[0008] The image processing method according to the present application is characterized in that a medical image captured by an imaging device is acquired, and a filter strength to be applied to the medical image is determined according to a degree of focus of the medical image. [Brief description of the drawings]

[0009] [Figure 1] FIG. 4 is a diagram showing an example of features of a medical image according to the embodiment. [Diagram 2] FIG. 13 is a diagram showing an example of a feature of a slide according to an embodiment. [Diagram 3] 1 is a diagram illustrating an example of the configuration of an image processing system according to an embodiment. [Figure 4] FIG. 13 is a diagram illustrating an example of a function indicating a blur feature amount according to the embodiment. [Diagram 5] FIG. 13 is a diagram illustrating an example of a function indicating the inverse of a blur feature amount according to the embodiment. [Figure 6] 10 is a diagram illustrating an example of a pixel range for calculating a blur feature amount according to the embodiment; [Figure 7] FIG. 4 is a diagram illustrating an example of a focus position according to the embodiment. [Figure 8] 10 is a diagram illustrating an example of a pixel range for calculating a blur feature amount according to the embodiment; [Figure 9] FIG. 1 is a diagram illustrating an example of the configuration of an imaging device according to an embodiment. [Figure 10] FIG. 1 is a diagram illustrating an example of the configuration of an image processing device according to an embodiment. [Figure 11] FIG. 4 is a diagram illustrating an example of a medical image storage unit according to the embodiment. [Figure 12] FIG. 4 is a diagram illustrating an example of an emphasis filter storage unit according to the embodiment. [Figure 13] 10 is a flowchart illustrating an example of information processing according to the embodiment. [Figure 14] 11A and 11B are diagrams illustrating an example of an effect of an emphasis filter according to the embodiment. [Figure 15] 11A and 11B are diagrams illustrating an example of an effect of an emphasis filter according to the embodiment. [Figure 16] FIG. 2 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the image processing apparatus. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, the image processing method, image processing device, and image processing system according to the present application will be described in detail with reference to the drawings. Note that the image processing method, image processing device, and image processing system according to the present application are not limited to the embodiments. In addition, the same parts in the following embodiments are given the same reference numerals, and duplicated explanations will be omitted.

[0011] The present disclosure will be described in the following order. 1. Characteristics of optical microscopes 2. Image Processing System Configuration 3. An example of information processing 4. Processing Variations 4-1.Tap range 4-2. Number of layers 4-3.Types of filters 4-4. Creating blocks with multiple pixels 4-5.Subject 4-6. Imaging device 4-7. Focus position 4-8. How to calculate features 4-9.Integration of the device 5. Configuration of the imaging device 6. Image Processing Device Configuration 7. Information processing flow 8. Variations 9. Hardware Configuration 10.Other

[0012] (Embodiment) [1. Features of optical microscopes] FIG. 1 shows the characteristics of an optical microscope image. An optical microscope image has different characteristics from an image observed through a digital microscope device. As shown in FIG. 1, there are several indexes related to the image quality of an optical microscope image. The characteristics of an optical microscope image will be explained using the indexes shown in FIG. 1. For example, there is an index indicating the brightness of the image quality (in this embodiment, this may be written as "sparkle"). For example, there is an index indicating the degree of edge of the image quality (in this embodiment, this may be written as "sharpness"). In FIG. 1, the indexes indicating the characteristics of an optical microscope image are, for example, "three-dimensional / flat", "transparent / dull", "sharp / blurred", etc.

[0013] The blur (blur) according to the embodiment refers to a state in which the image is not sharp. Specifically, the blur according to the embodiment refers to a state in which the image is out of focus and not sharp beyond the range of the depth of field. Note that the focus according to the embodiment refers to a state in which the image is in focus within the range of the depth of field. Note that the focus degree according to the embodiment is a value that is scored to indicate how well the image is in focus.

[0014] An optical microscope image is three-dimensional, as shown in Fig. 1. Here, the term "three-dimensional" according to the embodiment refers to a quality related to the visual contrast between blur and focus.

[0015] An optical microscope image has a sense of transparency. Here, the sense of transparency according to the embodiment refers to quality related to noise. Note that the noise according to the embodiment is unnecessary information other than the subject. Specifically, since an optical microscope image is not digitized, there is no noise emphasis, and the like, so that the image has a sense of transparency.

[0016] The optical microscope image has a sparkling effect. Here, the sparkling effect according to the embodiment refers to the quality of brightness caused by the interference fringes that are generated by scattered light when light is applied to the subject. Specifically, the optical microscope image has a sparkling effect because it emits light that is brighter than the light applied to the object due to the interference fringes.

[0017] The optical microscope image has a clear appearance. Here, the clear appearance according to the embodiment refers to quality related to sharpness.

[0018] Optical microscope images are three-dimensional, bright, and highly sharp, and therefore have a high ability to identify objects (hereinafter referred to as "object identification performance" for convenience).

[0019] Optical microscope images are three-dimensional, bright, and highly sharp, and therefore have high object recognition performance (hereinafter referred to as "object recognition performance" for convenience).

[0020] A method for making an image acquired by a digital microscope device closer to an optical microscope image will be described below. A specimen is placed on a glass slide. Cells and the like are distributed in the Z-axis direction (hereinafter referred to as the "Z direction" as appropriate) that indicates the direction of the thickness of the slide, and a medical image acquired by a digital microscope device contains a mixture of in-focus and out-of-focus areas. For example, if a high-frequency emphasis filter is applied to the entire medical image, low-frequency parts (e.g., noise) are also emphasized, and the sharpness of the out-of-focus areas increases. This results in an image with many high-frequency components, which deteriorates the object identification performance. Furthermore, if the strength of the filter is reduced, the emphasis of the areas to be emphasized will also be weakened.

[0021] An image close to an optical microscope image is an image that has the characteristics shown in Figure 1 and is formed from images with different degrees of focus. The effect of approaching an optical microscope image will be explained below.

[0022] For example, by making the medical image obtained by the digital microscope device closer to the optical microscope image, the structure of cells becomes easier to see. This can promote the use in diagnosis. For example, by making the medical image closer to the optical microscope image, the location of cells becomes easier to distinguish. This can increase the speed of diagnosis by pathologists and reduce fatigue. For example, by making the medical image closer to the optical microscope image, the visibility of overlapping cells increases. This can enable diagnosis of diseases in which identification of overlapping cells is important. For example, by making the medical image closer to the optical microscope image, pathologists can more easily adapt to diagnosis using medical images. For example, by making the medical image closer to the optical microscope image, small objects such as Helicobacter pylori can be prevented from being buried in noise. For example, by making the medical image closer to the optical microscope image, high compression efficiency can be ensured by area-limited emphasis.

[0023] FIG. 2 shows an example of a slide for imaging. FIG. 2 shows a view of the slide for imaging from a vertical direction. As shown in FIG. 2, the Z direction is the direction of the thickness of the slide. In other words, the Z direction is the direction of the thickness of the subject. Furthermore, since the subject is imaged from above the glass slide, the Z direction is perpendicular to the medical image. Furthermore, the Z direction is the direction of the optical axis when imaging. FIG. 2 shows a case where the subject is placed on a glass slide and covered with a cover glass. FIG. 2 shows a case where the subject is a slice of tissue. FIG. 2 shows a case where a specimen such as lymphocytes or macrophages contained in tissue is imaged.

[0024] As shown in Fig. 2, lymphocytes overlap each other and are therefore thick. Macrophages are present beneath the lymphocytes. Because the depth of field is shallow, it is not possible to focus on and image the entire lymphocytes and macrophages. Hereinafter, information processing by the image processing device 100 will be described as a process for correcting the decrease in visibility due to digitization.

[0025] 2. Image Processing System Configuration The configuration of the image processing system 1 will be described with reference to FIG. 3. FIG. 3 is a diagram showing an example of the configuration of the image processing system according to the embodiment. As shown in FIG. 3, the image processing system 1 includes an imaging device 10 and an image processing device 100. The imaging device 10 and the image processing device 100 are connected to each other via a predetermined communication network (network N) so as to be able to communicate with each other by wire or wirelessly. FIG. 3 is a diagram showing an example of the configuration of the image processing system according to the embodiment. Note that the image processing system 1 shown in FIG. 3 may include a plurality of imaging devices 10 and a plurality of image processing devices 100.

[0026] The imaging device 10 is an imaging device such as a microscope, and is used to capture an image of a specimen.

[0027] The image processing device 100 is used to determine information about a filter according to the degree of focus of a subject. The image processing device 100 is an information processing device such as a PC or a WS (Work Station), and performs processing based on information transmitted from an imaging device 10 or the like via a network N.

[0028] [3. An example of information processing] A case will be described below in which the image processing device 100 determines the filter strength depending on the degree of focus of the subject. A specimen such as a cell will be described below as an example of the subject.

[0029] The filter according to the embodiment is a filter that improves the image quality of a medical image. The filter according to the embodiment is applied to a medical image obtained by capturing an image of a subject. The type of filter according to the embodiment may be any type. That is, there is no limit to the region that the filter according to the embodiment emphasizes. For example, the filter according to the embodiment includes filters such as high-frequency emphasis, mid-frequency emphasis, low-frequency emphasis, and negative emphasis filters, i.e., smoothing filters.

[0030] The image processing device 100 calculates the degree of focus using a blur function (hereinafter, appropriately referred to as a "blur determination function" or a "blur amount determination function".) Hereinafter, a process in which the image processing device 100 generates the blur function will be described.

[0031] The blur function is generated by approximating the adjacent squared difference sum with a Lorentz function and calculating the inverse. Note that the approximation according to the embodiment is graph fitting (curve fitting). Here, Equation (1) shows the adjacent squared difference sum according to the embodiment.

[0032]

number

[0033] FIG. 4 shows a graph GR1 of the sum of squared adjacent differences. FIG. 4 shows a graph GR1 in which the sum of differences between a specific pixel and a pixel having a specific relationship is plotted as a feature value for all pixels in a medical image, and the value in the Z direction (hereinafter, referred to as "Z value" as appropriate) is plotted as a variable. Note that this feature value is referred to as a "blur feature value" as appropriate. Here, the pixel having a specific relationship is a pixel adjacent to the specific pixel. In this case, FIG. 4 shows a graph in which the sum of differences between a specific pixel and an adjacent pixel is plotted for all pixels in a medical image. In FIG. 4, the horizontal axis (X axis) of the graph GR1 is the Z value of the slide. Also, the vertical axis (Y axis) of the graph GR1 indicates the feature value. Note that in FIG. 4, the output value of the sum of squared adjacent differences is indicated as s(z). Also, in FIG. 4, the Z value at which the feature value is maximized is plotted to be 0. In other words, the maximum output value of the sum of squared adjacent differences is plotted to be s(0). Also, FIG. 4 shows that the greater the feature value, the higher the degree of focus.

[0034] 4 may show a graph in which the adjacent difference squared sum is approximated by a Lorentz function, where Equation (2) shows the Lorentz function according to the embodiment.

[0035]

number

[0036] In this case, the Y-axis of the graph GR1 in Fig. 4 may be f(z). That is, the Y-axis of the graph GR1 in Fig. 4 may be represented by f(z) as the output value of the Lorentz function.

[0037] Fig. 5 shows a graph GR2 which is the inverse of the Lorentz function f(z). Fig. 5 shows a graph GR2 in which the inverse of the Lorentz function f(z) is plotted. Note that the graph GR2 is a quadratic curve. Here, Equation (3) shows the quadratic curve according to the embodiment.

[0038]

number

[0039] The apex of the graph GR2 indicates the in-focus position, which is the Z value at which the degree of focus is maximized.

[0040] The image processing device 100 acquires image information of a predetermined focused image. The image processing device 100 acquires a predetermined number of pieces of image information focused on positions with different Z values ​​based on the predetermined focused image, together with the predetermined focused image. For example, the image processing device 100 acquires image information focused on a position that differs by several micrometers in the Z direction from the Z value of the predetermined focused image. As a specific example, the image processing device 100 acquires a total of three pieces of image information, including the predetermined focused image and two images that are focused on positions that differ by several micrometers in the Z direction from the Z value of the predetermined focused image.

[0041] The image processing device 100 calculates the blur feature amount of the acquired image information by using the adjacent squared difference sum. Specifically, the image processing device 100 calculates the blur feature amount of the acquired image information by approximating the adjacent squared difference sum to a Lorentz function.

[0042] The image processing device 100 calculates the inverse of the calculated blur feature amount. The image processing device 100 performs fitting to a quadratic curve based on the calculated inverse of the blur feature amount. The image processing device 100 estimates a focus position based on the fitted quadratic curve. Specifically, the image processing device 100 estimates the apex of the fitted quadratic curve as the focus position of the entire image.

[0043] The image processing device 100 calculates the degree of focus of the acquired image information based on the estimated focus position. Specifically, the image processing device 100 calculates the degree of focus of the acquired image information based on the difference between the estimated focus position and the Z value used in the blur function.

[0044] The image processing device 100 determines the strength of the filter according to the calculated degree of focus.

[0045] Hereinafter, the process of calculating the blur feature amount performed by the image processing device 100 will be described with reference to FIG.

[0046] The image processing device 100 uses a predetermined pixel in a medical image as a reference, applies taps around the predetermined pixel, and calculates a blur feature amount using the sum of adjacent squared differences. Here, the tap according to the embodiment indicates a range of pixels around the predetermined pixel. That is, the tap according to the embodiment indicates a range of peripheral pixels around a pixel of interest. For example, the tap according to the embodiment indicates a range of peripheral pixels around a pixel of interest to which a filter is applied. For example, a 3×3 tap indicates a range of 9 pixels in total, with 3 pixels in each of the vertical and horizontal directions of an image. The image processing device 100 applies similar taps to each image acquired with different Z values, and calculates the sum of adjacent squared differences as a blur feature amount. In FIG. 6, the image processing device 100 applies a 3×3 tap TA1 as an example. In FIG. 6, the image processing device 100 calculates a blur feature amount around S0, with S0 as a predetermined pixel. In this case, the image processing device 100 calculates a blur feature amount based on Equation (4).

[0047]

number

[0048] Specifically, when the image processing device 100 acquires three pieces of image information, a predetermined focused image and two images focused at different positions in the Z direction with respect to the Z value of the predetermined focused image, the image processing device 100 calculates the blur feature amount by tapping 3×3 from each of the three images and using the sum of adjacent squared differences from the center. In this case, the image processing device 100 calculates the blur feature amount based on the formula (5).

[0049]

number

[0050] The image processing device 100 calculates a blur feature amount using image information of a layer located at a predetermined Z value among the acquired image information. For example, the image processing device 100 calculates a blur feature amount F2 using image information of a higher layer among the acquired three pieces of image information. For example, the image processing device 100 calculates a blur feature amount F1 using image information of a middle layer among the acquired three pieces of image information. For example, the image processing device 100 calculates a blur feature amount F0 using image information of a lower layer among the acquired three pieces of image information.

[0051] FIG. 7 shows a graph GR3 obtained by fitting the reciprocals of the three values ​​calculated by the image processing device 100 to a quadratic curve. The image processing device 100 estimates the in-focus position based on the fitting. The image processing device 100 estimates the apex of the fitted quadratic curve as the in-focus position. In FIG. 7, Zc is the in-focus position. In FIG. 7, Zc is the in-focus position estimated by the blur feature. In FIG. 7, Z1 is the most in-focus position in the entire image. In FIG. 7, Z1 closest to Zc is the most in-focus position in the entire image. In addition, the image processing device 100 may apply a filter to Z1 with a strength that results in the same result as the result of applying the filter to be applied to Zc. In addition, the image processing device 100 may apply a filter with a strength equivalent to Zc to Z1 when Z1 is within a predetermined range that is considered to be equivalent to Zc.

[0052] The image processing device 100 calculates the degree of focus according to the distance from the in-focus position. In Fig. 7, the image processing device 100 calculates the degree of focus according to the distance from Zc.

[0053] The image processing device 100 determines the strength of the filter to be applied according to the calculated degree of focus. Then, the image processing device 100 applies the filter according to the determined strength. Specifically, the image processing device 100 determines the filter strength for each pixel and performs filter processing on the pixel. In this way, the image processing device 100 repeats the processing for each pixel to perform filter processing most suitable for the image.

[0054] This allows the image processing device 100 to change the strength of the filter depending on the degree of focus. This allows the image processing device 100 to apply a filter equivalent to 100% strength only to the focused area of ​​the image. If a filter with 100% strength is applied to the entire image, the sharpness of the out-of-focus area and noise will also increase. However, since the image processing device 100 applies a filter only to the focused area, the sharpness of only the focused area is improved. Since the sharpness of the out-of-focus area is not emphasized, the sense of depth of the image does not decrease, and overlapping of cells is easy to recognize. Since the sharpness of the noise is not emphasized, it is possible to suppress the burying of minute subjects such as Helicobacter pylori in noise. This allows the image processing device 100 to adjust the local contrast. This allows the image processing device 100 to correct an image in which the contrast has been reduced by the imaging optical system. This allows the image processing device 100 to partially adjust the contrast. This allows the image processing device 100 to confirm the situation even when mucus or the like is mixed into the cell nucleus and the genetic information is not located at the center of the cell nucleus, thereby improving the accuracy of diagnosis.

[0055] [4. Processing Variations] [4-1. Tap range] In the above example, the image processing device 100 selects a square tap centered on a predetermined pixel, but the range of the selected tap is not limited to a square such as 3×3, and any range may be selected as the tap. For example, the image processing device 100 may select a cross-shaped tap range centered on S0 as shown in FIG. 8. Furthermore, the number of pixels in the vertical and horizontal directions of the selected range may not be limited to 3 pixels. The image processing device 100 may select an 11×11 range with 11 pixels in each direction as the tap.

[0056] [4-2. Number of layers] In the above example, the image processing device 100 estimates the focus position using the reciprocals of three values ​​calculated using three pieces of image information with different Z directions, but the number of pieces of image information for estimating the focus position is not limited as long as it is equal to or greater than 3. For example, the image processing device 100 may estimate the focus position using the reciprocals of four values ​​calculated using four pieces of image information with different Z directions.

[0057] [4-3. Filter types] In the above example, the image processing device 100 determines the strength of the filter according to the degree of focus. However, the image processing device 100 may determine the type of the filter according to the degree of focus. For example, the image processing device 100 determines the type of the filter according to the estimated focus position. For example, when the degree of focus satisfies a predetermined condition, the image processing device 100 determines the type of a corresponding specific filter. For example, when the degree of focus satisfies a predetermined condition, the image processing device 100 determines the type of a filter that emphasizes a corresponding predetermined region. For example, when the degree of focus is equal to or greater than a predetermined threshold, the image processing device 100 determines to apply a mid-frequency emphasis filter. For example, when the degree of focus is less than a predetermined threshold, the image processing device 100 determines to apply a high-frequency emphasis filter. Alternatively, the image processing device 100 determines to apply a negative emphasis filter, i.e., a smoothing filter.

[0058] The image processing device 100 may determine the strength and type of the filter according to the degree of focus. The image processing device 100 may simultaneously determine both the strength and type of the filter according to the degree of focus. In this case, the image processing device 100 may apply a filter of the determined strength and type to the medical image. This allows the image processing device 100 to apply an optimal filter with optimal strength according to the degree of focus.

[0059] [4-4. Creating blocks with multiple pixels] In the above example, the image processing device 100 determines the strength and type of the filter for each pixel, but the image processing device 100 may configure a block with a plurality of pixels and determine the strength and type of the filter for each block. In this case, the image processing device 100 determines the plurality of pixels for configuring the block. The image processing device 100 may determine the plurality of pixels for configuring the block in any manner. For example, the image processing device 100 configures a block with a plurality of adjacent pixels. For example, the image processing device 100 configures a block with a predetermined plurality of pixels.

[0060] [4-5. Subject] In the above example, the image processing device 100 acquires a medical image having a specimen such as a cell as a subject, but the image processing device 100 may acquire an image having any subject as long as the subject is a living organism or an object collected from a living organism. For example, the image processing device 100 acquires an image having a specimen related to a living organism, a living organism, a material, a pathology, or the like in the medical field as a subject.

[0061] 4-6. Imaging device The imaging device according to the embodiment may be any device capable of capturing an image of a subject. For example, the imaging device according to the embodiment may be a microscope. When the imaging device according to the embodiment is a microscope, it may be any microscope.

[0062] [4-7.Focus position] In the above example, the image processing device 100 estimates the apex of the fitted quadratic curve as the focus position of the entire image, but it may also estimate a position within a predetermined range from the apex of the fitted quadratic curve as the focus position of the entire image.

[0063] [4-8. Calculation method of feature quantity] In the above embodiment, the pixel having the predetermined relationship is a pixel adjacent to the predetermined pixel, but this is not limited to this example. That is, the pixel having the predetermined relationship is not necessarily limited to an adjacent pixel. For example, the pixel having the predetermined relationship may be a pixel every other pixel or every third pixel. For example, the image processing device 100 may calculate the degree of focus by using the sum of the difference between the predetermined pixel and every other pixel as a feature. In this case, the image processing device 100 determines the strength of the filter to be applied to the medical image according to the degree of focus calculated by using the sum of the difference between the predetermined pixel and every other pixel as a feature.

[0064] [4-9. Integration of the device] In the above embodiment, the imaging device 10 and the image processing device 100 are separate devices, but the imaging device 10 and the image processing device 100 may be integrated. For example, the function of the image processing device 100 may be implemented in a computer that controls the operation of the imaging device 10, or may be implemented in any computer provided in the housing of the imaging device 10. In addition, the function of the image processing device 100 may be downloaded to a computer that controls the operation of the imaging device 10, or may be downloaded to any computer provided in the housing of the imaging device 10. This makes it possible to sell an imaging device 10 having the function of the image processing device 100.

[0065] 5. Configuration of the imaging device Next, the configuration of the imaging device 10 according to the embodiment will be described with reference to Fig. 9. Fig. 9 is a diagram showing an example of the configuration of the imaging device 10 according to the embodiment. As shown in Fig. 9, the imaging device 10 has a communication unit 11, a storage unit 12, and a control unit 13. Note that the imaging device 10 may also have an input unit (e.g., a keyboard, a mouse, etc.) that accepts various operations from an administrator of the imaging device 10, and a display unit (e.g., a liquid crystal display, etc.) that displays various information.

[0066] (Communications Department 11) The communication unit 11 is realized by, for example, a network interface card (NIC) etc. The communication unit 11 is connected to a predetermined network N by wire or wirelessly, and transmits and receives information to and from the image processing device 100 etc. via the predetermined network N.

[0067] (Storage unit 12) The storage unit 12 is realized by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 12 stores information related to medical images. Specifically, the storage unit 12 stores information related to medical images of an imaged subject.

[0068] (Control unit 13) The control unit 13 is a controller, and is realized, for example, by a CPU, an MPU, or the like, executing various programs stored in a storage device inside the imaging device 10 using a RAM as a working area. The control unit 13 is also a controller, and is realized, for example, by an integrated circuit such as an ASIC or an FPGA.

[0069] 9, the control unit 13 has an imaging unit 141, and realizes or executes the information processing action described below. Note that the internal configuration of the control unit 13 is not limited to the configuration shown in FIG. 9, and may be other configurations as long as they perform the information processing described below.

[0070] (Imaging unit 141) The imaging section 141 captures various types of information. The imaging section 141 captures an image of a subject on a slide. The imaging section 141 acquires various types of information. The imaging section 141 acquires a captured medical image.

[0071] 6. Configuration of Image Processing Device Next, the configuration of the image processing device 100 according to the embodiment will be described with reference to Fig. 10. Fig. 10 is a diagram showing an example of the configuration of the image processing device 100 according to the embodiment. As shown in Fig. 10, the image processing device 100 has a communication unit 110, a storage unit 120, and a control unit 130. Note that the image processing device 100 may have an input unit (e.g., a keyboard, a mouse, etc.) that accepts various operations from an administrator of the image processing device 100, and a display unit (e.g., a liquid crystal display, etc.) that displays various information.

[0072] (Communication unit 110) The communication unit 110 is realized by, for example, a NIC etc. The communication unit 110 is connected to a network N by wire or wirelessly, and transmits and receives information to and from the imaging device 10 etc. via the network N.

[0073] (Storage unit 120) The storage unit 120 is realized by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. As shown in FIG. 10, the storage unit 120 has a medical image storage unit 121 and an enhancement filter storage unit 122.

[0074] The medical image storage unit 121 stores information related to medical images. An example of the medical image storage unit 121 according to the embodiment is shown in Fig. 11. As shown in Fig. 11, the medical image storage unit 121 has items such as "medical image ID" and "medical image".

[0075] "Medical image ID" indicates identification information for identifying a medical image. "Medical image" indicates a medical image obtained by capturing an image of a subject. For example, "medical image" indicates a medical image captured by imaging device 10. In the example shown in FIG. 11, conceptual information such as "medical image #11" and "medical image #12" is stored in "medical image", but in reality, still images, videos, etc. are stored. In addition, still images and videos may be stored on a server or cloud different from the image processing device, and "medical image" may store a URL (Uniform Resource Locator) where the contents of the medical image are located, or a file path name indicating the storage location of these, etc.

[0076] The emphasis filter storage unit 122 stores information about emphasis filters to be applied to medical images. An example of the emphasis filter storage unit 122 according to the embodiment is shown in Fig. 12. As shown in Fig. 12, the emphasis filter storage unit 122 has items such as "emphasis filter ID" and "emphasis filter".

[0077] "Emphasis filter ID" indicates identification information for identifying an emphasis filter. "Emphasis filter" indicates information related to the emphasis filter. In the example shown in FIG. 12, conceptual information such as "emphasis filter #11" and "emphasis filter #12" is stored in "emphasis filter", but in reality, the strength and type of the emphasis filter, the range of pixels to which the emphasis filter is applied, etc. are stored.

[0078] (Control unit 130) The control unit 130 is a controller, and is realized, for example, by a CPU, an MPU, or the like, executing various programs stored in a storage device inside the image processing device 100 using a RAM as a working area. The control unit 130 is also a controller, and is realized, for example, by an integrated circuit such as an ASIC or an FPGA.

[0079] 10, the control unit 130 has an acquisition unit 131, a calculation unit 132, and a determination unit 133, and realizes or executes the information processing action described below. Note that the internal configuration of the control unit 130 is not limited to the configuration shown in FIG. 10, and may be other configurations as long as they perform the information processing described below.

[0080] (Acquisition part 131) The acquisition unit 131 acquires various types of information. The acquisition unit 131 acquires various types of information from an external information processing device. The acquisition unit 131 acquires various types of information from another information processing device such as the imaging device 10.

[0081] The acquisition unit 131 acquires various pieces of information from the storage unit 120. The acquisition unit 131 acquires various pieces of information from the medical image storage unit 121 and the enhancement filter storage unit 122.

[0082] The acquisition unit 131 stores the acquired various information in the storage unit 120. The acquisition unit 131 stores the various information in the medical image storage unit 121 and the enhancement filter storage unit 122.

[0083] The acquisition unit 131 acquires various pieces of information calculated and determined by other functional configurations.

[0084] The acquisition unit 131 acquires a medical image of a subject. For example, the acquisition unit 131 acquires a medical image of a subject captured by the imaging device 10. For example, the acquisition unit 131 acquires a medical image of a subject related to a living body, a living thing, a material, or a pathology in the medical field. For example, the acquisition unit 131 acquires a medical image captured by a microscope.

[0085] (Calculation unit 132) The calculation unit 132 calculates various pieces of information. The calculation unit 132 calculates various pieces of information from the storage unit 120. The calculation unit 132 calculates various pieces of information from the medical image storage unit 121 and the enhancement filter storage unit 122.

[0086] The calculation unit 132 stores the calculated various pieces of information in the storage unit 120. The calculation unit 132 stores the various pieces of information in the medical image storage unit 121 and the enhancement filter storage unit 122.

[0087] The calculation unit 132 calculates various pieces of information acquired and determined by other functional configurations. The calculation unit 132 calculates various pieces of information based on various pieces of information acquired and determined by other functional configurations.

[0088] The calculation unit 132 calculates the adjacent squared difference sum using the acquired image information. The calculation unit 132 calculates the adjacent squared difference sum within a tap range determined with a predetermined pixel as the center. For example, the calculation unit 132 calculates the adjacent squared difference sum based on a feature amount obtained by summing up the differences between a predetermined pixel and adjacent pixels in a medical image.

[0089] The calculation unit 132 calculates a Lorentz function from the calculated sum of squared adjacent differences. The calculation unit 132 calculates an approximation of the Lorentz function from the calculated sum of squared adjacent differences. The calculation unit 132 approximates the calculated sum of squared adjacent differences to the Lorentz function.

[0090] The calculation unit 132 calculates the inverse of the Lorentz function from the calculated Lorentz function. The calculation unit 132 estimates the in-focus position from the inverse of the Lorentz function. The calculation unit 132 estimates the apex of the inverse of the Lorentz function as the in-focus position.

[0091] The calculation unit 132 calculates the degree of focus. The calculation unit 132 calculates the degree of focus using the estimated focus position. The calculation unit 132 calculates the degree of focus according to the distance from the estimated focus position. The calculation unit 132 calculates the degree of focus according to the distance between the estimated focus position and a Z value corresponding to the calculated inverse of the Lorentz function.

[0092] (Decision unit 133) The determination unit 133 determines various pieces of information. The determination unit 133 determines various pieces of information from the storage unit 120. The determination unit 133 determines various pieces of information from the medical image storage unit 121 and the enhancement filter storage unit 122.

[0093] The determination unit 133 stores the determined various information in the storage unit 120. The determination unit 133 stores the various information in the medical image storage unit 121 and the enhancement filter storage unit 122.

[0094] The determination unit 133 determines various pieces of information acquired and calculated by other functional configurations. The determination unit 133 determines various pieces of information based on various pieces of information acquired and calculated by other functional configurations.

[0095] The determination unit 133 selects a pixel to which the filter is applied. The determination unit 133 selects the first pixel to which the filter is applied. For example, the determination unit 133 selects the next pixel after the filter is applied to a predetermined pixel. For example, the determination unit 133 selects the pixel to which the filter is applied based on a predetermined algorithm.

[0096] The determination unit 133 determines the strength of the filter to be applied to the medical image according to the calculated degree of focus. The determination unit 133 determines a synthesis rate indicating the ratio of synthesis of the medical image according to a plurality of filters with different strengths according to the calculated degree of focus. The determination unit 133 selectively determines a plurality of filters to be applied to the medical image according to the calculated degree of focus.

[0097] The determination unit 133 determines the type of filter to be applied to the medical image according to the calculated focus degree. The determination unit 133 determines a combination rate indicating the ratio of combination of the medical image according to a plurality of different types of filters according to the calculated focus degree.

[0098] The determining unit 133 applies a filter of the determined strength to the medical image. The determining unit 133 applies a filter of the determined type to the medical image.

[0099] The determination unit 133 applies a filter of a strength determined for a plurality of medical images having different Z values ​​according to the calculated degree of focus to each medical image. Then, the determination unit 133 determines a synthesis rate indicating a synthesis ratio of each medical image to which the filter has been applied according to the calculated degree of focus. Then, the determination unit 133 synthesizes each medical image based on the synthesis rate determined according to the calculated degree of focus.

[0100] The determination unit 133 determines whether the filter has been applied to all pixels. The determination unit 133 determines whether the filter process has been applied to all pixels in the medical image. The determination unit 133 determines whether the filter process has been applied to all pixels included in a predetermined range in the medical image.

[0101] [7. Information processing flow] Next, the procedure of information processing by the image processing system 1 according to the embodiment will be described with reference to Fig. 13. Fig. 13 is a flowchart showing the procedure of information processing by the image processing system 1 according to the embodiment.

[0102] As shown in FIG. 13, the image processing device 100 acquires a medical image (step S101). The image processing device 100 selects a predetermined pixel for performing a filter process (step S102). The image processing device 100 estimates a focus degree using a blur function (step S103). The image processing device 100 determines a type and strength of a filter from the estimated focus degree (step S104). The image processing device 100 applies a filter of the determined type and strength (step S105). The image processing device 100 determines whether the filter process has been performed on all pixels (step S106). When the image processing device 100 determines that the filter process has not been performed on all pixels (step S106; No), the image processing device 100 returns to the process of step S102. For example, the image processing device 100 returns to the process of step S102 and selects a pixel other than the pixel previously selected. On the other hand, when the image processing device 100 determines that the filter process has been performed on all pixels (step S106; Yes), the information processing ends.

[0103] 8. Modifications The image processing system 1 according to the embodiment described above may be implemented in various different forms other than the above embodiment, so other embodiments of the image processing system 1 will be described below.

[0104] In the above embodiment, the process of applying an optimal filter to each pixel of a plurality of medical images with different Z values ​​has been described. Here, a process that can obtain the same effect as the process of applying an optimal filter to each pixel of a plurality of medical images with different Z values ​​will be described with reference to Figs. 14 and 15.

[0105] Fig. 14 shows a case where the image processing device 100 applies one enhancement filter to an acquired medical image. In Fig. 14, image KG1 shows the image before the filter is applied, and image KG2 shows the image after the filter is applied. In this case, an image is generated in which the contrast and resolution of the entire image are improved, including objects other than the object of interest.

[0106] Here, the image processing device 100 may apply a plurality of enhancement filters to the acquired medical image. The image processing device 100 may generate a composite image to which a plurality of filters with different intensities are applied according to the degree of focus.

[0107] FIG. 15 shows a case where the image processing device 100 synthesizes an image obtained by applying a plurality of enhancement filters to an acquired medical image. In FIG. 15, image KG11 shows an image to which a first filter is applied, and image KG22 shows an image to which a second filter having a different strength from the first filter is applied. As shown in FIG. 15, image KG11 is an image with more blur than image KG22. Also, image KG33 shows a synthetic image obtained by synthesizing image KG11 and image KG22. In FIG. 15, target SJ11 is a first region indicating a region of interest in the image, and target SJ22 shows a second region that is not a region of interest. For example, target SJ22 is a noise portion in a medical image.

[0108] The image processing device 100 may determine a synthesis rate indicating the ratio of synthesis of medical images. For example, the image processing device 100 may determine a synthesis rate for a first region and a second region. In FIG. 15, the image processing device 100 synthesizes the image KG11 and the image KG22 with a synthesis rate of 1:9 for the region of the target SJ11. The synthesis rate ratio for the region of the target SJ11 may be any ratio as long as the ratio of the image KG22 is higher than that of the image KG11. Also, in FIG. 15, the image processing device 100 synthesizes the image KG11 and the image KG22 with a synthesis rate of 9:1 for the region of the target SJ22. The synthesis rate ratio for the region of the target SJ22 may be any ratio as long as the ratio of the image KG11 is higher than that of the image KG22.

[0109] 15 shows a case where two images to which an emphasis filter is applied are combined, but the number of images to be combined by the image processing device 100 is not limited. The image processing device 100 may selectively determine the number of images to be combined, or may selectively determine the images to be combined. For example, when there is an area that cannot be distinguished in an image to which a first filter is applied, the image processing device 100 selectively combines a plurality of images to which another filter is applied with an image to which a first filter is applied, thereby generating an image with improved visibility and distinguishability compared to an image to which only the first filter is applied.

[0110] In this way, the image processing device 100 can adjust the local contrast by synthesizing images processed by emphasis filters with different intensities, and the image processing device 100 can adjust the global and local contrast by adaptively applying an optimal filter, thereby improving visibility like an optical microscope.

[0111] 15 shows an example of synthesizing images to which a plurality of different filters are applied to one image. In this manner, the image processing device 100 may generate a synthetic image by synthesizing images to which a plurality of different filters are applied to one image. Also, as in the above embodiment, the image processing device 100 may generate a synthetic image by synthesizing images to which an optimal filter is applied according to the degree of focus for a plurality of medical images having different Z values ​​according to the degree of focus.

[0112] [9. Hardware Configuration] Moreover, the imaging device 10 and the image processing device 100 according to the above-described embodiment are realized, for example, by a computer 1000 configured as shown in Fig. 16. Fig. 16 is a hardware configuration diagram showing an example of a computer that realizes the functions of the imaging device 10 and the image processing device 100. The computer 1000 has a CPU 1100, a RAM 1200, a ROM 1300, a HDD 1400, a communication interface (I / F) 1500, an input / output interface (I / F) 1600, and a media interface (I / F) 1700.

[0113] The CPU 1100 operates and controls each unit based on a program stored in the ROM 1300 or the HDD 1400. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 is started up, programs that depend on the hardware of the computer 1000, and the like.

[0114] The HDD 1400 stores programs executed by the CPU 1100, data used by such programs, etc. The communication interface 1500 receives data from other devices via a predetermined communication network and sends it to the CPU 1100, and transmits data generated by the CPU 1100 to other devices via the predetermined communication network.

[0115] The CPU 1100 controls output devices such as a display and a printer, and input devices such as a keyboard and a mouse, via the input / output interface 1600. The CPU 1100 acquires data from the input devices via the input / output interface 1600. The CPU 1100 also outputs generated data to the output devices via the input / output interface 1600.

[0116] The media interface 1700 reads a program or data stored in the recording medium 1800 and provides it to the CPU 1100 via the RAM 1200. The CPU 1100 loads the program from the recording medium 1800 onto the RAM 1200 via the media interface 1700 and executes the loaded program. The recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase change rewritable Disc), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0117] For example, when the computer 1000 functions as the imaging device 10 and image processing device 100 according to the embodiment, the CPU 1100 of the computer 1000 executes programs loaded onto the RAM 1200 to realize the functions of the control units 13 and 130. The CPU 1100 of the computer 1000 reads and executes these programs from the recording medium 1800, but as another example, the CPU 1100 may obtain these programs from other devices via a predetermined communication network.

[0118] [10. Other] In addition, among the processes described in the above embodiments and modifications, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by a known method. In addition, the information including the processing procedures, specific names, various data and parameters shown in the above documents and drawings can be changed arbitrarily unless otherwise specified. For example, the various information shown in each drawing is not limited to the illustrated information.

[0119] In addition, each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads, usage conditions, etc.

[0120] Furthermore, the above-described embodiments and modifications can be appropriately combined as long as the processing contents are not contradictory.

[0121] Although some of the embodiments of the present application have been described in detail above with reference to the drawings, these are merely examples, and the present invention can be embodied in other forms that incorporate various modifications and improvements based on the knowledge of those skilled in the art, including the forms described in the Disclosure of the Invention section.

[0122] Furthermore, the above-mentioned "section, module, unit" can be read as "means" or "circuit", etc. For example, an acquisition section can be read as an acquisition means or an acquisition circuit.

[0123] The present technology can also be configured as follows. (1) Computer Acquire a medical image captured by an imaging device; A strength of a filter to be applied to the medical image is determined according to a degree of focus of the medical image. Image processing methods. (2) determining a strength of a filter to be applied to the medical image according to the degree of focus calculated by a blur function indicating a degree of blur of the medical image; The image processing method according to (1). (3) A filter strength to be applied to the medical image is determined according to the degree of focus calculated by the blur function according to a direction perpendicular to the medical image and a thickness direction of the subject. An image processing method according to (1) or (2). (4) A strength of a filter to be applied to the medical image is determined according to the degree of focus calculated from a feature amount obtained by summing up differences between a predetermined pixel in the medical image and surrounding pixels. The image processing method according to any one of (1) to (3). (5) A filter strength to be applied to the medical image is determined based on an estimated focus degree estimated based on the blur function and the focus degree calculated from a feature amount obtained by summing up differences between a predetermined pixel and surrounding pixels in the medical image. The image processing method according to any one of (1) to (4). (6) A composition ratio indicating a composition ratio of a composite image generated by applying a plurality of different filters to the medical image is determined according to a degree of focus of the subject. The image processing method according to any one of (1) to (5). (7) determining the blending ratio of the blended image generated by applying the plurality of filters having different strengths to the predetermined medical image; The image processing method according to any one of (1) to (6). (8) The synthesis rate of the synthetic image is determined by applying a filter according to the degree of focus of the subject to the plurality of medical images having different Z values. The image processing method according to any one of (1) to (7). (9) Selectively determining the plurality of filters to be applied to the medical images to generate the composite image in response to a focus state of the subject. The image processing method according to any one of (1) to (8). (10) Depending on the degree of focus of the subject, the type of region to be emphasized by the filter is determined to be high-frequency emphasis, mid-frequency emphasis, low-frequency emphasis, or a negative emphasis filter, i.e., a smoothing filter. The image processing method according to any one of (1) to (9). (11) Acquiring medical images of subjects related to living organisms, organisms, materials, or pathologies in the medical field The image processing method according to any one of (1) to (10). (12) The medical image is acquired by using a microscope as the imaging device. The image processing method according to any one of (1) to (11). (13) an acquisition unit for acquiring a medical image of a subject captured by a microscope; a determination unit that determines a strength of a filter to be applied to the medical image in accordance with a degree of focus of the subject, the filter improving image quality of the medical image; An image processing device comprising: (14) An imaging device that captures an image of a subject; and software used to process medical images corresponding to the object imaged by the imaging device, the software includes an image processing device that determines a filter strength to be applied to a medical image captured by the imaging device in accordance with a degree of focus of the subject; An image processing system comprising: [Explanation of symbols]

[0124] 1. Image Processing System 10. Imaging device 100 Image processing device 110 Communications Department 120 Storage section 121 Medical Image Storage Unit 122 Emphasis filter memory unit 130 Control section 131 Acquisition Department 132 Calculation section 133 Decision Section N Network

Claims

1. Computer Obtaining a medical image of a subject captured by an imaging device; determining a synthesis rate for the region of interest such that a first image obtained by applying a first filter to the medical image has a lower ratio than a second image obtained by applying a second filter to the medical image, the second filter having a different strength from the first filter; A blending rate for a noise region other than the region of interest is determined so that the first image has a higher blending rate than the second image. Image processing methods.

2. A composite image of the attention region is generated based on the synthesis rate determined for the attention region, and a composite image of the noise region is generated based on the synthesis rate determined for the noise region, thereby generating a composite image of the first image and the second image.

2. The image processing method according to claim 1.

3. Acquiring medical images of subjects related to living organisms, organisms, materials, or pathologies in the medical field 2. The image processing method according to claim 1.

4. The medical image is acquired by using a microscope as the imaging device.

2. The image processing method according to claim 1.

5. an acquisition unit for acquiring a medical image of a subject captured by a microscope; determining a synthesis rate for the region of interest such that a first image obtained by applying a first filter to the medical image has a lower ratio than a second image obtained by applying a second filter to the medical image, the second filter having a different strength from the first filter; a determination unit that determines a synthesis rate for a noise region other than the attention region such that the first image has a higher ratio than the second image; An image processing device comprising:

6. An image processing system including software used to process medical images corresponding to a subject captured by an imaging device, The imaging device for imaging a subject; The software determines a synthesis rate for the region of interest such that a first image obtained by applying a first filter to the medical image has a lower ratio than a second image obtained by applying a second filter to the medical image, the second filter having a different strength from the first filter; an image processing device that determines a synthesis rate for a noise region other than the attention region so that the first image has a higher ratio than the second image; An image processing system comprising:

Images