Image processing device
By setting multiple pixel groups and filters in the image processing device for angle configuration and smoothing, the problem of difficulty in extracting linear features under noise influence is solved, and the extraction accuracy and analysis accuracy of linear parts are significantly improved.
Patent Information
- Application Number
- CN202080077783.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2019-09-17
- Filing Date
- 2020-08-28
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2040-08-28
AI Technical Summary
In the prior art, when performing noise reduction processing on captured images, it is difficult to effectively retain information of linear parts, especially when linear features such as cell neurites are photographed, the noise influence leads to low extraction accuracy.
By setting multiple pixel groups in the image processing device, configuring and calculating candidate pixel values at different angles, the linear portions in the image are enhanced, and smoothing is performed with one-dimensional and two-dimensional filters, and binarization is performed by binarization processing. The part with low direction dependence is removed, and the extraction accuracy of the linear part is significantly improved.
Effectively enhance the linear part in the image, improve the extraction accuracy and analysis accuracy of linear features such as neurites, reduce the influence of noise, and enhance the ability to evaluate cell state.
Smart Images

Figure CN114651274B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an image processing device, an image processing method, and a program. Background Art
[0002] For example, a method of performing image processing on an image obtained by photographing an object including linear parts such as branches as morphological features has been proposed (for example, see Patent Document 1). When performing noise reduction processing in such image processing, it is necessary to perform the processing while retaining information about the linear parts.
[0003] Patent document 1: U.S. Patent Application Publication No. 2009 / 0154792. Summary of the invention
[0004] According to a first embodiment of the present invention, an image processing device includes a first image processing unit, which sets a pixel value of a second pixel of the second image located at a position corresponding to a first pixel of the first image in a second image obtained by performing image processing on a first image, and the first image processing unit includes: a first pixel group setting unit, which sets a plurality of first pixel groups, wherein the plurality of first pixel groups are a plurality of first pixel groups set corresponding to the first pixels, and the plurality of first pixel groups are arranged along a plurality of directions forming a plurality of angles different from each other with respect to a prescribed direction in the first image, or are arranged along a prescribed direction in each of a plurality of images obtained by moving the first image at a plurality of angles different from each other relative to the prescribed direction; a first calculation unit, which calculates a plurality of first candidate pixel values based on the magnitude of the pixel values of the pixels included in each of the plurality of first pixel groups; and a first pixel value setting unit, which sets the pixel value of the second pixel of the second image based on the plurality of first candidate pixel values.
[0005] According to a second embodiment of the present invention, an image processing method includes: in a second image obtained by performing image processing on a first image, setting a pixel value of a second pixel of the second image located at a position corresponding to a first pixel of the first image, the setting of the pixel value of the second pixel includes: setting a plurality of first pixel groups, the plurality of first pixel groups are a plurality of first pixel groups set corresponding to the first pixels, the plurality of first pixel groups are arranged along a plurality of directions forming a plurality of different angles relative to a prescribed direction in the first image, or are arranged along a prescribed direction in each of a plurality of images obtained by moving the first image at a plurality of different angles; calculating a plurality of first candidate pixel values based on the size of the pixel values of the pixels included in each of the plurality of first pixel groups; and setting the pixel value of the second pixel of the second image based on the plurality of first candidate pixel values.
[0006] According to a third aspect of the present invention, a program is a program for causing a processing device to perform a first image processing, wherein the first image processing sets a pixel value of a second pixel of the second image located at a position corresponding to a first pixel of the first image in a second image obtained by performing image processing on the first image, and the first image processing comprises: a first pixel group setting processing for setting a plurality of first pixel groups, wherein the plurality of first pixel groups are a plurality of first pixel groups set corresponding to the first pixels, and the plurality of first pixel groups are arranged along a plurality of directions forming a plurality of different angles relative to a prescribed direction in the first image, or are arranged along a prescribed direction in each of a plurality of images obtained by moving the first image at a plurality of different angles; a first calculation processing for calculating a plurality of first candidate pixel values based on the magnitude of the pixel values of the pixels included in each of the plurality of first pixel groups; and a second pixel setting processing for setting the pixel value of the second pixel of the second image based on the plurality of first candidate pixel values. BRIEF DESCRIPTION OF THE DRAWINGS
[0007] Figure 1 It is a schematic diagram showing the structure of the image processing device according to the first embodiment.
[0008] Figure 2 It is a schematic diagram showing the structure of the data processing unit according to the first embodiment.
[0009] Figure 3 It is a schematic diagram showing a captured image according to the first embodiment.
[0010] Figure 4 It is a schematic diagram for explaining the first image processing in the first embodiment.
[0011] Figure 5 It is a schematic diagram for explaining the second image processing in the first embodiment.
[0012] Figure 6 This is a flowchart showing the flow of the image processing method according to the first embodiment.
[0013] Figure 7 This is a flowchart showing the flow of the first image processing according to the first embodiment.
[0014] Figure 8 This is a flowchart showing the flow of the second image processing according to the first embodiment.
[0015] Fig. 9 It is a schematic diagram showing the structure of the data processing unit of Modification Example 2.
[0016] Fig.10 This is a schematic diagram for explaining the first image processing of Modification Example 2.
[0017] Fig.11 Schematic diagram showing the linear portion of a captured image and the position of the one-dimensional filter in each case where the angle of the one-dimensional filter is 0°, 30°, 60°, 90°, 120°, and 150°.
[0018] Fig.12 This is a graph showing the angle of a one-dimensional filter and the average value of the pixel values of a plurality of pixels corresponding to the filter.
[0019] Fig.13 This is a graph showing the angle of a one-dimensional filter and the average value of the pixel values of a plurality of pixels corresponding to the filter.
[0020] Fig.14 This is a flowchart showing the flow of the image processing method according to Modification Example 2.
[0021] Fig.15 It is a schematic diagram for explaining the setting of the range for selecting the first pixel.
[0022] Fig.16 2 is a schematic diagram showing an example of a display screen for displaying an enhanced image.
[0023] Fig.17 This is a schematic diagram showing an example of a display screen for setting parameters.
[0024] Fig.18 This is a flowchart showing the flow of the image processing method according to Modification Example 7.
[0025] Fig.19 This is a schematic diagram showing an example of a display screen of Modification Example 7.
[0026] Fig. 20 This is a schematic diagram showing an example of a display screen of Modification Example 7.
[0027] Fig.21 This is a schematic diagram showing an example of a display screen of Modification Example 7.
[0028] Fig. 22 This is a schematic diagram showing an example of a display screen of Modification Example 8.
[0029] Fig.23 is a schematic diagram provided for explaining the procedure. DETAILED DESCRIPTION
[0030] Hereinafter, a mode for carrying out the present invention will be described with reference to the drawings.
[0031] - First Embodiment -
[0032] Figure 1Schematic diagram showing the structure of the image processing device of the present embodiment. The image processing device 1 includes a culture unit 100 and an information processing unit 40. The culture unit 100 includes a culture chamber (culture cavity) 10, an observation sample stage (stage) 11, a drive unit 12, and a photographing unit 20. The information processing unit 40 includes an input unit 41, a communication unit 42, a storage unit 43, an output unit 44, and a control unit 50. The control unit 50 includes a data processing unit 51, an output control unit 52, and a device control unit 53.
[0033] The image processing device 1 is configured as a culture device, and captured image data, which is image data obtained by capturing images of samples (eg, cells) in the culture unit 100 , is input to the data processing unit 51 and processed.
[0034] In this embodiment, image processing is performed to enhance linear parts including straight lines, curves, broken lines or intersections based on the captured image corresponding to the captured image data. Here, "enhancement" means that the linear part and other parts (for example, the background part, the part other than the linear part) are easily distinguished by the pixel value, and is not limited to the case where the brightness of the linear part is improved. In the following, the cell Ce is used as a nerve cell, and the part corresponding to the neurite Nr is enhanced as an example of the linear part, but as long as the linear part as described above is included, the object of enhancement is not limited to this example, and can be any element constituting the image (for example, other cells, other than cells).
[0035] Furthermore, the above-described image processing may be performed on an image other than a captured image, such as a drawn image.
[0036] The culture unit 100 includes a culture chamber 10 for culturing cells Ce, and the imaging unit 20 images the cells Ce cultured in the culture chamber 10 .
[0037] The culture chamber 10 stores a culture container C for culturing cells Ce. The culture container C is, for example, a well plate or a dish. A temperature regulator (for example, a heater) and a temperature sensor (not shown) controlled by the control unit 50 are arranged inside the culture chamber 10. The interior of the culture chamber 10 is controlled to be maintained at a preset temperature by the temperature regulator and the temperature sensor, and the cells Ce are cultured under a preset environment. The drive unit 12 has an actuator that moves the culture container C at a predetermined time and places it on the observation sample stage 11 inside the culture chamber 10. In addition, in order to photograph the cells Ce, the drive unit 12 moves the photographing unit 20 or the observation sample stage 11 to an appropriate position (for example, an observation position) in the direction of the optical axis so that the cells Ce are arranged in the focal plane of the photographing unit 20.
[0038] The photographing unit 20 has a photographing device including a photographing element such as a CMOS or a CCD, and photographs the cell Ce, in particular, the neurite Nr of the cell Ce. The photographing method of the photographing unit 20 is not particularly limited as long as the pixel corresponding to the neurite Nr in the photographed image including the cell Ce can be distinguished from other parts with a desired accuracy by the brightness value of the pixel or a plurality of pixels around the pixel. For example, the photographing method of the photographing unit 20 can use a fluorescence observation method or a phase difference observation method. In the case of the fluorescence observation method, the photographing unit 20 obtains a fluorescence image of the sample, and the fluorescence image is used for the image analysis described later. In the case of the phase difference observation method, the photographing unit 20 obtains a phase difference image of the sample, and the phase difference image is used for the image analysis described later.
[0039] When the imaging unit 20 is imaging by fluorescence observation, fluorescent staining can be performed by causing the cell Ce to express a fluorescent protein such as green fluorescent protein (GFP) or a protein fused with a fluorescent protein and a protein locally present in the neurite Nr by gene introduction. If there is no problem with the utilization (e.g., culture, subculture, selection) of the cell Ce after imaging, other labeling methods such as immunostaining can also be performed.
[0040] The pixel signal obtained by the imaging unit 20 when imaging the cell Ce is converted into a digital signal and input into the information processing unit 40 ( Figure 1 ), stored in the storage unit 43.
[0041] The information processing unit 40 serves as an interface with a user of the image processing apparatus 1 (hereinafter simply referred to as a “user”), and also performs processes such as communication, storage, and calculation related to various data.
[0042] Furthermore, the information processing unit 40 may be configured as an information processing device physically separated from the culture unit 100. Furthermore, at least a part of the data used by the image processing device 1 may be stored in a remote server or the like connected via a network.
[0043] The input unit 41 includes an input device such as a mouse, a keyboard, various buttons, or a touch panel, etc. The input unit 41 receives images of the culture unit 100 and data required for data processing by the data processing unit 51 from the user.
[0044] The communication unit 42 includes a communication device capable of communicating via wireless or wired connection such as the Internet, and appropriately transmits and receives data related to control and processing in the image processing device 1 .
[0045] The storage unit 43 includes a nonvolatile storage medium, and stores a program for causing the control unit 50 to perform processing, image data related to processing by the data processing unit 51 , and the like.
[0046] The output unit 44 includes a display device such as a liquid crystal monitor, and outputs an image or the like representing information obtained by processing by the data processing unit 51 .
[0047] The control unit 50 is constituted by a processing device such as a CPU, functions as a main body for controlling the operation of the image processing device 1 , and performs various processes by executing a program loaded in the storage unit 43 .
[0048] The data processing unit 51 of the control unit 50 acquires the captured image data input from the imaging unit 20, processes it, and generates an image in which the linear portion of the captured image is enhanced. Hereinafter, this image is referred to as an enhanced image.
[0049] Figure 2 : is a schematic diagram showing the structure of the data processing unit 51. The data processing unit 51 includes a first image processing unit 61, a second image processing unit 62, a first removal unit 64, and a binarization unit 65. The first image processing unit 61 includes a first pixel group setting unit 611, a first calculation unit 612, and a first pixel value setting unit 613. The second image processing unit 62 includes a second pixel group setting unit 621, a second calculation unit 622, and a second pixel value setting unit 623. The first pixel group setting unit 611 and the second pixel group setting unit 621 constitute a setting unit.
[0050] The first image processing unit 611 of the data processing unit 51 performs image processing on the captured image data corresponding to the captured image, and generates first enhanced image data corresponding to the first enhanced image. In the following, the image processing based on the first image processing unit 611 is referred to as the first image processing. Here, the first enhanced image refers to an enhanced image in which the linear part (for example, the neurite Nr) in the captured image is enhanced by the first image processing. The first image processing unit 611 sets the pixel value of the first enhanced image corresponding to each pixel in the captured image. In the following, the pixel in the captured image that is the object of processing by the first image processing unit 61 is referred to as the first pixel, and the pixel in the first enhanced image corresponding to the first pixel is referred to as the second pixel. Here, the "correspondence" between the first pixel and the second pixel means that these pixels represent the brightness of the light from the same position of the subject in the captured image, but if the position obtains the desired analysis accuracy, a certain offset is allowed.
[0051] Figure 3 is a schematic diagram showing an example of a captured image. The captured image G1 is composed of a plurality of pixels Px. Figure 3 In the example above, darker shades represent brighter pixels. Figure 3In the captured image G1, the cell body So and the neurite Nr are shown as parts with high brightness. On the other hand, there are also scattered pixels with brightness caused by noise during shooting, that is, noise pixels Pxn. One of the reasons is that when shooting, the pixel signal corresponding to the linear part such as the neurite Nr is weak, so it cannot be photographed with sufficient accuracy. In particular, this tendency can be seen even in a fluorescence microscope that is often used in shooting cells, etc.
[0052] The method of expressing the image in the captured image data is not particularly limited, and for example, the portion with a larger pixel value and higher brightness or chroma may be used, or the portion with a smaller pixel value and higher brightness or chroma may be used.
[0053] In the culture of neural cells, the neural cells are evaluated by measuring the length (or width) of the neurite Nr and the number of branches. In order to accurately evaluate the neural cells, it is necessary to appropriately extract the neurite from the captured image. Figure 3 In the captured image G1 of such noise pixels Pxn, it is sometimes difficult to extract the linear portion with high accuracy. The image processing device 1 of the present embodiment makes it easy to appropriately extract the neurites Nr by generating an image that enhances the linear portion in the captured image.
[0054] In addition to the evaluation of neurites, the image processing device 1 of this embodiment can also be used for various purposes such as utilizing, extracting or analyzing linear parts in an image. In addition, for the cell body So, the part corresponding to the cell body So can be detected from the captured image in advance, and the part can be masked to perform the following first image processing. The detection of the cell body So can be appropriately performed by eliminating the details of the captured image by using opening processing and closing processing.
[0055] The first pixel group setting unit 611 of the first image processing unit 61 sets a plurality of first pixel groups corresponding to first pixels of the captured image G1 .
[0056] Figure 4 Schematic diagram for explaining the first image processing. The first pixel group setting unit 611 selects the first pixel Px1 from the pixels Px included in the captured image G1. The first pixel group setting unit 611 selects and sets the first pixel Px1 from the pixels included in the area (hereinafter referred to as the target area) in which the first pixel Px1 is selected in the captured image G1 and is pre-set by the first image processing unit 61.
[0057] The object area may be the entire captured image. For example, when the pixel at the edge of the captured image G1 is set as the first pixel Px1, there may be no pixel corresponding to the one-dimensional filter described later. Therefore, in this case, it is preferable to appropriately add virtual pixels around the captured image G1 to perform the image processing described later with the desired accuracy. The object area can be set to a portion obtained by removing the edge of the captured image G1 that is about half the length of the one-dimensional filter, so that there will be no situation where there is no pixel corresponding to the one-dimensional filter described later.
[0058] The first pixel group setting unit 611 sets a pixel group including the first pixel Px1 and arranged along a plurality of directions forming a predetermined angle with respect to a reference direction as a first pixel group Pg1. The predetermined angle is hereinafter referred to as a first angle θ1. Figure 4 In the figure, it is the left-right direction of the captured image G1 (see arrow A10), but it is not particularly limited and may be any direction. Figure 4 , the first pixel group Pg1 includes a plurality of pixels Px11, Px12, Px13, Px14 when the first angle θ1 is 0°. The first pixel group Pg1 preferably includes a plurality of pixels arranged in one direction and adjacent to each other. The first pixel group setting unit 611 can set a plurality of pixels arranged in an oblique direction and in a longitudinal direction and adjacent to each other in the captured image G1 as the first pixel group Pg1 when the first angle θ1 is 45° and 90°.
[0059] exist Figure 4 , partial areas of the captured image G1 corresponding to the first pixel group Pg1 when the first angle θ1 is 0°, 30°, and 60° are schematically shown using one-dimensional filters F1, F11, and F12, respectively. When the first pixel group Pg1 corresponding to the one-dimensional filters F11 and F12 is set, pixels overlapping with these one-dimensional filters and pixels through which the straight lines corresponding to these one-dimensional filters pass can be included in the first pixel group Pg1. However, the pixels set in this way are not arranged on a straight line, so the accuracy is slightly reduced.
[0060] Therefore, in actual calculation, the first pixel group setting unit 611 preferably sets a plurality of pixels arranged along a prescribed direction with the pixel corresponding to the first pixel as the center in the image obtained by rotating the captured image G1 in a direction forming a first angle θ1 relative to the reference direction as the first pixel group Pg1. The prescribed direction is not particularly limited and may be any direction such as the reference direction. Figure 4In the example of FIG. 1 , the first pixel group setting unit 611 rotates the captured image G1 clockwise by θ1 while fixing the one-dimensional filter F1, and sets the pixels of 1 vertical × 5 horizontal pixels within the range of the one-dimensional filter F1 in the obtained image as the first pixel group Pg1. Thus, compared with the case of rotating the one-dimensional filter to set the first pixel group Pg1, the first pixel group Pg1 corresponding to the one-dimensional filter F11 in the captured image G1 before rotation can be accurately set.
[0061] The first pixel group setting unit 611 sets the first pixel group Pg1 for each of the plurality of first angles θ1. The plurality of first angles θ1 are set for each predetermined angle selected from a range of 1° to 45°, for example. For example, the first pixel group setting unit 611 can set the plurality of first angles θ1 from 0° to 179° at intervals of 1°, and set the first pixel group Pg1 for each first angle θ1.
[0062] The number of pixels constituting the first pixel group Pg1 is not particularly limited as long as it is two or more. The number of pixels can be set based on the curvature or curvature radius of the linear portion in the captured image G1.
[0063] Furthermore, the first pixel group Pg1 may be a pixel group consisting of a plurality of columns, such as a pixel block of 2 pixels×5 pixels. In this case, the number of pixels arranged in one direction is preferably more than twice the number of pixels arranged in the other direction.
[0064] The first calculation unit 612 of the first image processing unit 61 calculates a first candidate pixel value based on the pixel values of the pixels included in the first pixel group Pg1. The first candidate pixel value is a value that is a candidate for the pixel value set in the second pixel corresponding to the first pixel Px1 in the first enhanced image. In other words, the pixel value set in the second pixel is selected from a plurality of first candidate pixel values.
[0065] The first calculation unit 612 calculates the average value of the pixel values of the pixels included in the first pixel group Pg1 and sets it as the first candidate pixel value. For example, the first calculation unit 612 calculates the arithmetic average of the pixels of the pixels Px11, Px12, Px1, Px13 and Px14 corresponding to the one-dimensional filter F1 and sets it as the first candidate pixel value of the first pixel group Pg1 corresponding to the one-dimensional filter F1. The first calculation unit 612 calculates the first candidate pixel value for each first pixel group Pg1 set by the first pixel group setting unit 611. The first calculation unit 612 calculates the first candidate pixel value for each first pixel group Pg1 corresponding to the plurality of first angles θ1.
[0066] In addition, the first calculation unit 612 may calculate the average value of the pixel values of the pixels included in the first pixel group Pg1 as the first candidate pixel value by an average other than an arithmetic average such as a geometric average. Alternatively, the first calculation unit 612 may set the sum of the pixel values of the pixels included in the first pixel group Pg1 or a value that has been appropriately adjusted by normalization or the like based on a numerical range in which the sum can be set as a pixel value as the first candidate pixel value. As long as the first candidate pixel value can be set based on the overall size of the pixel values of the pixels included in the first pixel group Pg1, as in the above-mentioned sum or average, the calculation method of the first candidate pixel value set by the first calculation unit 612 is not particularly limited.
[0067] The first pixel value setting unit 613 of the first image processing unit 61 sets the pixel value of the second pixel in the first enhanced image corresponding to the first pixel Px1 based on the plurality of first candidate pixel values calculated for the first pixel Px1.
[0068] The first pixel value setting unit 613 can set the maximum value among the multiple first candidate pixel values of the first pixel Px1 as the pixel value of the corresponding second pixel. This means that in the first image processing, the pixel value of the one-dimensional filter corresponding to the direction in which the pixels with the highest pixel value are arranged in the one-dimensional filters F, F11, and F12 in multiple directions is reflected in the first enhanced image. As a result, when the pixel value is higher, the possibility that the pixel corresponds to the linear part of the neurite Nr or the like is higher, the linear part can be smoothed and enhanced. The first pixel value setting unit 613 can set the minimum value among the multiple first candidate pixel values of the first pixel Px1 as the pixel value of the corresponding second pixel. As a result, when the pixel value is lower, the possibility that the pixel corresponds to the linear part is higher, the linear part can be smoothed and enhanced.
[0069] Furthermore, the pixel value of the second pixel may be set to a value in the range of 0.8 to 1.2 times, preferably 0.9 to 1.1 times, the maximum value or the minimum value. In such a case, the linear portion can be enhanced with a certain degree of accuracy.
[0070] When the first image processing unit 61 completes the first image processing by the first pixel group setting unit 611, the first calculation unit 612, and the first pixel value setting unit 613 for a certain first pixel Px1, the first image processing unit 61 performs the first image processing on a pixel different from the first pixel Px1 in the target area of the captured image G1 as the first pixel Px1. The first image processing unit 61 performs the first image processing on each pixel included in the target area.
[0071] In addition, the first pixel group setting unit 611 and the first calculation unit 612 may also fix the first angle θ1, and calculate the first candidate pixel value of the one-dimensional filter based on the direction corresponding to the first angle θ1 for each first pixel Px1 in the object area. Then, the first pixel group setting unit 611 and the first calculation unit 612 change the first angle θ1 and fix it again, and calculate the first candidate pixel value of each first pixel Px1 in the same manner. This process is repeated, and after the first candidate pixel value is calculated for each first angle θ1 and each first pixel Px1, the first pixel value setting unit 613 can calculate the pixel value of the second pixel corresponding to each first pixel Px1. In this way, as long as the pixel value of the corresponding second pixel can be calculated for each pixel Px in the set area, the order of calculation is not particularly limited.
[0072] The second image processing unit 62 of the data processing unit 51 performs image processing on the captured image data corresponding to the captured image G1, and generates two-dimensional smoothed image data corresponding to the two-dimensional smoothed image. Hereinafter, the image processing by the second image processing unit 62 is referred to as the second image processing. Here, the two-dimensional smoothed image refers to an image obtained by smoothing the captured image G1 using a two-dimensional filter corresponding to pixels arranged in at least two directions. In the two-dimensional filter, the number of pixels arranged in one direction is preferably less than twice the number of pixels arranged in another direction. The second image processing unit 62 sets the pixel value of the two-dimensional smoothed image corresponding to each pixel in the captured image G1. Hereinafter, the pixel in the two-dimensional smoothed image corresponding to the first pixel Px1 in the captured image G1 is referred to as the third pixel.
[0073] The second pixel group setting unit 621 of the second image processing unit 62 sets a plurality of second pixel groups corresponding to the first pixels Px1 of the captured image G1 .
[0074] Figure 5 2 is a schematic diagram for explaining the second image processing. The second pixel group setting unit 621 selects a first pixel Px1 from the pixels Px included in the captured image G1. The second pixel group setting unit 621 selects and sets the first pixel Px1 from the pixels included in the target area.
[0075] The second pixel group setting unit 621 sets the pixel group corresponding to the two-dimensional filter obtained by moving the two-dimensional filter F2 with respect to the reference direction including a rotation of a specified angle as the second pixel group Pg2. In other words, the second pixel group Pg2 is a pixel group configured along at least two directions after rotating the at least two directions by a specified angle. The second pixel group setting unit 621 sets a plurality of second pixel groups Pg2 for each of the plurality of the above-mentioned specified angles. The specified angle is hereinafter referred to as the second angle θ2. The reference direction is Figure 5The reference direction is the left-right direction of the captured image G1 (see arrow A20), but is not particularly limited. The reference direction may be any direction, and may be different from the reference direction in the first image processing described above.
[0076] exist Figure 5 , a two-dimensional filter F2 (second angle θ2 = 0°) and a two-dimensional filter F21 (second angle θ2 = approximately 20°) obtained by rotating the two-dimensional filter F2 by approximately 20° are shown. Figure 5 In the example, the two-dimensional filter F2 corresponds to a square pixel block of 5 pixels in length and 5 pixels in width, and the second pixel group Pg2 corresponding to the two-dimensional filter F2 includes the 5×5 pixels centered on the first pixel Px1. The shape of the two-dimensional filter F2 corresponding to the second pixel group Pg2 is not particularly limited. The second pixel group Pg2 preferably includes a plurality of pixels adjacent to each other.
[0077] exist Figure 5 , partial areas of the captured image G1 corresponding to the second pixel group Pg2 when the second angle θ2 is 0° and about 20° are schematically shown using two-dimensional filters F2 and F21, respectively. When the second pixel group Pg2 corresponding to the two-dimensional filter F21 is set, pixels overlapping with the two-dimensional filter F21 can be included in the second pixel group Pg2. However, for the pixels set in this way, the shape of the pixel block corresponding to the second pixel group Pg2 is deviated, so the accuracy is slightly reduced.
[0078] Therefore, in actual calculation, the second pixel group setting unit 621 preferably sets a plurality of pixels arranged in at least two directions with the pixel corresponding to the first pixel as the center in the image obtained by rotating the captured image G1 in the direction forming the second angle θ2 relative to the reference direction as the second pixel group Pg2. Figure 5 In the example of FIG. 1 , the second pixel group setting unit 621 rotates the captured image G1 clockwise by θ2 while fixing the two-dimensional filter F2, and sets the pixels of 5 vertical × 5 horizontal pixels within the range of the two-dimensional filter F2 in the obtained image as the second pixel group Pg2. Thus, compared with the case where the two-dimensional filter F2 is rotated to set the second pixel group Pg2, the second pixel group Pg2 corresponding to the two-dimensional filter F21 in the captured image G1 before rotation can be set more accurately.
[0079] The second pixel group setting unit 621 sets the second pixel group Pg2 for each of the plurality of second angles θ2. The plurality of second angles θ2 are set, for example, at a plurality of predetermined angles selected from a range of 1° to 45°. For example, the second pixel group setting unit 621 can set the plurality of second angles θ2 from 0° to 179° at intervals of 1°, and set the second pixel group Pg2 for each second angle θ2.
[0080] The second calculation unit 622 of the second image processing unit 62 calculates a second candidate pixel value based on the pixel values of the pixels included in the second pixel group Pg2. The second candidate pixel value is a candidate value for the pixel value set in the third pixel corresponding to the first pixel Px1 in the second enhanced image. In other words, the pixel value set in the third pixel is selected from a plurality of second candidate pixel values.
[0081] The second calculation unit 622 calculates the average value of the pixel values of the pixels included in the second pixel group Pg2 and sets it as the second candidate pixel value. For example, the second calculation unit 612 calculates the arithmetic mean of the pixel values of 5×5 pixels corresponding to the two-dimensional filter F2 and sets it as the second candidate pixel value of the second pixel group Pg2 corresponding to the two-dimensional filter F2. The second calculation unit 622 calculates the second candidate pixel value for each second pixel group Pg2 set by the second pixel group setting unit 621. The second calculation unit 622 calculates the second candidate pixel value for the second pixel group Pg2 corresponding to each second angle θ2 of the plurality of second angles θ2. In addition, the second calculation unit 612 may calculate the average value of the pixel values of the pixels included in the second pixel group Pg2 as the second candidate pixel value by an average other than an arithmetic average such as a geometric average. Alternatively, the second calculation unit 622 may set the sum of the pixel values of the pixels included in the second pixel group Pg2, and an adjusted value appropriately normalized according to a numerical range in which the sum can be set as a pixel value, as the second candidate pixel value. As long as the second candidate pixel value can be set based on the overall magnitude of the pixel values of the pixels included in the second pixel group Pg2 as described above, such as the sum or the average, the calculation method of the second candidate pixel value set by the second calculation unit 622 is not particularly limited.
[0082] The second pixel value setting unit 623 of the second image processing unit 62 sets the pixel value of the third pixel in the two-dimensional smoothed image corresponding to the first pixel Px1 based on the plurality of second candidate pixel values calculated for the first pixel Px1.
[0083] The second pixel value setting unit 623 can set the maximum value among the multiple second candidate pixel values of the first pixel Px1 as the pixel value of the corresponding third pixel. This means that in the second image processing, the pixel value of the two-dimensional filter F2 and F21 corresponding to the multiple second angles θ2, which includes the pixel with the highest pixel value as a whole, is reflected in the two-dimensional smoothed image. As a result, when the pixel value is higher and the brightness is higher, smoothing without biasing to a specific direction can be effectively performed. The second pixel value setting unit 623 can set the minimum value among the multiple second candidate pixel values of the first pixel Px1 as the pixel value of the corresponding third pixel. As a result, when the pixel value is lower and the brightness is higher, smoothing without biasing to a specific direction can be effectively performed.
[0084] Furthermore, the pixel value of the third pixel may be set to a value in the range of 0.8 to 1.2 times, preferably 0.9 to 1.1 times, the maximum value or the minimum value. In such a case, smoothing without biasing in a specific direction can be performed with a certain degree of accuracy.
[0085] When the second image processing unit 62 completes the second image processing by the second pixel group setting unit 621, the second calculation unit 622, and the second pixel value setting unit 623 for a certain first pixel Px1, the second image processing unit 62 performs the second image processing on a pixel different from the first pixel Px1 in the target area of the captured image G1 as the first pixel Px1. The second image processing unit 61 performs the second image processing on each pixel included in the target area.
[0086] As in the case of the second image processing, as long as the pixel value of the corresponding third pixel can be calculated for each pixel Px in the set area, the order of calculation and the like are not particularly limited.
[0087] Return to Figure 2 The first removing unit 64 removes at least a part of the non-linear part in the first enhanced image based on the two-dimensional smoothed image data, and generates the second enhanced image data corresponding to the second enhanced image. Here, "removal" means removing the pixel value components that do not contribute to the representation of the non-linear part in the first enhanced image. Specifically, the first removing unit 64 can generate an image as the second enhanced image, in which the absolute value of the difference obtained by subtracting the pixel value of the two-dimensional smoothed image from the pixel value of the first enhanced image is set as the pixel value. In this way, the brightness spread and noise with low direction dependence are removed from the first enhanced image, and the enhanced image in which the linear part is further enhanced, that is, the second enhanced image, is obtained.
[0088] The binarization unit 65 binarizes the pixel values of the second enhanced image using a plurality of threshold values that are appropriately different for each portion based on the pixel values of the pixels included in each portion of the second enhanced image. The binarization unit 65 divides the second enhanced image into a plurality of regions. These regions are referred to as segmented regions. The binarization unit 65 calculates an average value such as an arithmetic average of the pixel values of the pixels included in the segmented region and a standard deviation of the pixel values for each segmented region. The threshold value based on the binarization of the binarization unit 65 is referred to as a binarization threshold value Th. The binarization unit 65 sets the average value as avg and the standard deviation as σ, and calculates the binarization threshold value Th in each segmented region using the following formula (1).
[0089] Th=avg+α×σ…(1)
[0090] Here, α is a constant. In this way, by determining the binarization threshold Th based on the average value and deviation of the pixel values in each segmented area, it is possible to more accurately enhance the linear portion in the entire image.
[0091] In the image obtained by the binarization, different values are used to distinguish between the portion corresponding to the neurite Nr and the portion not corresponding to the neurite Nr. The image is referred to as a detection result image, and the data corresponding to the detection result image is referred to as detection result image data. The binarization unit 65 stores the detection result image data obtained by binarizing the second enhanced image data in the storage unit 43 or the like.
[0092] In addition, the binarization unit 65 may store the data obtained by binarizing the first enhanced image data as the detection result image data in the storage unit 43 or the like. In this case, it is not necessary to perform the second image processing and the processing of the first removal unit 64. This is because even if the second image processing and the processing of the first removal unit 64 are not performed, the linear portion can be enhanced with the desired accuracy by the first image processing. In addition, the data processing unit 51 can appropriately perform image processing such as level correction to remove the relatively dark portion in the image before the binarization unit 65 performs the binarization processing.
[0093] The data processing unit 51 detects the neurites Nr in the detection result image, and can appropriately perform analysis such as detection of intersections or which cell a linear fragment is connected to. Furthermore, the data processing unit 51 can analyze the length of the neurites Nr, and based on the data obtained from the analysis, evaluate the state of the cultured or culturing cell Ce.
[0094] Return to Figure 1 The output control unit 52 generates an output image to be output to the output unit 44 based on the detection result image data, and outputs it to the output unit 44. The output control unit 52 displays the detection result image on the display monitor of the output unit 44.
[0095] The device control unit 53 of the control unit 50 controls each unit of the culture unit 100 (arrow A2) based on input from the input unit 41. The device control unit 53 performs control related to cell culture (for example, control of the culture chamber 10 for managing temperature and humidity, control of the drive unit 12), and causes the imaging unit 20 to perform imaging.
[0096] Figure 61 is a flowchart showing the flow of the image processing method of this embodiment. In step S101, the data processing unit 51 obtains the captured image data obtained by the imaging unit 20. If step S101 is completed, step S103 is started. In step S103, the first image processing unit 61 performs the first image processing on the captured image data to generate the first enhanced image data. If step S103 is completed, step S105 is started.
[0097] In step S105, the second image processing unit 62 performs the second image processing on the captured image data to generate two-dimensional smoothed image data. If step S105 is completed, step S107 is started. In step S107, the first removal unit 64 generates second enhanced image data based on the first enhanced image data and the two-dimensional smoothed image data. If step S107 is completed, step S109 is started.
[0098] In step S109, the binarization unit 65 binarizes the second enhanced image data to generate detection result image data. If step S109 is completed, step S111 is started. In step S111, the output control unit 52 displays the detection result image on the output unit 44. If step S111 is completed, the processing is terminated.
[0099] Figure 7 Yes means Figure 6 Flowchart of the process of step S103 (first image processing) in the flowchart of . If step S101 is completed, step S201 is started. In step S201, the first pixel group setting unit 611 selects a first pixel Px1 whose pixel value is not set to a corresponding second pixel from the target area. If step S201 is completed, step S203 is started.
[0100] In step S203, the first pixel group setting unit 611 sets the first pixel group Pg1 included in the one-dimensional filter F1 extending along the reference direction, and the first calculation unit 612 calculates the first candidate pixel value for the reference direction. If step S203 is completed, step S205 is started. In step S205, the first pixel group setting unit 611 generates data for rotating the one-dimensional filter or rotating the captured image, sets the first pixel group Pg1 for each first angle θ1, and the first calculation unit 612 calculates the first candidate pixel value for each first angle θ1. If step S205 is completed, step S207 is started.
[0101] In step S207, the first pixel value setting unit 613 sets the pixel value of the second pixel corresponding to the first pixel Px1 based on the first candidate pixel value of the first pixel Px1. When step S207 is completed, step S209 is started.
[0102] Furthermore, the order of steps S203 , S205 , and S207 may be changed as appropriate.
[0103] In step S209, the data processing unit 51 determines whether the pixel values of the second pixels corresponding to the first pixels Px1 are set for all the first pixels Px1 in the target area. If the pixel values of the second pixels corresponding to all the first pixels Px1 are set, the data processing unit 51 makes an affirmative determination in step S209 and starts step S105. If there are first pixels Px1 for which the pixel values of the second pixels corresponding to the first pixels Px1 are not set, the data processing unit 51 makes a negative determination in step S209 and starts step S201.
[0104] Figure 8 Yes means Figure 6 Flowchart of the process of step S105 (second image processing) in the flowchart of . If step S103 is completed, step S301 is started. In step S301, the second pixel group setting unit 621 selects the first pixel Px1 whose pixel value of the corresponding third pixel is not set from the target area. If step S301 is completed, step S303 is started.
[0105] In step S303, the second pixel group setting unit 621 sets the second pixel group Pg2 included in the two-dimensional filter F2 in the reference direction, and the second calculation unit 622 calculates the second candidate pixel value in the reference direction. If step S303 is completed, step S305 is started. In step S305, the second pixel group setting unit 621 generates data for rotating the two-dimensional filter or rotating the captured image, sets the second pixel group Pg2 for each second angle θ2, and the second calculation unit 622 calculates the second candidate pixel value for each second angle θ2. If step S305 is completed, step S307 is started.
[0106] In step S307, the second pixel value setting unit 623 sets the pixel value of the third pixel corresponding to the first pixel Px1 based on the second candidate pixel value of the first pixel Px1. When step S307 is completed, step S309 is started.
[0107] The order of steps S303, S305 and S307 may be changed appropriately.
[0108] In step S309, the data processing unit 51 determines whether the pixel value of the third pixel corresponding to the first pixel Px1 is set for all the first pixels Px1 in the target area. When the pixel value of the corresponding third pixel is set for all the first pixels Px1, the data processing unit 51 makes an affirmative determination on step S309 and starts step S107. When there is a first pixel Px1 for which the pixel value of the corresponding third pixel is not set, the data processing unit 51 makes a negative determination on step S309 and starts step S301.
[0109] According to the above-described embodiment, the following effects can be obtained.
[0110] (1) The image processing device 1 of the present embodiment includes a first image processing unit 61, which sets the pixel value of the second pixel of the enhanced image located at a position corresponding to the first pixel Px1 of the captured image G1 in the enhanced image (second image) obtained by performing image processing on the captured image (first image) G1. The first image processing unit 61 includes: a first pixel group setting unit 611, which sets a plurality of first pixel groups Pg1, wherein the first pixel groups Pg1 are a plurality of first pixel groups Pg1 set corresponding to the first pixel Px1, and the plurality of first pixel groups Pg1 are arranged along the lines corresponding to the captured image. The first pixel group Pg1 is configured to form a plurality of directions at a plurality of different angles (first angle θ1) with respect to a predetermined direction (reference direction of the image) in the image G1, or is configured along a predetermined direction in each of a plurality of images obtained by rotating the captured image G1 at a plurality of different angles (first angle θ1) relative to the predetermined direction (reference direction); the first calculation unit 612 calculates a plurality of first candidate pixel values based on the magnitude of the pixel values of the pixels included in each of the plurality of first pixel groups Pg1; and the first pixel value setting unit 613 sets the pixel value of the second pixel of the enhanced image based on the plurality of first candidate pixel values. Thus, the linear portion of the cell in the image can be effectively enhanced.
[0111] (2) In the image processing device 1 of this embodiment, the first calculation unit 612 can calculate a plurality of first candidate pixel values based on the sum or average of the pixel values of the pixels included in each of the plurality of first pixel groups Pg1. This makes it possible to detect the direction in which pixels with high pixel values are arranged, and more effectively enhance the linear portion of the image.
[0112] (3) In the image processing device 1 of this embodiment, the first calculation unit 612 calculates the first candidate pixel value for the first pixel group Pg1 using the smoothing filters F1, F11, and F12. This makes it possible to detect the linear portion based on the length of the smoothing filter and more effectively enhance the linear portion in the image.
[0113] (4) In the image processing device 1 of this embodiment, the one-dimensional filters F1, F11, and F12 are one-dimensional smoothing filters. Thus, the linear portion can be detected by the shape of the one-dimensional filter (for example, the shape including length and thickness), and the linear portion in the image can be enhanced more effectively.
[0114] (5) In the image processing device 1 of this embodiment, the first pixel group setting unit 611 is provided to set the first pixel Px1 according to the pixels constituting the captured image G1, and the first pixel value setting unit 613 can set the pixel value of the second pixel to be 0.8 to 1.2 times the maximum value or 0.8 to 1.2 times the minimum value among the plurality of first candidate pixel values. Thus, the direction in which pixels with high or low pixel values are arranged can be detected, and the linear portion in the image can be enhanced more effectively.
[0115] (6) In the image processing device 1 of the present embodiment, the second pixel group setting unit 621 sets a plurality of second pixel groups Pg2 arranged along at least two directions obtained by rotating at least two directions of the pixel arrangement in the two-dimensional filter at different angles (second angles θ2) relative to a predetermined direction (reference direction) in the captured image G1, or arranged along at least two directions in each of the captured images G1 rotated at different angles (second angles θ2), the second image processing unit 62 includes a second calculation unit 622 that calculates a plurality of second candidate pixel values based on the sum or average of the pixel values of the pixels included in each of the plurality of second pixel groups Pg1, and the second pixel value setting unit 623 sets the pixel value of the third pixel based on the plurality of second candidate pixel values. Thus, image data for removing the pixel value corresponding to the portion with low directional dependence in the subject can be generated.
[0116] (7) The image processing device 1 of this embodiment includes a first removing unit 64, which removes at least a portion of the non-linear portion in the enhanced image based on the two-dimensional smoothed image. This can remove pixel values corresponding to portions of the subject with low directional dependence, and further effectively enhance the linear portion in the image.
[0117] (9) The image processing device 1 of this embodiment includes a binarization unit 65, which binarizes the pixel values of the enhanced image using a plurality of different binarization threshold values Th based on the pixel values of the pixels included in each portion of the enhanced image. Thus, it is possible to easily understand whether each pixel in the image corresponds to a linear portion through binarization, and to facilitate analysis.
[0118] (10) In the image processing device 1 of this embodiment, the captured image G1 is an image of the neurite Nr. Thus, the portion of the image corresponding to the neurite Nr can be effectively enhanced. In addition, using the image in which the neurite Nr is enhanced, the length of the neurite Nr can be analyzed more accurately.
[0119] (11) The imaging device according to the present embodiment includes the above-mentioned image processing device 1 and the imaging unit 20 for imaging cells and the like including the linear portion. Thus, the linear portion in the imaged image can be effectively enhanced.
[0120] (12) The above-mentioned image processing apparatus 1 as a culture apparatus according to the present embodiment includes a culture section 100 for culturing cells. This can effectively enhance the linear portion in an image obtained by capturing cells Ce cultured in an undifferentiated state or a differentiated state.
[0121] (13) The image processing method according to the present embodiment includes: in an enhanced image (second image) obtained by performing image processing on a captured image (first image) G1, setting a pixel value of a second pixel of the captured image G1 located at a position corresponding to a first pixel Px1 of the captured image G1, wherein the setting of the pixel value of the second pixel comprises: setting a plurality of first pixel groups Pg1, the plurality of first pixel groups Pg1 being a plurality of first pixel groups Pg1 set corresponding to the first pixel Px1, the plurality of first pixel groups Pg1 being arranged along a plurality of directions forming a plurality of different angles (first angles θ1) with respect to a predetermined direction (reference direction) in the captured image G1, or being arranged along a predetermined direction in each of a plurality of images obtained by rotating the captured image G1 at a plurality of different angles (first angles θ1); calculating a plurality of first candidate pixel values based on the magnitude of the pixel values of the pixels included in each of the plurality of first pixel groups Pg1; and setting the pixel value of the second pixel based on the plurality of first candidate pixel values. Thus, a linear portion in an image can be effectively enhanced.
[0122] The following modifications are also within the scope of the present invention and can be combined with the above-mentioned embodiment. In the following modifications, parts and the like representing the same structure and function as those in the above-mentioned embodiment are referenced using the same reference numerals, and description thereof is appropriately omitted.
[0123] (Variant 1)
[0124] In the second image processing of the above-mentioned embodiment, the second image processing unit 62 may not perform the process of moving and rotating the two-dimensional filter or the captured image G1, but may set the second candidate pixel value calculated based on the pixels included in one two-dimensional filter as the pixel value of the third pixel. This is because even in such a case, if the two-dimensional filter originally has a shape with a small deviation in a specific direction, smoothing without deflection in a specific direction can be performed with a certain degree of accuracy.
[0125] The image processing device 1 of this modified example includes a second image processing unit 62, which sets the pixel value of the third pixel located at a position corresponding to the first pixel Px1 in the two-dimensional smoothed image obtained by performing image processing on the captured image G1. The second image processing unit 62 includes: a second pixel group setting unit 621, which sets the second pixel group Pg2 set corresponding to the first pixel Px1 and includes pixels arranged in at least two directions; and a second pixel value setting unit 623, which sets the pixel value of the third pixel based on the sum or average of the pixel values of the pixels included in the second pixel group Pg2. Thus, the captured image G1 can be smoothed based on the shape of the pixel block constituting the second pixel group Pg1, and the linear part in the image can be effectively enhanced.
[0126] (Variant 2)
[0127] In the above-mentioned embodiment, the image processing device 1 may be configured to generate information on whether the first pixel Px1 in the captured image G1 corresponds to a linear portion using a one-dimensional filter. Hereinafter, this information is referred to as corresponding information on pixels of the linear portion (eg, neurite Nr).
[0128] Fig. 9 2 is a schematic diagram showing the structure of a data processing unit 51 a in the image processing device according to the present modification. The data processing unit 51 a is different from the above-mentioned data processing unit 51 in that it includes a pixel information processing unit 63 .
[0129] The pixel information processing section 63 includes a third pixel group setting section 631, an information generating section 632, and a processing control section 633. The pixel information processing section 63 acquires information from the captured image data using a one-dimensional filter, and processes the information.
[0130] The third pixel group setting unit 631 sets a plurality of third pixel groups Pg3 corresponding to the first pixel Px1 in the captured image G1. The third pixel group setting unit 631 sets a plurality of third pixel groups Pg3 using one-dimensional filters F1, F11, and F12 corresponding to a plurality of angles, similar to the setting of the plurality of first pixel groups Pg1 by the first pixel group setting unit 611 described above.
[0131] Fig.10 2 is a schematic diagram for explaining the third pixel group Pg3. The third pixel group setting unit 631 selects a first pixel Px1 from the pixels Px included in the captured image G1. The third pixel group setting unit 631 selects and sets the first pixel Px1 from the pixels included in the target area.
[0132] The third pixel group setting unit 631 sets a pixel group including the first pixel Px1 and arranged along a plurality of directions forming a predetermined angle with respect to the reference direction as a third pixel group Pg3. The predetermined angle is hereinafter referred to as a third angle θ3. Fig.10 In the figure, it is the left-right direction of the captured image G1 (see arrow A30), but it is not particularly limited and may be any direction. Fig.10 , a plurality of pixels Px31, Px32, Px1, Px33, and Px34 included in the third pixel group Pg3 are shown when the third angle θ3 is 0°. The third pixel group Pg3 preferably includes a plurality of pixels arranged in one direction and adjacent to each other.
[0133] In actual calculation, the third pixel group setting unit 631 preferably sets a plurality of pixels arranged along a predetermined direction with the pixel corresponding to the first pixel as the center in the image obtained by performing a movement including a rotation in the direction forming the third angle θ3 with respect to the reference direction as the third pixel group Pg3. The predetermined direction is not particularly limited and can be set to any direction such as the reference direction. Thus, compared with the case where the third pixel group Pg3 is set by performing a movement including a rotation on the one-dimensional filter F1, the third pixel group Pg3 can be set more precisely.
[0134] The third pixel group setting unit 631 sets the third pixel group Pg3 for each of the plurality of third angles θ3. The plurality of third angles θ3 are set, for example, at a plurality of predetermined angles selected from a range of 1° to 45°. For example, the third pixel group setting unit 631 can set the plurality of third angles θ3 from 0° to 179° at intervals of 1°, and set the third pixel group Pg3 for each third angle θ3. The third pixel group setting unit 631 sets the third pixel group Pg3 for each first pixel Px1 in the target area.
[0135] The information generation unit 632 generates the above-mentioned corresponding information. The information generation unit 632 calculates the value representing the overall magnitude of the pixel values of the pixels included in each of the plurality of third pixel groups Pg3, that is, the sum or average of the pixel values. The average can be appropriately arithmetic mean or the like. In the following, the calculated sum or average of the pixel values is referred to as a one-dimensional filter value.
[0136] Fig.11 3 is a schematic diagram showing a one-dimensional filter and a linear portion LP corresponding to a plurality of third angles θ3. Fig.11 In the figure, there is shown a one-dimensional filter F100 when the third angle θ3 is 0°, a one-dimensional filter F110 when the third angle θ3 is 30°, a one-dimensional filter F120 when the third angle θ3 is 60°, a one-dimensional filter F130 when the third angle θ3 is 90°, a one-dimensional filter F140 when the third angle θ3 is 120°, and a one-dimensional filter F150 when the third angle θ3 is 150°.
[0137] exist Fig.11 In the image, the first pixel Px1 is located on the linear portion LP, and is therefore a pixel corresponding to the linear portion. If the light from the subject is stronger, a higher pixel value is set in the captured image data, and the pixel value of the pixel corresponding to the linear portion LP is higher than the pixel value of the nearby pixel that does not correspond to the linear portion. If the linear portion LP and the one-dimensional filter overlap, the sum or average of the pixel values of the pixels corresponding to the one-dimensional filter, that is, the one-dimensional filter value, is the highest.
[0138] exist Fig.11 In the case where the third angle θ3 is 0°, 120° and 150°, the overlap between the one-dimensional filters F100, F140, F150 and the linear portion LP is small, and the one-dimensional filter value is smaller than that in other cases of the third angle θ3. When the third angle θ3 is 30° or 90°, the one-dimensional filter F110 or the one-dimensional filter F130 and the linear portion LP overlap by about 3 pixels out of the 5 pixels constituting the one-dimensional filter. Therefore, the one-dimensional filter value is larger than that in the case where the third angle θ3 is 0°, 120° or 150°. When the third angle θ3 is 60°, the one-dimensional filter F120 and the linear portion LP roughly overlap, and in Fig.11 In the case shown, the one-dimensional filter has the highest value.
[0139] The information generation unit 632 generates data that makes the third angle θ3 correspond to the one-dimensional filter value. This data is hereinafter referred to as filter value data. The filter value data may also be appropriately smoothed or generated based on curve approximation. This can reduce the influence of noise and the like and improve accuracy.
[0140] Fig.12 θ is a graph showing a one-dimensional filter value for the third angle θ3 in the first pixel Px1 that can be generated based on the filter value data. Fig.12 assumed Fig.11 The one-dimensional filter value shown in the figure has angle dependence. Fig.12In the graph of , the one-dimensional filter value takes values from Vmin to Vmax, and the one-dimensional filter value is the highest when the third angle θ3 is about 60°. The information generation unit 632 can calculate the third angle θ3 at which the one-dimensional filter value is the highest and the peak width of the peak P1 associated with the third angle θ3. The information generation unit 632 can detect the peak value based on whether the difference between Vmax and Vmin, the S / N ratio, etc. are greater than a predetermined value, etc.
[0141] Fig.13 θ is a graph showing the one-dimensional filter value for the third angle θ3 when there is no significant peak. Fig.13 In the example, the change in the one-dimensional filter value is smaller than the S / N ratio, so no peak is detected.
[0142] In the graph representing the one-dimensional filter value for the third angle θ3, the characteristics of the subject in the first pixel Px1 can be analyzed by the number of peaks. When the number of peaks is 0, the first pixel Px1 becomes a portion that does not correspond to a line or an intersection. When the number of peaks is one, the first pixel Px1 corresponds to a linear portion. When the number of peaks is two, the first pixel Px1 corresponds to an intersection. The information generating unit 632 generates corresponding information on whether the first pixel Px1 corresponds to a linear portion by analyzing the one-dimensional filter value. The information generating unit 632 can use a binary value to represent whether each pixel in the captured image corresponds to a linear portion and generate corresponding information associated with each pixel, and store it in the storage unit 43 or the like. In addition, the information generating unit 632 can also generate information on whether each pixel in the captured image corresponds to an intersection.
[0143] The information generating unit 632 generates filter value data and corresponding information for each first pixel Px1 in the target area.
[0144] Furthermore, in calculating the one-dimensional filter value, initially, the third pixel group setting unit 631 increases the intervals of the third angles θ3 to set the third pixel group Pg3, and if a peak is detected, the intervals of the third angles θ3 around the peak and its vicinity can be reduced to set the third pixel group Pg3. Thus, the information generating unit 632 can accurately calculate the one-dimensional filter value around the peak and its vicinity, and can efficiently analyze the angular dependency of the one-dimensional filter value.
[0145] Processing control unit 633 ( Fig. 9) Based on the corresponding information, the first pixel group setting unit 611 is controlled to set the processing of the first pixel group Pg1. The processing control unit 633 changes the number of third angles θ3 set in the first image processing through the position of the first pixel Px1 based on the corresponding information. The processing control unit 633 can reduce the number of third angles θ3 set in pixels that do not correspond to the linear portion compared to pixels that correspond to the linear portion. Alternatively, the processing control unit 633 can be set so that the first image processing is not performed in pixels that do not correspond to the linear portion. In other words, the processing control unit 633 can set the first image processing so that the first pixel group setting unit 611 selects the first pixel Px1 from the pixels corresponding to the linear portion. In this way, by setting the first image processing to be performed more precisely on the pixels corresponding to the linear portion, the linear portion in the image can be efficiently enhanced.
[0146] In addition, when the information generating unit 632 generates information on whether each pixel in the captured image G1 corresponds to an intersection, the processing control unit 633 can appropriately set the conditions for the first image processing based on the information, such as not performing the first image processing on the pixels corresponding to the intersection.
[0147] Fig.14 is a flowchart showing the flow of the image processing method of this variation. Step S401 and Figure 6 The information generation unit 632 generates information (correspondence information) on whether the pixel included in the captured image G1 corresponds to the linear portion. If step S403 is completed, step S405 is started.
[0148] In step S405, the processing control unit 633 sets the conditions for the first image processing based on the corresponding information generated in step S403. If step S405 is completed, step S407 is started. In step S407, the first image processing unit 61 performs the first image processing based on the conditions set in step S405 to generate the first enhanced image data. If step S407 is completed, step S409 is started. Steps S409 to S415 are similar to the steps in step S416. Figure 6 These correspond to steps S105 to S111 in the flowchart of , and therefore their description is omitted.
[0149] (1) In the image processing device of this modified example, there are provided: a third pixel group setting unit 631 for setting a plurality of third pixel groups Pg3, wherein the plurality of third pixel groups Pg3 are set corresponding to the first pixel Px1, and the plurality of third pixel groups Pg3 are arranged along a plurality of directions forming a plurality of angles (third angles θ3) with respect to a predetermined direction (reference direction) of the captured image (first image) G1, or are arranged along a predetermined direction in each of a plurality of images obtained by rotating the captured image G1 at a plurality of angles (third angles θ3); and an information generating unit 632 for generating information (correspondence information) on whether the first pixel Px1 corresponds to a linear portion in the captured image G1 based on the sum or average of the pixel values of the pixels included in each of the plurality of third pixel groups Pg3. Thus, image processing can be efficiently performed using the generated information.
[0150] (2) The image processing device of this modification includes a processing control unit that controls the first pixel group setting unit 611 to set the first pixel group Pg1 based on the corresponding information generated by the information generating unit 632. This allows efficient first image processing using the generated information.
[0151] (3) In the image processing device of this modified example, the number of the plurality of angles (first angles θ1) when the first pixel group setting unit 611 sets the plurality of first pixel groups Pg1 varies depending on the position of the first pixel Px1, and the processing control unit 633 can set the number of the plurality of angles at each position of the captured image G1 based on the corresponding information. Thus, the precision of the first image processing can be controlled by the position in the captured image using the generated information.
[0152] (4) In the image processing device of this modification, the processing control unit 633 can determine whether to perform the process of setting the first pixel group Pg1 by the first pixel group setting unit 611 at each position of the captured image G1 based on the corresponding information. This makes it possible to perform the first image processing more efficiently using the generated information.
[0153] Furthermore, the data processing unit 51a may include a second removing unit that removes at least a portion of the non-linear portion in the first enhanced image or the second enhanced image based on the corresponding information. The second removing unit sets the pixel value to a value corresponding to the absence of the subject, such as 0, for pixels that are set not to correspond to the linear portion in the corresponding information. In this way, the linear portion in the image can be further enhanced.
[0154] (Variant 3)
[0155] In the above-described embodiment, a configuration may be adopted in which the user sets the target area via the input unit 41 .
[0156] Fig.15 Schematic diagram showing the target region R1 . The user can input the range of the target region R1 by dragging the mouse as the input unit 41 , for example, while viewing the display screen of the output unit 44 displaying the captured image G1 .
[0157] The image processing device of this modification includes an input unit 41 for inputting the position or range of the first pixel Px1 to be processed by the first image processing unit 61. Thus, the user sets the target of image processing, and linear portions in the image can be efficiently enhanced.
[0158] (Variant 4)
[0159] In the above-mentioned embodiment, the subject of the captured image G1 is set as the cell Ce, and the enhanced linear portion is set as the neurite Nr. However, the enhanced object is not limited to this example as long as the linear portion is included, and it is also preferably set as a blood vessel. Furthermore, it is more preferable to set the subject of the captured image G1 as the fundus, and the enhanced linear portion as the blood vessel of the fundus, and more preferably to set as the blood vessel of the retina or the choroid.
[0160] In the image processing device of this modification, the captured image G1 is an image of a blood vessel, so that the linear portion in the image of the blood vessel can be effectively enhanced.
[0161] (Variant 5)
[0162] In the above-mentioned embodiment, the output unit 44 may output a display screen for allowing the user to select and display at least one of the captured image G1 and the enhanced image under the control of the output control unit 52. This display screen is referred to as a first display screen. The enhanced image may be a first enhanced image or a second enhanced image.
[0163] Fig.16: is a schematic diagram showing an example of the first display screen of this variant. The first display screen 70 includes an image selection unit 71 and a first image display unit 72 that displays at least one of a captured image G1 and an enhanced image. The image selection unit 71 is composed of a combo box, and the combo box is configured to switch the display or non-display of a list 71b each time an icon 71a is selected. The list 71b includes a first input element 71c, a second input element 71d, and a third input element 71e. If the user clicks the first input element 71c using a mouse or the like, the captured image G1 as an input image is displayed in the first image display unit 72, and the enhanced image G2 is not displayed. If the user clicks the second input element 71d, the enhanced image G2 is displayed in the first image display unit 72, and the captured image G1 is not displayed. If the user clicks the third input element 71e, the captured image G1 and the enhanced image G2 are displayed in the first image display unit 72.
[0164] Furthermore, the image selection unit 71 is not limited to a combo box, and may be configured to include any image element as long as the displayed image can be selected.
[0165] As in the present modification, the user appropriately selects and displays the captured image G1 or the enhanced image G2 , so that the effect of the image processing method of the above-described embodiment can be easily understood by the user.
[0166] (Variant 6)
[0167] In the above-mentioned embodiment, the user may set the number of the first angles θ1. The output unit 44 may output a display screen for the user to set the number of the first angles θ1 under the control of the output control unit 52. This display screen is referred to as a second display screen.
[0168] Fig.17 2 is a schematic diagram showing an example of the second display screen of the present modification. The second display screen 80 includes a first angle input unit 81 , a second image display unit 82 for displaying the captured image G1 , and a first execution unit 83 . The first angle input unit 81 includes a text box 810 .
[0169] The first angle input unit 81 is an element for inputting the number of first angles θ1. The user can input a numerical value indicating the number of first angles θ1 into the text box 810 using a keyboard or the like. The user can input while appropriately viewing the captured image G1 displayed on the second image display unit 82.
[0170] Furthermore, as long as the number of first angles θ1 can be input, the image element used for input is not particularly limited to the text box, and the user may select from displayed numerical value candidates.
[0171] The first execution unit 83 is a button for setting the number of first angles θ1 input by the user and causing the image processing device 1 to start the first image processing. When the user clicks the first execution unit 83, the first pixel group setting unit 611 sets a plurality of first angles θ1 for each angle obtained by, for example, dividing 180° by the value input to the text box 810, and performs the first image processing.
[0172] In the present modification, the user sets the parameters (setting information) related to the first angle θ1 while appropriately viewing the captured image G1 , thereby being able to set the parameters more appropriately.
[0173] In addition, as long as the user can set the conditions regarding the first angle θ1 , the second angle θ2 , or the third angle θ3 , the number of input angles is not limited to the case of the present modification.
[0174] (Variant 7)
[0175] In the above-described embodiment, the configuration may be such that, with respect to the first image processing, the user can appropriately observe the display screen of the output unit 44 and input or change parameters related to the first image processing.
[0176] Fig.18 is a flowchart showing the flow of the image processing method of this variation. Step S501 and Figure 6 The step S101 of the flowchart is the same as that of the flowchart of FIG. 1 , and therefore the description thereof is omitted. When the step S501 is completed, the step S503 is started.
[0177] In step S503, the first image processing unit 61 sets the parameters in the first image processing based on the user's input. This parameter is called the first parameter. The first parameter can be set to a numerical value representing the length and thickness of the one-dimensional filters F1, F11, and F12 in the first image processing. The user observes the captured image G1 and then observes the detection result image when the above-mentioned first image processing to binarization processing has been performed, and can input the first parameter. When step S503 is completed, step S505 is started.
[0178] In addition, it can also be configured that the second image processing unit 62 sets the parameters in the second image processing, that is, the second parameters, based on the user's input. The second parameter can include, for example, a numerical value representing the width and shape of the two-dimensional filter. Alternatively, it can also be configured that the binarization unit 65 can change the parameters (for example, numerical values) such as α of the formula (1) in the binarization in the above-mentioned embodiment based on the user's processing. In this way, by appropriately setting the binarization conditions, the linear part of the neurite Nr etc. can be detected with high precision.
[0179] In step S505, the first image processing unit 61 performs the first image processing using the value of the first parameter set in step S503 to generate the first enhanced image data. If step S505 is completed, step S507 is started. Steps S507 to S513 are the same as Figure 6 The steps S105 to S111 of the flowchart correspond to each other, and therefore the description thereof is omitted. When step S513 is completed, step S515 is started.
[0180] In step S515, the data processing unit 51 determines whether the user has input an instruction to perform the processing related to the above-mentioned image processing method, such as the first image processing, again via the input unit 41. If such an instruction has been input, the data processing unit 51 makes an affirmative determination in step S515 and starts step S503. If such an instruction has not been input, the data processing unit 51 makes a negative determination in step S515 and ends the processing.
[0181] In the following, regarding the image processing method of this variant example, an example of a display screen displayed on the output unit 44 is shown, but the form of the image in each display screen and the shape of each display element such as a button or text box are not limited to the contents shown in the following figures.
[0182] Fig.19 This is an example of a screen displayed on the output unit 44 by the output control unit 52 when the first parameter is selected, and this screen is set as the basic screen 701. The basic screen 701 includes an image display unit 800, a first input element display unit 710, and a video operation unit 810. The video operation unit 810 includes: a rewind button 811 for rewinding when the image display unit 800 displays a video, a play button 812 for playing, a pause button 813 for pausing, and a fast-forward button 814 for fast-forwarding. Fig.19 The image display unit 800 of the basic screen 701 shows the captured image G1 , but the image displayed on the image display unit 800 can be changed as appropriate.
[0183] The first input element display unit 710 includes an execution button 711 and a parameter selection button 712 as input elements for the user to point a cursor at and click with a mouse or a touch pad to cause the image processing device 1 to perform a predetermined operation.
[0184] The execution button 711 is an input element for the data processing unit 51 to execute the processing from the first image processing to the binarization, and to change the display screen to the screen for displaying the detection result of the neurite Nr, that is, the result display screen. Fig.21The result display screen 702 is displayed on the output unit 44. At this time, when the user does not change the parameters, it is preferable to perform the first image processing using the first parameters set in advance.
[0185] The parameter selection button 712 is an input element for displaying a display element, and the display element is used to input the first parameter. If the user clicks the parameter selection button 712, the first text box 713 and the second text box 714 ( Fig. 20 ) is displayed on the first input element display unit 710.
[0186] Fig. 20 713 and the second text box 714. Fig. 20 In the basic screen 701 of FIG. 701 , the first input element display unit 710 includes a first text box 713 and a second text box 714 in which a user can input a numerical value using a keyboard or the like. The first text box 713 is configured to allow input of the length of the one-dimensional filter F1, F11, and F12 as the number of pixels. The second text box 714 is configured to allow input of the thickness of the one-dimensional filter F1, F11, and F12 as the number of pixels.
[0187] When the first text box 713 and the second text box 714 are displayed, the parameter selection button 712 is displayed in a different manner than when the first text box 713 and the second text box 714 are not displayed, by changing the hue, chroma, or brightness. Fig. 20 In FIG. 7 , this point is indicated by shading the parameter selection button 712 .
[0188] Alternatively, the first text box 713 and the second text box 714 may be displayed from the beginning when the basic screen image 701 is displayed.
[0189] Fig.21 Result display screen 702 is a schematic diagram showing an example of a result display screen. Result display screen 702 is displayed on the display screen of output unit 44 through output control unit 52. Result display screen 702 includes image display unit 800, second input element display unit 720, and video operation unit 810. Fig.21 The image display unit 800 displays a detection result image G3.
[0190] The second input element display unit 720 includes an image superimposition button 721 , a re-execution button 722 , and an end button 723 as input elements for the user to point a cursor at and click with a mouse or a touch pad to cause the image processing device 1 to perform a predetermined action.
[0191] Furthermore, the parameter selection button 712 is displayed on the second input element display unit 720. Fig.21In the example of FIG. 702 , the user selects the parameter selection button 712, and the first text box 713 and the second text box 714 are displayed. In the result display screen 702, the values set in the first image processing performed previously are preferably displayed as the initial setting values of the first text box 713 and the second text box 714. Thus, the user can easily understand the relationship between the obtained detection result of the neurite Nr and the first parameter.
[0192] The image overlap button 721 is an input element for displaying an overlapped image of the captured image G1 and the detection result image G3 on the image display unit 800. If the image overlap button 721 is clicked, the output control unit 52 displays the overlapped image on the image display unit 800 of the result display screen 702. Thus, the user can compare the captured image G1 before image processing and the detection result image G3 after image processing, and can more accurately understand the effect of the image processing method in the above-mentioned embodiment. If the image overlap button 721 is clicked in a state where the overlapped image is displayed on the image display unit 800, it can be configured to display either the captured image G1 or the detection result image G3.
[0193] The re-execution button 722 is an input element for the data processing unit 51 to re-execute the processing from the first image processing to the binarization, and update the result display screen 702 so that the display screen displays the detection result of the neurite Nr obtained by the re-processing. If the re-execution button 722 is clicked, the output control unit 52 displays the updated result display screen 702 on the output unit 44. At this time, the first image processing unit 61 performs the first image processing using the first parameters input to the first text box 713 and the second text box 714.
[0194] The end button 723 is a button for ending the processing of the image processing method of this modification by clicking the end button 723. When the end button 723 is clicked, the data processing unit 51 ends the processing as appropriate.
[0195] In the image processing method of this variation, the user sets parameters in the above-mentioned image processing method while appropriately checking the captured image G1, the detection result image G3, etc., so that linear parts can be enhanced with high accuracy according to the image form.
[0196] (Variant 8)
[0197] In the above-mentioned modification 7, when the numerical values input to the first text box 713 and the second text box 714 are inappropriate in terms of the above-mentioned image processing, it can be configured so that a notification is displayed on the display screen.
[0198] Fig. 22710 of the basic screen 701 includes a notification display unit 715. When the values input to the first text box 713 and the second text box 714 do not satisfy the specified conditions, the output control unit 52 displays a notification on the notification display unit 715. The specified conditions can be, for example, set to a case where the length L of the one-dimensional filter F1, F11, F12 is less than or equal to twice the thickness W of the one-dimensional filter F1, F11, F12. In this case, the output control unit 52 can Fig. 22 In this way, a notification such as “W is too large relative to L” is displayed on the notification display unit 715 .
[0199] The form of the notification is not particularly limited, and the output control unit 52 may display the content of the notification using an image or the like instead of text, or may issue an audible warning or the like.
[0200] (Variant 9)
[0201] It is also possible to record a program for realizing the information processing function of the information processing unit 40 of the above-mentioned embodiment in a computer-readable recording medium, and to make the computer system read the program recorded in the recording medium and related to the processing of the data processing unit 51, 51a such as the first image processing, the second image processing and binarization, and execute it. In addition, the so-called "computer system" here includes the OS (Operating System) and the hardware of the peripheral devices. In addition, the so-called "computer-readable recording medium" refers to a portable recording medium such as a floppy disk, a magneto-optical disk, an optical disk, a memory card, and a storage device such as a hard disk built into the computer system. In addition, the so-called "computer-readable recording medium" may also include a structure that dynamically retains the program for a short period of time, such as a communication line when the program is sent via a network such as the Internet or a communication line such as a telephone line, and a structure that retains the program for a certain period of time, such as a volatile memory inside a computer system that is a server or a client in this case. In addition, the above-mentioned program may also be a structure for realizing a part of the above-mentioned function, and further, it may be a structure that realizes the above-mentioned function by combining it with a program already recorded in the computer system.
[0202] In addition, when applied to a personal computer (hereinafter referred to as a PC) or the like, the program related to the above-mentioned control can be provided via a recording medium such as a CD-ROM or a data signal via the Internet or the like. Fig.23950 is a diagram showing this situation. PC950 receives the program via CD-ROM953. In addition, PC950 has a connection function with communication line 951. Computer 952 is a server computer that provides the above-mentioned program, and stores the program in a recording medium such as a hard disk. Communication line 951 is a communication line such as the Internet, computer communication, or a dedicated communication line. Computer 952 uses a hard disk to read the program and sends the program to PC950 via communication line 951. That is, the program is transmitted as a data signal through a carrier wave and sent via communication line 951. In this way, the program can be provided as a computer-readable computer program product in various forms such as a recording medium and a carrier wave.
[0203] The program for realizing the above-mentioned information processing function includes the following program, which is used to make the processing device (control unit 50) perform the first image processing (with Figure 6 The first image processing sets the pixel value of the second pixel of the enhanced image located at a position corresponding to the first pixel Px1 of the captured image G1 in the enhanced image (second image) obtained by performing image processing on the captured image (first image) G1. The first image processing comprises: a first pixel group setting processing, setting a plurality of first pixel groups Pg1, wherein the plurality of first pixel groups Pg1 are a plurality of first pixel groups Pg1 set corresponding to the first pixel Px1, and the plurality of first pixel groups Pg1 form different directions along a specified direction (reference direction) in the captured image G1. The first pixel group Pg1 is configured in multiple directions at different angles (first angle θ1), or is configured along a specified direction in each of multiple images obtained by moving the captured image G1 at different angles (first angle θ1); the first calculation process calculates multiple first candidate pixel values based on the size of the pixel values of the pixels included in each of the multiple first pixel groups Pg1 (the first pixel group setting process and the first calculation process correspond to steps S203 and S205); and the first pixel value setting process sets the pixel value of the second pixel based on the multiple first candidate pixel values (corresponding to step S207). As a result, the linear part in the image can be effectively enhanced.
[0204] In addition, when applied to an information processing system having a website server (hereinafter, also referred to as a server) and a personal computer (hereinafter, referred to as a user terminal or PC), the information processing unit 40 of the image processing device 1 is a website server having a communication unit 42, a storage unit 43, an output unit 44, and a data processing unit 51. The information processing system outputs the analysis results including the above-mentioned images (for example, the first image, the second image, the third image, etc.) to the user terminal via a network by cloud computing. For example, the information processing system includes a server, and the server includes: a communication unit (for example, the communication unit 42) for acquiring the above-mentioned first image etc. photographed in the culture unit 100; a storage unit (for example, the storage unit 43) for storing the above-mentioned images (for example, the first image, the second image, the third image, etc.) and information required for the above-mentioned image analysis; a data processing unit having the same function as the above-mentioned data processing unit 51; and an output unit (for example, the output unit 44) for outputting the analysis results of the images based on the data processing unit to the user terminal.
[0205] The present invention is not limited to the contents of the above-mentioned embodiment, and other aspects that can be considered within the scope of the technical concept of the present invention are also included in the scope of the present invention.
[0206] The disclosures of the following priority basic applications are incorporated herein by reference.
[0207] Japan Special Application No. 2019-168631 (applied on September 17, 2019).
[0208] Description of Reference Numerals
[0209] 1…image processing device; 10…culture chamber; 20…shooting unit; 40…information processing unit; 43…storage unit; 44…output unit; 50…control unit; 52…output control unit; 51, 51a…data processing unit; 100…culture unit; 61…first image processing unit; 62…second image processing unit; 63…pixel information processing unit; 64…first removal unit; 65…second removal unit; 70…first display screen; 80…second display screen; 611…first pixel group setting unit; 612…first calculation unit; 613…first pixel value setting unit; 621…second pixel group setting unit; 622…second calculation unit; 623…second 2. Pixel value setting unit; 701…basic screen; 702…result display screen; 800…image display unit; Ce…cell; F1, F11, F12, F100, F110, F120, F130, F140, F150…one-dimensional filter; F2, F21…two-dimensional filter; G1…captured image; G2…enhanced image; G3…detection result image; LP…linear part; P1…peak; Pg1…first pixel group; Pg3…third pixel group; Px…pixel; Px1…first pixel; R1…object area; So…cell body; θ1…first angle; θ2…second angle; θ3…third angle.
Claims
1. An image processing device, in, have: a first image processing unit that sets, in a second image obtained by performing image processing on a first image obtained by photographing, a pixel value of a second pixel of the second image located at a position corresponding to a first pixel of the first image; a second image processing unit that sets a pixel value of a third pixel located at a position corresponding to the first pixel in a third image obtained by performing image processing on the first image; as well as a third image processing unit that generates an enhanced image based on the pixel value of the second pixel processed by the first image processing unit and the pixel value of the third pixel processed by the second image processing unit, The first image processing unit comprises: a first pixel group setting unit configured to set a first pixel group, wherein the first pixel group is a pixel group set corresponding to the first pixels and includes the first pixels arranged along one direction; A first calculation unit calculates a first candidate pixel value for each of the plurality of first pixel groups based on the magnitude of pixel values of pixels included in each of the plurality of first pixel groups; as well as a first pixel value setting unit that sets a pixel value of a second pixel of the second image based on a plurality of first candidate pixel values; The second image processing unit comprises: a second pixel group setting unit configured to set a second pixel group, wherein the second pixel group is a pixel group set corresponding to the first pixel and includes the first pixel and is a pixel group configured along at least two directions after rotating the at least two directions by a predetermined angle; a second calculation unit, calculating a second candidate pixel value for each of the plurality of second pixel groups based on the magnitude of pixel values of pixels included in each of the plurality of second pixel groups; as well as a second pixel value setting unit that sets a pixel value of a third pixel of the third image based on the plurality of second candidate pixel values; The third image processing unit includes a first removing unit configured to obtain a difference obtained by subtracting a pixel value of the third pixel of the third image from a pixel value of the second pixel of the second image as a pixel value of the enhanced image.
2. The image processing device according to claim 1, in, The first pixel group setting unit sets a plurality of first pixel groups, wherein the first pixel groups are arranged along a plurality of directions in each of a plurality of images obtained by moving the first image at a plurality of angles different from each other relative to a predetermined direction. The second pixel group setting unit sets a plurality of second pixel groups, wherein the plurality of second pixel groups are arranged along the at least two directions in each of a plurality of images obtained by moving the first image at a plurality of angles different from each other relative to a predetermined direction. The first calculation unit calculates a plurality of first candidate pixel values based on a sum or an average of pixel values of pixels included in each of the plurality of first pixel groups. The second calculation unit calculates a plurality of second candidate pixel values based on a sum or an average of pixel values of pixels included in each of the plurality of second pixel groups.
3. The image processing device according to claim 1, in, The first calculation unit calculates the first candidate pixel value using a one-dimensional filter for the first pixel group. The second calculation unit calculates the second candidate pixel value using a two-dimensional filter for the first pixel group.
4. The image processing device according to claim 2, in, The first image processing unit generates a first enhanced image in which a linear portion in the second image is enhanced based on the first candidate pixel value. The second image processing unit generates a two-dimensional smoothed image in which pixel values in the third image are not biased in a specific direction based on the second candidate pixel value. The third image processing unit generates a second enhanced image in which a linear portion of the first enhanced image is enhanced, based on the pixel values of the first enhanced image and the pixel values of the two-dimensional smoothed image.
5. The image processing device according to claim 1, in, The first pixel value setting unit sets a value that is 0.8 to 1.2 times a maximum value or 0.8 to 1.2 times a minimum value among the plurality of first candidate pixel values as the pixel value of the second pixel.
6. The image processing device according to claim 1, in, The first pixel group setting unit sets a plurality of the first pixel groups, the plurality of the first pixel groups being a plurality of the first pixel groups set corresponding to the first pixels, the plurality of the first pixel groups being arranged along a plurality of directions forming a plurality of angles different from each other with respect to a prescribed direction in the first image, or being arranged along a prescribed direction in each of a plurality of images obtained by moving the first image at a plurality of angles different from each other with respect to the prescribed direction, The second pixel group setting unit sets a plurality of second pixel groups, and the plurality of second pixel groups are arranged along at least two directions obtained by rotating the at least two directions at different multiple angles relative to a specified direction in the first image, or are arranged along the at least two directions in each first image of the first image rotated at different multiple angles.
7. The image processing device according to claim 1, in, The image processing device comprises: a third pixel group setting unit configured to set a plurality of third pixel groups including the first pixel, wherein the plurality of third pixel groups are a plurality of third pixel groups set corresponding to the first pixel, and the plurality of third pixel groups are arranged along a plurality of directions forming a plurality of angles with a predetermined direction of the first image, or are arranged along a predetermined direction in each of a plurality of images obtained by rotating the first image at a plurality of angles; an information generating unit, which generates corresponding information on whether the first pixel corresponds to a linear portion in the first image based on a sum or an average of pixel values of pixels included in each of the plurality of third pixel groups; as well as The processing control unit controls the processing of setting the first pixel group by the first pixel group setting unit based on the corresponding information generated by the information generating unit.
8. The image processing device according to claim 7, in, When the first pixel group setting unit sets the plurality of first pixel groups, the number of the plurality of angles varies depending on the position of the first pixel. The processing control unit sets the number in each position of the first image based on the correspondence information.
9. The image processing device according to claim 8, in, The processing control unit determines whether to perform the process of setting the first pixel group by the first pixel group setting unit at each position of the first image based on the correspondence information.
10. The image processing device according to claim 7, in, A second removing unit is provided for removing at least a part of a non-linear portion in the second image based on the correspondence information generated by the information generating unit.
11. The image processing device according to claim 4, in, The third image processing unit includes a binarization unit that binarizes pixel values of the second enhanced image using a plurality of different threshold values based on pixel values of pixels included in each portion of the second enhanced image.
12. The image processing device according to claim 1, in, An input unit is provided for inputting a position or a range of the first pixel to be processed by the first image processing unit.
13. The image processing device according to claim 1, in, The first image is an image of neurites or blood vessels.
Citation Information
Patent Citations
Toner
JP2019168631A
Linear Feature Detection Method and Apparatus
US20090154792A1
Image processor
JP2001283215A
