Strip artifact removal method and system for optical microscopic imaging, readable storage medium and equipment

Abstract

The invention relates to the technical field of medical images, in particular to a bar artifact removal method and system for optical microscopic imaging, a readable storage medium and equipment. The method comprises the following steps: projecting and averaging image data to obtain an intensity curve and a fitting curve so as to obtain artifact distribution, and multiplying the obtained artifact distribution by an image of which the bar artifacts are not removed to obtain an image of which the bar artifacts are removed. According to the scheme provided by the invention, the artifact part can be effectively compensated through the algorithm on the premise of not changing the overall structure and intensity distribution of the image, so that the bar-shaped artifacts are remarkably reduced. And the accuracy of methods such as nerve projection reconstruction, registration or cell segmentation is improved.

Description

Technical Field

[0001] The present invention relates to the field of medical imaging technology, and particularly relates to a method, a system, a readable storage medium, and a device for removing strip artifacts in optical microscopy imaging. Background Art

[0002] With the rapid development and integration of high-resolution optical microscopy technology, tissue clearing technology, and sparse labeling technology, the industry has been able to achieve three-dimensional super-high-resolution imaging at the single-cell level. However, currently, the main method for obtaining large-scale high-resolution microscopic images is still to continuously stitch multiple images or strips. In this way, the stitched optical microscopic images are often affected by uneven illumination of the optical device or uncertainty factors of the sample itself, resulting in the appearance of stripes or artifacts, which seriously damages the image quality and affects subsequent quantitative analysis work.

[0003] Currently, the methods for solving strip artifacts are mainly divided into two categories: optical method solutions and image post-processing methods. In optical solutions, increasing laser power and optimizing beam quality are common techniques. Increasing laser power can stimulate fluorescence signals to a greater extent, but high power will increase the risk of light damage to the sample. In optimizing beam quality, wavefront shaping techniques such as Bessel beam self-reconstruction technology and DOE multi-focus fusion will increase the complexity and construction cost of the optical system.

[0004] As an alternative, digital processing of images provides a solution that does not increase the complexity of optical devices, which relies on algorithms to process strip artifacts in images. Common image processing methods include median filtering and fast Fourier filtering, etc., but these methods may damage the original image and have problems such as difficult parameter adjustment.

[0005] In summary, there is currently a lack of an accurate and non-destructive method for optical microscopy imaging Summary of the Invention

[0006] The present invention provides a method for removing strip artifacts applicable to optical microscopy imaging, which significantly reduces strip artifacts without changing the overall structure and intensity distribution of the original image.

[0007] The present invention provides the following technical solutions:

[0008] A method for removing strip artifacts in optical microscopy imaging, comprising the following steps:

[0009] Project the image data horizontally and / or vertically and average to obtain the projection values in the projected direction, and the top curve of the projection values in the projected direction is the intensity curve;

[0010] Find the local maximum values in the projection values in the projected direction;

[0011] Using each local maximum as a vertex for fitting to obtain a fitted curve;

[0012] Dividing the fitted curve by the intensity curve in the projection direction to obtain the artifact distribution in the projection direction;

[0013] Multiplying the obtained artifact distribution by the two-dimensional image without removing the bar artifacts to obtain an image with bar artifacts removed.

[0014] Furthermore, a third-order Bessel curve is used to fit the vertices.

[0015] Furthermore, before projecting the image array, the image is first downsampled.

[0016] Furthermore, the image obtained by the optical microscopy is a three-dimensional image. Before projecting the image array in the horizontal and / or vertical directions and averaging, the following steps are further included:

[0017] Obtaining an image imaging object mask;

[0018] Removing background noise in the image through the image imaging object mask;

[0019] Projecting the image after removing background noise onto the plane of the bar artifact intersection and averaging to obtain an average projection image.

[0020] Furthermore, projecting the average projection image in the horizontal and vertical directions and averaging to obtain projection values in the horizontal and vertical directions; integrating the artifact distributions in the horizontal and vertical directions, and multiplying the integrated artifact distribution by the two-dimensional slice of the three-dimensional image without removing the bar artifacts to obtain an image with bar artifacts removed.

[0021] Furthermore, if the resolution of the original three-dimensional image is anisotropic, before obtaining the image imaging object mask, the three-dimensional image is first downsampled in the two-dimensional slice to make the section resolution consistent in the slice direction, becoming isotropic.

[0022] Furthermore, the image obtained by the optical microscopy is a two-dimensional image, and the image array is projected in a direction perpendicular to the bar artifacts and averaged.

[0023] The present invention also provides a bar artifact removal system for optical microscopy, including:

[0024] An image projection unit for projecting and averaging the image array in the horizontal and / or vertical directions to obtain the intensity curve and the fitted curve of the image in the horizontal and / or vertical directions;

[0025] An artifact distribution acquisition unit for obtaining the artifact distribution of the bar artifacts in the image according to the intensity curve and the fitted curve;

[0026] An artifact processing unit for multiplying the artifact distribution by a two-dimensional image with bar artifacts not removed to obtain an image with bar artifacts removed.

[0027] The present invention also provides a readable storage medium storing an executable program, and when the executable program is executed by a processor, the steps of the method for removing bar artifacts described above are implemented.

[0028] The present invention also provides a bar artifact removal device for optical microscopy imaging, including a memory and a processor. The memory stores an executable program, and when the processor executes the executable program, the steps of the method for removing bar artifacts described above are implemented.

[0029] The beneficial effects of the present invention are as follows: The bar artifacts in optical microscopy images are a type of spatially related multiplicative noise, and their generation is related to the attenuation and scattering of the excitation light by substances in the sample and the diffraction limit of the optical system. The solution of the present invention can effectively compensate for the artifact part through an algorithm without changing the overall structure and intensity distribution of the image, thereby significantly reducing bar artifacts; it can excellently remove bar artifacts from two-dimensional or larger three-dimensional optical microscopy images at the single-cell level of resolution, and further improve the accuracy of methods such as neural projection reconstruction, registration, or cell segmentation; it can be applied to multiple optical microscopy modalities, such as cytoarchitecture, fluorescence labeling, etc. Moreover, it can robustly remove artifacts without adjusting complex parameters for different modality images; for image artifact processing at the single-cell level of resolution, it takes a short time and has a low computational cost. Description of the Drawings

[0030] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0031] Figure 1 is a schematic flowchart of the present invention;

[0032] Figure 2 is a schematic diagram of projecting and averaging image data in the X direction;

[0033] Figure 3 is a schematic diagram of the intensity curve C(x) and the fitting curve C b (x);

[0034] Figure 4 is a two-dimensional image after removing bar artifacts;

[0035] Figure 5 is a three-dimensional image with bar artifacts not removed;

[0036] Figure 6Schematic diagram of planar projection and averaging of bar artifact intersections;

[0037] Figure 7 Average projection image obtained after planar projection and averaging of bar artifact intersections;

[0038] Figure 8 Schematic diagram of intensity curve C(y) and fitted curve C b (y);

[0039] Figure 9 Schematic structural diagram of a bar artifact removal system for optical microscopy imaging. Specific implementation manners

[0040] The specific implementation manners of the present invention will be described below with reference to the accompanying drawings.

[0041] Refer to Figure 1 , for the bar artifact removal method applicable to optical microscopy imaging in this embodiment, for two-dimensional images, the following steps are adopted:

[0042] 1) Downsample the original optical microscopy image to a resolution of 50×50 μm²;

[0043] The resolution of optical microscopy images can usually reach the single-cell level. In order to significantly improve the calculation efficiency and reduce the consumption of computing resources, under the premise of ensuring that the image intensity distribution is not damaged, the image is downsampled.

[0044] 2) Refer to Figure 2 , project and average the downsampled image data in the vertical or horizontal direction of the bar artifact distribution;

[0045] The main function of projection is to reduce the influence of other signals on the image, enhance the artifacts, and make it easier to obtain the true distribution of the artifacts.

[0046] After projection, the top curve of the obtained projection values is the intensity curve. In this embodiment, project and average along the vertical direction X of the bar artifacts to obtain the intensity curve C(x);

[0047] 3) Refer to Figure 3 , find all local maxima in the projection values in this direction according to the preset minimum height, minimum distance between peaks, and peak width, fit with each local maximum as the vertex, and use a third-order Bessel curve for fitting to obtain the fitted curve C b (x);

[0048] Limited by the diffraction limit of the optical system, the intensity distribution of the line illumination modulation beam used on the focal plane of the objective lens is not an ideal line without width. Instead, it has a certain width in the direction perpendicular to the line illumination beam. Usually, the intensity distribution of the laser light source used in line confocal illumination microscopy is Gaussian distribution, with the property of being bright in the middle and dark around. This results in different illumination intensities at different positions, and thus produces strip artifacts that are bright in the middle and darker on both sides on the section. And the local maximum of C(x) is the center of the light source. By fitting the local maximum, a more ideal intensity distribution can be obtained, and then the correction function can be obtained.

[0049] 4) Divide the obtained curve C b (x) by the intensity curve C(x), and then obtain the approximate correction function ratio(x) = C b (x) / C(x);

[0050] 5) Multiply the obtained artifact correction function by the original two-dimensional image without removing the strip artifacts, and then restore the strip artifacts that are bright in the middle and dark on both sides caused by the optical diffraction limit, and obtain the image with strip artifacts removed, see Figure 4 .

[0051] When performing three-dimensional imaging with an optical microscope, due to reasons such as the change in the intensity of the laser light source caused by long-term acquisition of the device, the intensity change of the intensity distribution on the section usually occurs, resulting in artifacts as shown in Figure 5 . For another embodiment of the present invention, for the three-dimensional image obtained by optical microscopy imaging, the following steps are adopted:

[0052] 1) If the resolution of the original three-dimensional image is anisotropic, before obtaining the mask of the imaging object of the image, first downsample the resolution of the section in the two-dimensional slice of the three-dimensional image to be the same in the slice direction to become isotropic; downsample the isotropic image to 50*50*50μm 3 , and reconstruct it from the two-dimensional slice into a three-dimensional image; if it is not anisotropic, no processing of this step is required.

[0053] 2) Downsample the reconstructed three-dimensional image to 250*250*250μm 3 , and obtain the mask of the imaging object at a low resolution. In this embodiment, it is the brain mask;

[0054] Fill the holes in the brain mask and upsample it to 50*50*50μm 3 ;

[0055] 4) At a resolution of 50*50*50μm 3 , remove the background noise in the image through the upsampled brain mask;

[0056] 5) Project and average the 3D image after removing background noise onto the plane of the intersection of bar artifacts to obtain an average projection image, see Figure 6 and 7 ;

[0057] To improve the processing efficiency, for the two types of artifacts present in the image, we project and average the artifacts in the 3D space along a specific direction, such that the two types of artifacts form a cross-shaped distribution on the projection plane.

[0058] 6) See Figure 8 , select a side Y of the bar artifact where the projection image is perpendicular to the Gaussian distribution, project the projection image onto this side, and obtain the intensity curve C(y) along the Y direction;

[0059] 7) See Figure 8 , find all local maxima in the projection values in this direction according to the preset minimum height, minimum distance between peaks, and width of the peak, fit with each local maximum as the vertex, and use a third-order Bessel curve for fitting to obtain the fitted curve C b (y);

[0060] 8) Divide the obtained curve C b (y) by the intensity curve C(y), and then obtain the approximate artifact distribution ratio(y) = C b (y) / C(y) in the Y direction;

[0061] 9) According to the method in steps 6 - 8), obtain the artifact distribution ratio(x) = C b (x) / C(x) in the X direction;

[0062] 10) Integrate the artifact distributions in the two axial directions to obtain the artifact distribution ratio(x,y) of the average projection image;

[0063] 11) Multiply the obtained artifact distribution by each 2D slice of the 3D image without removing bar artifacts to obtain the image with bar artifacts removed.

[0064] See Figure 9 , the bar artifact removal system for optical microscopy imaging, includes:

[0065] An image projection unit, configured to project and average the image array horizontally and / or vertically to obtain the intensity curve and the fitted curve of the image in the horizontal and / or vertical direction;

[0066] An artifact distribution acquisition unit, configured to obtain the artifact distribution of bar artifacts in the image according to the intensity curve and the fitted curve; and

[0067] An artifact processing unit, configured to multiply an artifact distribution by a two-dimensional image with bar artifacts not removed to obtain an image with bar artifacts removed.

[0068] A readable storage medium, on which an executable program is stored, and when the executable program is executed by a processor, the steps of the foregoing method for removing bar artifacts are implemented.

[0069] An optical microscopy bar artifact removal device, including a memory and a processor, where the memory stores an executable program, and characterized in that when the processor executes the executable program, the steps of the foregoing method for removing bar artifacts are implemented.

[0070] Artifacts on an optical microscopy image usually exhibit periodicity, with the characteristics of being bright in the middle and dark around. During the image scanning process, the ideal image of the laser center is used as a reference standard. This method accurately fits the laser centers of the focal planes in two axial directions of the image, so as to obtain the multiplicative noise distribution of bar artifacts on the image.

[0071] Those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention also intends to include these changes and modifications.

Claims

1. Method for removing strip artifacts in optical microscopic imaging, characterized in that, It includes the following steps: Project and average the image data in the horizontal and / or vertical directions to obtain the projection values in the projected direction, and the top curve of the projection values in the projected direction is the intensity curve; Find the local maxima in the projection values in the projected direction; Perform fitting with each local maximum as the vertex to obtain the fitting curve; Divide the fitting curve by the intensity curve in the projected direction to obtain the artifact distribution in the projected direction; Multiply the obtained artifact distribution by the two-dimensional image without removing the bar artifacts to obtain the image with bar artifacts removed.

2. The method for removing strip artifacts in optical microscopic imaging according to claim 1, wherein: Use a third-order Bezier curve to fit the vertices.

3. The method for removing strip artifacts in optical microscopy imaging according to claim 2, wherein: Before projecting the image array, first downsample the image.

4. The method for removing strip artifacts in optical microscopy imaging according to any one of claims 1-3, characterized in that, The image obtained by the optical microscopy imaging is a three-dimensional image. Before projecting and averaging the image array in the horizontal and / or vertical directions, the following steps are further included: Obtain the mask of the image imaging object; Eliminate the background noise in the image through the mask of the image imaging object; Project and average the image after eliminating the background noise onto the plane of the intersection of the bar artifacts to obtain the average projection image.

5. The method for removing strip artifacts in optical microscopy imaging according to claim 4, wherein: Project and average the average projection image in the horizontal and vertical directions to obtain the projection values in the horizontal and vertical directions; Integrate the artifact distributions in the horizontal and vertical directions, and multiply the integrated artifact distribution by the two-dimensional slice of the three-dimensional image without removing the bar artifacts to obtain the image with bar artifacts removed.

6. The method for removing bar artifacts in optical microscopy imaging according to claim 5, wherein: If the resolution of the original three-dimensional image is anisotropic, before obtaining the mask of the image imaging object, first downsample the resolution of the section in the two-dimensional slice of the three-dimensional image to make the section directions consistent and become isotropic.

7. The method for removing strip artifacts in optical microscopic imaging according to any one of claims 1-3, characterized in that: The image obtained by the optical microscopy imaging is a two-dimensional image. Project and average the image array in the direction perpendicular to the bar artifacts.

8. Bar artifact removal system for optical microscopy imaging, characterized in that: It includes: An image projection unit for projecting and averaging the image array horizontally and / or vertically to obtain the intensity curve and the fitting curve of the image in the horizontal and / or vertical direction; An artifact distribution acquisition unit for obtaining the artifact distribution of the bar artifacts in the image according to the intensity curve and the fitting curve; An artifact processing unit for multiplying the artifact distribution by the two-dimensional image without removing the bar artifacts to obtain the image with bar artifacts removed.

9. A readable storage medium having an executable program stored thereon, characterized in that, When the executable program is executed by the processor, it implements the steps of the method for removing bar artifacts described in any one of claims 1 to 7.

10. An apparatus for removing bar artifacts in optical microscopic imaging, comprising a memory and a processor, wherein the memory stores an executable program, characterized in that, When the processor executes the executable program, it implements the steps of the method for removing bar artifacts described in any one of claims 1 to 7.

Images