A relief sample microscopic imaging method and system

Abstract

This invention relates to a method and system for microscopic imaging of undulating samples. In the image acquisition stage, several sample planes are imaged separately, and each sample plane is imaged using both low numerical aperture (NFA) and high numerical aperture (NFA) illumination. In the preprocessing stage, the parfocality score of each field of view is estimated using the low NFA image, and the parfocal plane of each field of view is determined based on this score. In the multi-layer image fusion processing stage, high NFA images are stitched together for regions with continuous parfocal planes; low NFA images are stitched together for discontinuous boundaries. This method, for multi-layer microscopic imaging, achieves both a stable and unambiguous parfocal plane selection method and an imaging effect that balances high resolution and visual continuity, enabling the acquisition of high-quality image data.

Description

Technical Field

[0001] This invention relates to the field of microscopic optics, and more specifically to a method for microscopic imaging of undulating samples. Background Technology

[0002] Observing tissue or cell samples under a microscope is the gold standard for medical diagnosis. This observation typically targets tissue sections or cell smears that are only about one layer thick with cells, so digital imaging of the sample plane can record them perfectly.

[0003] However, in specific applications, it is necessary to image samples with uneven surfaces, such as observing special cells extracted from artificially processed filter membranes. Because the thickness of the membrane itself (several micrometers) is greater than the depth of field of the microscope (less than 1 micrometer), and wrinkles and local elevation by impurity particles are inevitable during processing, the sample appears on different sample planes under the microscope.

[0004] At this point, imaging only on a fixed sample plane will mean that areas on other sample planes cannot be correctly recorded. Summary of the Invention

[0005] To address this issue, the present invention provides a method for microscopic imaging of undulating samples. In the image acquisition stage, several sample planes are imaged separately, and each sample plane is imaged using both low numerical aperture (NFA) and high numerical aperture (HFA) illumination. In the preprocessing stage, the parfocality score of each field of view is estimated using the low NFA image, and the parfocal plane of each field of view is determined based on this score. In the multi-layer image fusion processing stage, regions with continuous parfocal planes are stitched together using HFA images; for discontinuous boundaries, low NFA images are stitched together.

[0006] This method is designed for microscopic imaging of undulating samples. It achieves a stable and unambiguous parfocal plane selection method, as well as imaging results that balance high resolution and visual continuity, and can acquire high-quality image data.

[0007] The present invention proposes a method for solving the above-mentioned technical problems: a multi-slice microscopy imaging method, comprising the following steps:

[0008] The sample to be imaged is divided into multiple imaging regions on a horizontal plane;

[0009] For each of the imaging regions, microscopic imaging is performed and recorded at multiple preset imaging distances;

[0010] The microscopic images of each region are fused into a full sample image;

[0011] The step of performing microscopic imaging and recording at multiple imaging distances includes:

[0012] Move the sample to the first imaging distance.

[0013] Illuminate the imaging with a low numerical aperture light source and add the image to the low numerical aperture image sequence of the region.

[0014] Illuminate the imaging with a high numerical aperture light source and add the image to the high numerical aperture image sequence of the region.

[0015] Move to the next imaging distance and repeat the illumination imaging and image sequence recording until all preset imaging distances have been traversed;

[0016] The homofocal index sequence of the region is obtained by sequentially calculating the homofocal index of each image in the low numerical aperture image sequence of the region.

[0017] Search for the maximum value in the homofocality index sequence and record the corresponding sequence number as the homofocal surface number of the region.

[0018] Save the image with the parfocal plane number from the low numerical aperture image sequence and the high numerical aperture image sequence of the region, and denot them as the low numerical aperture parfocal plane image and the high numerical aperture parfocal plane image of the region, respectively.

[0019] The step of fusing the microscopic images of each region into a full sample image includes:

[0020] According to the horizontal plane position relationship, compare the parfocal plane number of each imaging region with the adjacent imaging regions. If they are all the same, the imaging region is determined to be a non-transition region; otherwise, the region is determined to be a transition region.

[0021] For the transition region, the low numerical aperture parfocal image of that region is retained as the parfocal image of that region; for the non-transition region, the high numerical aperture parfocal image of that region is retained as the parfocal image of that region.

[0022] Based on the horizontal plane positional relationship, the parfocal plane images of each of the imaging regions are stitched together to form a full sample image.

[0023] The present invention, in order to solve the above-mentioned technical problems, also includes a microscopic imaging system for undulating samples, comprising:

[0024] The system includes a microscopic imaging module, a translational motion module, a numerical aperture variable light source module, a control module, a parfocality index calculation module, a parfocal plane selection and numerical aperture selection module, and an image stitching module.

[0025] Among them, microscopic imaging is used to image samples microscopically;

[0026] The translational motion module is used to move the sample to image at different horizontal positions and heights;

[0027] The numerical aperture variable light source module is used to provide illumination sources with multiple numerical apertures.

[0028] The control module is used to coordinate the synchronous operation of the microscopic imaging module, the translational motion module, and the numerical aperture variable light source module;

[0029] The homofocality index calculation module is used to calculate the homofocality index of the resulting microscopic image.

[0030] The parfocal plane selection and numerical aperture selection modules are used to select the parfocal plane of each region, and to select either a high numerical aperture or a low numerical aperture image as the parfocal plane image of that region.

[0031] The image stitching module is used to stitch together the parfocal images of each sample region into a complete sample image according to their horizontal positional relationship. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained from these drawings without creative effort.

[0033] Figure 1 It is a typical microscopic imaging system.

[0034] Figure 2 This is a typical curve showing the parfocality index as a function of the imaging plane height in single-layer thin-sample microscopy.

[0035] Figure 3 This is a detailed explanation of the light source-sample section when using a low numerical aperture light source in a microscopic imaging system.

[0036] Figure 4 This is a detailed explanation of the light source-sample section when using a high numerical aperture light source in a microscopic imaging system.

[0037] Figure 5 This is a comparison of the curves showing the parfocality index as a function of the image plane height under low-NA and high-NA illumination.

[0038] Figure 6 This is a flowchart of the workflow for a multilayer microscopy imaging method that combines low-NA and high-NA light sources.

[0039] Figure 7 This is a distribution diagram of the imaging plane numbers of the parfocal plane when using a high NA light source.

[0040] Figure 8This is a distribution diagram of the imaging plane numbers of the parfocal plane when using a low-NA light source. Detailed Implementation

[0041] Now, reference will be made to the accompanying drawings and embodiments described in detail. Numerous specific details are set forth in the following detailed description to provide a full understanding of the invention. However, those skilled in the art will understand that the invention can be practiced without these specific details. In other instances, well-known methods, processes, components, and circuits are not described in detail to avoid unnecessarily obscuring the embodiments.

[0042] In this embodiment, as Figure 1 Observing a typical microscope imaging system from a direction parallel to the optical axis, light emitted from light source 100 illuminates a thin sample 200 and is transmitted through a magnifying optical path 300 to form an image on image sensor 400. Two optical paths passing through a point on the sample are shown by the dashed line 500. Since these optical paths intersect on the sample 200 and also intersect on the image sensor 400, it can be known that this point lies on the parfocal plane of the microscope imaging system.

[0043] According to the basic principles of geometrical optics, when the position of the optical device remains unchanged, the image sensor is conjugate with the sample only when the thin sample is exactly on the parfocal plane, thus producing a clear and sharp image on the image sensor. When the thin sample is not on the parfocal plane, the image on the image sensor becomes blurry, and the degree of blurriness increases non-linearly with the distance of this region from the parfocal plane.

[0044] If the sample is in the direction parallel to the light path ( Figure 1 If the image is divided into multiple layers (such as the stacked cells in biological tissue), then the sample layer on the parfocal plane will appear as a clear and sharp image, while the other layers will appear as a blurry image. The image sensor reads the effect of the superposition of each layer; if there is no layer on the parfocal plane, then the image should be completely blurry.

[0045] Furthermore, based on the fundamental principles of geometric optics, the imaging of each region on the plane perpendicular to the optical axis is approximately independent of each other. That is, when a certain region of the sample is out of focus due to unevenness, it does not affect the imaging of other regions on the parfocal plane.

[0046] Therefore, for example Figure 1 The sample 200 shown is in the direction parallel to the optical axis ( Figure 1 By moving a sample with undulations in the vertical direction (parallel to the optical axis) to several heights and taking images of it separately, it is possible to achieve clarity in certain areas at one height and in other areas at another height. Then, the images of different areas on the horizontal plane that are clear at different heights can be extracted and merged into a single image in which all areas are clearly imaged.

[0047] However, how to accurately extract clear layers and how to handle images with multiple layers in local areas caused by sample fluctuations are fundamental principles that are not explicitly stated, and this invention will focus on solving these problems.

[0048] First, consider a simpler case: the thin sample has overall undulations, but after dividing the sample into many regions, each region is locally perfectly flat, meaning it can be clearly imaged at just one height. Then, if this region is moved to various preset imaging planes (typically evenly spaced within a certain range) and imaged separately, and its parfocal index (using, for example, sharpness, which is the sum of the absolute values ​​of the grayscale gradients of each pixel) is calculated separately, its distribution should appear as follows: Figure 2 The curve represents the parfocal index, which peaks at a certain height when the imaging plane is used, and gradually decreases at other heights. This height is the parfocal plane of that region.

[0049] In this simple case, it is only necessary to plot the imaging plane number-parfocality index curve for each region, i.e. Figure 2 By searching for the imaging surface corresponding to the maximum value of the parfocality index, we can find the parfocal surface for this region and select the parfocal image.

[0050] Because the homofocality index calculation relies on the presence of random image features within a region, the homofocality calculation region cannot be divided too small to avoid missing image features. Therefore, a more complex and common scenario is that the same region may still be uneven, or contain multiple layers. Consequently, the aforementioned image plane number-homofocality index curve may exhibit multiple maximum values, making it difficult to find a unique optimal solution, or requiring complex decision-making logic. Furthermore, even if a relatively accurate image plane is determined, discontinuities between image layers may still occur, hindering subsequent processing.

[0051] To address this issue, this method proposes using multi-numerical aperture light source imaging fusion. The following section first briefly introduces the concept of numerical aperture and its impact on imaging.

[0052] like Figure 3 As shown, the angle θ formed by the two points farthest apart on the light source illuminating the same point on the parfocal plane is the angular aperture of the light source, which is an important parameter of the imaging system. Typically, the angular aperture is equivalently expressed by the numerical aperture (NA): when air is the medium, the formula is NA = sin(θ / 2).

[0053] The resolution of the imaging system is determined by both the light source NA and the NA of the amplification optical path 300: the smaller of the two NAs is the NA of the entire system.

[0054] The smaller the NA of an optical system, the worse the imaging resolution, but the greater the depth of field. Figure 3 The light source 110, marked as having a low aperture (NA), has an angular aperture denoted as θLNA (Low NA). Correspondingly, Figure 4 The light source 120 marked as High NA has an angular aperture denoted as θHNA (High NA).

[0055] By comparison Figure 3 and Figure 4 It can be seen that, with other components remaining unchanged, by changing Figure 1 By changing the light emission range of the light source 100, the NA of the light source can be changed, which is equivalent to switching between the light source 110 and the light source 120, thereby controlling the NA of the optical system and adjusting the balance between imaging resolution and depth of field.

[0056] Now consider the case where there are two layers of samples in the same area due to sample undulations. The general case for illuminating the image with low-NA and high-NA light sources and calculating the parfocality index is as follows: Figure 5 As shown. Traditional methods use only a single, high-NA light source for illumination. Due to the low depth of field of a high-NA light source, the two sample layers have little mutual influence, thus each exhibits a high homofocal index on its corresponding imaging plane and has little impact on other imaging planes. Therefore, the corresponding imaging plane number-homofocal index is as follows: Figure 5 As shown by the solid curve, two local maximum peaks appear. It should be noted that the relative height of these two peaks may be random due to differences in the features of the two image layers, sensor noise, layer height position errors, etc. Therefore, the result of choosing one of the two is random, which may cause the final image to appear as abrupt changes at the boundary of the sample fluctuations, resulting in distortion. Further examples will be provided below.

[0057] Correspondingly, the method of this invention uses a low-NA light source to detect the parfocal plane, resulting in a large depth of field and some fusion of the images of the two sample layers. Therefore, the corresponding imaging plane number-parfocality index is as follows: Figure 5 As shown by the dashed curve, only one peak appears, and it is not located at either of the two solid curve peaks. This means that this region is the transition boundary between two surfaces. Taking the layer corresponding to the peak of the dashed curve, that is, the position between the two peaks of the solid curve, can cover both layers well in terms of height.

[0058] In addition, by using a low-NA light source to create a large depth-of-field image at the discontinuity of the parfocal plane, local image details are sacrificed, but the layer boundary at this point can be recorded more accurately in terms of image quality.

[0059] Based on this, the flowchart of the method of the present invention is as follows: Figure 6As shown in the diagram. In this process, the imaging plane number of the parfocal plane in each region is recorded, allowing us to obtain the distribution of these imaging plane numbers across the entire sample by region. This imaging plane distribution map is essentially the distribution of sample undulation height in units of imaging plane height intervals, which helps to intuitively understand the beneficial effects of this method.

[0060] In traditional methods, systems using high-NA light sources for imaging and imaging surface detection obtain an imaging surface distribution map on a sample as follows: Figure 7 As shown; in the method of this invention, a low-NA light source is used for imaging to detect the imaging surface, and the imaging surface distribution map obtained on the same sample is shown in the figure. Figure 8 As shown, this sample exhibits undulations near the main diagonal marked in gray, and the area marked N has the highest parfocality index on the Nth imaging plane.

[0061] Figure 7 This intuitively reflects two shortcomings of using only high-NA for imaging and image plane detection. First, because the samples are concentrated at two heights, with few pixels at the transition height, almost all pixels are at one of the two heights. This causes a jump in the detected layer height at the image plane boundary, jumping directly from 1 to 4, resulting in a discontinuous transition in the acquired image. Second, as... Figure 5 The explanation is that the relative magnitudes of the parfocality indices of the two imaging planes exhibit a certain degree of randomness in the transition region. As a result, areas that have transitioned to layer 1 may revert to layer 4, as shown in the circled area, further exacerbating the discontinuity of the obtained image.

[0062] Figure 8 The image shown employs low-NA light source imaging to achieve uniformity and continuity in image plane detection. Layer numbers marked with an underline indicate areas where adjacent regions have different image plane numbers, thus being identified as transitional regions, and these are used in the final image.

[0063] The present invention is not limited to the above embodiments. The technical solutions of the above embodiments of the present invention can be combined with each other to form new technical solutions. In addition, all technical solutions formed by equivalent substitutions fall within the protection scope claimed by the present invention.

Claims

1. A multi-slice microscopy imaging method, comprising the following steps: The sample to be imaged is divided into multiple imaging regions on a horizontal plane; For each of the imaging regions, microscopic imaging is performed and recorded at multiple preset imaging distances; The microscopic images of each region are fused into a full sample image; The step of performing microscopic imaging and recording at multiple preset imaging distances includes: Move the sample to the first imaging distance. Illuminate the imaging with a low numerical aperture light source and add the image to the low numerical aperture image sequence of the region. Illuminate the imaging with a high numerical aperture light source and add the image to the high numerical aperture image sequence of the region. Move to the next imaging distance and repeat the illumination imaging and image sequence recording until all preset imaging distances have been traversed; The homofocal index sequence of the region is obtained by sequentially calculating the homofocal index of each image in the low numerical aperture image sequence of the region. Search for the maximum value in the homofocality index sequence and record the corresponding sequence number as the homofocal surface number of the region; Save the image with the parfocal plane number from the low numerical aperture image sequence and the high numerical aperture image sequence of the region, and denot them as the low numerical aperture parfocal plane image and the high numerical aperture parfocal plane image of the region, respectively. The step of fusing the microscopic images of each region into a full sample image includes: According to the horizontal plane position relationship, compare the parfocal plane number of each imaging region with the adjacent imaging regions. If they are all the same, the imaging region is determined to be a non-transition region; otherwise, the region is determined to be a transition region. For the transition region, the low numerical aperture parfocal image of that region is retained as the parfocal image of that region; for the non-transition region, the high numerical aperture parfocal image of that region is retained as the parfocal image of that region. Based on the horizontal plane positional relationship, the parfocal plane images of each of the imaging regions are stitched together to form a full sample image.

2. A multi-slice microscopy imaging system for performing the multi-slice microscopy imaging method as described in claim 1, comprising: The system includes a microscopic imaging module, a translational motion module, a numerical aperture variable light source module, a control module, a parfocality index calculation module, a parfocal plane selection and numerical aperture selection module, and an image stitching module. Among them, microscopic imaging is used to image samples microscopically; The translational motion module is used to move the sample to image at different horizontal positions and heights; The numerical aperture variable light source module is used to provide lighting sources with different numerical apertures. The control module is used to coordinate the synchronous operation of the microscopic imaging module, the translational motion module, and the numerical aperture variable light source module; The homofocality index calculation module is used to calculate the homofocality index of the resulting microscopic image. The parfocal plane selection and numerical aperture selection modules are used to select the parfocal plane of each region, and to select either a high numerical aperture or a low numerical aperture image as the parfocal plane image of that region. The image stitching module is used to stitch together the parfocal images of each sample region into a complete sample image according to their horizontal positional relationship.

Images