Differential confocal microscopic imaging system and method based on custom aperture

Abstract

The present application belongs to the field of optical measurement technology, and particularly relates to a differential confocal microscopic imaging system and method based on a self-defined aperture. The method comprises the following steps: S1: calculating optimal amplitude distribution parameters of the self-defined aperture based on an entrance pupil of an objective lens in a focusing imaging module, and adjusting the self-defined aperture based on the calculation result; S2: rotating the adjusted self-defined aperture circumferentially relative to an optical axis of the objective lens in the focusing imaging module, and collecting pre-focal images and post-focal images under different rotation angles; S3: respectively performing image fusion on the pre-focal images and the post-focal images under different rotation angles, and correspondingly obtaining pre-focal fusion images and post-focal fusion images; S4: constructing a multi-focal fusion image and a pre-focal post-focal difference image based on the pre-focal fusion images and the post-focal fusion images; and S5: constructing a differential enhancement image based on the multi-focal fusion image and the pre-focal post-focal difference image. The present application can effectively improve the lateral resolution of the differential confocal microscope.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of optical measurement, and particularly relates to a differential confocal microscopic imaging system and method based on a self-defined aperture. BACKGROUND

[0002] A confocal microscope is a high-resolution microscopic imaging technology that realizes high-resolution imaging by scanning a sample with a point light source and combining optical sectioning technology. A traditional confocal microscope controls the focusing of a light beam by using a pinhole and the numerical aperture of an objective lens, thereby realizing high-resolution images in three-dimensional space. In recent years, as the demand for imaging increases, higher requirements have been placed on the lateral resolution of confocal microscopes.

[0003] In view of the problem of limited lateral resolution of a confocal microscope, existing technologies have proposed various improvement methods. One is to improve the focusing performance by changing the parameters of the incident light beam, such as reducing the diameter of the incident light beam and adjusting the filling mode of the light beam in the aperture of the objective lens, so as to reduce the size of the focal spot to some extent and improve the lateral resolution. Although this type of method has a relatively simple structure, the resolution improvement effect is strictly limited by the diffraction limit, and at the same time, the incident energy utilization rate is reduced, and the imaging signal-to-noise ratio is easily affected. The other is to introduce super-resolution microscopic imaging technology, such as structured light illumination microscopy and stimulated emission depletion microscopy, which breaks through the traditional diffraction limit through nonlinear optical equations or frequency domain modulation methods. Although this type of method can obtain higher lateral resolution under theoretical and experimental conditions, the optical path structure is complex, the requirements for light source stability, modulation precision and synchronous control are high, the system cost and implementation difficulty are large, and it is difficult to be directly applied to high-speed scanning and engineering measurement scenes.

[0004] Therefore, the above-mentioned solutions generally have the following deficiencies in practical application: first, the system structure is complex and the cost is high, which makes it difficult to realize high integration while maintaining system stability; second, additional light energy loss is easily introduced in the modulation process, which affects the signal strength and imaging signal-to-noise ratio; third, it is sensitive to environmental disturbances and system parameter changes, which is not conducive to high-speed scanning and engineering application. Therefore, the existing technology has not yet provided a solution that can effectively improve the lateral resolution of differential confocal microscopic imaging while ensuring light energy utilization and system stability. SUMMARY

[0005] Therefore, the present application aims to provide a differential confocal microscopic imaging system and method based on a self-defined aperture, which can effectively improve the lateral resolution of differential confocal microscopy while maintaining the signal strength.

[0006] To achieve the above object, the technical scheme of the application is implemented as follows:

[0007] A differential confocal microscopic imaging system based on a self-defined aperture comprises a light source, a polarization beam splitter, a self-defined aperture, a 1 / 4 wave plate, a focusing imaging module, a beam splitter, a pre-focal detection module, a post-focal detection module and a signal processing module, wherein the single-wavelength laser emitted by the light source is transmitted through the polarization beam splitter to output p-direction linearly polarized light, the p-direction linearly polarized light is incident on the 1 / 4 wave plate through the self-defined aperture, the 1 / 4 wave plate converts the p-direction linearly polarized light into circularly polarized light, the circularly polarized light is focused on the sample to be measured after the action of the focusing imaging module, the circularly polarized light reflected by the sample to be measured is incident on the 1 / 4 wave plate through the focusing imaging module, the 1 / 4 wave plate converts the circularly polarized light into s-direction linearly polarized light, the s-direction linearly polarized light is incident on the polarization beam splitter through the self-defined aperture, the polarization beam splitter reflects the s-direction linearly polarized light to the beam splitter for beam splitting to obtain a first light beam and a second light beam, the first light beam is processed by the pre-focal detection module to obtain a pre-focal image, the second light beam is processed by the post-focal detection module to obtain a post-focal image, and the signal processing module outputs a differential enhancement image based on the pre-focal image and the post-focal image at different circumferential rotation angles of the self-defined aperture relative to the optical axis of the objective lens in the pre-focal detection module.

[0008] Further, the focusing imaging module comprises an objective lens and an objective lens driver, the objective lens driver drives the objective lens to move relative to the surface of the sample to be measured to realize scanning of the sample to be measured.

[0009] Further, the pre-focal detection module and the post-focal detection module have the same structure and each comprises a focusing lens, a pinhole and a face array camera arranged in sequence along the light path direction; wherein the pinhole of the pre-focal detection module is arranged in front of the focal length of the focusing lens of the pre-focal detection module, and the pinhole of the post-focal detection module is arranged behind the focal length of the focusing lens of the post-focal detection module.

[0010] Further, the signal processing module comprises an image fusion module, a multi-focus fusion and differential module and a differential enhancement module, wherein the image fusion module is used for image fusion of the pre-focal images and the post-focal images at different rotation angles respectively to correspondingly obtain pre-focal fusion images and post-focal fusion images; the multi-focus fusion and differential module is used for constructing a multi-focus fusion image and a pre-focal post-focal differential image based on the pre-focal fusion images and the post-focal fusion images; and the differential enhancement module is used for constructing a differential enhancement image based on the multi-focus fusion image and the pre-focal post-focal differential image.

[0011] A differential confocal microscopic imaging method based on a self-defined aperture is implemented by using a differential confocal microscopic imaging system based on a self-defined aperture, and specifically comprises the following steps:

[0012] S1: calculate the optimal amplitude distribution parameters of the self-defined aperture based on the entrance pupil of the objective lens in the focusing imaging module, and adjust the self-defined aperture based on the calculation results;

[0013] S2: circumferentially rotate the adjusted self-defined aperture relative to the optical axis of the objective lens in the focusing imaging module, and collect pre-focal images and post-focal images under different rotation angles;

[0014] S3: the image fusion module respectively performs image fusion on the pre-focal images and the post-focal images under different rotation angles, and correspondingly obtains pre-focal fusion images and post-focal fusion images;

[0015] S4: the multi-focus fusion and difference module constructs a multi-focus fusion image and a pre-focal post-focal difference image based on the pre-focal fusion images and the post-focal fusion images;

[0016] S5: the difference enhancement module constructs a difference enhancement image based on the multi-focus fusion image and the pre-focal post-focal difference image.

[0017] Further, step S1 specifically includes the following steps:

[0018] S11: divide the entrance pupil of the objective lens in the focusing imaging module into N sub-regions, and construct a to-be-optimized aperture mask function :

[0019] ;

[0020] ;

[0021] wherein, is the amplitude distribution parameter of the i-th sub-region, , is the indicator function of the i-th sub-region, is the normalized polar coordinate defined based on the entrance pupil surface of the objective lens, is the half opening angle of the objective lens, is the transmittance of the objective lens;

[0022] S12: in the three-dimensional PSF corresponding to the focusing spot on the to-be-tested sample, the focusing spot is located in the XOY plane of the three-dimensional PSF, the full width at half maximum value of the normalized intensity curve corresponding to the straight line on which the minimum width of the focusing spot in the y-axis direction is located is defined as the lateral resolution evaluation index, and the effective light flux of the self-defined aperture is defined as the light flux energy index, and based on the lateral resolution evaluation index and the light flux energy index, a target function is constructed:

[0023] ;

[0024] ;

[0025] ;

[0026] ;

[0027] wherein, is a lateral resolution evaluation index, is a light energy passing index, is a set of amplitude distribution parameters of each sub-region, and are weight coefficients, , , are corresponding reference values under a circular aperture, is a normalized polar coordinate corresponding to the light intensity distribution of the focused light spot on the to-be-tested sample, is the light intensity distribution of the focused light spot on the to-be-tested sample corresponding to the normalized polar coordinate (0, 0);

[0028] S13: randomly generating an initial set of amplitude distribution parameters, adjusting the amplitude distribution parameters of the custom aperture based on a genetic algorithm, and obtaining an optimal set of amplitude distribution parameters.

[0029] Further, step S3 specifically comprises:

[0030] S31: obtaining a pre-focal image under the kth rotation angle, and calculating the gradient amplitude of the pre-focal image under the kth rotation angle :

[0031] ;

[0032] ;

[0033] wherein, is the intensity distribution of the light reflected by the to-be-tested sample; is a point spread function of the pre-focal detection module under the kth rotation angle, is a convolution operator; is a noise term;

[0034] S32: combining the to-be-optimized aperture mask function under the kth rotation angle in the Cartesian coordinate system to construct an unnormalized weight value corresponding to the pre-focal image under the kth rotation angle:

[0035] ;

[0036] wherein, is the unnormalized weight value corresponding to the pre-focal image under the kth rotation angle;

[0037] ​S33: obtaining the fusion weight corresponding to the pre-focus image under the kth rotation angle based on the calculation result of step S32 :

[0038] ;

[0039] wherein, to prevent the positive number of the denominator being zero, is the non-normalized weight value corresponding to the pre-focus image under the jth rotation angle, and K is the total number of rotation angles;

[0040] S34: calculating the pre-focus high-lateral-resolution fusion image based on the calculation result of step S33 by the following formula :

[0041] ;

[0042] wherein, K is the total number of rotation angles;

[0043] S35: calculating the pre-focus fusion image based on the calculation result of step S34 by the following formula:

[0044] ;

[0045] wherein, D() is a processing operator;

[0046] S36: replacing the pre-focus image under the kth rotation angle with the post-focus image under the kth rotation angle, repeating steps S31-S35 to obtain the post-focus fusion image.

[0047] Further, in step S35, the processing operator is specifically a bilateral filter processing operator, a wavelet denoising processing operator or a contrast-limited adaptive histogram equalization processing operator.

[0048] Further, step S4 specifically comprises the following steps:

[0049] S41: calculating the local sharpness of the pre-focus fusion image and the post-focus fusion image respectively by using a Laplace operator:

[0050] ;

[0051] wherein, is the local sharpness of the pre-focus fusion image, is the local sharpness of the post-focus fusion image;

[0052] S42: constructing the fusion weight of the pre-focus fusion image and the post-focus fusion image respectively based on the calculation result of step S41 by the following formula:

[0053] ;

[0054] wherein, is the fusion weight of the pre-focus fusion image, is the fusion weight of the post-focus fusion image;

[0055] S43: based on the calculation results of step S41 and step S42, a multi-focus fusion image is calculated by the following formula :

[0056] ;

[0057] a pre-focus post-focus difference image is calculated by the following formula :

[0058] .

[0059] Further, in step S5, a difference enhancement image is constructed based on the multi-focus fusion image and the pre-focus post-focus difference image The calculation formula used is:

[0060] ;

[0061] wherein, is the difference enhancement weight coefficient.

[0062] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0063] (1) The differential confocal microscopic imaging system and method based on a self-defined aperture of the present invention, compared with the traditional confocal microscope, significantly improves the lateral resolution and imaging quality of the confocal microscope by introducing adaptive optical aperture technology: by rotating the aperture, a series of images at different angles are obtained, and the region with the strongest lateral resolution of each image is extracted to synthesize a high lateral resolution image. Enhance the detail performance of the overall image. Finally, through the multi-focus fusion technology, the image clarity and contrast are further improved, avoiding focus blur and improving image quality.

[0064] (2) The differential confocal microscopic imaging system and method based on a self-defined aperture of the present invention can significantly improve the lateral resolution while maintaining the beam intensity and imaging clarity, and effectively enhance the contrast and details of the image, providing an innovative solution for high-resolution imaging of differential confocal microscopes. BRIEF DESCRIPTION OF DRAWINGS

[0065] The accompanying drawings, which form a part of the present invention, are included to provide a further understanding of the present invention and are incorporated in and constitute a part of this specification. The illustrations are provided to explain the present invention and are not intended to limit the present invention. In the drawings:

[0066] Figure 1 Structure diagram of a differential confocal microscopic imaging system based on a self-defined aperture according to an embodiment of the present invention;

[0067] Figure 2 Structure block diagram of a signal processing module according to an embodiment of the present invention;

[0068] Figure 3 Flowchart of a differential confocal microscopic imaging method based on a self-defined aperture according to an embodiment of the present invention;

[0069] Figure 4 Normalized intensity curve corresponding to a straight line on which the width of a focused light spot passing through a circular aperture in the y-axis direction lies according to an embodiment of the present invention;

[0070] Figure 5 Normalized intensity curve corresponding to a straight line on which the width of a focused light spot passing through a self-defined aperture in the y-axis direction lies according to an embodiment of the present invention.

[0071] Explanation of reference signs:

[0072] 1, light source; 2, polarization beam splitter; 3, self-defined aperture; 4, 1 / 4 wave plate; 5, focusing imaging module; 6, sample to be measured; 7, test table; 8, beam splitter; 9, pre-focal detection module; 10, post-focal detection module; 11, focusing lens; 12, pinhole; 13, area array camera; 14, image fusion module; 15, multi-focal fusion and difference module; 16, difference enhancement module. DETAILED DESCRIPTION

[0073] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, and do not constitute a limitation on the present invention.

[0074] It should be noted that the embodiments in the present invention and the features in the embodiments can be combined with each other without conflict.

[0075] In the description of the present application, it is to be understood by the terms "center", "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. In addition, the terms "first", "second" and the like are only for the purpose of description and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined with "first", "second" and the like can explicitly or implicitly include one or more of the features. In the description of the present application, unless otherwise specified, the meaning of "a plurality of" is two or more.

[0076] In the description of the present application, it should be noted that unless otherwise specified and limited, the terms "mounting", "connecting", "connecting" should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium, or it can be connected inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0077] The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0078] As Figure 1As shown, the application provides a differential confocal microscopic imaging system based on a self-defined aperture, which comprises a light source 1, a polarization beam splitter 2, a self-defined aperture 3, a 1 / 4 wave plate 4, a focusing imaging module 5, a beam splitter mirror 8, a pre-focal detection module 9, a post-focal detection module 10 and a signal processing module. The single-wavelength laser emitted by the light source 1 is transmitted through the polarization beam splitter 2 to output p-direction linearly polarized light. The p-direction linearly polarized light is incident on the 1 / 4 wave plate 4 through the self-defined aperture 3. The 1 / 4 wave plate 4 converts the p-direction linearly polarized light into circularly polarized light. The circularly polarized light is focused on the sample 6 to be measured on the test table 7 after the action of the focusing imaging module 5. The circularly polarized light reflected by the sample 6 to be measured is incident on the 1 / 4 wave plate 4 through the focusing imaging module 5. The 1 / 4 wave plate 4 converts the circularly polarized light into s-direction linearly polarized light. The s-direction linearly polarized light is incident on the polarization beam splitter 2 through the self-defined aperture 3. The polarization beam splitter 2 reflects the s-direction linearly polarized light to the beam splitter mirror 8 for beam splitting to obtain a first light beam and a second light beam. The first light beam is processed by the pre-focal detection module 9 to obtain a pre-focal image. The second light beam is processed by the post-focal detection module 10 to obtain a post-focal image. The signal processing module outputs a differential enhanced image based on the pre-focal images and the post-focal images of the self-defined aperture 3 at different circumferential rotation angles relative to the optical axis of the objective lens in the pre-focal detection module 9.

[0079] It should be noted that the application introduces an optimally designed self-defined aperture 3 in the system optical path, changes the distribution of part of the light beam on the self-defined aperture 3 under the condition that the self-defined aperture 3 ensures the optimal amplitude distribution parameters, thereby realizing the optimization of the focal spot morphology.

[0080] Specifically, the stable single-wavelength light source 1 is collimated and converted into linearly polarized light by the polarization beam splitter 2. The linearly polarized light is modulated by the self-defined aperture 3 and then sequentially passes through the 1 / 4 wave plate 4, the objective lens and the objective lens driver to be focused on the surface of the sample 6 to be measured. The reflected light of the surface of the sample 6 to be measured returns along the original optical path, passes through the 1 / 4 wave plate 4 and the self-defined aperture 3, is reflected by the polarization beam splitter 2 and guided to the beam splitter mirror 8. After light splitting, the signals enter the pre-focal detection module 9 and the post-focal detection module 10, respectively, and finally a differential enhanced image with higher resolution and better contrast is obtained through differential processing and image optimization.

[0081] The focusing imaging module 5 comprises an objective lens and an objective lens driver. The objective lens driver drives the objective lens to move relative to the surface of the sample 6 to be measured, thereby realizing the scanning of the sample 6 to be measured.

[0082] The pre-focal detection module 9 and the post-focal detection module 10 have the same structure and each comprises a focusing lens 11, a pinhole 12 and a face array camera 13 arranged in sequence along the optical path. The pinhole 12 of the pre-focal detection module 9 is arranged in front of the focal length of the focusing lens 11 of the pre-focal detection module 9, and the pinhole 12 of the post-focal detection module 10 is arranged behind the focal length of the focusing lens 11 of the post-focal detection module 10.

[0083] As shown in Figure 2 , the signal processing module includes an image fusion module 14, a multi-focus fusion and difference module 15, and a difference enhancement module 16, wherein the image fusion module 14 is configured to perform image fusion on the pre-focus images and the post-focus images at different rotation angles respectively, thereby obtaining pre-focus fusion images and post-focus fusion images; the multi-focus fusion and difference module 15 is configured to construct a multi-focus fusion image and a pre-focus post-focus difference image based on the pre-focus fusion images and the post-focus fusion images; and the difference enhancement module 16 is configured to construct a difference enhancement image based on the multi-focus fusion image and the pre-focus post-focus difference image.

[0084] It should be noted that, in the signal processing link, the present application first optimizes the self-defined aperture 3 to make the energy distribution of the light beam on the self-defined aperture 3 more reasonable, thereby achieving good beam matching and improving the quality of the focal spot. Subsequently, the pre-focus images and the post-focus images collected are optimized to obtain images with higher resolution. Finally, the pre-focus fusion images and the post-focus fusion images are fused through difference processing to further enhance the contrast and clarity of the images, thereby realizing high-resolution and high-quality imaging, i.e., obtaining a difference enhancement image.

[0085] As shown in Figure 3 , the present application further provides a differential confocal microscopic imaging method based on a self-defined aperture, which is realized by using a differential confocal microscopic imaging system based on a self-defined aperture, and specifically includes the following steps:

[0086] S1: calculating the optimal amplitude distribution parameters of the self-defined aperture 3 based on the entrance pupil of the objective lens in the focusing imaging module 5, and adjusting the self-defined aperture 3 based on the calculation results;

[0087] S2: rotating the adjusted self-defined aperture 3 circumferentially relative to the optical axis of the objective lens in the focusing imaging module 5, and collecting pre-focus images and post-focus images at different rotation angles;

[0088] S3: the image fusion module 14 performs image fusion on the pre-focus images and the post-focus images at different rotation angles respectively, thereby obtaining pre-focus fusion images and post-focus fusion images;

[0089] S4: the multi-focus fusion and difference module 15 constructs a multi-focus fusion image and a pre-focus post-focus difference image based on the pre-focus fusion images and the post-focus fusion images;

[0090] S5: the difference enhancement module 16 constructs a difference enhancement image based on the multi-focus fusion image and the pre-focus post-focus difference image.

[0091] Further, step S1 specifically includes the following steps:

[0092] S11: divide the entrance pupil of the objective lens in the focusing imaging module 5 into N sub-regions, and construct an aperture mask function to be optimized :

[0093] ;

[0094] ;

[0095] wherein, is the amplitude distribution parameter of the i-th sub-region, , is the indicator function of the i-th sub-region, is the normalized polar coordinate defined based on the entrance pupil surface of the objective lens, is the half opening angle of the objective lens, is the transmittance of the objective lens;

[0096] S12: in the three-dimensional PSF corresponding to the focused spot on the sample to be tested 6, the focused spot is located in the XOY plane of the three-dimensional PSF, the definition of the full width at half maximum value of the normalized intensity curve corresponding to the straight line on which the minimum width of the focused spot in the y-axis direction is located is defined as the lateral resolution evaluation index, and the effective light flux of the self-defined aperture 3 is defined as the light energy index, and based on the lateral resolution evaluation index and the light energy index, a target function is constructed :

[0097] ;

[0098] ;

[0099] ;

[0100] ;

[0101] wherein, is the lateral resolution evaluation index, is the light energy index, is the set of amplitude distribution parameters of each sub-region, and are weight coefficients, , , are all corresponding reference values under the circular aperture, is the normalized polar coordinate corresponding to the light intensity distribution of the focused spot on the sample to be tested 6, is the light intensity distribution of the focused spot on the sample to be tested 6 corresponding to the normalized polar coordinate (0, 0);

[0102] The XOY plane of the three-dimensional PSF is used to represent the image coordinate position of each pixel point of the focused light spot, the Y axis is used to represent the normalized intensity curve corresponding to the image coordinate position corresponding to the pixel point, as shown in Figure 4 and Figure 5 , and the normalized intensity curve corresponding to the straight line where the width of the focused light spot in the y-axis direction is located.

[0103] S13: randomly generate an initial amplitude distribution parameter set, adjust the amplitude distribution parameters of the custom aperture 3 based on a genetic algorithm, and obtain an optimal amplitude distribution parameter set.

[0104] It should be noted that in the confocal microscope, the numerical aperture of the objective lens determines the highest spatial frequency that the optical system can pass, thereby determining the theoretical lateral resolution. Assuming that the imaging wavelength of the optical system is , the refractive index of the medium on the side of the sample 6 to be measured is , and the half opening angle of the objective lens is , then:

[0105] ;

[0106] Under the ideal aberration-free, scalar diffraction approximation condition, the lateral focusing field of the sample 6 surface is uniquely determined by the complex amplitude distribution at the objective lens entrance pupil.

[0107] The normalized polar coordinates are defined at the objective lens entrance pupil surface:

[0108] ;

[0109] The complex amplitude transmission function of the custom aperture 3 is defined as :

[0110] ;

[0111] Wherein: represents the amplitude transmittance; represents the phase distribution (in the present application, the value of )

[0112] The lateral light spot intensity distribution of the sample 6 surface is defined as :

[0113] ;

[0114] Wherein, represents the lateral complex amplitude field of the sample 6 surface under the Debye scalar approximation.

[0115] The present application adopts an irregular custom aperture 3 to control the entrance pupil amplitude distribution , which is mathematically expressed as:

[0116] ;

[0117] wherein, is the aperture mask function to be optimized, satisfying:

[0118] ;

[0119] The lateral focused spot under the irregular custom aperture 3 can be expressed as:

[0120] ;

[0121] The entrance pupil is divided into sub-regions, and the definition is:

[0122] ;

[0123] The initial population of the genetic algorithm is expressed as , and the constraint function used by the genetic algorithm is:

[0124] ;

[0125] The individual fitness function is defined as:

[0126] ;

[0127] wherein, t is the iteration number, is the objective function of the i-th sub-region under the t-th iteration;

[0128] Through selection, crossover and mutation operations, is constantly updated, and the iteration termination condition is:

[0129] ;

[0130] ;

[0131] wherein, and are intermediate parameters used to express the termination condition, and have no physical meaning.

[0132] The vector is taken as input to describe the parameter set of the irregular aperture amplitude distribution, and under the given objective lens parameters and imaging conditions, the algorithm is iteratively optimized by constructing the comprehensive evaluation objective function of lateral resolution and light energy. The output of the algorithm is a set of optimal aperture amplitude distribution parameters that satisfy the optimization objective, which is used to generate the corresponding custom aperture 3, so as to realize the improvement of the lateral resolution and imaging quality of the confocal microscopic imaging system.

[0133] Further, step S3 specifically comprises:

[0134] S31: Obtain the pre-focus image under the kth rotation angle, and calculate the gradient amplitude of the pre-focus image under the kth rotation angle

[0135]

[0136]

[0137] wherein, I(x,y) is the intensity distribution of the light reflected by the sample 6 to be tested; is the point spread function of the pre-focus detection module 9 under the kth rotation angle, is a convolution operator; is a noise term;

[0138] S32: Combine the aperture mask function to be optimized in the Cartesian coordinate system corresponding to the kth rotation angle to construct the non-normalized weight corresponding to the pre-focus image under the kth rotation angle:

[0139]

[0140] wherein, Wk(x,y) is the non-normalized weight corresponding to the pre-focus image under the kth rotation angle; It should be noted that the aperture optimization and spot analysis process are described in the polar coordinate system, and in the image fusion and sharpness calculation process, the Cartesian coordinate system (x, y) is used for uniform representation for the convenience of processing discrete images. The consistency between the two is maintained through coordinate transformation. The coordinate conversion between the Cartesian coordinate system and the polar coordinate system is a prior art, which will not be described here.

[0141] S33: Based on the calculation result of step S32, obtain the fusion weight corresponding to the pre-focus image under the kth rotation angle

[0142]

[0143] wherein, Wk(x,y) is the non-normalized weight corresponding to the pre-focus image under the kth rotation angle;

[0144] is a positive number to prevent the denominator from being zero, is the non-normalized weight corresponding to the pre-focus image under the jth rotation angle; S34: Based on the calculation result of step S33, calculate the pre-focus high lateral resolution fusion image by the following formula

[0145]

[0146]

[0147] ​​​​​​​​​​​Wherein, K is the total number of rotation angles;

[0148] S35: based on the calculation result of step S34, the pre-focus fusion image is calculated by the following formula:

[0149] ;

[0150] Wherein, D() is a processing operator;

[0151] S36: the post-focus image under the kth rotation angle is used to replace the pre-focus image under the kth rotation angle, and steps S31-S35 are repeated to obtain the post-focus fusion image.

[0152] Further, in step S35, the processing operator is specifically a bilateral filter processing operator, a wavelet denoising processing operator or a contrast limited adaptive histogram equalization processing operator.

[0153] To improve the signal-to-noise ratio of the pre-focus fusion image and the post-focus fusion image and the subsequent fusion stability, the pre-focus fusion image and the post-focus fusion image can be subjected to image enhancement processing, wherein the processing operator can be but is not limited to a bilateral filter processing operator, a wavelet denoising processing operator or a contrast limited adaptive histogram equalization processing operator.

[0154] It should be noted that in the present application, the custom irregular aperture is realized by an algorithm, so that the lateral spot in a certain fixed direction is the narrowest, and the lateral resolution in the corresponding direction is the highest. The direction is referred to as the local "preferred resolution direction". By changing the rotation angle of the irregular aperture , a plurality of images are sequentially obtained on the full field of view of the sample 6. Through design, for each k, there is always a direction corresponding to the highest lateral resolution.

[0155] To extract high-resolution information from the plurality of images, a local sharpness index of the image corresponding to the kth rotation angle is defined, and the local sharpness index is represented by the gradient amplitude. To highlight the "region near the corresponding preferred resolution direction" in the image corresponding to each rotation angle, a rotation angle related direction optimization aperture mask function satisfying:

[0156]

[0157] And in theory, the region corresponding to the preferred resolution direction is larger, and the other regions are smaller.

[0158] The contribution of the image corresponding to the kth rotation angle at each pixel point is determined by , and if the spatial position of a certain image is , the contribution of the image corresponding to the kth rotation angle is ​The image gradient at the rotation angle is stronger, and the direction matches the preferred resolution direction, so The details of the image are preferentially written into the fusion result; the multi-angle images are superimposed according to the 'local clearest principle', so that a high-lateral-resolution fusion image with improved overall lateral resolution is obtained in the full field of view. The high-lateral-resolution fusion image is further processed for denoising and enhancement, for example, linear filtering or nonlinear enhancement operators, to obtain the pre-focal fusion image.

[0159] Further, step S4 specifically includes the following steps:

[0160] S41: The Laplace operator is used to calculate the local sharpness of the pre-focal fusion image and the post-focal fusion image, respectively:

[0161] ;

[0162] Wherein, is the local sharpness of the pre-focal fusion image, is the local sharpness of the post-focal fusion image;

[0163] S42: Based on the calculation results of step S41, the fusion weights of the pre-focal fusion image and the post-focal fusion image are constructed respectively by the following formula:

[0164] ;

[0165] Wherein, is the fusion weight of the pre-focal fusion image, is the fusion weight of the post-focal fusion image;

[0166] S43: Based on the calculation results of step S41 and step S42, the multi-focal fusion image is calculated by the following formula :

[0167] ;

[0168] The pre-focal and post-focal difference image is calculated by the following formula :

[0169] .

[0170] It should be noted that on the basis of obtaining high-lateral-resolution images by aperture rotation, in order to further improve the axial focusing quality and contrast of imaging, the present application combines the differential confocal idea to perform differential analysis and fusion on the pre-focal fusion image and the post-focal fusion image.

[0171] After aperture rotation and high-lateral-resolution fusion processing, two fusion images and Two fused images respectively reflect the imaging information of the sample 6 to be measured at the positions before and after the focal point, and the clarity distribution and detail characteristics thereof are different, thereby providing a basis for subsequent differential analysis and multi-focus fusion.

[0172] In order to measure the clarity before and after the focal point and adjust the regional weight, the clear regions are adaptively selected from the two fused images and A focal point evaluation function is constructed based on local sharpness or gradient. A Laplace operator is used to define the local clarity of the pre-focal fused image and the post-focal fused image, and then a fusion weight is constructed. If a certain region is clearer in the post-focal image, the fusion weight is larger; if a certain region is clearer in the pre-focal image, the fusion weight is larger. The pre-focal fused image and the post-focal fused image are fused to obtain a multi-focus fused image , The multi-focus fused image is clearer at each pixel position , and the side with clearer focal point contributes more according to the local focal point evaluation function, thereby obtaining the effects of focal point region expansion and fuzzy region suppression on the whole image. Finally, a differential enhancement is performed to further highlight the detail changes near the focal point, and a pre-focal / post-focal differential image is constructed.

[0173] Further, in step S5, a differential enhancement image is constructed based on the multi-focus fused image and the pre-focal / post-focal differential image. The calculation formula used is as follows:

[0174] ;

[0175] wherein, a is a differential enhancement weight coefficient, which is used to balance the degree of overall brightness and local contrast improvement.

[0176] Through the combination of the above aperture rotation multi-angle fusion algorithm and the pre-focal / post-focal differential image processing algorithm, the present application realizes the directional compression of the transverse spot at the optical level, realizes the adaptive fusion of multi-angle, high-definition regions and the comprehensive utilization of multi-focus information at the image level, and thereby obtains a final imaging result with high transverse resolution, high focal point clarity and high contrast.

[0177] It should be understood that the various forms of flow shown above can be used to reorder, add or delete steps. For example, the steps described in the present disclosure can be executed in parallel, in sequence or in different orders, as long as the desired results of the technical solutions of the present disclosure can be achieved, and the present disclosure is not limited herein. ​​​​

[0178] The foregoing detailed description has set forth various embodiments of the application via specific examples. But it is noted that this description is illustrative only and not limiting of the application's scope, which is defined by the appended claims. Changes in or substitutions of equivalents of parts and materials can be made that are within the skill of the art and as such can occur to those skilled in the art. Any such changes or substitutions are intended to fall within the scope of the application.

Claims

1. A differential confocal microscopy imaging method based on a custom aperture, implemented using a differential confocal microscopy imaging system based on a custom aperture, characterized in that: The differential confocal microscopy imaging system based on a custom aperture includes a light source, a polarizing beam splitter, a custom aperture, a quarter-wave plate, a focusing imaging module, a beam splitter, a pre-focus detection module, a post-focus detection module, and a signal processing module. The single-wavelength laser emitted from the light source is transmitted through the polarizing beam splitter, outputting p-direction linearly polarized light. This p-direction linearly polarized light is then incident on the quarter-wave plate through the custom aperture. The quarter-wave plate converts the p-direction linearly polarized light into circularly polarized light. This circularly polarized light is then focused onto the sample under test by the focusing imaging module. The circularly polarized light reflected from the sample is then focused into the sample by the focusing imaging module. The light is incident on a quarter-wave plate, which converts the circularly polarized light into linearly polarized light in the S-direction. The linearly polarized light in the S-direction is incident on a polarizing beam splitter through a custom aperture. The polarizing beam splitter reflects the linearly polarized light in the S-direction to a beam splitter mirror for splitting, resulting in a first beam and a second beam. The first beam is processed by a front-focus detection module to obtain a front-focus image, and the second beam is processed by a back-focus detection module to obtain a back-focus image. The signal processing module outputs a differential enhancement image based on the front-focus and back-focus images of the custom aperture relative to the objective optical axis in the front-focus detection module at different circumferential rotation angles. The method specifically includes the following steps: S1: Calculate the optimal amplitude distribution parameters of the custom aperture based on the entrance pupil of the objective lens in the focusing imaging module, and adjust the custom aperture based on the calculation results; Step S1 specifically includes the following steps: S11: Divide the entrance pupil of the objective lens in the focusing imaging module into N sub-regions and construct the aperture mask function to be optimized. : ； ； in, Let be the amplitude distribution parameters of the i-th sub-region. , For the indicator function of the i-th sub-region, Normalized polar coordinates defined based on the entrance pupil plane of the objective lens. For the objective lens at half-open angle, The transmittance of the objective lens; S12: In the three-dimensional PSF corresponding to the focused spot on the sample under test, the focused spot is located in the XOY plane of the three-dimensional PSF. The full width at half maximum (FWHM) of the normalized intensity curve corresponding to the line containing the minimum width of the focused spot in the y-axis direction is defined as the lateral resolution evaluation index, and the effective light transmission of the custom aperture is defined as the light transmission energy index. Based on the lateral resolution evaluation index and the light transmission energy index, an objective function is constructed. : ； ； ； ; in, It serves as a metric for evaluating horizontal resolution. The indicator is the light transmission energy. This is the set of amplitude distribution parameters for each sub-region. and All are weighting coefficients. , , These are all reference values ​​corresponding to the diameter of a circular hole. Normalized polar coordinates The corresponding light intensity distribution of the focused spot on the sample under test. The light intensity distribution of the focused spot on the sample under test corresponding to the normalized polar coordinates (0, 0); S13: Randomly generate an initial set of amplitude distribution parameters, and adjust the amplitude distribution parameters of the custom aperture based on a genetic algorithm to obtain the optimal set of amplitude distribution parameters; S2: Rotate the adjusted custom aperture circumferentially relative to the objective optical axis in the focusing imaging module to acquire front-focus and back-focus images at different rotation angles; S3: The image fusion module fuses the foreground and background images at different rotation angles to obtain foreground fused images and background fused images respectively. S4: The multifocal fusion and difference module constructs multifocal fusion images and pre-focus and post-focus difference images based on the pre-focus fusion image and the post-focus fusion image; S5: The differential enhancement module constructs a differential enhancement image based on the multi-focus fused image and the front-focus and back-focus differential images.

2. The differential confocal microscopy imaging method based on a custom aperture according to claim 1, characterized in that: The focusing imaging module includes an objective lens and an objective lens driver. The objective lens driver drives the objective lens to move relative to the surface of the sample to be tested, thereby scanning the sample.

3. The differential confocal microscopy imaging method based on a custom aperture according to claim 1, characterized in that: The front-focus detection module and the back-focus detection module have the same structure, both including a focusing lens, a pinhole, and an area array camera arranged sequentially along the optical path; wherein, the pinhole of the front-focus detection module is located in front of the focal length of the focusing lens of the front-focus detection module, and the pinhole of the back-focus detection module is located behind the focal length of the focusing lens of the back-focus detection module.

4. The differential confocal microscopy imaging method based on a custom aperture according to claim 1, characterized in that: The signal processing module includes an image fusion module, a multifocal fusion and difference module, and a difference enhancement module. The image fusion module is used to fuse the foreground and background images at different rotation angles to obtain foreground and background fused images, respectively. The multifocal fusion and difference module is used to construct multifocal fused images and foreground and background difference images based on the foreground and background fused images. The difference enhancement module is used to construct difference enhanced images based on the multifocal fused images and foreground and background difference images.

5. The differential confocal microscopy imaging method based on a custom aperture according to claim 1, characterized in that: Step S3 specifically includes: S31: Obtain the focal foreground image at the k-th rotation angle, and calculate the focal foreground image at the k-th rotation angle. gradient magnitude : ； ； in, This represents the intensity distribution of light reflected from the sample under test. Let be the point spread function of the focus front detection module at the k-th rotation angle. This is the convolution operator; Noise term; S32: Combining the aperture mask function to be optimized in the Cartesian coordinate system corresponding to the k-th rotation angle, construct the unnormalized weights corresponding to the focal front image at the k-th rotation angle: ； in, The unnormalized weights are the focal front images corresponding to the k-th rotation angle; S33: Based on the calculation results of step S32, obtain the fusion weights corresponding to the focal foreground image at the k-th rotation angle. : ； in, To prevent positive numbers with a denominator of zero, The unnormalized weights are the focal front images corresponding to the j-th rotation angle, and K is the total number of rotation angles; S34: Based on the calculation results of step S33, calculate the high lateral resolution fused image in the focal front using the following formula. : ； Where K is the total number of rotation angles; S35: Based on the calculation results of step S34, calculate the focus-front fused image using the following formula: ； Where D() is the processing operator; S36: Replace the foreground image at the kth rotation angle with the back-focus image at the kth rotation angle, and repeat steps S31-S35 to obtain the back-focus fused image.

6. The differential confocal microscopy imaging method based on a custom aperture according to claim 5, characterized in that: In step S35, the processing operator is specifically a bilateral filtering operator, a wavelet denoising operator, or a contrast-limited adaptive histogram equalization operator.

7. The differential confocal microscopy imaging method based on a custom aperture according to claim 5, characterized in that: Step S4 specifically includes the following steps: S41: The Laplacian operator is used to calculate the local sharpness of the front-focus fused image and the back-focus fused image respectively: ； in, To improve the local sharpness of the focus-front fused image, For the local sharpness of the post-focus fused image; S42: Based on the calculation results of step S41, construct the fusion weights for the foreground fused image and the background fused image respectively using the following formula: ； in, The fusion weights are for the focus-front fusion images. The fusion weights are used for the post-focus fused images; S43: Based on the calculation results of steps S41 and S42, calculate the multifocal fused image using the following formula. : ； The pre-focus and post-focus difference images are calculated using the following formula. : 。 8. The differential confocal microscopy imaging method based on a custom aperture according to claim 7, characterized in that: In step S5, a differential enhancement image is constructed based on the multi-focus fused image and the pre-focus and post-focus difference images. The calculation formula used is: ； in, The weighting coefficients are for differential enhancement.

Images