A method for decoupling reflection information of dark-field confocal microscopy images

Abstract

A method for decoupling reflection information of a dark field confocal microscopic image, which relates to a method for decoupling reflection information of a microscopic image. The present invention aims to solve the problem that in the dark field confocal imaging process, the reflection signal that is not completely filtered out is mixed in the scattered signal, resulting in uneven brightness of the acquired image, large background noise, and further interference with the observation of sub-surface defects. The present invention uses the bright field image as a priori, calculates the priori gradient template, extracts the dark field scattered signal through the priori gradient template, and filters out the reflection information noise. The present invention belongs to the field of image processing technology.

Description

Technical Field

[0001] The invention relates to a microscopic image reflection information decoupling method, belonging to the technical field of image processing. Background Art

[0002] Subsurface defects are the key to limiting the damage threshold of optical components. Imaging of subsurface defects is conducive to exploring the characterization of subsurface defects and the physical process that causes laser damage. Optical scattering imaging is one of the most promising detection methods. Laser confocal microscopy has become a non-destructive testing technology for obtaining subsurface information due to its higher lateral resolution and unique axial tomography capabilities. However, according to bright-field and dark-field confocal microscopy experiments of subsurface defects of optical components, the intensity of subsurface scattering signals is only one-tenth of the intensity of incident light. Therefore, due to the strong suppression of reflected light on the surface of optical components, traditional bright-field confocal microscopy is not very effective. Microscopy cannot obtain information on sub-surface defects. Dark-field microscopy collects scattered signals from sub-surface defects for imaging by shielding strong reflected signals from the surface of optical components. Confocal microscopy uses point illumination and point detection, has axial tomographic imaging capabilities, and can achieve three-dimensional detection of sub-surface defects. Dark-field confocal microscopy combines the advantages of the two imaging methods and has become an important method for non-destructive detection of sub-surface defects. However, in the dark-field confocal imaging process, the reflected signals that are not completely filtered out are mixed in the scattered signals, resulting in uneven brightness of the acquired image and large background noise, which interferes with the observation of sub-surface defects. Summary of the invention

[0003] The present invention aims to solve the problem that during dark-field confocal imaging, reflected signals that are not completely filtered out are mixed in scattered signals, resulting in uneven brightness of the acquired image, large background noise, and further interference with sub-surface defect observation. A method for decoupling reflected information of dark-field confocal microscopic images is proposed.

[0004] The technical solution adopted by the present invention to solve the above-mentioned problem is: the steps of the present invention include:

[0005] Step 1: Obtain bright field and dark field confocal microscopic images of surface defects of optical components through a dark field confocal microscopic system;

[0006] Step 2: Establish a mathematical model of the image acquired by the dark-field confocal microscopy system;

[0007] Step 3, input the dark field confocal sub-surface defect image D and the bright field confocal image L at the same depth;

[0008] Step 4, normalize the grayscale values ​​of the dark field confocal sub-surface defect image D and the bright field confocal image L at the same depth;

[0009] Step 5: Calculate the bright field image prior A;

[0010] Step 6: Calculate the sum of squares of grayscale value differences of the eight neighborhoods of each pixel (i, j) in the bright field confocal image L at the same depth, i.e., the gradient value grad(L(i, j));

[0011] Step 7: For each pixel (i, j) of the dark field confocal sub-surface defect image D, A is applied as a weighting coefficient to obtain a repair value;

[0012] Step 8: Output dark field confocal decoupled image

[0013] Furthermore, for each pixel (i, j) of the bright field confocal image L at the same depth:

[0014]

[0015] In formula (1), α controls the influence of the sum of squares of adjacent pixel differences on the prior value, and β controls the nonlinear change of the image prior value. The convolution template is applied to the dark field image D:

[0016]

[0017] In formula (2), λ controls the weight between the original pixel value and the restored value.

[0018] Furthermore, the mathematical model of the image acquired by the dark field confocal microscope system in step 2 is:

[0019] f(x,y)=n(x,y)+t(x,y)(3),

[0020] In formula (3), n(x, y) represents the inherent noise of the system, which is expressed as additive noise, and t(x, y) represents the imaging of the missed reflection signal.

[0021] Furthermore, the main sources of inherent noise in dark-field confocal systems include: Poisson scattering noise caused by the random process of counting collected photons on the photodetector, Poisson noise P(λ P ) P represents the average dark flux, and the inherent noise is converted into Gaussian noise for noise reduction. The inherent noise of the dark field confocal image is a Poisson-Gaussian mixed noise. The Poisson and Gaussian random processes are independent. Assume that the noisy data at the observation point (x, y) is The noise-free data is m(x,y), and the relationship between the two is:

[0022]

[0023] In formula (4), α represents the gain constant, ρ□P(λ s ) Strength parameter λ s =m(x,y) Poisson random process, for the observation point:

[0024]

[0025] In formula (5), g represents the gray value, and parameter λ s is directly related to the number of photons reaching each pixel and depends on the exposure time used to acquire the image, v□N(0,σ 2 ) is a normal distribution with zero mean:

[0026]

[0027] Combining formula (4) and (6), we can get:

[0028]

[0029] Furthermore, in the dark field confocal image noise reduction process, the Poisson-Gaussian mixed noise is first converted into Gaussian noise with unit variance through the generalized Anscombe variance stabilization transformation. The process is:

[0030]

[0031] In formula (8), σ = 1, μ = 0, then the Gaussian noise is denoised, and the denoised image is obtained by inverse transformation. The optimal inverse of Anscombe transform is applied, and the closed form approximation is:

[0032]

[0033] In formula (9), D represents the data of the denoised image, and I(D) is a family of inverse transforms whose closed form approximation is obtained by the closed form approximation of I~(D). The estimation is improved by applying the exact unbiased inverse transform to minimize the transformation deviation.

[0034] The beneficial effects of the present invention are:

[0035] 1. The present invention uses the bright field image as a priori, calculates the prior gradient template, extracts the dark field scattering signal through the prior gradient template, and filters out the reflection information noise;

[0036] 2. The present invention utilizes the idea of ​​gradient prior filtering, and calculates the gradient of the bright field confocal image at the same depth as a convolution template and applies it to the dark field image to decouple the reflection information;

[0037] 3. The present invention extracts a bright field confocal image prior A of the designed sample and applies it to the dark field confocal image D. During the imaging process of the dark field confocal image, the background noise caused by the leakage of reflected light is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is a schematic diagram of the dark-field confocal image decoupling algorithm;

[0039] Figure 2 It is the dark field image reflection information noise model;

[0040] Figure 3 Schematic diagram of sample design in the embodiment, wherein Figure 3 (a) is a schematic diagram of digital and rectangular structure design. Figure 3 (b) is a schematic diagram of the rectangular array structure design. Figure 3 (c) is a schematic diagram of the sample as a whole;

[0041] Figure 4 is a schematic diagram of the result of reflective wide-field imaging of the sample of the embodiment, Figure 4 (a) is a schematic diagram of the imaging results of a rectangular array. Figure 4 (b) is a schematic diagram of the overall imaging results of the sample. Figure 4 (c) is a schematic diagram of the imaging result of a 4-digital rectangular structure;

[0042] Figure 5 This is a schematic diagram of the decoupling results of the reflection information of the dark field confocal image. Figure 5 (a) is a bright field confocal image, Figure 5 (b) is a dark field confocal image. Figure 5 (c) is a schematic diagram of the processing structure. Figure 5 (d) Yes Figure 5 (b) and Figure 5 (c) Schematic diagram of the processing structure. DETAILED DESCRIPTION

[0043] Specific implementation method 1: Combination Figure 1 and Figure 2 This embodiment describes a method for decoupling reflection information of a dark field confocal microscopic image, which includes the following steps:

[0044] Step 1: Obtain bright field and dark field confocal microscopic images of surface defects of optical components through a dark field confocal microscopic system;

[0045] Step 2: Establish a mathematical model of the image acquired by the dark-field confocal microscopy system;

[0046] Step 3, input the dark field confocal sub-surface defect image D and the bright field confocal image L at the same depth;

[0047] Step 4, normalize the grayscale values ​​of the dark field confocal sub-surface defect image D and the bright field confocal image L at the same depth;

[0048] Step 5: Calculate the bright field image prior A;

[0049] Step 6: Calculate the sum of squares of grayscale value differences of the eight neighborhoods of each pixel (i, j) in the bright field confocal image L at the same depth, i.e., the gradient value grad(L(i, j));

[0050] Step 7: For each pixel (i, j) of the dark field confocal sub-surface defect image D, A is applied as a weighting coefficient to obtain a repair value;

[0051] Step 8: Output dark field confocal decoupled image

[0052] In the actual dark field imaging process, the imaging result can be regarded as the coupling of two groups of layers that have no correlation with each other, including the imaging result of the sub-surface scattering signal and the imaging result of the surface reflection signal. The imaging result of the reflection signal includes the specular reflection component and the diffuse reflection component. The total imaging result is:

[0053] f(x,y)=S(x,y)+R s (x,y)+R d (x,y)(10),

[0054] In formula (10), f(x, y) represents the dark field imaging result, S(x, y) represents the sub-surface scattering signal imaging result, and R s (x, y) represents the imaging result of the surface specular reflection signal, R d (x, y) represents the imaging result of the surface diffuse reflection signal;

[0055] The diffuse reflection component and the sub-surface scattering signal are very similar and cannot be distinguished according to the spatial or frequency domain characteristics of the image block. Therefore, the distinction and separation of the surface diffuse reflection component and the sub-surface scattering signal component are not considered. Assuming that the surface reflection noise is mainly smooth mirror reflection, the dark field imaging result can be written as:

[0056] f(x,y)=S(x,y)+R s (x,y).

[0057] Specific implementation method 2: Combination Figure 1 and Figure 2 This embodiment is described. For each pixel (i, j) of a bright field confocal microscopic image L at the same depth, a dark field confocal microscopic image reflection information decoupling method described in this embodiment is:

[0058]

[0059] In formula (1), α controls the influence of the sum of squares of adjacent pixel differences on the prior value, and β controls the nonlinear change of the image prior value. The convolution template is applied to the dark field image D:

[0060]

[0061] In formula (2), λ controls the weight between the original pixel value and the restored value.

[0062] Specific implementation method three: Combination Figure 1 and Figure 2 To illustrate this embodiment, in step 2 of the dark field confocal microscopy image reflection information decoupling method described in this embodiment, a mathematical model of an image acquired by a dark field confocal microscopy system is established as follows:

[0063] f(x,y)=n(x,y)+t(x,y)(3),

[0064] In formula (3), n(x, y) represents the inherent noise of the system, which is expressed as additive noise, and t(x, y) represents the imaging of the missed reflection signal.

[0065] Specific implementation method four: Combination Figure 1 and Figure 2 The present embodiment is described. The main sources of the inherent noise of the dark field confocal system of the dark field confocal microscopic image reflection information decoupling method described in the present embodiment include: Poisson scattering noise caused by the random process of counting collected photons on the photodetector, Poisson noise P(λ P ) P represents the average dark flux, and the inherent noise is converted into Gaussian noise for noise reduction. The inherent noise of the dark field confocal image is a Poisson-Gaussian mixed noise. The Poisson and Gaussian random processes are independent. Assume that the noisy data at the observation point (x, y) is The noise-free data is m(x,y), and the relationship between the two is:

[0066]

[0067] In formula (4), α represents the gain constant, ρ□P(λ s ) Strength parameter λ s =m(x,y) Poisson random process, for the observation point:

[0068]

[0069] In formula (5), g represents the gray value, and parameter λ s is directly related to the number of photons reaching each pixel and depends on the exposure time used to acquire the image, v□N(0,σ 2 ) is a normal distribution with zero mean:

[0070]

[0071] Combining formula (4) and (6), we can get:

[0072]

[0073] Specific implementation method five: Combination Figure 1 and Figure 2 This embodiment is described. In the dark field confocal microscopy image reflection information decoupling method described in this embodiment, during the dark field confocal image noise reduction process, the Poisson-Gaussian mixed noise is first converted into Gaussian noise with unit variance through the generalized Anscombe variance stable transformation, and the process is:

[0074]

[0075] In formula (8), σ = 1, μ = 0, then the Gaussian noise is denoised, and the denoised image is obtained by inverse transformation. The optimal inverse of Anscombe transform is applied, and the closed form approximation is:

[0076]

[0077] In formula (9), D represents the data of the denoised image, and I(D) is a family of inverse transforms whose closed form approximation is obtained by the closed form approximation of I~(D). The estimation is improved by applying the exact unbiased inverse transform to minimize the transformation deviation.

[0078] Example

[0079] The mask pattern is designed using L-Edit software, and the laser direct writing technology is used to process samples with a certain structure on a 130-micron-thick Sitai microscope cover glass substrate. The process flow is as follows: spin-coat the photoresist on the substrate that has been pre-treated by cleaning and drying, and then pre-bake to volatilize the organic solvent in the photoresist and make it solid. Then, photolithography technology is used to etch the designed mask pattern on the photoresist, and the photoresist in the exposed area is removed by development and then cleaned to obtain the desired sample, such as Figure 3 (c) shows the simulation of a sub-surface structure, where the laser-written structure is placed facing the opposite direction of the objective lens.

[0080] Figure 4 This is the result of the sample being imaged by reflective wide-field imaging. Figure 4 (b) is the overall imaging result of the sample (objective lens is 10×, NA=0.13). The sample consists of a rectangular array structure in the upper part and a digital and rectangular structure in the lower part. Figure 4 (a) is the imaging result of a rectangular array (objective lens is 100×, NA=0.90), the width of the rectangle is 3 μm, the length is 10 μm, and the distance is 5 μm, which is used to simulate the structure of sub-surface defects such as pits. Figure 4 (c) 4 is the imaging result of a digital rectangular structure (objective lens is 50×, NA=0.50). The rectangle is 3 microns wide and 30 microns long, which is relatively large and is used to locate the structure in the field of view.

[0081] Through the confocal microscope, the subsurface bright field image and dark field image of the prepared sample are obtained. It is particularly important to note that due to the tomographic capability of the confocal microscope, the depths corresponding to the acquired bright field image and dark field image should be consistent. The bright field image is given a suitable coefficient to reduce it to the noise level, and the coupled reflection information is removed through the algorithm. Figure 5 As shown, Figure 5 (a) Figure 5 (b) are the bright field confocal image and dark field confocal image of the sample at the same depth. The presence of reflected signals that are not completely filtered out by the aperture results in large background noise in the dark field confocal image. Figure 5 (c) is the dark field image processed by the gradient filtering prior method. It is observed that the overall background noise of the image is reduced. Figure 5 (d) is the result of subtracting the algorithm-processed image from the dark field confocal image. It can be seen that it is basically background noise, and the loss of details is very small. The image processing effect is evaluated by the signal-to-noise ratio (SNR). The average value of the image brightness is taken as the signal s, and the standard deviation of the image is taken as the noise n, which is defined as:

[0082] SNR = 20log 10 (s / n),

[0083] The SNR of the original dark-field confocal image is 17.30, and the SNR of the image after decoupling processing is 18.55, which is 7.23% higher than the original image.

[0084] The above is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent replacement and improvement made to the above embodiments without departing from the content of the technical solution of the present invention, based on the technical essence of the present invention, within the spirit and principles of the present invention, still fall within the protection scope of the technical solution of the present invention.

Claims

1. A method for decoupling reflection information of dark field confocal microscopy images, characterized in that: The steps of the dark field confocal microscopy image reflection information decoupling method include: Step 1: Obtain bright field and dark field confocal microscopic images of surface defects of optical components through a dark field confocal microscopic system; Step 2: Establish a mathematical model of the image acquired by the dark-field confocal microscopy system; Step 3, input the dark field confocal sub-surface defect image D and the bright field confocal image L at the same depth; Step 4, normalize the grayscale values ​​of the dark field confocal sub-surface defect image D and the bright field confocal image L at the same depth; Step 5: Calculate the sum of squares of grayscale value differences of the eight neighborhoods of each pixel (i, j) in the bright field confocal image L at the same depth, i.e., the gradient value grad(L(i, j)); Step 6: Calculate the bright field image prior A; Step 7: For each pixel (i, j) of the dark field confocal sub-surface defect image D, A is applied as a weighting coefficient to obtain a repair value; Step 8: Output dark field confocal decoupling image For each pixel (i, j) of the bright field confocal image L at the same depth: In formula (1), α controls the influence of the sum of squares of adjacent pixel differences on the prior value, β controls the nonlinear change of the image prior value, and x mid is the 8-neighborhood gradient value of pixel (i, j), and the bright field image prior A is applied to the dark field image D: In formula (2), λ controls the weight between the original pixel value and the restored value; The mathematical model of the image acquired by the dark field confocal microscopy system in step 2 is: f(x,y)=n(x,y)+t(x,y) (3), In formula (3), n(x, y) represents the inherent noise of the system, which is expressed as additive noise, and t(x, y) represents the imaging of the missed reflection signal.

Images