Multi-focus structured light reconstruction algorithm based on Hough transform

Through Hough transformation and digital pinhole technology, the Gaussian spot center is directly positioned on the original sequence diagram, solving the low signal-to-noise ratio problem under the influence of noise in traditional algorithms, achieving efficient multi-focus structured light reconstruction, and simplifying the process And improves accuracy.

CN119067851APending Publication Date: 2024-12-03SOUTH CHINA NORMAL UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411181581.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-27
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The traditional multifocal structured light-illuminating microscope reconstruction algorithm is susceptible to noise when processing sample images, resulting in a low signal-to-noise ratio and requires additional shooting to calibrate the light field to locate the center of the spot, which is cumbersome.

Method used

Using a multi-focus structured light reconstruction algorithm based on Hough transform, noise is removed through mean filtering and binarization, the Gaussian spot center is positioned using Hough circle detection, digital pinholes are directly applied, and resolution is improved through RL deconvolution , eliminate the frequency domain conversion step.

Benefits of technology

The signal-to-noise ratio of the reconstructed image is improved, the experimental process is simplified, the possibility of reconstruction errors is reduced, and the operation is simple and efficient.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119067851A_ABST
    Figure CN119067851A_ABST
Patent Text Reader

Abstract

The invention discloses a multi-focus structured light reconstruction algorithm based on Hough transform, and specifically, a Gaussian spot center is directly positioned in a spatial domain by using a Hough transform circle detection method, and then pixel repositioning, sequence image fusion and RL deconvolution are performed to obtain a super-resolution image. In a traditional algorithm, a high-quality excitation light field needs to be additionally shot to assist in positioning, data needs to be further processed in a frequency domain, the process is complex, and the phenomenon that the center of a Gaussian light spot is positioned mistakenly easily occurs. According to the method, each Gaussian light spot in the original data is accurately estimated by using the Hough transform circle detection method, an additional calibration light field is not needed, and compared with a traditional algorithm, the process is simple, and the signal-to-noise ratio of the reconstructed image is enhanced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of optical microscopic imaging, and in particular to a multi-focus structured light reconstruction algorithm based on Hough transform. Background Art

[0002] Optical super-resolution microscopy is one of the important tools in life science research. One of the most prominent methods in this field is structured light microscopy (SIM). Multi-focus structured light microscopy (MSIM) is one of the technical means of structured light microscopy. It has great application potential due to its larger imaging range and higher imaging depth. In MSIM, the structured light is no longer the stripe structured light in traditional SIM, but a plurality of diffraction-limited Gaussian light spot arrays. MSIM can also be regarded as a parallel version of image scanning (confocal) microscopy ISM.

[0003] The reconstruction algorithm is also an important step in obtaining super-resolution images. In the traditional algorithm, a dot light field is obtained by shooting a thick sample. The lattice vector, offset vector, and motion vector are used to obtain the light field to reconstruct the digital pinhole. The digital pinhole is applied to the sample image sequence to obtain a pixel repositioned image sequence. The pixel repositioned image sequence is fused, and RL deconvolution is performed to obtain a super-resolution image.

[0004] The lattice vector is obtained by using the regular distribution of the Fourier spectrum of the periodic signal. The noise signal will affect the harmonic component after the Fourier transform, so the signal-to-noise ratio of the light field will affect the estimation of the lattice vector, resulting in errors in locating the center of the light spot. Before using the traditional algorithm for reconstruction, it is necessary to additionally shoot a high-quality calibration light field when relocating pixels. In addition, the traditional algorithm processes both the noise signal and the sample signal at the same time, resulting in a poor signal-to-noise ratio of the reconstructed image. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention aims to provide a multi-focus structured light reconstruction algorithm based on Hough transform, so that the signal-to-noise ratio of the sample image can be improved without the need for additional shooting of the calibration light field.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0007] A multi-focus structured light reconstruction algorithm based on Hough transform, the method comprising the following steps:

[0008] 1) Obtain the original sequence images, perform mean filtering preprocessing on each original sequence image, the purpose of which is to remove noise and smooth the Gaussian spot, and then perform binarization processing by mean filtering.

[0009] 2) The Hough circle detection process is to extract the edge of the preprocessed image, use the Canny operator to extract the edge, and obtain the binary image of edge detection. Use the Sobel operator to obtain the gradient of the binary image and determine the circumference line, where the gradient of the circumference is its normal. In the two-dimensional Hough space, draw the gradient straight lines of all figures. The larger the coordinate point, the more times the straight lines intersect at this point, that is, the more likely it is the center of the circle. Set the threshold of the cumulative sum, and the point in the Hough space where the cumulative sum is greater than the threshold corresponds to the center of the circle.

[0010] 3) Traverse the position of each Gaussian spot in the image, and generate a digital pinhole at the position corresponding to the center of the Gaussian spot on the new blank image, and apply the digital pinhole to the original image sequence.

[0011] 4) All images are superimposed to obtain a reconstructed image with 1.4 times the resolution, and then RL deconvolution is performed to obtain an image with 2 times the resolution.

[0012] It should be noted that there is no need for additionally photographed calibration light fields to assist in positioning the excitation dot matrix, which saves tedious light field acquisition steps and simplifies the experimental process.

[0013] It should be noted that a reasonable threshold can be set to avoid the digital pinhole being mistakenly positioned at a noise point. Signals greater than the threshold are determined to be true sample signals, and signals less than the threshold are determined to be noise signals and discarded.

[0014] It should be noted that since the digital pinhole is only applied to the located sample signal position and the noise signal is discarded after algorithm judgment, the signal-to-noise ratio of the reconstructed image of this algorithm is better than that of the traditional algorithm.

[0015] It should be noted that this algorithm is a pure spatial domain reconstruction process and does not require Fourier transform to convert into frequency domain spatial positioning lattice vectors. Compared with the traditional algorithm process, this algorithm operation is simpler and reduces the possibility of reconstruction errors.

[0016] It should be noted that the number of RL deconvolution iterations is ten.

[0017] The beneficial effect of the present invention is that the present invention provides a multi-focus structured light reconstruction algorithm based on Hough transform, which directly locates the center of the excitation spot on the original sequence image through the technical means of Hough transform circle detection, does not require additional calibration light field, can improve the signal-to-noise ratio of the reconstructed image, and eliminates the step of converting frequency domain estimation, thereby simplifying the reconstruction process and improving the accuracy of reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is the algorithm flow chart of the present invention;

[0019] Figure 2 a is a schematic diagram of the scanning mode of the multi-focus excitation dot array in the multi-focus wide-field super-resolution microscopy imaging method required to obtain the original sequence image;

[0020] Figure 2 b is an experimental diagram of the multi-focus excitation dot matrix in the multi-focus wide-field super-resolution microscopy imaging method required to obtain the original sequence image;

[0021] Figure 3 is the program interface of the present invention;

[0022] Figure 4 a is a line simulation wide-field image when the number of frames of multi-focus scanning simulated by computer is 224 in Example 1;

[0023] Figure 4 b is a line simulation diagram of the conventional multi-focus pixel repositioning reconstruction algorithm when the number of scanning frames is 224 frames through computer simulation in Example 1;

[0024] Figure 4 c The line simulation diagram of the present invention is used when the number of scanning frames is 224 frames through computer simulation in Example 1;

[0025] Figure 5 a is a single particle wide-field image when the experimental scanning frame number is 224 frames in Example 2.

[0026] Figure 5 b is a single particle reconstruction image using the present invention when the experimental scanning frame number is 224 frames in Example 2;

[0027] Figure 5 c is a single particle super-resolution image obtained by using the RL deconvolution method of the present invention when the experimental scanning frame number is 224 in Example 2;

[0028] Figure 5 d is the value of Figure 5 Gaussian function fitting curves of the FWHW of the selected single particles in a, b, c. DETAILED DESCRIPTION

[0029] The specific implementation modes of the present invention are further described below in conjunction with the accompanying drawings, but the present invention is not limited thereto.

[0030] The present invention is a multi-focus structured light reconstruction algorithm based on Hough transform, and the algorithm comprises the following steps:

[0031] 1) Obtain the original sequence images, perform mean filtering preprocessing on each original sequence image, the purpose of which is to remove noise and smooth the Gaussian spot, and then perform binarization processing by mean filtering.

[0032] 2) The Hough circle detection process is to extract the edge of the preprocessed image, use the Canny operator to extract the edge, and obtain a binary image for edge detection. Use the Sobel operator to obtain the gradient of the binary image and determine the circumference line, where the gradient of the circumference is its normal. In the two-dimensional Hough space, draw the gradient straight lines of all figures. The larger the cumulative sum value at a certain coordinate point, the more times the straight lines intersect at this point, that is, the more likely it is the center of the circle. Set a cumulative sum threshold, and the point in the Hough space where the cumulative sum is greater than the threshold corresponds to the center of the circle.

[0033] 3) Traverse the position of each Gaussian spot in the image, and generate a digital pinhole at the position corresponding to the center of the Gaussian spot on the new blank image, and apply the digital pinhole to the original image sequence.

[0034] 4) All images are superimposed to obtain a reconstructed image with 1.4 times the resolution, and then RL deconvolution is performed to obtain an image with 2 times the resolution.

[0035] It should be noted that there is no need for additionally photographed calibration light fields to assist in positioning the excitation dot matrix, which saves tedious light field acquisition steps and simplifies the experimental process.

[0036] It should be noted that a reasonable threshold can be set to avoid the digital pinhole from being mistakenly positioned at a noise point. Signals greater than the threshold are determined to be true sample signals, and signals less than the threshold are determined to be noise signals and discarded.

[0037] It should be noted that since the digital pinhole is only applied to the located sample signal position and the noise signal is discarded after algorithm judgment, the signal-to-noise ratio of the reconstructed image of this algorithm is better than that of the traditional algorithm.

[0038] It should be noted that this algorithm is a pure spatial domain reconstruction process and does not require Fourier transform to convert into frequency domain spatial positioning lattice vectors. Compared with the traditional algorithm process, this algorithm operation is simpler and reduces the possibility of reconstruction errors.

[0039] It should be noted that the number of RL deconvolution iterations is ten.

[0040] The beneficial effect of the present invention is that the present invention provides a multi-focus structured light reconstruction algorithm based on Hough transform, which directly locates the center of the excitation spot on the original sequence diagram through the technical means of Hough transform circle detection, does not require additional calibration of the light field, can improve the signal-to-noise ratio after reconstruction, and omits the step of converting frequency domain estimation, thereby simplifying the reconstruction process and improving the accuracy of reconstruction.

[0041] Example 1

[0042] The MSIM formula is:

[0043]

[0044] · represents dot product, and * represents convolution. is the sample structure distribution, is the image on the detection surface, The excitation mode can be regarded as is the dot matrix illumination mode, i.e. the excitation point spread function of the system is convolved on the illumination dot matrix, is the detection point spread function of the system.

[0045] Taking the excitation wavelength of 480nm, the detection wavelength of 520nm, and the numerical aperture NA of the system objective as 1.45 as an example, the full width at half maximum (FWHM) of the point spread function is calculated, FWHM = 0.61lamda / NA, so as to obtain the standard deviation σ of the Gaussian distribution corresponding to the PSF = FWHM / 2.35, and then according to the Gaussian function PSF(x,y) = A exp[-((x-x0) 2 )+(y-y0) 2 / (2·σ 2 )] calculate the excitation PSF and radiation PSF respectively.

[0046] The image line diagram simulates a Gaussian spot with a single excitation wavelength limited by diffraction and a real sample with a size of 256×256. The line diagram has ten pairs of lines with an even spacing increasing from 60nm to 330nm from left to right.

[0047] Then, according to the MSIM formula, a multi-focus experiment with 224 frames was performed to obtain simulated data, and the simulated data were superimposed to obtain a wide-field image. Figure 5 a; Use traditional algorithms to obtain the reconstructed image of the simulated data. Figure 5 b; using the reconstruction diagram of the present invention to simulate the data, Figure 5 c.

[0048] It is obvious from the figure that the present invention can best separate line pairs with a spacing of 150nm, while the conventional algorithm can only best separate line pairs with a spacing of 180nm. Compared with the conventional algorithm under the simulation, the effect of using the algorithm of the present invention is not only improved in resolution, but also in image signal-to-noise ratio.

[0049] Example 2

[0050] Use a single particle sample in the built visible light multi-focus optical path, with an excitation wavelength of 488nm, a radiation wavelength of 520nm, and a scanning mode such as Figure 2 a, dot matrix lighting mode such as Figure 2 b, The scanning frame number is 224 frames, and 224 frames of original sequence data are collected through synchronous control.

[0051] After obtaining the original sequence data, use the Figure 3 In the program shown, sen sets the sensitivity of determining whether it is a Gaussian spot of the sample signal, which is generally set between 0.8 and 0.9. The threshold value is the binarization threshold in the pre-processed image, which is used to filter out noise signals. It is generally set between 0 and 0.1. The iter value is set to the number of scan frames. The SIGMA value sets the radius of Gaussian spots above which they are retained, which is used to filter out Gaussian spots of noise signals. Then enter the folder and file name where the processed data needs to be saved, as well as the file location where the processed data needs to be saved. Decon selects whether to use RL deconvolution processing, the lamda value is set to the excitation wavelength, the NA value is set to the numerical aperture of the objective lens, and the scale value is set to the camera pixel size. Click to start processing, and after the processing progress bar is finished reading, you can see three pictures. They are as follows: Figure 5 shown Figure 5 a Wide-field single particle image, Figure 5 b Super-resolution image reconstructed by the algorithm of the present invention, Figure 5 c is the super-resolution image after RL deconvolution processing. Figure 5 d represents the FWHW of the selected single particle in wide field, after reconstruction, and after deconvolution. The FWHW of the wide field single particle is 300nm, the FWHW of the reconstructed image is 215nm, and the FWHW of the deconvolution is 145nm.

[0052] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. A multi-focus structured light reconstruction algorithm based on Hough transform, Features: The algorithm comprises the following steps: S1: Get the original sequence graph, perform mean filter preprocessing on each original sequence graph, and then perform binarization processing on the mean filter; S2: Perform Hough circle detection, including edge extraction of the image preprocessed in S1 to obtain a binary image, using the Sobel operator to obtain the gradient of the binary image, and determining the circumference line. In the two-dimensional Hough space, the gradient straight lines of all the figures are drawn to determine the center of the circle; S3: traverse the position of each Gaussian spot in the image, and generate a digital pinhole at the position corresponding to the center of the Gaussian spot on the new blank image, and apply the digital pinhole to the original image sequence; S4: Image sequence fusion, superimpose all images to obtain a reconstructed image with 1.4 times the resolution, and then use RL deconvolution to obtain an image with 2 times the resolution.

2. According to the multi-focus structured light reconstruction algorithm based on Hough transform in claim 1, Features: The determination of the circle center in S2 includes setting a threshold of the accumulated value of the coordinate point in the two-dimensional Hough space, recording the accumulated value of the coordinate point in the two-dimensional Hough space, and when the accumulated value of the coordinate point is greater than the threshold of the accumulated value of the coordinate point, the coordinate point is the circle center.

3. According to the multi-focus structured light reconstruction algorithm based on Hough transform in claim 1, Features: The S3 also includes setting a threshold; a signal greater than the threshold is determined as a true signal of the sample, and a signal less than the threshold is determined as a noise signal and discarded.

4. The multi-focus structured light reconstruction algorithm based on Hough transform according to claim 3, Features: Since the digital pinhole is only applied to the located sample signal position and the noise signal is discarded after algorithm judgment, the signal-to-noise ratio of the reconstructed image of this algorithm is better than that of the traditional algorithm.

5. The multi-focus structured light reconstruction algorithm based on Hough transform according to claim 1, Features: The number of RL deconvolution iterations in S4 is ten.