A multi-target automatic focusing method for holographic imaging in complex environment

Abstract

The application discloses a kind of multi-target automatic focusing methods for holographic imaging under complex environment, comprising: step 1, the original holographic image is carried out Gaussian attenuation processing of background light source;Step 2, the original holographic image is carried out full image reconstruction, obtains corresponding reconstructed image data set;Step 3, filtering and binarization are carried out, and corresponding binarization image data set is obtained;Step 4, the analysis of binarization image data connected component is carried out, and the centroid information corresponding to binarization image data is obtained;Step 5, repeat step 4 process, and the centroid information set corresponding to binarization image data set is obtained;Step 6, the binarization image data with centroid distance less than threshold value is carried out measure and centroid coordinate merging, and the measure change curve is obtained as image focusing curve;Step 7, the distance where the centroid maximum value on measure curve is located is extracted, and the corresponding focusing distance is obtained.The method provided by the application does not need to extract and cut the target area, avoids the problem that target is missed due to segmentation algorithm.

Description

Technical Field

[0001] This invention belongs to the field of holographic image processing, and in particular relates to a multi-target autofocus method for holographic imaging in complex environments. Background Technology

[0002] Currently, autofocus algorithms for digital holographic images mainly target single targets or single focusing distances. These methods include grayscale gradient algorithms and edge sharpness algorithms for general images. However, because they are designed for general images, their effectiveness is unstable when holographic images have many diffraction spots. While complex amplitude criterion methods for holographic images perform well for single-target focusing, their effectiveness for multi-target focusing at multiple focusing distances in complex water environments still needs improvement. Therefore, some research has proposed segmenting the target in the image before focusing. However, segmentation algorithms can lead to incorrect or missed segmentation, requiring sophisticated algorithms and failing to suppress noise in complex environments, resulting in significant deviations in focusing distance determination and poor actual focusing performance.

[0003] Patent document CN109828444A discloses an automatic focusing method for multi-target digital holographic measurement. The method includes the following steps: Step 1: Acquiring a hologram; Step 2: Separating the holograms; Step 3: Setting the automatic focusing search range; Step 4: Obtaining the automatic focusing curve; Step 5: Using the set automatic focusing search range as the abscissa and the Ci value as the ordinate, plotting the Ci curves obtained from all holographic sub-images in a unified coordinate system; Step 6: Reconstructing the original hologram based on the specific online distances of the multiple targets obtained in Step 5, and placing all reconstructed images in a unified spatial coordinate system to obtain the spatial distribution of the multiple targets.

[0004] Patent document CN106502074A discloses an automatic focal length method for digital holographic microscopy measurement of the image plane. This method includes: moving the sample to be measured along the principal optical axis of the objective lens and recording a series of equally spaced holographic images; obtaining the object light intensity image corresponding to each holographic image through operations such as Fourier transform, angular spectrum filtering, and inverse Fourier transform; calculating the autocorrelation operator for each intensity image; fitting the highest point of the curve (i.e., the maximum value of the autocorrelation operator) to the corresponding image position using a polynomial fitting method; and moving the surface of the sample to be measured to the focal length plane position using a motorized lifting stage controlled by a program to obtain the focal length image of the image plane. This method provides a new way to measure the surface morphology of samples in real time, greatly facilitating post-processing. However, the process of controlling the motorized stage to capture multiple holograms is relatively cumbersome, increasing the error. Summary of the Invention

[0005] To address the aforementioned problems, this invention provides a convenient, non-manually selectable method for multi-target autofocus, which can solve the multi-target focusing problem in complex environments present in existing digital holographic imaging autofocus systems.

[0006] A multi-target autofocusing method for holographic imaging in complex environments includes:

[0007] Step 1: Perform Gaussian attenuation processing on the background light source of the original holographic image;

[0008] Step 2: Based on the preset step size, perform full-image reconstruction on the original holographic image processed in Step 1 within the reconstruction distance range to obtain the corresponding reconstructed image dataset;

[0009] Step 3: Filter and binarize the reconstructed image dataset obtained in Step 2 to obtain the corresponding binarized image dataset;

[0010] Step 4: Analyze the connected components of all targets in the binarized image data obtained in Step 3 to obtain the centroid information corresponding to the binarized image data;

[0011] Step 5: Repeat step 4 to obtain the centroid information set corresponding to the binarized image dataset;

[0012] Step 6: Based on the reconstruction step size, merge the metric and centroid coordinates of the binarized image data with centroid distance less than the threshold to obtain the metric change curve as the image focusing curve. The metric represents the number of pixels in the connected component.

[0013] Step 7: Extract the distance of the centroid to the maximum value on the focusing curve to obtain the focusing distance of each target in the original holographic image for reconstruction.

[0014] This invention automatically divides the focus target by the grayscale changes of the reconstructed image and uses its measurement curve to solve the crosstalk problem between targets, thereby eliminating the need for manual or algorithmic extraction and cropping of the target area and realizing the automated generation of the focus distance.

[0015] Specifically, in step 1, the specific formula for the Gaussian attenuation process is as follows:

[0016]

[0017] Among them, I opt This is the preprocessed hologram. The division here is bitwise division. Gaussian is the Gaussian blur function. For an image of size 2048, the convolution kernel size is 150 and the variance is 100.

[0018] Specifically, in step 2, the reconstructed image dataset is obtained by reconstructing the original holographic image for each reconstruction step using the angular spectrum reconstruction algorithm.

[0019] Specifically, in step 3, the filtering and binarization include: performing minimum value filtering on the image twice with a kernel size of 4 and binarizing the image according to a preset value.

[0020] Specifically, in step 4, the binarized image data needs to be preprocessed before analysis. The preprocessing includes noise removal and information enhancement to avoid crosstalk between different connected components.

[0021] Specifically, the noise removal operation involves eliminating connected components with a measure less than a preset value.

[0022] Specifically, the information enhancement operation involves performing an opening operation on the noise-removed binarized image data.

[0023] Specifically, in step 4, the analysis of the connected components includes recording the measures of the preprocessed connected components and calculating the corresponding centroid coordinates. The formula for calculating the centroid coordinates is as follows:

[0024]

[0025] In the formula, [x o y o [x] represents the centroid coordinates. i y i ] represents the pixel coordinates in the connected component, and M is the measure of the connected component.

[0026] Specifically, in step 6, the merging is calculated based on the coordinates of the two centroids and the measure of the corresponding connected components to obtain the centroid coordinates of the corresponding connected components.

[0027] Specifically, the specific formula for the merging is as follows:

[0028]

[0029] In the formula, [X new Y new [X1, Y1] and [X2, Y2] are the centroids of the connected components, [X1, Y1] and [X2, Y2] are the coordinates of the centroids to be merged, and M1 and M2 are the measures of the corresponding connected components.

[0030] Specifically, the original holographic image is an image of a marine copepod mixed with other organisms and impurities.

[0031] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0032] (1) Focus curves are generated for each of the multiple target objects. The peak values ​​of the focus curves of each object are clear and there is no crosstalk between objects.

[0033] (2) No image segmentation of the hologram is required beforehand, and no manual or algorithmic extraction and cropping of the target region is required, which reduces the omission of the target due to the segmentation algorithm.

[0034] (3) It has a good noise suppression effect in the imaging area, reducing the impact of noise on the determination of the optimal reconstruction distance. Attached Figure Description

[0035] Figure 1 A flowchart illustrating a multi-target autofocusing method for holographic imaging in complex environments provided by the present invention;

[0036] Figure 2 The original holographic image provided in this embodiment;

[0037] Figure 3 The optimal focusing distance of 0.0280672m provided in this embodiment is illustrated by the measurement curve and the corresponding image reconstruction results.

[0038] Figure 4 A schematic diagram showing the optimal focusing distance of 0.032605m for this embodiment, along with the corresponding image reconstruction results;

[0039] Figure 5 The optimal focusing distance of 0.0517467m provided in this embodiment is shown in the measurement curve and the corresponding image reconstruction result diagram. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] like Figure 1 As shown, a multi-target autofocusing method for holographic imaging in complex environments includes:

[0042] Step 1: Acquire holograms using a single-arm or dual-arm coaxial holographic system and perform preprocessing, specifically including: removing background inhomogeneities caused by Gaussian attenuation of the imaging light source in the holographic image using the following formula:

[0043]

[0044] Among them, Iopt This is the preprocessed hologram. The division here is bitwise division. Gaussian is the Gaussian blur function. For an image of size 2048, the kernel size is 150 and the variance is 100.

[0045] Step 2: Perform full image reconstruction on the holographic image obtained in Step 1 within the reconstruction distance range according to a certain step size, and obtain the corresponding reconstructed image at each reconstruction step size;

[0046] Step 3: Filter and binarize the reconstructed image from Step 2. Specifically, this includes performing two minimum value filters on the image with a kernel size of 4, and then binarizing it according to a threshold.

[0047] Step 4: Remove noise and enhance information in the binary image from Step 3. Specifically, this includes discarding all connected components with a measure less than a predetermined threshold. It should be noted that the measure refers to the number of pixels in the connected component.

[0048] Next, a binary image opening operation is performed to enhance the image information and make the focus curve easier to distinguish. Since the background environment is relatively complex, in order to avoid crosstalk between different connected components, the degree of opening operation here needs to be as small as possible to ensure that environmental interference can be filtered out in the following steps.

[0049] Step 5: Repeat step 4 to obtain the centroid information set corresponding to the binarized image dataset;

[0050] Step 6: Based on the reconstruction step size, merge the measurement and centroid coordinates of the binarized image data with centroid distance less than the threshold to obtain the measurement change curve as the image focusing curve;

[0051] The formula for calculating the centroid coordinates is as follows:

[0052]

[0053] In the formula, [x o y o [x] represents the centroid coordinates, [x] represents the centroid coordinates. i y i ] represents the pixel coordinates in the connected component, and n is the measure of the connected component.

[0054] The specific formula for merging is as follows:

[0055]

[0056] In the formula, [X new Y new [X1, Y1] and [X2, Y2] are the centroids of the connected components, [X1, Y1] and [X2, Y2] are the coordinates of the centroids to be merged, and M1 and M2 are the measures of the corresponding connected components.

[0057] Step 7: Extract the distance of the centroid to the maximum value on the metric curve to obtain the focusing distance of each target in the original holographic image for reconstruction.

[0058] Based on the obtained focusing distance, the original holographic image is reconstructed.

[0059] The example image is a marine copepod containing other organisms and impurities, captured using a single-arm coaxial digital holographic system. First, the... Figure 2 The holographic image shown is preprocessed according to the formula in step one to remove the Gaussian attenuation of the light source of the single-arm coaxial system, so as to reduce the judgment of the focus of the reconstructed image. The preprocessed image is reconstructed in the interval [0.02, 0.08] (unit: m) with a step size of 0.0005m to obtain 120 reconstructed images.

[0060] Each image is subjected to two minimum filtering operations with a kernel size of 4 to enhance the information of points on the focal plane. Then, binarization is performed to extract the information of the enhanced image and obtain a binarized image. The binarized image is inverted to facilitate measurement calculation.

[0061] Calculate the measure of all connected components in the graph, discard connected components with a distance less than the threshold, and calculate the centroid and record the measure for the remaining connected components. Compare the centroid information at the reconstructed distance with the centroid information at the previous reconstructed distance. For target centroids with a distance less than the threshold, they are considered to be the same object, their measures are merged, and the centroid is updated.

[0062] Based on the final centroid-measure information, the corresponding centroid measure change curve is plotted as the image focusing curve:

[0063] like Figure 3 As shown, the image focusing curves indicate that the maximum distance between the two centroids is 0.0280672m. Based on this distance, the hologram is reconstructed to obtain a clear reconstructed image.

[0064] like Figure 4 As shown, the image focusing curves indicate that the maximum distance between the centroids is 0.032605 mm. Based on this distance, the hologram is reconstructed to obtain a clear reconstructed image.

[0065] like Figure 5 As shown, the image focusing curves indicate that the maximum distance between the two centroids is 0.0517467m. Based on this distance, the hologram is reconstructed to obtain a clear reconstructed image.

Claims

1. A multi-target autofocusing method for holographic imaging in complex environments, characterized in that, include: Step 1: Perform Gaussian attenuation processing on the background light source of the original holographic image; Step 2: Based on the preset step size, perform full-image reconstruction on the original holographic image processed in Step 1 within the reconstruction distance range to obtain the corresponding reconstructed image dataset; Step 3: Filter and binarize the reconstructed image dataset obtained in Step 2 to obtain the corresponding binarized image dataset; Step 4: Analyze the connected components of all targets in the binarized image data obtained in Step 3 to obtain the centroid information corresponding to the binarized image data; Step 5: Repeat step 4 to obtain the centroid information set corresponding to the binarized image dataset; Step 6: Based on the reconstruction step size, merge the metric and centroid coordinates of the binarized image data with centroid distance less than the threshold to obtain the metric change curve as the image focusing curve. The metric represents the number of pixels in the connected component. The merging is calculated based on the coordinates of the two centroids and the measure of the corresponding connected components to obtain the centroid coordinates of the corresponding connected components. The specific formula for the merging is as follows: In the formula, [ [ is the centroid of the connected component, [ ]and[ [ ] represents the centroid coordinates to be merged. For the measure of the corresponding connected components; Step 7: Extract the distance of the centroid to the maximum value on the focusing curve to obtain the focusing distance of each target in the original holographic image for reconstruction.

2. The multi-target autofocusing method for holographic imaging in complex environments according to claim 1, characterized in that, In step 3, the filtering and binarization specifically include: performing minimum value filtering on the image twice with a kernel size of 4 and binarizing the image according to a preset value.

3. The multi-target autofocusing method for holographic imaging in complex environments according to claim 1, characterized in that, In step 4, the binarized image data needs to be preprocessed before analysis. The preprocessing includes noise removal and information enhancement.

4. The multi-target autofocusing method for holographic imaging in complex environments according to claim 3, characterized in that, The specific operation for noise removal is to remove connected components whose measure is less than a preset value.

5. The multi-target autofocusing method for holographic imaging in complex environments according to claim 3, characterized in that, The specific operation of the information enhancement is to perform an opening operation on the noise-removed binarized image data.

6. The multi-target autofocusing method for holographic imaging in complex environments according to claim 1, characterized in that, In step 4, the analysis of the connected components includes recording the measures of the preprocessed connected components and calculating the corresponding centroid coordinates. The formula for calculating the centroid coordinates is as follows: In the formula, [ ] represents the centroid coordinates, [ [] represents the pixel coordinates in the connected component. It is a measure of connected components.

7. The multi-target autofocusing method for holographic imaging in complex environments according to claim 1, characterized in that, The original holographic image is an image of a marine copepod mixed with other organisms and impurities.

Images