Multi-height lensless holographic microscopic image registration method based on combination of frequency domain and space domain

Abstract

The application discloses a multi-height lensless holographic microscopic image registration method based on frequency domain and space domain combination. Firstly, a Fourier phase correlation method is used to find a correlation peak value to coarsely calibrate a space translation error; then, for the coarsely corrected hologram, a method for quickly and accurately searching a matching feature point in a space domain is provided to obtain an accurate scaling factor, an image morphological technology is used, and the sample itself is used as the feature point, so that any marking on the sample is not needed; after the image scaling calibration is completed, the Fourier phase correlation method is used again to perform accurate translation calibration. Compared with the existing registration algorithm, the application can more accurately and quickly register and align the multi-height lensless holographic microscopic images.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and more particularly to a multi-height lensless holographic microscopic image registration method. BACKGROUND

[0002] In recent years, due to the rapid development of low-cost image sensing devices, modern digital computers, light-emitting diodes (LEDs), etc., it provides rich opportunities for the development of computational imaging. Lensless digital holographic microscopic imaging has become an important branch of computational imaging technology. Compared with the traditional optical microscope, the lensless holographic microscope does not need bulky and expensive lenses and optical connection structures, but only needs low-cost light sources and imaging sensing devices, which greatly reduces the cost of the microscope and makes the structure simple, small and portable, and can be used directly outdoors, which has broad prospects in practical applications.

[0003] At present, the holographic microscopic image registration algorithm is generally divided into three categories: region-based matching method, feature point-based matching method and transform domain-based matching method. Among them, the region-based matching method is to select one block as a template and move it iteratively on the standard image by image segmentation, and then determine the position of the template on the standard image by calculating the correlation coefficient between the two, so as to realize the digital registration of the two images. However, this method has the disadvantages of large amount of calculation and slow operation speed; the feature point-based matching method is to detect and match the feature points between two images for image registration. The SIFT operator is currently recognized as the most stable feature matching operator, which has many advantages such as scale invariance, rotation invariance and illumination invariance. However, due to the repeatability of the feature points of biological samples, the matching between feature points is prone to error. Although ransac and other methods can be used to eliminate the wrong matching pairs, the running time is still long and the registration accuracy is low; the transform domain-based matching method is to transform the image into the frequency domain by Fourier transform and use the phase correlation method to register the image. This method is generally used for registration between images with only translation. The improved Fourier-Mellin algorithm can obtain the rotation and scaling factor of the image. Although the Fourier-Mellin algorithm runs fast, the image registration accuracy is not high.

[0004] Therefore, how to effectively improve the registration accuracy and registration speed of the holographic microscopic image registration algorithm is a problem that needs to be solved by those skilled in the art. SUMMARY

[0005] Therefore, the present application provides a multi-height lensless holographic microscopic image registration method based on the combination of frequency domain and spatial domain to solve the technical problems existing in the background art.

[0006] In order to achieve the above object, the present application adopts the following technical solutions:

[0007] S1, obtaining multiple frames of lensless holographic microscopic images of different heights, using Fourier phase correlation method to find correlation peak value, and preliminarily correcting spatial translation error of holographic images of different heights;

[0008] S2, using image morphology-based method to find matched feature points, and correcting scaling error of the preliminarily corrected holographic images;

[0009] S3, using Fourier phase correlation method again to correct translation error of the scaling error corrected holographic images, and realizing accurate matching of original holographic images.

[0010] Preferably, the S1 specifically comprises:

[0011] S11, obtaining multiple frames of holographic microscopic images of different heights, focusing holographic images of different heights to obtain focused intensity images containing conjugate images;

[0012] S12, using Fourier phase correlation calculation on the focused intensity images to obtain spatial translation amount of correlation peak value;

[0013] S13, preliminarily correcting translation error between holographic images according to the spatial translation amount.

[0014] Preferably, the step S2 specifically comprises:

[0015] S21, setting image intensity threshold value, and pre-processing the preliminarily corrected focused intensity images;

[0016] S22, using inflation template to perform inflation operation on the pre-processed intensity images, so that the feature shape of the sample to be tested in the intensity image becomes the shape of the inflation template;

[0017] S23, performing skeleton extraction operation on the intensity images after the inflation operation;

[0018] S24, using the inflation template to perform "hit or miss" transformation on the skeleton-extracted intensity images, the skeleton that is "hit" is taken as a feature point and is reserved, and the pixel coordinates of the position where the feature point center is located are obtained, and the pixel in the remaining area is set to zero;

[0019] S25, determining a standard holographic image, and selecting a target feature point in the feature points of the focused intensity image of the standard holographic image;

[0020] S26, obtaining corresponding matching feature points in the holographic image to be registered according to the target feature point;

[0021] S27, respectively calculate the distance L1 between the two target feature points and the distance L2 between the two matching feature points, and calculate the scaling factor of the standard holographic image and the to-be-registered holographic image according to L1 and L2;

[0022] S28, perform scaling error correction on the to-be-registered holographic image by using the scaling factor.

[0023] Preferably, the step S21 specifically comprises: setting an image intensity threshold, and setting the region below the image intensity threshold in each intensity image after the preliminary correction to zero according to the image intensity threshold.

[0024] Preferably, the dilation template in the steps S22 and S24 is a cross-shaped template.

[0025] Preferably, the standard holographic image in the step S25 is the holographic image with the smallest recording distance in the different-height framed holographic images.

[0026] Preferably, the two target feature points selected on the focused intensity map of the standard holographic image in the step S25 are the feature points close to the different edge positions of the image, that is, the distance L1 of the two target feature points should be as large as possible.

[0027] Preferably, the step S26 specifically comprises:

[0028] After the "hit or miss" transformation operation, a 3-pixel region is expanded around the target feature points on the focused intensity map of the standard holographic image, the pixel values in the region are set to 1, the pixel values outside the region are set to 0, then the "and" operation is performed between the intensity map of the to-be-registered holographic image and the region, so as to obtain the matching feature points in the to-be-registered holographic image.

[0029] According to the above technical solution, compared with the prior art, the application discloses a multi-height lensless holographic microscopic image registration method based on the combination of frequency domain and space domain. First, the Fourier phase correlation method is used to find the correlation peak value to coarsely calibrate the spatial translation error; then, for the holographic image after the coarse correction, a method of quickly and accurately searching for matching feature points in the space domain is proposed to obtain an accurate scaling factor, the image morphological technology is used, and the sample itself is used as the feature point, without the need of marking the sample; after the image scaling calibration is completed, the Fourier phase correlation method is used again for accurate translation calibration. Compared with the existing registration algorithm, the application can more accurately and more quickly register and align the multi-height lensless holographic microscopic image. BRIEF DESCRIPTION OF DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative effort based on the provided drawings.

[0031] Figure 1 The algorithm flowchart provided by the present application;

[0032] Figure 2 The multi-height lensless holographic microscopic device structure schematic diagram provided by the embodiment of the present application;

[0033] Figure 3 The standard polystyrene microsphere test result diagram provided by the embodiment of the present application. Among them Figure 3 (a) is a hologram with the minimum recording distance; Figure 3 (b) is a local enlarged schematic diagram of the image in (a); Figure 3 (c) is a focusing intensity diagram of the hologram; Figure 3 (d) is a difference diagram of the normalized intensity values of the focusing intensity diagrams of the holograms with the maximum and minimum recording distances; Figure 3 (e) and 3(f) are the intensity image and phase image obtained by directly using the multi-height hologram phase recovery method on the hologram without registration processing; Figure 3 (g) is a difference diagram of the normalized intensity values of the focusing intensity diagrams of the holograms with the maximum and minimum recording distances after registration by the registration method of the present application; Figure 3 (h) and 3(i) are the intensity image and phase image obtained by using the multi-height hologram phase recovery method to reconstruct the hologram after registration processing;

[0034] Figure 4 The red blood cell intensity images reconstructed after using different registration methods provided by the embodiment of the present application. Among them Figure 4 (a) is a red blood cell intensity image obtained after registration using the Fourier-Mellin algorithm and reconstruction using the phase recovery method; Figure 4 (b) is a red blood cell intensity image obtained after registration using the sift method and reconstruction using the phase recovery method; Figure 4 (c) is a red blood cell intensity image obtained after registration using the method combining spatial domain and frequency domain proposed by the present application and reconstruction using the phase recovery method; Figure 4 (d) is Figure 4 (a) is an enlarged view of the boxed portion in (a);

[0035] Figure 4 (e) is Figure 4 (b) is an enlarged view of the boxed portion in (b); Figure 4 (f) is Figure 4 (c) is an enlarged view of the boxed portion in (c). Detailed Implementation

[0036] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] This invention discloses a multi-height lensless holographic microscopy image registration method based on frequency-domain and spatial-domain combination, such as... Figure 1 As shown, it includes the following steps:

[0038] S1. Acquire multiple frames of lensless holographic microscopic images at different heights, use the Fourier phase correlation method to find the correlation peak, and perform preliminary correction on the spatial translation error of the holographic images at different heights.

[0039] like Figure 2 As shown, in this embodiment, holographic microscopic images can be acquired using a multi-height lensless digital holographic microscopic imaging experimental device. The sample to be tested is placed on the sample stage, and the height of the imaging sensor device along the z-axis from the sample to be tested is adjusted in the experimental device to record coaxial holographic images at different heights. By using the frequency-domain and spatial-domain combined multi-height lensless holographic microscopic image registration method proposed in this invention, accurate registration of holographic images can be achieved.

[0040] Furthermore, in this embodiment, step S1 specifically includes:

[0041] S11. Acquire multiple frames of holographic microscopic images at different heights, focus the holographic images at different heights, and obtain a focus intensity map containing the conjugate image.

[0042] S12. Calculate the spatial shift of the correlation peak using Fourier phase correlation on the focused intensity map;

[0043] S13. Perform preliminary correction on the translation error between holograms based on the spatial translation amount.

[0044] S2. Use image morphology-based methods to find matching feature points and perform scaling error correction on the pre-corrected holographic image;

[0045] The coaxial holographic image directly captured by the multi-height lensless digital holographic microscopy imaging experimental device is used to obtain a focused hologram after optical field propagation and focusing calculation. However, there will be interference from conjugate images, which will interfere with registration. Therefore, this embodiment of the invention uses image morphology technology, using the sample itself as feature points, to correct the scaling error of the holographic image after preliminary correction of translation error. Specifically, it includes:

[0046] S21, set an image intensity threshold to pre-process the preliminary corrected focus intensity image; the specific process is as follows: set an image intensity threshold, and according to the image intensity threshold, set the region below the intensity threshold in the preliminary corrected focus intensity image to zero, so as to reduce the interference of conjugate images and scattered samples to be detected on registration.

[0047] S22, perform an inflation operation on the pre-processed intensity image by using an inflation template, so that the feature shape of the sample to be detected becomes the shape of the inflation template.

[0048] Through the inflation operation, some samples to be detected that are very close to each other can be connected into one piece and no longer serve as an identification target. On the other hand, the shape of the sample to be detected is also changed by the inflation template. Regardless of the shape of the sample to be detected, the feature shape of the unknown sample to be detected can be changed into the shape of the known inflation template after the inflation operation. In the embodiment of the present application, a cross-shaped template is used to perform the inflation operation on the pre-processed focus intensity image.

[0049] S23, perform a skeleton extraction operation on the intensity image after the inflation operation.

[0050] After the inflation and skeleton extraction, the samples to be detected that are very close to each other no longer maintain the shape of the inflation model, while the samples to be detected that are relatively discrete still maintain the shape of the inflation model after the skeleton extraction. In this way, even if the samples to be detected are very dense, the relatively discrete samples to be detected can still be found.

[0051] S24, perform a "hit or miss" transformation on the focus intensity image after the skeleton extraction by using the inflation template, the skeleton that is "hit" is taken as a feature point and is reserved, and the pixel coordinates of the position where the feature point center is located are obtained, and the pixels in the remaining region are set to zero.

[0052] The purpose of this step is to obtain the feature point and the position where the feature point is located. The method used here is "hit or miss" transformation. If the matching is successful, it is "hit" and is taken as a feature point, otherwise it is "miss". The inflation template used here is a cross-shaped template.

[0053] S25, determine a standard holographic image, and select a target feature point in the feature points of the focus intensity image of the standard holographic image.

[0054] In this embodiment, the holographic image with the smallest recording distance among the multiple holographic images with different heights is defined as a standard image, and the other images are taken as holographic images to be registered.

[0055] Preferably, in order to improve the calculation accuracy of the scaling factor, the present application selects the feature points that are as much as possible on the edges and occupy different edges in the focus intensity image corresponding to the standard holographic image as the target feature points.

[0056] S26, searching for corresponding matching feature points in the holographic image to be registered according to the target feature points; the specific process is that: after the "hit or miss" transformation operation, a 3-unit pixel area (3 pixels in the range of 7*7 are set, and the rest of the pixel area is set to zero) is expanded around the target feature point on the focused intensity map of the standard holographic image, the processed intensity image is subjected to an "and" operation with the focused intensity map of the holographic image to be registered, and the corresponding matching feature points on the holographic image to be registered are obtained.

[0057] S27, the distances L1 between the two target feature points and the distances L2 between the two matching feature points are calculated respectively, and the scaling factor of the standard holographic image and the holographic image to be registered is calculated according to L1 and L2.

[0058] In order to improve the accuracy, a plurality of target feature points and matching feature points can be selected, and a plurality of scaling factors can be calculated, and finally the average value is taken as the scaling factor.

[0059] S28, the scaling error correction is performed on the holographic image to be registered by using the scaling factor.

[0060] S3, the Fourier phase correlation method is used again to correct the translation error of the holographic image after the scaling error correction, and the accurate matching of the original holographic image is realized.

[0061] In the multi-height lensless holographic microscopic imaging system, there are inevitably various error sources such as translation and scaling. Therefore, effective digital registration and alignment operation of these coaxial holograms is a key problem for realizing iterative phase retrieval reconstruction. Through the multi-height lensless holographic image registration method based on the combination of frequency domain and space domain proposed in the application, the multi-height lensless holographic image can be registered and aligned more accurately and at a higher speed.

[0062] The invention principle of the application will be described in further detail below.

[0063] Firstly, the phase correlation method based on Fourier transform is used to preliminarily correct the translation error between the holograms recorded at different distances, but since the recorded holograms are defocused holograms, the light field distributions at different distances are different, so the translation amount cannot be directly calculated, and the defocused hologram needs to be focused, and then the translation amount is calculated through the correlation calculation of the focused intensity map, and the translation error of the original hologram is corrected. It is assumed that there is a translation (x0, y0) between two focused intensity maps f1(x, y) and f2(x, y), that is, f1(x, y)=f2(x-x0, y-y0). The normalized cross power spectrum calculation expression between the two images is:

[0064]

[0065] where F1(u,v) and F2(u,v) are the Fourier transforms of f1(x,y) and f2(x,y) respectively, F2 * (u,v) is the complex conjugate of F2(u,v), j is the imaginary factor, |F1(u,v)F2 * (u,v)| is the modulus of F1(u,v)F2 * (u,v), M and N are the number of pixels of the images. The correlation peak between f1(x,y) and f2(x,y) can be obtained by inverse Fourier transform of (1), i.e.

[0066] q(x,y) = δ(x-x0,y-y0) (2)

[0067] The pixel-level translation (x0,y0) between f1(x,y) and f2(x,y) can be determined by searching the position of the peak. The translation is used to correct the translation error of the hologram.

[0068] Then, the image morphological based method is used to quickly find the matched feature points to correct the scaling error of the holograms of different heights. Since the illumination source of the multi-height lensless digital holographic microscopy experimental device is a spherical wave, there is a scaling error between the holograms recorded at different heights. However, since the distance between the illumination source and the sample to be measured is much larger than the distance from the sample to the image sensor plane, and the recording distance of each hologram changes very little, in this embodiment, the change is 50 um, so the scaling error between the holograms is very small. In this embodiment, for a 1024*1024 pixel holographic image, after the phase correlation method is used to preliminarily correct the translation error, the pixel position difference caused by scaling is about 2-3 pixels. However, the phase retrieval algorithm in the multi-height digital holographic microscopy system has very high registration requirements. The key of the feature point matching method is to accurately obtain the feature points. Dust particles or substrate defects are ideal candidate feature points in most samples. However, for samples with high density of particles, or when the dust particles or substrate defects are not clear, this method is not feasible. In addition, isolated spatial markers (such as dots) can be placed in the corners of the sample or cover glass as predetermined feature points. However, it is not convenient to prepare the sample, and the sample is damaged to a certain extent. Therefore, in this embodiment, the sample to be measured is used as the feature point.

[0069] The reconstructed intensity map of the on-axis hologram after reconstruction and focusing will have the interference of conjugate image, which will disturb the registration. First, a suitable image intensity threshold is selected to make the image intensity less than the threshold to be zero. In this embodiment, the intensity threshold can be about 150 for the 256 gray scale image. Then a cross-shaped template is used to dilate the processed intensity image. Because even if the threshold is used to make the area less than the threshold to be zero, there will still be some scattered interference to affect the subsequent registration. In addition, because the measured sample is very dense, some samples that are very close to each other are difficult to identify, so the dilatation can make some samples that are very close to each other to be connected, so that these samples that are very close to each other are no longer the identification target. In addition, the shape of the sample will also be changed by the dilated template after the dilatation of the image by the template with a known shape, so no matter what the shape of the sample is, the unknown shape can be changed to the known template shape after the dilatation. Then the skeleton of the image is extracted by the morphological method. The samples that are very close to each other will no longer maintain the cross-shaped shape after the dilatation and the skeleton extraction, while the relatively discrete samples can still maintain the cross-shaped shape after the dilatation and the skeleton extraction. Then the feature points and their position information are obtained. The method used here is the "hit or miss" transformation. The matching success is called "hit", otherwise it becomes "miss". Only the cross-shaped template is used to do the "hit or miss" transformation on the previous processed result image, so the skeleton of the cross-shaped shape is retained as the feature point, and the remaining area is cleared to zero, and the pixel coordinates of the center of the feature point are obtained. Finally, the target feature point and the matching feature point are found. The hologram with the minimum recorded distance is selected as the standard hologram. After the "hit or miss" transformation operation, only the cross-shaped skeleton of the feature point is retained in each focused intensity map. The feature points in the focused intensity map of the standard hologram that are as close as possible to the different edge regions of the image are selected as the target feature points. Because a rough registration has been done, the difference between the corresponding target points in different images is limited to within 2-3 pixels, so as long as the target feature point is taken as the center to expand the 3-pixel region around it, the pixel value in this expanded region is set to 1, and the remaining pixels outside this expanded region are set to zero, and then the "and" operation is performed with the image to be registered, the matching feature point corresponding to the target feature point can be found in the image to be registered. Finally, the scaling factor is calculated according to the two target feature points and the matching feature points found. In order to accurately calculate the scaling factor, multiple sets of target feature points and matching feature points can be searched and the scaling factor can be calculated. The average value is taken as the final scaling factor to correct the scaling error of the hologram.

[0070] Finally, the hologram with the corrected scaling error is used to correct the translation error by the Fourier phase correlation method, and the precise registration of the image is finally realized.

[0071] In order to verify the accuracy and the wide applicability of the method, the following experiment was carried out:

[0072] In the experimental device, the light source is a monochromatic LED light source, the center wavelength is 456 nm, and the spectral bandwidth is about 10 nm. The light is coupled into a multimode optical fiber, and the light emitted from the end of the optical fiber propagates and irradiates to the sample plane, which can be regarded as uniform illumination. The sample is placed on a plane less than 1 mm away from the CMOS image sensor chip (MT9P031, Micron, 2.2 μm pixel size, 5 million pixels). The image sensor is fixed on a cheap displacement platform. The platform can be vertically moved to change the distance between the image sensor and the sample, which is used to obtain multiple holograms at different recording distances. The collection of these holograms provides a series of measurement constraints for the multi-height phase retrieval algorithm.

[0073] Firstly, standard polystyrene microspheres were selected as the measured sample, the microspheres had a diameter of 10 ± 5% μm and a refractive index of 1.59. A small amount of microspheres was scattered on the carrier glass, and after the water was evaporated, the microspheres were immersed in immersion oil with a refractive index of 1.58, and then a cover glass was covered. The standard sample was placed on the stage. A total of 8 holograms at different heights were collected. The hologram with the smallest recording distance is shown in Figure 3 (a), and the part of the hologram in the rectangular frame is enlarged and displayed in Figure 3 (b), which can clearly show the interference fringes. The focused intensity image distribution of the object light field reconstructed directly by using the angular spectrum-based light field propagation algorithm is shown in Figure 3 (c). Since it is an on-axis hologram, the reconstructed image is seriously disturbed by the conjugate image. Figure 3 (d) is the difference image of the focused intensity image intensity value of the hologram with the largest recording distance and the hologram with the smallest recording distance after normalization, which shows that there is an alignment error in the holograms recorded at different heights. Therefore, if the recorded holograms are not registered, the intensity image and the phase image obtained by directly using the multi-height hologram phase retrieval method are shown in Figure 3 (e) and (f) respectively, that is, the correct reconstructed image cannot be obtained. After the recorded 8 holograms are registered by using the hologram registration method combining spatial and frequency domains proposed in the present application, the difference image of the focused intensity image intensity value of the hologram with the largest recording distance and the hologram with the smallest recording distance after normalization is shown in Figure 3 (g), which clearly shows that the image has been accurately registered. After the registered hologram image is reconstructed by using the multi-height hologram phase retrieval method, the intensity image and the phase image Figure 3 (h) and (i) are obtained. Therefore, after the collected holograms are registered by using the method proposed in the present application, the intensity image and the phase image of high quality can be obtained by using the phase retrieval method, which confirms that the registration method proposed in the present application can accurately register the holograms collected at different heights.

[0074] To further confirm the accuracy and rapidity of the method proposed in the present application for registration of lensless holographic microscopic images, further comparative experiments were performed. A little blood was taken from the fingertips of a healthy adult male in the laboratory, dropped into 0.9% physiological saline to ensure the activity and normal physiological activity of the red blood cells. 1 ul of the solution was taken by a pipette, dropped onto a glass slide, and covered with a cover glass to make a sample. The sample was placed on the stage. A total of 8 holograms at different heights were collected. After registration of the holograms recorded at different heights by the Fourier-Mellin method, reconstruction was performed by the multi-height hologram phase retrieval method, and the reconstructed intensity image was as shown in Figure 4 (a), Figure 4 (d) is Figure 4 (a) is an enlarged view of the frame portion in (a). After registration of the holograms by the method proposed in the present application, reconstruction was performed by the multi-height hologram phase retrieval method, and the intensity image obtained was as shown in Figure 4 (b), Figure 4 (e) is Figure 4 (b) is an enlarged view of the frame portion in (b). After registration of the holograms by the method proposed in the present application, reconstruction was performed by the multi-height hologram phase retrieval method, and the intensity image obtained was as shown in Figure 4 (c), Figure 4 (f) is Figure 4 (c) is an enlarged view of the frame portion in (c). By comparing these results, it can be seen that the method proposed in the present application can accurately register lensless holographic microscopic images at different heights, while the sift and Fourier-Mellin methods cannot meet the registration requirements of the multi-height hologram phase retrieval image.

[0075]

[0076] Table 1

[0077] Table 1 is a comparison table of running time of the three registration methods, the size of the registered image is 1024*1024, and the number of registered images is 8. The registration was performed in the Matlab environment on the same personal computer, and it can be seen from the table that the registration method proposed in the present application is much faster than the sift algorithm, and also faster than the Fourier-Mellin algorithm, and in terms of registration accuracy, it is much better than the sift and Fourier-Mellin algorithms.

[0078] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts of each embodiment can be referred to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the related parts can be referred to the method part.

[0079] The foregoing description of the disclosed embodiments enables a person skilled in the art to make or use the application. Modifications of these embodiments will occur to persons of skill in the art, and that the appended claims are intended to cover all such modifications that do not depart from the true spirit and scope of the application. Therefore, the application is not limited to the embodiments shown but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multi-height lensless holographic microscopic image registration method based on frequency domain and spatial domain combination, characterized in that, The method comprises the following steps: S1, obtaining multiple frames of lensless holographic microscopic images at different heights, and using a Fourier phase correlation method to find a correlation peak value to preliminarily correct spatial translation errors of the holographic images at different heights; S2, using an image morphology-based method to find matched feature points to correct scaling errors of the preliminarily corrected holographic images; specifically comprising: S21, setting an image intensity threshold value to pre-process each focus intensity image after the preliminary correction; S22, using an inflation template to perform an inflation operation on the pre-processed intensity image, so that the feature shape of the measured sample in the intensity image becomes the shape of the inflation template; S23, performing a skeleton extraction operation on the intensity image after the inflation operation; S24, using the inflation template to perform a "hit or miss" transformation on the intensity image after the skeleton extraction, the skeleton that is "hit" is taken as a feature point and is retained, and the pixel point coordinates of the position where the feature point center is located are obtained, and the pixel values of the remaining regions are set to zero; S25, determining a standard holographic image, and selecting target feature points in the feature points of the focus intensity graph of the standard holographic image; S26, obtaining matched feature points in a holographic image to be registered according to the target feature points; S27, respectively calculating distances L1 between two target feature points and distances L2 between two matched feature points, and calculating a scaling factor of the standard holographic image and the holographic image to be registered according to L1 and L2; S28, using the scaling factor to correct scaling errors of the holographic image to be registered; S3, using the Fourier phase correlation method again to correct translation errors of the holographic image after the scaling error correction, and realizing accurate matching of the original holographic image.

2. The multi-height lensless holographic microscopic image registration method based on the combination of frequency domain and spatial domain according to claim 1, characterized in that, The step S1 specifically comprises: S11, obtaining multiple frames of holographic microscopic images at different heights, focusing the holographic images at different heights to obtain focus intensity graphs containing conjugate images; S12, using a Fourier phase correlation calculation to obtain a spatial translation amount of a correlation peak value for the focus intensity graphs; S13, preliminarily correcting translation errors between the holographic images according to the spatial translation amount.

3. The multi-height lensless holographic microscopic image registration method based on the combination of frequency domain and spatial domain according to claim 1, characterized in that, The step S21 specifically comprises: setting an image intensity threshold value, and setting regions below the image intensity threshold value in each focus intensity image after the preliminary correction to zero according to the image intensity threshold value.

4. The multi-height lensless holographic microscopic image registration method based on the combination of frequency domain and spatial domain according to claim 1, characterized in that, The inflation template in steps S22 and S24 is a cross-shaped template.

5. The multi-height lensless holographic microscopic image registration method based on the combination of frequency domain and spatial domain according to claim 1, characterized in that, The standard holographic image in step S25 is a holographic image recording the smallest distance among the multiple frames of holographic images at different heights.

6. The multi-height lensless holographic microscopic image registration method based on the combination of frequency domain and spatial domain according to claim 1, characterized in that, The two target feature points selected in the focus intensity graph of the standard holographic image in step S25 are feature points close to different edge positions of the image, that is, the distance L1 between the two target feature points should be as large as possible.

7. The method of claim 1, wherein, The step S26 specifically comprises: After the "hit or miss" transformation operation, a 3-unit pixel region is expanded around the target feature point in the focus intensity graph of the standard holographic image, the pixel values in this region are set to one, the pixels outside this region are set to zero, and then the focus intensity graph of the holographic image to be registered is operated with the focus intensity graph of the standard holographic image to obtain matched feature points in the holographic image to be registered.

Images