A digital holographic phase image stitching method based on deep learning

Abstract

This invention discloses a deep learning-based digital holographic phase image stitching method. The method includes: acquiring multi-field holographic interferograms, obtaining a single-field phase image through numerical reconstruction, and then linearly mapping it to a grayscale image to create a synthetic dataset; using semi-supervised training of a noise-resistant registration network, calling the obtained registered grayscale image pairs and their binary masks, and then inputting them into a baseline compensation fusion network, using unsupervised training of the fusion model; inputting the registered grayscale image pairs into the trained image stitching model to obtain a stitched grayscale image, which, after grayscale-phase conversion, yields a large field-of-view phase image. This invention uses only a synthetic dataset to train the network, avoiding the difficulty of manually annotating the registration matrix. Through a semi-supervised training strategy and noise-resistant registration network design, it greatly improves registration accuracy, and baseline compensation further optimizes the fusion effect, showing great application potential in digital holographic microscopy quantitative measurement field-of-view stitching.

Description

Technical Field

[0001] This invention relates to an image stitching method, which falls under the field of digital holography, and specifically to a deep learning-based digital holographic phase image stitching method. Background Technology

[0002] In digital holographic microscopy (DHM) systems, lateral resolution and imaging field of view are mutually constrained. A key challenge in DHM technology is achieving a large field of view while maintaining high resolution. Image stitching techniques can be used to stitch phase images from multiple DHM fields of view, effectively expanding the imaging field of view and measurement area.

[0003] However, existing stitching methods face severe challenges when processing DHM phase images with unique physical characteristics. The inherent low contrast and complex three-dimensional structure of phase-type samples such as biological tissues, combined with the structured noise introduced by DHM optical aberrations and laser coherence, severely degrade the performance of traditional image stitching algorithms. These algorithms (such as those based on Scale Invariant Feature Transform (SIFT) combined with Random Sample Consistency (RANSAC)) heavily rely on stable and rich image features. In DHM phase images, noise and optical aberrations significantly reduce the repeatability and consistency of feature descriptors, leading to inaccurate estimated geometric transformations (homography matrices), ultimately resulting in registration bias, stitching seams, and structural artifacts in the stitching results. Although subsequent studies have proposed various improvement schemes, such as phase stitching algorithms combining phase correlation and corner detection, or phase stitching strategies using reconstructed intensity maps for guided stitching, their performance improvements on complex biological samples are limited due to either excessive sensitivity to noise and aberrations or reliance on suboptimal guiding information. They fail to fundamentally resolve the inherent conflict between traditional feature extraction paradigms and the physical characteristics of DHM phase images.

[0004] In summary, achieving high-precision, high-fidelity DHM phase image stitching faces three major challenges: (1) coherent noise significantly reduces the signal-to-noise ratio and reliability of phase data, interfering with the stability of feature extraction; (2) inherent aberrations of the optical system and the scanning process may introduce asymmetric and nonlinear geometric distortions between sub-fields of view, destroying the assumption of spatial continuity of phase data; (3) random phase baseline drift between different sub-fields of view will introduce systematic errors in the large-field stitched phase image, destroying the physical consistency of quantitative phase.

[0005] When directly transferring existing deep learning frameworks to DHM phase image stitching tasks, their data-driven paradigm itself constitutes a major limitation. Supervised learning paradigms essentially rely on large-scale, precisely labeled training data, but the acquisition process of real DHM data is cumbersome and the labeling cost is high. Simulation datasets built based on ideal assumptions struggle to reproduce the complex coherent noise distribution and optical aberration characteristics of real DHM systems, resulting in significantly insufficient generalization ability of trained models in real-world scenarios. While unsupervised learning can effectively circumvent the data labeling problem, its core optimization relies on the assumption of overlapping region consistency, which conflicts with the inherent physical characteristics of DHM phase images. This makes it difficult for consistency constraints to effectively guide model convergence, leading to registration failures. More fundamentally, most existing methods treat phase images as ordinary images, focusing only on spatial geometric registration while completely ignoring explicit modeling and correction of the global continuity of the physical quantity "phase." This results in visually coherent stitching results that lose quantitative accuracy due to implicit phase steps, rendering them unsuitable for serious biomedical quantitative analysis. Summary of the Invention

[0006] To address the problems existing in the background technology, this invention provides a deep learning-based digital holographic phase image stitching method. This invention constructs a noise suppression and registration model based on semi-supervised learning, which is trained through multi-intensity noise simulation and consistency constraints across noise intensities, overcoming the limitations of traditional feature-dependent methods. A fusion network integrating phase baseline correction is designed to eliminate systematic shifts between sub-fields of view, achieving seamless fusion of phase images.

[0007] The technical solution adopted in this invention is: The deep learning-based digital holographic phase image stitching method of the present invention includes: Step 1: Acquire holographic interferograms of several biological tissue isophase samples under multiple fields of view using an off-axis digital holographic microscopy system. After reconstruction and mapping, obtain single-field grayscale images. After cropping each single-field grayscale image, obtain several pairs of overlapping sub-grayscale images as adjacent field-of-view image pairs to be stitched together, and then construct a synthetic dataset.

[0008] Step 2: Establish an image stitching model that includes a noise-resistant registration network and a baseline compensation fusion network connected in sequence. Input the synthetic dataset into the noise-resistant registration network for semi-supervised training to obtain the trained noise-resistant registration network. Output the registered synthetic dataset and its binary mask and input them into the baseline compensation fusion network for unsupervised training to obtain the trained fusion network. Finally, obtain the trained image stitching model.

[0009] Step 3: Acquire the holographic interferogram of the phase-type sample to be registered, and perform the same operation as in Step 1 to obtain the adjacent field-of-view image pairs to be stitched. Then, input them into the trained image stitching model for processing and output the stitched grayscale image. After grayscale-phase conversion, obtain the single field-of-view phase image after field-of-view expansion, thus realizing image stitching and field-of-view expansion.

[0010] In step one, for each holographic interferogram, aberration correction is first performed using a numerical reconstruction method to obtain a single-field-of-view phase map. Then, the single-field-of-view phase map is scaled using a preset coefficient. k A single-view grayscale image is obtained after linear mapping.

[0011] In step one, when cropping each single-view grayscale image, several pixels in the single-view grayscale image are randomly selected as starting positions. An offset position is generated for each starting position according to a preset field-of-view overlap rate range. For each starting position, the starting position and its offset position are respectively used as the origin. Two sub-grayscale images with overlapping regions of the cropped target field-of-view size are used as adjacent field-of-view image pairs to be stitched together. The origin is located at the upper left corner of each image. The offset position information is used as the label of the adjacent field-of-view image pairs to be stitched together. The offset position information includes the starting position and the offset amount of the offset position.

[0012] In step two, the Focus module in the registration network of the image stitching DHS (DunHuangStitch) model is replaced with a noise suppression module to obtain an improved registration network, which is then used as an anti-noise registration network. The adjacent field-of-view image pairs to be stitched are input into the anti-noise registration network for processing and output as registered grayscale image pairs and their binary masks.

[0013] The noise suppression module comprises a noise attention focus layer, an oriented convolutional layer, a refined convolutional layer, and a downsampling operation connected in sequence. The noise attention focus layer includes spatial rearrangement to expand channels (image block encoding and channel information). The noise attention focus layer consists of a third convolutional layer (conv), a fourth convolutional layer (conv), a first diagonal convolutional layer (dil1), a second diagonal convolutional layer (dil2), and a channel attention mechanism. The output of the noise attention focus layer is first fed into the spatial rearrangement expansion channel, and the processed output is fed into the channel attention mechanism (SENet) and the second convolutional layer for further processing. The outputs of the first and second convolutional layers are then fed into the first convolutional layer (conv) for further processing. The output of the first and second convolutional layers (conv) is then fed into the first convolutional layer (conv) for further processing. The directional convolutional layer consists of a third convolutional layer (conv), a fourth convolutional layer (conv), a first diagonal convolutional layer (dil1), a second diagonal convolutional layer (dil2), and a channel attention mechanism. The output of the noise attention focus layer is fed into the third convolutional layer (conv), the fourth convolutional layer (conv), the first diagonal convolutional layer (dil1), and the second diagonal convolutional layer (dil2). The processed outputs are concatenated along the channel dimension and then weighted and fused by the channel attention mechanism to serve as the output of the directional convolutional layer. The refined convolutional layer is the fifth convolutional layer (conv).

[0014] In step two, during semi-supervised training of the noise-resistant registration network, for each adjacent field-of-view image pair, high-intensity Perlin noise and Gaussian noise are first mixed and added to the adjacent field-of-view image pair to obtain a high-noise image pair, and low-intensity Perlin noise and Gaussian noise are mixed and added to the adjacent field-of-view image pair to obtain a low-noise image pair. Under the low-intensity mixed noise branch, each low-noise image pair is input into the noise-resistant registration network for training. During the forward propagation process, the gradient of the noise-resistant registration network is updated, and the noise-resistant registration network outputs the registration matrix parameters under the low-intensity mixed noise branch. Based on the registration matrix parameters under the low-intensity mixed noise branch and the labels of each low-noise image pair, L... 1. Loss calculation: Obtain the label loss and stop after reaching the preset first iteration number. Then, under the high-intensity mixed noise branch, train the input noise-resistant registration network with each high-noise image. Perform only forward propagation and freeze the gradient of the noise-resistant registration network. The noise-resistant registration network outputs the registration matrix parameters under the high-intensity mixed noise branch. Calculate the L1 loss based on the registration matrix parameters under the high and low-intensity mixed noise branches to obtain the contrast consistency loss. Directly optimize the model parameters of the noise-resistant registration network through backpropagation using the label loss. Then, obtain the total registration loss of the noise-resistant registration network by weighted summation of the label loss and the contrast consistency loss. Stop training when the total registration loss converges or reaches the second iteration number to obtain the trained noise-resistant registration network.

[0015] In step two, the baseline compensation fusion network includes a baseline correction module and an image fusion module connected in sequence. The image fusion module includes two weight-shared convolutional neural networks (CNN), a U-net convolutional neural network, and a seam generator. Specifically, the image fusion module adopts the fusion network in the DHS (DunHuangStitch) model. The input of the baseline compensation fusion network is first processed by the baseline correction module, which then outputs a baseline compensation value. The baseline compensation value is then applied to the registered grayscale image obtained by cropping the sub-grayscale image at the offset position to obtain the compensated image. Finally, the compensated image is... The compensated image, the registered grayscale image obtained from the sub-grayscale image cropped from the starting position, and the binary mask of the registered grayscale image are further processed by the image fusion module. Shallow feature extraction is performed by two convolutional neural networks (CNNs). The processed outputs are jointly input into the U-net CNN for feature extraction and reconstruction, and then input into the seam generator to re-encode the binary mask. Finally, two seam masks are output. The two seam masks, the compensated image, and the registered grayscale image obtained from the sub-grayscale image cropped from the starting position are weighted and fused to obtain the stitched grayscale image.

[0016] The baseline correction module first extracts the overlapping region of the low-noise grayscale image pair and calculates the average brightness difference between the two as the reference compensation value. cbase Simultaneously, structural difference features between low-noise grayscale image pairs are extracted using two layers of standard convolutional blocks, and then the structural difference features are used to predict the brightness shift using the tanh activation function. c offset Finally, the baseline compensation value is obtained. c , c=c base + γ · c offset , γ To adjust c offset The hyperparameter of amplitude.

[0017] In step two, during unsupervised training of the baseline compensation fusion network, low-intensity Perlin noise and Gaussian noise are first mixed and added to the registered grayscale image pairs to obtain low-noise grayscale image pairs. Then, a phase baseline shift simulated by random brightness shift is superimposed, and then input into the baseline correction module and image fusion module for processing. The unsupervised fusion loss function of the baseline compensation fusion network, which includes seam boundary loss and seam smoothness loss, is calculated until the total fusion loss converges or the third iteration is reached, and then training is stopped to obtain the trained baseline compensation fusion network.

[0018] In step three, a preset inverse proportionality coefficient 1 / k Perform grayscale-phase conversion on the stitched grayscale image.

[0019] The beneficial effects of this invention are: 1) A Method for Generating Physically Constrained Data and Automatically Constructing Labels for Phase Image Registration: This invention provides an end-to-end supervised learning data construction scheme for digital holographic microscopy systems. Based on actually acquired phase images, sample pairs are synthesized through controllable random translation, and corresponding simplified homography matrices are generated as accurate labels. This scheme effectively solves the problem of difficulty in labeling real transformation parameters in holographic image registration. Furthermore, by conforming to the characteristics of mechanical scanning displacement and simplifying the matrix degrees of freedom to two translations, it significantly reduces the difficulty of model learning, providing a reliable foundation for training high-precision, high-efficiency registration models.

[0020] 2) A multi-level, noise-resistant, robust registration model was constructed: This invention designed a structured noise suppression module, achieving noise suppression and structure preservation during the feature extraction stage through attention calibration and residual feature fusion. Furthermore, a semi-supervised contrastive training mechanism based on dual-intensity noise simulation was proposed. By applying consistency constraints between low-intensity and high-intensity noise branches, the model learns robustness to changes in noise intensity, rather than simply fitting a specific noise pattern. This approach systematically improves the stability and accuracy of the registration model under complex holographic noise and aberration interference from both feature learning and training strategy perspectives.

[0021] 3) Achieved joint optimization of post-registration baseline correction and fusion: This invention proposes a joint optimization method for post-registration baseline correction and fusion. Global compensation is performed by calculating the average brightness difference in overlapping regions, and combined with the local offset predicted by the convolutional network, forming a global-local two-level correction. Furthermore, the baseline correction module and the image fusion module are jointly optimized in an unsupervised framework, enabling baseline correction and stitching quality to evolve synergistically. Ultimately, while eliminating baseline differences between fields of view, seamless and coherent large field-of-view phase stitching is achieved.

[0022] The method of this invention uses only synthetic datasets to train the network, avoiding the difficulty of manually annotating the registration matrix. Through semi-supervised training strategy and noise-resistant registration network design, the registration accuracy is greatly improved. Baseline compensation further optimizes the fusion effect, and it has great application potential in digital holographic microscopy quantitative measurement field stitching. Attached Figure Description

[0023] Figure 1 This is a flowchart of the deep learning-based digital holographic phase image stitching method provided in this embodiment of the invention, wherein... Figure 1 (a) shows the overall process of stitching together digital holographic phase images. Figure 1 (b) is the registration model flowchart. Figure 1 (c) is the flowchart of the fusion model; Figure 2 This is a schematic diagram of the noise suppression module of the registration network of the present invention; Figure 3 The figure shows the experimental results of a specific embodiment of the present invention. Figure 3 (a) is a two-dimensional phase map of the resolution plate of adjacent fields of view acquired by the actual digital holographic microscopy system. Figure 3 (b) is the three-dimensional phase map of the resolution plate of the adjacent field of view. Figure 3 (c) is a two-dimensional spliced ​​phase map. Figure 3 (d) is a three-dimensional stitched phase map. Detailed Implementation

[0024] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0025] like Figure 1 of (a) Figure 1 (b) and Figure 1 As shown in (c), the deep learning-based digital holographic phase image stitching method of the present invention is as follows: Step 1: Acquire holographic interferograms of several biological tissue isophase samples under multiple fields of view using an off-axis digital holographic microscopy system. After reconstruction and mapping, obtain single-field grayscale images. After cropping each single-field grayscale image, obtain several pairs of overlapping sub-grayscale images as adjacent field-of-view image pairs to be stitched together, and then construct a synthetic dataset.

[0026] For each holographic interferogram, aberration correction is first performed using numerical reconstruction methods to obtain a single-field-of-view phase map. Specific numerical reconstruction methods include Fourier transform, spectrum extraction, inverse Fourier transform, least squares phase unwrapping, cubic curve fitting, and alternating direction method for aberration correction. Then, the single-field-of-view phase map is scaled using a preset coefficient. k After linear mapping, a single-field grayscale image is obtained. For single-field phase images of the same biological tissue under different fields of view, a preset scaling factor is used. k A linear mapping is performed from the phase interval to the grayscale value interval, and each single-view phase map has an adaptive constant. b Map the background or subject phase to the same grayscale value.

[0027] When cropping each single-view grayscale image, several pixels in the single-view grayscale image are randomly selected as starting positions. An offset position is generated for each starting position according to a preset field-of-view overlap rate range. For each starting position, the starting position and its offset position are respectively used as the origin. Two sub-grayscale images with overlapping regions of the cropped target field-of-view size are used as adjacent field-of-view image pairs to be stitched together. The origin is located at the upper left corner of each image. The offset position information is used as the label of the adjacent field-of-view image pairs to be stitched together. The offset position information includes the starting position and the offset amount of the offset position.

[0028] Step 2: Establish an image stitching model that includes a noise-resistant registration network and a baseline compensation fusion network connected in sequence. Input the synthetic dataset into the noise-resistant registration network for semi-supervised training to obtain the trained noise-resistant registration network. Output the registered synthetic dataset and its binary mask and input them into the baseline compensation fusion network for unsupervised training to obtain the trained fusion network. Finally, obtain the trained image stitching model.

[0029] An improved registration network is obtained by replacing the Focus module in the registration network of the DHS (DunHuangStitch) image stitching model with a noise suppression module, which then serves as the noise-resistant registration network. Specifically, the noise-resistant registration network adopts the registration network of the DHS (DunHuangStitch) model from DunHuangStitch: Unsupervised Deep Image Stitching of Dunhuang Murals, replacing the Focus module (YOLO) in the registration model with a noise suppression module. The noise-resistant registration network includes a weight-shared feature extractor and a registration parameter predictor connected in sequence. The weight-shared feature extractor contains a noise suppression module, six feature extraction blocks, and three patch merging blocks. The feature extraction blocks include a parallel convolutional neural network (CNN) and a visual transformer (Swin Transformer, Shifted Windows Transformer). The registration parameter predictor consists of three optical flow estimation modules, three registration parameter prediction modules, three direct linear transformation solving modules, and three image registration operations. The noise-resistant registration network first uses a noise suppression module to perform spatial rearrangement, feature enhancement, and interference suppression to improve the signal-to-noise ratio. Then, it uses CNN and ViT to extract semantic information features, extracting features in three layers. Next, it uses a multi-scale feature pyramid based on optical flow estimation to construct the relationship between features and spatial location. Then, it uses this relationship to predict the four-parameter offset. Finally, it uses the direct linear transformation algorithm to solve for the registration matrix parameters. The adjacent field-of-view images to be stitched are input into the noise-resistant registration network, processed, and output as a registered grayscale image pair and its binary mask.

[0030] like Figure 2As shown, the noise suppression module includes a noise attention focus layer, an oriented convolutional layer, a refined convolutional layer, and a downsampling operation connected in sequence. The noise attention focus layer includes a spatial rearrangement expansion channel, a channel attention mechanism SENet, a first 3×3 convolutional layer (conv), and a second 1×1 convolutional layer (conv). The input of the noise attention focus layer is first fed into the spatial rearrangement expansion channel for slicing and channel concatenation. Based on the focus structure, the number of channels is increased and the resolution is reduced. The processed outputs are then fed into the channel attention mechanism SENet and the second convolutional layer for further processing. The channel attention mechanism SENet is embedded to adaptively calibrate the feature channels to suppress noise. The outputs of the channel attention mechanism SENet and the spatial rearrangement expansion channel are added together and then fed into the first convolutional layer (conv) for further processing. The outputs of the first convolutional layer (conv) and the second convolutional layer (conv) are then added together and used as the output of the noise attention focus layer. Simultaneously, a parallel residual branch is constructed, and the outputs are processed sequentially. The 1×1 convolutional module Conv aligns the channel dimensions, and the 3×3 convolutional module Conv extracts local features. These features are then fused with attention-weighted features to maintain structural integrity and enhance robustness against noise during dimensionality reduction. The directional convolutional layers consist of a third 1×3 convolutional layer conv, a fourth 3×1 convolutional layer conv, a first 45° diagonal convolutional layer dil1, a second 135° diagonal convolutional layer dil2, and a channel attention mechanism. The output of the noise attention focus layer is fed into the third convolutional layer conv, the fourth convolutional layer conv, the first diagonal convolutional layer dil1, and the second diagonal convolutional layer dil2, respectively. Four types of convolutional kernels are used to extract multi-directional spatial features. The processed outputs are concatenated along the channel dimensions and then weighted and fused using the channel attention mechanism to output the directional convolutional layers, thus strengthening edge structures and suppressing noise and aberration interference. The refined convolutional layer is the fifth 1×1 convolutional layer conv, which doubles the number of channels (C→2C) to increase the feature representation capability.

[0031] Furthermore, the 3x3 registration matrix has 9 parameters and 8 degrees of freedom, requiring the model to predict 8 parameters. The simplified model only needs to predict two parameters, which are translation parameters. Other parameters include scaling, rotation, shearing, and perspective distortion parameters. The registration network's prediction parameters only return two translation parameters, effectively suppressing the interference introduced by noise and aberrations in the eight-degree-of-freedom estimation, thus improving the robustness of registration.

[0032] The initial channel of the noise suppression module input image, as well as the length H and width W, are [1,512,512]. After processing, they become [16,256,256], [16,256,256], [32,256,256], and [32,128,128].

[0033] During semi-supervised training of the noise-resistant registration network, for each adjacent field-of-view image pair, high-intensity Perlin noise and Gaussian noise are first mixed and added to the adjacent field-of-view image pair to obtain a high-noise image pair, and low-intensity Perlin noise and Gaussian noise are mixed and added to the adjacent field-of-view image pair to obtain a low-noise image pair, in order to simulate the noise of a real digital holographic system. The amplitudes of the low-intensity mixed noise and the high-intensity mixed noise are set to ±10% and ±20% of the sample grayscale baseline value, respectively; the noise random seed is generated by the registration network training... The training consists of three parts: the global base seed, the sample index, and the current number of training epochs. In the low-intensity mixed noise branch, each low-noise image pair is input into the noise-resistant registration network for training. During forward propagation, the gradient of the noise-resistant registration network is updated. The noise-resistant registration network outputs the registration matrix parameters under the low-intensity mixed noise branch. Based on the registration matrix parameters under the low-intensity mixed noise branch and the labels of each low-noise image pair, L1 loss is calculated to obtain the label loss. The training stops after reaching a preset first iteration number, which can be 50. Then, in the high-intensity mixed noise branch... Each high-noise image is used as input to train the noise-resistant registration network. Only forward propagation is performed, and the network's gradients are frozen. The network outputs registration matrix parameters under the high-intensity mixed noise branch. L1 loss is calculated based on these parameters to obtain the contrast consistency loss, whose weights gradually increase with the number of training epochs. The label loss is then used to directly optimize the network's model parameters through backpropagation, dominating early gradient updates. Model parameters include weights and biases (such as kernel shape) in the neural network. At the start of training, these parameters are usually randomly initialized. The gradient is the vector of partial derivatives of the loss function with respect to each model parameter. The training process is as follows: current parameters → forward calculation of loss → backpropagation calculation of gradient → optimizer updates parameters → obtaining the next generation of parameters. The label loss and contrast consistency loss are weighted and summed to obtain the total registration loss of the noise-resistant registration network. These two losses are integrated through a dynamic weight mechanism to jointly guide and optimize the parameter updates of the model until the total registration loss converges or the second iteration number is reached, which can be 200 iterations, to obtain the trained noise-resistant registration network.

[0034] The baseline compensation fusion network comprises a baseline correction module and an image fusion module connected in sequence. The image fusion module includes two weight-shared convolutional neural networks (CNNs), a U-net convolutional neural network, and a seam generator. Specifically, the image fusion module adopts the fusion network in the DHS (DunHuangStitch) model. The input of the baseline compensation fusion network is first fed into the baseline correction module for processing, and the output is the baseline compensation value. The baseline compensation value is then used to compensate the registered grayscale image obtained by cropping the sub-grayscale image at the offset position to obtain the compensation image. The compensation image, the registered grayscale image obtained by cropping the sub-grayscale image at the starting position, and the binary mask of the registered grayscale image are then processed by the image fusion module. Shallow feature extraction is performed by the two CNNs, and the processed outputs are jointly input into the U-net convolutional neural network for feature extraction and reconstruction. Then, the outputs are input into the seam generator to re-encode the binary mask, and finally, two seam masks are output. The two seam masks, the compensation image, and the registered grayscale image obtained by cropping the sub-grayscale image at the starting position are weighted and fused to obtain the stitched grayscale image.

[0035] The baseline correction module first extracts the overlapping region of low-noise grayscale image pairs and calculates the average brightness difference between them as the baseline compensation value. c base Simultaneously, structural difference features between low-noise grayscale image pairs are extracted using two layers of standard convolutional blocks, and then the structural difference features are used to predict the brightness shift using the tanh activation function. c offset Finally, the baseline compensation value is obtained. c , c=c base + γ · c offset , γ To adjust c offset The hyperparameter of amplitude.

[0036] During unsupervised training of the baseline compensation fusion network, low-intensity Perlin noise and Gaussian noise are first mixed and added to the registered grayscale image pairs to obtain low-noise grayscale image pairs. Then, a phase baseline shift simulated by random brightness shift is superimposed. The phase baseline shift is a constant, which is the brightness shift relative to the grayscale image and can be understood as a random constant. The data is then sequentially input into the baseline correction module and the image fusion module for processing. The unsupervised fusion loss function of the baseline compensation fusion network, which includes seam boundary loss and seam smoothness loss, is calculated. Training stops when the total fusion loss converges or reaches the third iteration, which can be 50 iterations, to obtain the trained baseline compensation fusion network.

[0037] The baseline correction module and the image fusion module are jointly optimized using unsupervised seam boundary loss and seam smoothness loss. After the compensation value predicted by the baseline correction module is applied to the registered non-reference image, the pixel difference between the seam and the edge of the overlapping area of ​​the two images, as well as the pixel difference between the images on both sides of the seam, are calculated as the total loss. During network training, the weights and biases of the baseline correction module and the fusion network module are optimized simultaneously.

[0038] Step 3: Acquire holographic interferograms of the phase-type samples to be registered, and perform the same operations as in Step 1 to obtain adjacent field-of-view image pairs to be stitched. Then, input these pairs into the trained image stitching model for processing and output the stitched grayscale image. Use a preset inverse scaling factor 1 / k A grayscale-phase transformation is performed on the stitched grayscale image to obtain a single-field-of-view phase image with expanded field of view, thus realizing image stitching and field of view expansion.

[0039] Specific embodiments of the present invention are as follows: (1) Create a synthetic dataset, specifically: In practice, the collected holographic interferogram values ​​will be reconstructed to 1024. × For a 1024-pixel phase map, using an appropriate and identical scaling factor for phase maps of the same biological tissue under different fields of view. k A linear mapping is performed from the phase range to the grayscale range (50~205); each phase map has an adaptive constant. b Map the lowest phase value of the background or subject to the same grayscale value (50). First, randomly generate a reference image according to the preset field-of-view overlap range (10%~100%). I r The top-left corner coordinates are used as the starting position, and then the corresponding offset position is randomly generated. t x , t y The image to be registered can then be obtained. I t The coordinates of the top left corner, t x and t y These are the offsets for the x-axis and y-axis, respectively.

[0040] From the experimental grayscale image (1024×1024 pixels), based on randomly generated... I r and I t Using the coordinates of the top left corner and the preset field of view size (512×512 pixels), two sub-grayscale images (512×512 pixels) with overlapping areas are cropped, and the relative displacement between the two sub-images is recorded.t x , t y ) as registration labels.

[0041] (2) Training the registration network, specifically: When loading the synthetic dataset, a mixture of Perlin and Gaussian noise of two intensities is added to simulate the noise of a real digital holographic system. In the low-intensity mixed noise branch, the gradient is calculated using the registration model during the forward propagation. The registration matrix parameters output by the registration model are then used to calculate the L1 loss with the labels, yielding the label loss. L H The branch with high-intensity mixed noise performs only forward propagation and freezes gradient calculations. The registration matrix parameters output by the registration model are compared with the output of the branch with low-intensity mixed noise using L1 loss calculation to obtain the contrast consistency loss. L c .

[0042] Label loss L H Backpropagation is used to directly optimize model parameters, dominating early gradient updates. This is compared to consistency loss. L c Loss statistics are introduced after the 50th training epoch, with their weights gradually increasing with each additional epoch. Ultimately, the total loss of the registration model is... L t The total loss of the registration model is composed of a weighted sum of label loss and contrast consistency loss. These two losses are integrated through a dynamic weighting mechanism to jointly guide and optimize the model's parameter updates. L t Represented as: in, m To compare consistency loss L c The overall scaling factor, in specific implementation m = 0.1; E This represents the current training cycle number. E 0 represents the number of warm-up periods for semi-supervised learning. E 0 = 50; E total This represents the total number of training cycles.

[0043] Noise Random Seed Seed Global base seeds related to registration network training S base Sample Index S ID With the current number of training cyclesE It consists of three parts. Seed It can be represented as: Where, Δ offset Represents reference image I r and the image to be registered I t Noise seed offset between.

[0044] Two noise intensities are generated according to the following configuration: Low-intensity mixed noise: Perlin noise has a fixed fundamental frequency grid size of 2×2 and a frequency band of 2. Its spatial pattern and amplitude coefficient are... a (Samples within the interval [1, 5]) are all generated under the control of a noise seed, and the corresponding superimposed Gaussian noise follows a certain order. N (0, ( a ×0.001) 2 )distributed.

[0045] High-intensity mixed noise: Perlin noise has a fixed base grid size of 4×4 and a frequency band of 4. Its spatial pattern and amplitude coefficient are... a (Samples within the interval [5, 10]) are all generated under the same deterministic seed as the low-intensity sample, and the corresponding superimposed Gaussian noise follows the same rules. N (0, ( a ×0.005) 2 The two types of noise intensity generated are added to the current sample grayscale image respectively, normalized, and then input into the registration model.

[0046] The registration network features are as follows: two sub-grayscale images with added noise are first passed through a noise suppression module with shared weights to enhance features and suppress interference to improve the signal-to-noise ratio; then, a feature extraction module is used to extract features in three layers and construct a multi-scale feature pyramid; the 4D cost volume is calculated, and the four-parameter offset is predicted from the multi-scale features through a progressive regression network; finally, the registration matrix parameters are solved using a direct linear transformation algorithm.

[0047] The noise reduction measures in the registration network are as follows: Noise Suppression Module: Based on the Focus structure, spatial rearrangement is implemented to expand channels and reduce resolution. The SENet channel attention mechanism is embedded to adaptively calibrate feature channels to suppress noise. Simultaneously, a parallel residual branch is constructed, sequentially aligning channel dimensions through a 1×1 Conv convolution module and extracting local features through a 3×3 Conv convolution module. Finally, these features are fused with attention-weighted features, maintaining structural integrity and enhancing robustness against noise during dimensionality reduction. Subsequently, four types of convolution kernels—1×3 conv, 3×1 conv, 3×3 dil1, and 3×3 dil2—are used to extract multi-directional spatial features. These features are concatenated along the channel dimensions and then weighted and fused using the channel attention mechanism to strengthen edge structures and suppress noise and aberration interference.

[0048] Furthermore, the registration network's prediction parameters return only two translation parameters, effectively suppressing the interference introduced by noise and aberrations in the eight-degree-of-freedom estimation, thereby improving the robustness of registration.

[0049] (3) Training the fusion network, specifically: First, load the two registered grayscale images. I wr and I wt and its overlapping region binary mask M br and M bt and in I wr and I wt A fixed low-intensity mixture of Perlin and Gauss noise, along with a brightness offset of [-5,5], is added to simulate baseline shift. This is then input into the baseline correction module to obtain the predicted baseline compensation value. c Baseline corrected I wt + c ,and I wr , M br and M bt Shallow features are extracted using a shallow convolutional network, followed by feature extraction and reconstruction using a U-Net network. Finally, a seam generator outputs two seam masks. M cr and M ct .Will I wt + c , I wr , M cr andM ct Weighted fusion is performed to reconstruct the final stitched image. I S Based on this, the unsupervised fusion loss function is calculated. L stitch And optimize the fusion network through backpropagation. in, α and β Boundary loss L boundary and smoothness loss L smoothness The weighting coefficients are taken as follows: α= 10 4 , β = 10 3 .

[0050] Boundary loss L boundary The start and end points of the constraint seam are located at the physical boundaries of the overlapping area. Baseline compensation values ​​are introduced. c Then, it is defined as: Smoothing loss L smoothness From the photometric smoothing term L d and splicing smooth item L s It is used to constrain the seam path and ensure a smooth and natural transition in the splicing area. Wherein, D is the field-to-field difference map after baseline compensation c; i, j Represents image coordinates.

[0051] The specific characteristics of the fusion network for baseline compensation are as follows: Extract the overlapping region of the registered grayscale image pairs and calculate the average brightness difference between them as the baseline compensation value. c base Simultaneously, structural difference features between the two are extracted using two standard convolutional modules, and the brightness shift is predicted based on these features using the tanh activation function. c offset The final output baseline compensation value c=c base + γ · c offset , γ To adjust c offset The hyperparameter of amplitude. The baseline correction module and the image fusion module are integrated through unsupervised seam boundary loss. L boundary and loss of seam smoothness L smoothness By performing joint optimization, baseline differences between fields of view are eliminated, while achieving seamless and coherent large field of view stitching.

[0052] (4) Use the trained fusion model, specifically: Registered grayscale image pairs ( I wr and I wt ) and its overlapping region binary mask ( M br and M bt The input is fed into the trained fusion model to obtain the fused and stitched grayscale image. I S Using the aforementioned phase-to-grayscale ratio coefficient k Mapping grayscale values ​​to phase intervals in the stitched grayscale image (1 / k The final output is a large field-of-view phase map.

[0053] like Figure 3 As shown in (a), a1 and a2 in the figure are two-dimensional phase maps of the resolution plate of adjacent fields of view acquired by the actual digital holographic microscopy system, as shown in (a). Figure 3 As shown in (b), b1 and b2 in the figure are the three-dimensional phase maps of the resolution plate for adjacent fields of view, as... Figure 3 As shown in (c), this is a two-dimensional spliced ​​phase map, as... Figure 3 As shown in (d), this is a three-dimensional stitched phase image. As can be seen from this implementation, this invention deeply integrates deep learning with phase stitching technology, systematically solving key problems such as noise interference, residual aberrations, and baseline inconsistencies between fields of view. By introducing multi-intensity noise simulation and semi-supervised contrastive training, an end-to-end mapping from spatial features to registration parameters is achieved, directly learning from the data, eliminating the dependence on explicit feature extraction. Simultaneously, through a strategy of synergistic optimization of baseline correction and image fusion, baseline drift and seam exposure problems in the phase image are effectively suppressed. This invention provides a robust and fully automated phase image stitching solution for large field-of-view digital holographic microscopy.

[0054] The above specific embodiments are used to explain and illustrate the present invention, but not to limit the present invention. Any modifications and changes made to the present invention within the spirit and scope of the claims shall fall within the protection scope of the present invention.

Claims

1. A deep learning-based digital holographic phase image stitching method, characterized in that, include: Step 1: Acquire holographic interferograms of several phase-type samples under multiple fields of view using an off-axis digital holographic microscopy system. After reconstruction and mapping, obtain single-field grayscale images. After cropping each single-field grayscale image, obtain several pairs of overlapping sub-grayscale images as adjacent field-of-view image pairs to be stitched together, and then construct a synthetic dataset. Step 2: Establish an image stitching model that includes a noise-resistant registration network and a baseline compensation fusion network connected in sequence. Input the synthetic dataset into the noise-resistant registration network for semi-supervised training to obtain the trained noise-resistant registration network. Output the registered synthetic dataset and its binary mask and input them into the baseline compensation fusion network for unsupervised training to obtain the trained fusion network. Finally, obtain the trained image stitching model. Step 3: Acquire the holographic interferogram of the phase-type sample to be registered, and perform the same operation as in Step 1 to obtain the adjacent field-of-view image pairs to be stitched. Then, input them into the trained image stitching model for processing and output the stitched grayscale image. After grayscale-phase conversion, obtain the single field-of-view phase image after field-of-view expansion, thus realizing image stitching and field-of-view expansion.

2. The deep learning-based digital holographic phase image stitching method according to claim 1, characterized in that: In step one, for each holographic interferogram, aberration correction is first performed using a numerical reconstruction method to obtain a single-field-of-view phase map. Then, the single-field-of-view phase map is scaled using a preset coefficient. k A single-view grayscale image is obtained after linear mapping.

3. The deep learning-based digital holographic phase image stitching method according to claim 1, characterized in that: In step one, when cropping each single-view grayscale image, several pixels in the single-view grayscale image are randomly selected as starting positions. An offset position is generated for each starting position according to a preset field-of-view overlap rate range. For each starting position, the starting position and its offset position are respectively used as the origin. The two sub-grayscale images that overlap with the area of ​​the cropped target field-of-view size are used as adjacent field-of-view image pairs to be stitched together. The offset position information is used as the label of the adjacent field-of-view image pairs to be stitched together.

4. The deep learning-based digital holographic phase image stitching method according to claim 1, characterized in that: In step two, the Focus module in the registration network of the image stitching DHS model is replaced with a noise suppression module to obtain an improved registration network, which is then used as an anti-noise registration network. The adjacent field-of-view image pairs to be stitched are input into the anti-noise registration network for processing and output as registered grayscale image pairs and their binary masks.

5. The deep learning-based digital holographic phase image stitching method according to claim 4, characterized in that: The noise suppression module comprises a noise attention focus layer, an oriented convolutional layer, a refined convolutional layer, and a downsampling operation connected in sequence. The noise attention focus layer includes a spatial rearrangement extended channel, a channel attention mechanism SENet, a first convolutional layer (conv), and a second convolutional layer (conv). The input of the noise attention focus layer is first fed into the spatial rearrangement extended channel. The processed output is then fed into the channel attention mechanism SENet and the second convolutional layer for further processing. The outputs of the channel attention mechanism SENet and the spatial rearrangement extended channel are added together and then fed into the first convolutional layer (conv) for further processing. The first convolutional layer (conv) and the second convolutional layer (conv) are then fed into the second convolutional layer (conv). The outputs of the convolutional layers (conv) are summed and then used as the output of the noise attention focus layer. The directional convolutional layers include the third convolutional layer (conv), the fourth convolutional layer (conv), the first diagonal convolutional layer (dil1), the second diagonal convolutional layer (dil2), and a channel attention mechanism. The outputs of the noise attention focus layer are input into the third convolutional layer (conv), the fourth convolutional layer (conv), the first diagonal convolutional layer (dil1), and the second diagonal convolutional layer (dil2), respectively. The processed outputs are concatenated along the channel dimension and then weighted and fused by the channel attention mechanism to serve as the output of the directional convolutional layers. The refined convolutional layer is the fifth convolutional layer (conv).

6. The deep learning-based digital holographic phase image stitching method according to claim 3, characterized in that: In step two, during semi-supervised training of the noise-resistant registration network, for each adjacent field-of-view image pair, high-intensity Burmester noise and Gaussian noise are first mixed and added to the adjacent field-of-view image pair to obtain a high-noise image pair, and low-intensity Burmester noise and Gaussian noise are mixed and added to the adjacent field-of-view image pair to obtain a low-noise image pair. Under the low-intensity mixed noise branch, each low-noise image pair is input into the noise-resistant registration network for training. During the forward propagation process, the gradient of the noise-resistant registration network is updated. The noise-resistant registration network outputs the registration matrix parameters under the low-intensity mixed noise branch. Based on the registration matrix parameters under the low-intensity mixed noise branch and the labels of each low-noise image pair, L1 loss is calculated to obtain the label loss, and the process stops after reaching the preset first iteration number. Then, under the high-intensity mixed noise branch, each high-noise image is used as input to train the noise-resistant registration network. Only forward propagation is performed and the gradient of the noise-resistant registration network is frozen. The noise-resistant registration network outputs the registration matrix parameters under the high-intensity mixed noise branch. L1 loss is calculated based on the registration matrix parameters under the high and low-intensity mixed noise branches to obtain the contrast consistency loss. The label loss is used to directly optimize the model parameters of the noise-resistant registration network through backpropagation. The label loss and contrast consistency loss are weighted and summed to obtain the total registration loss of the noise-resistant registration network. Training stops when the total registration loss converges or the second iteration number is reached, and the trained noise-resistant registration network is obtained.

7. The deep learning-based digital holographic phase image stitching method according to claim 4, characterized in that: In step two, the baseline compensation fusion network includes a baseline correction module and an image fusion module connected in sequence. The image fusion module includes two weight-shared convolutional neural networks (CNNs), a U-net convolutional neural network, and a seam generator. The input of the baseline compensation fusion network is first processed by the baseline correction module and outputs a baseline compensation value. The baseline compensation value is then used to compensate the registered grayscale image obtained by cropping the sub-grayscale image at the offset position to obtain a compensation image. The compensation image, the registered grayscale image obtained by cropping the sub-grayscale image at the starting position, and the binary mask of the registered grayscale image are then processed by the image fusion module. Shallow feature extraction is performed by the two CNNs, and the processed outputs are input into the U-net convolutional neural network for feature extraction and reconstruction. Then, the outputs are input into the seam generator to re-encode the binary mask, and finally, two seam masks are output. The two seam masks, the compensation image, and the registered grayscale image obtained by cropping the sub-grayscale image at the starting position are weighted and fused to obtain a stitched grayscale image.

8. The deep learning-based digital holographic phase image stitching method according to claim 7, characterized in that: The baseline correction module first extracts the overlapping region of the low-noise grayscale image pair and calculates the average brightness difference between the two as the reference compensation value. c base Simultaneously, structural difference features between low-noise grayscale image pairs are extracted using two layers of standard convolutional blocks, and then the structural difference features are used to predict the brightness shift using the tanh activation function. c offset Finally, the baseline compensation value is obtained. c , c=c base + γ · c offset , γ This is a hyperparameter.

9. The deep learning-based digital holographic phase image stitching method according to claim 7, characterized in that: In step two, during unsupervised training of the baseline compensation fusion network, low-intensity Berlin noise and Gaussian noise are first mixed and added to the registered grayscale image pairs to obtain low-noise grayscale image pairs. Then, phase baseline shift is superimposed, and the data is sequentially input into the baseline correction module and the image fusion module for processing. The unsupervised fusion loss function of the baseline compensation fusion network, which includes seam boundary loss and seam smoothness loss, is calculated until the total fusion loss converges or the third iteration is reached, at which point training stops, and the trained baseline compensation fusion network is obtained.

10. The deep learning-based digital holographic phase image stitching method according to claim 2, characterized in that: In step three, a preset inverse proportionality coefficient 1 / k Perform grayscale-phase conversion on the stitched grayscale image.

Images