A system and method for dual view quantitative phase imaging and cell artifact removal

Abstract

The present application relates to a kind of system and method for double view quantitative phase imaging and honeycomb artifact removal based on multi-core optical fiber transmission.The system includes: light source, pinhole diaphragm, condenser lens, spatial light modulator, mirror, multi-core optical fiber, objective lens, tube lens, beam splitter, first CMOS camera and second CMOS camera.The system for double view quantitative phase imaging and honeycomb artifact removal based on multi-core optical fiber transmission described in the present application, the phase distortion of multi-core optical fiber is corrected by spatial light modulator, different illumination angles generated using beam splitter, combined with the transmission characteristics of multi-core optical fiber, can be quickly collected under the premise of not changing sample position, phase information under two different views, then the following method is used to process the two groups of data, realize three-dimensional reconstruction, improve imaging depth and accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of optical fiber imaging, and in particular to a system and method for dual-view quantitative phase imaging and honeycomb artifact removal. Background Art

[0002] Due to their flexibility and compactness, fiber optic endoscopes have broad clinical application potential in in situ diagnostics and interventional procedures. They have already been applied in optical biopsies of the gastrointestinal tract, urinary system, and respiratory tract, as well as in cancer diagnosis. However, fiber optic microscopy relies on imaging methods that only provide amplitude contrast. Observing "phase objects" and measuring various morphological parameters of tissue (such as refractive index changes or material density) requires quantitative phase information. Conventional detectors can only record the amplitude of light and cannot obtain phase information. Therefore, other technical means are needed to obtain phase information.

[0003] Quantitative phase imaging (QPI) is a label-free technique that can simultaneously provide morphological and quantitative biophysical information in biomedical applications. The Transport of Intensity Equation (TIE) uses the stacking of through-focus intensity profiles under partially coherent illumination to recover phase information. TIE phase retrieval relies on the assumption of irrotational flow, which results in ambiguity in absorbing phase objects.

[0004] Multi-core fiber (MCF) enables ultrathin probes for in vivo imaging, but images obtained through fiber bundles are affected by artifacts such as honeycomb patterns. Honeycomb artifacts are caused by the difference in light transmittance between the core and cladding, as well as the presence of tiny gaps between densely packed single-filament fibers. These gaps and discontinuities appear as honeycomb-like artifacts in the reconstructed image, affecting image quality and clarity.

[0005] Therefore, there is an urgent need to realize a system and method for dual-view quantitative phase imaging and honeycomb artifact removal based on multi-core optical fiber transmission. Summary of the Invention

[0006] The purpose of the present invention is to provide a system and method for dual-view quantitative phase imaging and honeycomb artifact removal, which is used to perform phase recovery on biological tissue and use a neural network to suppress honeycomb artifacts in the sample phase recovered by the dual-view quantitative phase imaging system.

[0007] To achieve the above object, the present invention adopts the following technical solutions:

[0008] The present invention includes a system for dual-view quantitative phase imaging and honeycomb artifact removal based on multi-core optical fiber transmission, the system comprising: a light source, a pinhole aperture, a condensing mirror, a spatial light modulator, a mirror, a multi-core optical fiber, an objective lens, a tube lens, a spectroscope, a first CMOS camera and a second CMOS camera; a light beam emitted by the light source passes through the pinhole aperture, the condensing mirror, the spatial light modulator, the mirror, the sample, the multi-core optical fiber, the objective lens, the tube lens and the spectroscope in sequence, and the light beam is split into two beams by the spectroscope and then collected by the first CMOS camera and the second CMOS camera respectively.

[0009] The light beam emitted by the light source enters a condensing mirror after the shape and range of the incident light beam are adjusted by a pinhole aperture; the condensing mirror focuses the light beam; the light beam focused by the condensing mirror enters a spatial light modulator, which corrects the phase distortion generated by the multi-core optical fiber; the mirror is used to guide the light path and adjust the direction of the light beam so that the light beam is irradiated onto the sample; the multi-core optical fiber is used to transmit the light signal, the objective lens preliminarily amplifies the light signal transmitted through the multi-core optical fiber, the tube lens further amplifies the image, and guides the light beam to a spectroscope, which splits the light path into two: one part is transmitted to a first CMOS camera to record an out-of-focus plane image, and the other part is transmitted to a second CMOS camera to record an over-focus plane image. The first CMOS camera and the second CMOS camera are connected to the same computer, and the computer records the images simultaneously.

[0010] The present invention also includes a method for dual-view quantitative phase imaging and honeycomb artifact removal based on multi-core optical fiber transmission. The method is implemented based on the above system and includes the following steps:

[0011] S1, using spatial light modulator to correct the phase distortion produced by multi-core optical fiber;

[0012] S2, using a CMOS camera to capture the intensity image of the sample, and converting the sample intensity image captured by the first CMOS camera into and a second CMOS camera capturing the sample intensity image , the initial phase of the sample is calculated by the TIE calculation equation .

[0013] S3, build a PINN network, and set the initial phase of the sample As the input of the PINN network, the sample phase is iteratively optimized using the PINN network.

[0014] S4, build HAR-CNN network, and transform the optimized sample phase As the input of the HAR-CNN network, the HAR-CNN network is used to remove the honeycomb artifacts in the optimized phase image.

[0015] As a further improvement of the above technical solution, in step S1, using a spatial light modulator to correct the phase distortion generated by the multi-core optical fiber includes:

[0016] S11, spatial light modulator uses off-axis holography to measure the phase distortion of multi-core optical fiber;

[0017] S12. Loading the phase conjugate of the multi-core optical fiber phase distortion onto the spatial light modulator to compensate for the distorted phase.

[0018] As a further improvement of the above technical solution, in step S2, the intensity image of the sample is captured by the CMOS camera, and the intensity image of the sample captured by the first CMOS camera is converted to and a second CMOS camera capturing the sample intensity image , the initial phase of the sample is calculated by the TIE calculation equation ,include:

[0019] In step S2, the intensity image of the sample is captured by the CMOS camera, and the intensity image of the sample captured by the first CMOS camera (11) is converted into and the sample intensity image captured by the second CMOS camera (12) , the initial phase of the sample is calculated by the TIE calculation equation ,include:

[0020] S21, using a CMOS camera to capture an intensity image of the sample, the image captured by the first CMOS camera (11) is the sample intensity image , the image captured by the second CMOS camera (12) is the sample intensity image .

[0021] S22, Sample intensity image of the sample recorded by two CMOS cameras and sample intensity image , use formula (1) to calculate the average intensity of the sample intensity image :

[0022] (1)

[0023] S23, Sample intensity image of the sample recorded by two CMOS cameras and sample intensity image , use formula (2) to calculate the intensity gradient of the sample intensity image :

[0024] (2)

[0025] S24, based on the average intensity and intensity gradient of the sample intensity image, calculate the initial phase of the sample using formula (3) :

[0026] (3)

[0027] In formulas (1) to (3), 、 Perpendicular to the optical axis The transverse coordinate of the plane, is the axial distance between the camera and the imaging plane, 、 are the spectral coordinates in the Fourier domain, are the Fourier transform operator and the inverse Fourier transform operator, respectively. Wave number and wavelength Correlation, defined as .

[0028] As a further improvement of the above technical solution, the PINN network includes a data consistency layer and a CNN layer for learning network input features;

[0029] The CNN layer performs nonlinear correction on the input through learned features, capturing complex information that is difficult to express in physical models.

[0030] The data consistency layer calculates the gradient through the physical model and provides correction information that conforms to the physical constraints. The CNN layer and the data consistency layer are jointly optimized. Through back propagation, the network parameters are optimized, and the optimized sample phase is finally obtained through iterative training. .

[0031] As a further improvement of the above technical solution, in step S3, the PINN network is constructed to convert the initial phase of the sample As the input of the PINN network, the sample phase is iteratively optimized using the PINN network, including:

[0032] S31, using formula (4) to calculate the initial phase Perform nonlinear optimization:

[0033] (4)

[0034] in, is the data fidelity term, R(φ) is the regularizer, and the parameter γ controls the weight balance between them.

[0035] Initial phase The optimization is a nonlinear optimization process, which needs to minimize the objective function J( ) to complete the nonlinear optimization. Equation (4) is the objective function that defines the phase optimization problem. The goal is to optimize the predicted intensity With true strength The error is minimized, and the regularization term is added to constrain optimization.

[0036] S32. Based on TIE phase inversion, the defocus intensity transmitted by the light field is obtained, and the phase is iteratively optimized.

[0037] S321, phase inversion based on TIE, the forward operator Expressed as defocus intensity , using the Fresnel diffraction integral to obtain the propagation distance d = +z The near-field light field at , and the defocus intensity is obtained by taking the square of the complex field using formula (5) :

[0038] (5)

[0039] Calculate the defocus intensity using formula (5) , that is, the propagation distance is +z This is the basic step to achieve phase inversion and provides input for further optimization. represents the defocus intensity, 、 Perpendicular to the optical axis The transverse coordinate of the plane, is the axial distance between the camera and the imaging plane, Wave number and wavelength Correlation, defined as .

[0040] S322, from the initial phase First, use formula (6) to calculate the phase profile in each iteration Iterate until the marginal gain decreases at the local minimum and then stop:

[0041] (6)

[0042] Formula (6) is used to update the phase of the current iteration, combined with the gradient of the loss function , gradually optimize the phase to make it close to the real phase. is the iteration step size, The phase calculated by equations (1) to (3) , is the regularization term.

[0043] S33. Use a convolutional neural network to perform regularization optimization on the iteratively optimized phase, and convert the recovered phase result into an image close to the true phase.

[0044] The performance of regularized optimization is severely affected by the choice of regularizer and hyperparameters in R. Instead of using off-the-shelf regularizers as in some previous TIE retrieval algorithms, in this work, a deep learning based framework is employed to identify prior features of objects from training data.

[0045] S331. Determine the regularization function based on the convolutional neural network and use formula (7) to perform regularization constraints;

[0046] (7)

[0047] Formula (7) introduces the convolutional neural network based on formula (6): As a regularization term The present invention uses convolutional neural network (CNN) to learn complex regularization constraints instead of traditional gradient regularization, which improves the adaptability of the model to different data distributions and features. Reformulating the regularization term To keep the structure of the cascaded neural network fixed, the total number of iterations was pre-selected to be 3.

[0048] S332. Use formula (8) to train the regularization function and optimize the convolutional neural network.

[0049] By minimizing the network output and the corresponding truth value The mean square error between the two is used to train a parameterized The optimal regularizer for .

[0050] (8)

[0051] Formula (8) is used for training The parameters of , ensure that the regularization constraints learned by CNN can accurately reflect the true characteristics of the data. Where k represents a training dataset with a total of k samples.

[0052] S333. Based on the trained regularization function, a cascade neural network is used, combined with physical models and feature learning, to convert the recovered phase result into an image close to the real phase.

[0053] As a further improvement of the above technical solution, the HAR-CNN network consists of three convolutional layers, namely a feature extraction layer, a nonlinear mapping layer and a reconstruction layer;

[0054] The feature extraction layer is used to extract local image features, covering the sample phase , forming an initial feature map; the nonlinear mapping layer is used to map the high-dimensional feature vector into a low-dimensional feature, and realize feature filtering by capturing the complex nonlinear characteristics of the artifact; the reconstruction layer is used to generate a clear image without honeycomb artifacts.

[0055] As a further improvement of the above technical solution, in step S4, the HAR-CNN network is constructed to optimize the sample phase As the input of the HAR-CNN network, the HAR-CNN network is used to remove honeycomb artifacts in the optimized phase image, including:

[0056] S41. Image segmentation and feature extraction

[0057] The image is divided into blocks, and features are extracted from the blocked images. The rectangular area containing the cellular artifacts is extracted from the blocked phase image, and the high-dimensional data is converted into low-dimensional data to process the nonlinear features in the image. The image is convolved and the neighborhood pixels are used to output a high-resolution non-cellular image.

[0058] S42. Network design and configuration

[0059] The first and second layers of the HAR-CNN network respectively use activation functions (such as ReLU) as loss functions to achieve nonlinear mapping; the mean square error (MSE) is used as the optimization target of the HAR-CNN network; the filter size of the first layer of the HAR-CNN network is 9 to ensure that the pattern of honeycomb artifacts is included, the convolution kernel spacing is 5-6 pixels, the filter size of the nonlinear mapping layer is set to 1, and the number of feature maps is 64 to balance computational efficiency and filtering capability.

[0060] S43. Reconstruction layer configuration and model training

[0061] The HAR-CNN network is configured with a reconstruction layer. In the reconstruction layer, the filter size used by the HAR-CNN network is 5, and the number of feature maps is set to 32 to optimize the output image quality and computational efficiency. The HAR-CNN network is trained using a stochastic gradient descent optimizer to complete the model optimization.

[0062] S44. Use the optimized HAR-CNN network to remove honeycomb artifacts in the optimized phase image.

[0063] Compared with the prior art, the advantages of the present invention are:

[0064] The present invention combines two neural network models with a dual-view quantitative phase imaging system based on multi-core fiber to restore the phase of the sample under test and remove honeycomb artifacts, overcoming the problem of honeycomb artifacts in phase images produced by multi-core fiber in the prior art. The system and method for dual-view quantitative phase imaging and honeycomb artifact removal based on multi-core fiber transmission described in the present invention, through the different illumination angles generated by a spatial light modulator and combined with the transmission characteristics of multi-core fiber, can quickly acquire phase information from two different perspectives without changing the sample position. The two sets of data are then processed using the method described below to achieve three-dimensional reconstruction, improving imaging depth and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 This is a schematic diagram of a system for dual-view quantitative phase imaging and honeycomb artifact removal based on multi-core optical fiber transmission in the present invention;

[0066] Figure 2 It is a flow chart of the method for dual-view quantitative phase imaging and honeycomb artifact removal based on multi-core optical fiber transmission in the present invention;

[0067] Figure 3 This is the network architecture diagram of the PINN network;

[0068] Figure 4 This is the network architecture diagram of the HAR-CNN network.

[0069] in:

[0070] 1. Light source, 2. Pinhole aperture, 3. Condenser mirror, 4. Spatial light modulator, 5. Mirror, 6. Sample, 7. Multi-core optical fiber, 8. Objective lens, 9. Tube lens, 10. Beam splitter, 11. First CMOS camera, 12. Second CMOS camera, 13. Computer. DETAILED DESCRIPTION

[0071] The present invention will be further described below with reference to the accompanying drawings:

[0072] like Figure 1The system shown here is for dual-view quantitative phase imaging and honeycomb artifact removal based on multi-core fiber transmission. The system includes: a light source 1, a pinhole aperture 2, a condensing mirror 3, a spatial light modulator 4, a mirror 5, a multi-core fiber 7, an objective lens 8, a tube lens 9, a beam splitter 10, a first CMOS camera 11, and a second CMOS camera 12. The light beam emitted by the light source 1 passes through the pinhole aperture 2, the condensing mirror 3, the spatial light modulator 4, the mirror 5, the sample 6, the multi-core fiber 7, the objective lens 8, the tube lens 9, and the beam splitter 10. The beam is then split into two beams by the beam splitter 10, which are then captured by the first CMOS camera 11 and the second CMOS camera 12, respectively. The light source 1 controls the shape and range of the incident light beam through the pinhole aperture 2 to ensure that the light enters the condensing mirror 3 accurately; the condensing mirror 3 is used to focus the light beam and improve the beam quality; the light beam focused by the condensing mirror 3 enters the spatial light modulator 4, and the spatial light modulator 4 is used to correct the phase distortion generated by the multi-core optical fiber 7; the mirror 5 is used to guide the light path and adjust the direction of the light beam so that the light beam is irradiated on the sample 6; the multi-core optical fiber 7 is used to transmit the light signal, the objective lens 8 preliminarily amplifies the light signal transmitted through the multi-core optical fiber 7, the tube lens 9 further amplifies the image and guides the light beam to the spectroscope 10, which divides the light path into two: one part is transmitted to the first CMOS camera 11 to record the defocused plane image, and the other part is transmitted to the second CMOS camera 12 to record the overfocused plane image. The first CMOS camera 11 and the second CMOS camera 12 are connected to the same computer 13 to record the image at the same time.

[0073] Specifically, the light source 1 provides uniform and stable illumination for the entire system, and a laser light source or LED light source with good coherence is usually selected. The pinhole aperture 2 controls the size of the light beam entering the subsequent optical components, improving the spatial resolution of the system. The condenser mirror 3 focuses the light from the light source 1 onto the sample 6 to ensure that the sample surface is uniformly illuminated. The spatial light modulator 4 is used to dynamically adjust the incident light wavefront, which can achieve complex light field control, such as phase modulation. In this system, the spatial light modulator 4 can be used to generate different illumination angles to support dual-view imaging. The mirror 5 is used to change the direction of the light path and guide the light to propagate along a predetermined path. As one of the key components, the multi-core optical fiber 7 can simultaneously transmit multiple independent light channels. Each core carries information from a different perspective, which makes it possible to obtain multi-angle image data without moving the sample, thereby achieving three-dimensional reconstruction and honeycomb artifact removal. The objective lens 8 is used to magnify the sample and transmit the details of the sample to the subsequent optical components. The tube lens 9 is used in conjunction with the objective lens 8 to further adjust the imaging ratio and optimize the image quality. A beam splitter 10 splits the light reflected from the sample into two paths, each of which is fed to two CMOS cameras. This allows images of the same area to be captured from two different perspectives. A first CMOS camera 11 and a second CMOS camera 12 are used to record the images formed by the two beams separated by the beam splitter 10. These two cameras operate synchronously to ensure temporal consistency of the acquired data.

[0074] The system for dual-view quantitative phase imaging and honeycomb artifact removal based on multi-core fiber transmission described in the present invention, through the different illumination angles generated by the spatial light modulator 4, combined with the transmission characteristics of the multi-core fiber 7, can quickly collect phase information at two different perspectives without changing the sample position, and then use the method described below to process these two sets of data to achieve three-dimensional reconstruction and improve imaging depth and accuracy.

[0075] like Figure 2 As shown, the present invention also includes a method for dual-view quantitative phase imaging and honeycomb artifact removal based on multi-core optical fiber transmission, the method comprising the following steps:

[0076] S1. Correct the phase distortion generated by the multi-core optical fiber 7 using the spatial light modulator 4.

[0077] The spatial light modulator 4 is used to correct the phase distortion produced by the multi-core optical fiber 7 to eliminate the influence of the properties of the multi-core optical fiber itself on the phase of the measured sample.

[0078] Specifically, after the light beam emitted by the light source is processed sequentially by the spectral filter, the pinhole aperture, and the condensing mirror, the phase distortion caused by the multi-core optical fiber 7 can be corrected by using digital optical phase conjugation of the spatial light modulator 4. The spatial light modulator 4 first uses off-axis holography to measure the phase distortion of the multi-core optical fiber 7, and then loads the phase conjugate of the phase distortion of the multi-core optical fiber 7 onto the spatial light modulator 4 to achieve compensation for the distorted phase.

[0079] S2, using a CMOS camera to capture the intensity image of the sample, and converting the intensity image of the sample captured by the first CMOS camera 11 into and the sample intensity image captured by the second CMOS camera 12 , the initial phase of the sample is calculated by the TIE calculation equation .

[0080] S21, using a CMOS camera to capture the intensity image of the sample, the image captured by the first CMOS camera 11 is the sample intensity image , the image captured by the second CMOS camera 12 is the sample intensity image .

[0081] S22, Sample intensity image of the sample recorded by two CMOS cameras (i.e., overfocus image) and sample intensity image (defocused image), calculate the average intensity of the sample intensity image using formula (1) :

[0082] (1)

[0083] S23, Sample intensity image of the sample recorded by two CMOS cameras (i.e., overfocus image) and sample intensity image (defocused image), the intensity gradient of the sample intensity image is calculated using formula (2) :

[0084] (2)

[0085] S24. Substitute the calculated average intensity and intensity gradient into formula (3) to calculate the initial phase of the sample

[0086] (3)

[0087] In formulas (1) to (3), 、 Perpendicular to the optical axis The transverse coordinate of the plane, is the axial distance between the camera and the imaging plane, 、 are the spectral coordinates in the Fourier domain, are the Fourier transform operator and the inverse Fourier transform operator, respectively. Wave number and wavelength Correlation, defined as .

[0088] S3, build a PINN network, and set the initial phase of the sample As the input of the PINN network, the sample phase is iteratively optimized using the PINN network.

[0089] The PINN network consists of a data consistency layer (DC) and a CNN layer that learns network input features. The CNN layer uses the learned features to perform nonlinear corrections on the input, capturing complex information that is difficult to express in physical models. The data consistency layer calculates gradients using the physical model and provides correction information that conforms to physical constraints. The CNN layer and the data consistency layer are jointly optimized, and the network parameters are optimized through backpropagation. Iterative training ultimately obtains the optimized sample phase. .

[0090] The present invention uses PINN (physics-informed neural network) network to perform iterative optimization of phase. The PINN network architecture is as follows: Figure 3 shown.

[0091] S31, initial phase The optimization is a nonlinear optimization process, which needs to minimize the objective function J( ) to complete the nonlinear optimization. Using formula (4) to calculate the initial phase Perform nonlinear optimization:

[0092] (4)

[0093] in, is the data fidelity term, R(φ) is the regularizer, and the parameter γ controls the weight balance between them.

[0094] S32. Based on TIE phase inversion, the defocus intensity transmitted by the light field is obtained, and the phase is iteratively optimized.

[0095] S321, for phase inversion based on TIE, the present invention uses the forward operator Expressed as the defocus diffraction intensity , first use the Fresnel diffraction integral to get the propagation distance d = +z The near-field light field at , and then square the complex field to get the defocus intensity Using formula (5) to square the complex field, we can get the defocus intensity :

[0096] (5)

[0097] Among them, * represents convolution, represents the defocus intensity, 、 Perpendicular to the optical axis The transverse coordinate of the plane, is the axial distance between the camera and the imaging plane, Wave number and wavelength Correlation, defined as .

[0098] S322, The regularization optimization in can be solved by iterative gradient descent algorithm. First, use formula (6) to calculate the phase profile in each iteration Iterate until the marginal gain decreases at the local minimum and then stop:

[0099] (6)

[0100] Formula (6) is used to update the phase of the current iteration, combined with the gradient of the loss function , gradually optimize the phase to make it close to the real phase. is the iteration step size, The phase calculated by equations (1) to (3) , is the regularization term;

[0101] S33. Use a convolutional neural network to perform regularization optimization on the iteratively optimized phase, and convert the recovered phase result into an image close to the true phase.

[0102] The performance of regularized optimization is severely affected by the choice of regularizer and hyperparameters in R. Instead of using off-the-shelf regularizers as in some previous TIE retrieval algorithms, in this work, a deep learning based framework is employed to identify prior features of objects from training data.

[0103] S331. Determine the regularization function based on the convolutional neural network and use formula (7) to perform regularization constraints;

[0104] (7)

[0105] Formula (7) introduces the convolutional neural network based on formula (6): As a regularization term The approximation of CNN is used to learn complex regularization constraints instead of traditional gradient regularization, which improves the adaptability of the model to different data distributions and characteristics.

[0106] The present invention uses convolutional neural network (CNN) Reformulate the gradient descent of R. In order to keep the structure of the cascaded neural network fixed, the total number of iterations is pre-selected to be 3.

[0107] S332, use formula (8) to train the regularization function and optimize the convolutional neural network; by minimizing the network output and the corresponding truth value The mean square error between the two is used to train a parameterized The optimal regularizer for .

[0108] (8)

[0109] Formula (8) is used for training The parameters of , ensure that the regularization constraints learned by CNN can accurately reflect the true characteristics of the data. Where k represents a training dataset with a total of k samples.

[0110] S333. Based on the trained regularization function, a cascade neural network is used, combined with physical models and feature learning, to convert the recovered phase result into an image close to the real phase.

[0111] S4, build HAR-CNN network, and transform the optimized sample phase As the input of the HAR-CNN network, the HAR-CNN network is used to remove the honeycomb artifacts in the optimized phase image.

[0112] The HAR-CNN network consists of three convolutional layers, namely the feature extraction layer, the nonlinear mapping layer and the reconstruction layer. The feature extraction layer is used to extract local image features, covering the sample phase , forming an initial feature map. The nonlinear mapping layer is used to map high-dimensional feature vectors into low-dimensional features, and implement feature filtering by capturing the complex nonlinear characteristics of artifacts. The reconstruction layer is used to generate a clear image without honeycomb artifacts.

[0113] Since multi-core optical fibers introduce honeycomb artifacts during the imaging process, these artifacts will significantly affect the quality and clarity of the collected images, reducing the accuracy and usability of the imaging. In order to solve this problem, the present invention adopts a deep neural network HAR-CNN (Honeycomb Artifact Removal-Convolutional Neural Network), which is specifically used to remove honeycomb artifacts and can effectively improve the clarity and fidelity of the image. By utilizing the powerful feature extraction and processing capabilities of the HAR-CNN network, the network can identify and remove artifacts caused by multi-core optical fiber transmission, thereby improving image quality without losing image details. This can achieve clearer imaging results while maintaining high-precision phase information. The HAR-CNN network structure is as follows: Figure 4 shown.

[0114] S41. Image segmentation and feature extraction

[0115] The image is divided into blocks, and features are extracted from the divided images. The rectangular area containing the honeycomb artifacts is extracted from the phase image after the blocks, and the high-dimensional data is converted into low-dimensional data to process the nonlinear features in the image. The image is convolved and the neighborhood pixels are used to output a high-resolution non-honeycomb image.

[0116] S42. Network design and configuration

[0117] The first and second layers of the HAR-CNN network each use an activation function (e.g., ReLU) as a loss function to achieve nonlinear mapping. The mean squared error (MSE) is used as the optimization objective of the HAR-CNN network. The filter size of the first layer of the HAR-CNN network is 9 to ensure that the pattern of honeycomb artifacts is included. The convolution kernel spacing is 5-6 pixels. The filter size of the nonlinear mapping layer is set to 1, and the number of feature maps is 64 to balance computational efficiency and filtering capability.

[0118] S43. Reconstruction layer configuration and model training

[0119] The HAR-CNN network is configured with a reconstruction layer. In the reconstruction layer, the HAR-CNN network uses a filter size of 5 and a number of feature maps of 32 to optimize output image quality and computational efficiency. The HAR-CNN network is trained using a stochastic gradient descent optimizer to complete model optimization.

[0120] S44. Use the optimized HAR-CNN network to remove honeycomb artifacts in the optimized phase image.

[0121] The patch extraction layer creates a set of image patches surrounding individual honeycomb regions. The nonlinear mapping layer addresses the nonlinear characteristics of honeycomb artifacts by converting high-dimensional vectors into low-dimensional vectors. The reconstruction layer outputs a high-resolution honeycomb-free image through convolution of adjacent pixels. The mean squared error (MSE) is used as the loss function for the HAR-CNN network. Rectified linear units (ReLUs) are used as activation functions for the first and second layers. The network model is trained using a stochastic gradient descent optimizer. Because the hyperparameter settings for each layer require a trade-off between speed and performance, the spatial size of the convolution kernels and the number of feature maps must be appropriately selected. For example, the filter size of the first layer is crucial for removing honeycomb artifacts. The filter size of the first layer is set to 9 to adequately contain the honeycomb artifact pattern, with a distance of approximately 5-6 pixels between adjacent kernels. The filter size of the nonlinear mapping layer is set to 1, and the number of feature maps is set to 64, providing sufficient nonlinear filtering while also being computationally efficient. Finally, the reconstruction layer uses a filter size of 5 and 32 feature maps.

[0122] The above-described embodiments are merely descriptions of preferred implementations of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should fall within the scope of protection determined by the claims of the present invention.

Claims

1. A system for dual-view quantitative phase imaging and honeycomb artifact removal based on multi-core optical fiber transmission, characterized in that: The system comprises: a light source (1), a pinhole aperture (2), a condensing mirror (3), a spatial light modulator (4), a mirror (5), a multi-core optical fiber (7), an objective lens (8), a tube lens (9), a spectroscope (10), a first CMOS camera (11), and a second CMOS camera (12); a light beam emitted by the light source (1) passes through the pinhole aperture (2), the condensing mirror (3), the spatial light modulator (4), the mirror (5), the sample (6), the multi-core optical fiber (7), the objective lens (8), the tube lens (9), and the spectroscope (10) in sequence, and the light beam is split into two beams by the spectroscope (10) and collected by the first CMOS camera (11) and the second CMOS camera (12) respectively; The light beam emitted by the light source (1) enters the condensing mirror (3) after the shape and range of the incident light beam are adjusted by the pinhole aperture (2); the condensing mirror (3) focuses the light beam; the light beam focused by the condensing mirror (3) enters the spatial light modulator (4), and the spatial light modulator (4) corrects the phase distortion generated by the multi-core optical fiber (7); the mirror (5) is used to guide the light path and adjust the direction of the light beam so that the light beam is irradiated onto the sample (6); the multi-core optical fiber (7) is used to transmit the light signal, the objective lens (8) preliminarily amplifies the light signal transmitted through the multi-core optical fiber (7), the tube lens (9) further amplifies the image and guides the light beam to the spectroscope (10), the spectroscope (10) divides the light path into two: one part is transmitted to the first CMOS camera (11) to record the defocused plane image, and the other part is transmitted to the second CMOS camera (12) to record the overfocused plane image. The first CMOS camera (11) and the second CMOS camera (12) are connected to the same computer (13), and the computer (13) records the images at the same time.

2. A method for dual-view quantitative phase imaging and honeycomb artifact removal based on multi-core optical fiber transmission, characterized in that: The method is implemented based on the system according to claim 1, and the method comprises the following steps: S1, using a spatial light modulator (4) to correct the phase distortion generated by the multi-core optical fiber (7); S2, using a CMOS camera to capture the intensity image of the sample, and the intensity image of the sample captured by the first CMOS camera (11) and the sample intensity image captured by the second CMOS camera (12) , the initial phase of the sample is calculated by the TIE calculation equation ; S3, build a PINN network, and set the initial phase of the sample As the input of the PINN network, the sample phase is iteratively optimized using the PINN network; S4, build HAR-CNN network, and transform the optimized sample phase As the input of the HAR-CNN network, the HAR-CNN network is used to remove the honeycomb artifacts in the optimized phase image.

3. The method for dual-view quantitative phase imaging and honeycomb artifact removal based on multi-core optical fiber transmission according to claim 2, characterized in that: In step S1, the spatial light modulator (4) is used to correct the phase distortion generated by the multi-core optical fiber (7), including: S11, the spatial light modulator (4) uses off-axis holographic technology to measure the phase distortion of the multi-core optical fiber (7); S12, loading the phase conjugate of the phase distortion of the multi-core optical fiber (7) onto the spatial light modulator (4) to compensate for the distortion phase.

4. The method for dual-view quantitative phase imaging and honeycomb artifact removal based on multi-core optical fiber transmission according to claim 2, characterized in that: In step S2, the intensity image of the sample is captured by the CMOS camera, and the intensity image of the sample captured by the first CMOS camera (11) is converted into and the sample intensity image captured by the second CMOS camera (12) , the initial phase of the sample is calculated by the TIE calculation equation ,include: S21, using a CMOS camera to capture an intensity image of the sample, the image captured by the first CMOS camera (11) is the sample intensity image , the image captured by the second CMOS camera (12) is the sample intensity image ; S22, Sample intensity image of the sample recorded by two CMOS cameras and sample intensity image , use formula (1) to calculate the average intensity of the sample intensity image : （1） S23, Sample intensity image of the sample recorded by two CMOS cameras and sample intensity image , use formula (2) to calculate the intensity gradient of the sample intensity image : （2） S24, based on the average intensity and intensity gradient of the sample intensity image, calculate the initial phase of the sample using formula (3) : （3） In formulas (1) to (3), 、 Perpendicular to the optical axis The transverse coordinate of the plane, is the axial distance between the camera and the imaging plane, 、 are the spectral coordinates in the Fourier domain, are the Fourier transform operator and the inverse Fourier transform operator, respectively. Wave number and wavelength Correlation, defined as .

5. The method for dual-view quantitative phase imaging and honeycomb artifact removal based on multi-core optical fiber transmission according to claim 2, characterized in that: The PINN network includes a data consistency layer and a CNN layer for learning network input features; The CNN layer uses learned features to perform nonlinear corrections on the input, capturing complex information that is difficult to express in physical models. The data consistency layer calculates gradients through physical models and provides correction information that complies with physical constraints; The CNN layer and the data consistency layer are jointly optimized. Through back propagation, the network parameters are optimized, and iterative training finally obtains the optimized sample phase. .

6. The method for dual-view quantitative phase imaging and honeycomb artifact removal based on multi-core optical fiber transmission according to claim 5, characterized in that: In step S3, the PINN network is constructed, and the initial phase of the sample is As the input of the PINN network, the sample phase is iteratively optimized using the PINN network, including: S31, using formula (4) to calculate the initial phase Perform nonlinear optimization: （4） in, is the data fidelity term, R(φ) is the regularizer, and the parameter γ controls the weight balance between them; S32. Based on TIE phase inversion, obtain the defocus intensity transmitted by the light field and iteratively optimize the phase; S321, phase inversion based on TIE, the forward operator Expressed as defocus intensity , using the Fresnel diffraction integral to obtain the propagation distance d = +z The near-field light field at , and the defocus intensity is obtained by taking the square of the complex field using formula (5) : （5） Among them, * represents convolution, represents the defocus intensity, 、 Perpendicular to the optical axis The transverse coordinate of the plane, is the axial distance between the camera and the imaging plane, Wave number and wavelength Correlation, defined as ; S322, from the initial phase First, use formula (6) to calculate the phase profile in each iteration Iterate until the marginal gain decreases at the local minimum and then stop: （6） in, is the iteration step size, The phase calculated by equations (1) to (3) , is the regularization term; S33, using a convolutional neural network to perform regularization optimization on the iteratively optimized phase, and converting the recovered phase result into an image close to the true phase; S331. Determine the regularization function based on the convolutional neural network and use formula (7) to perform regularization constraints; （7） S332, use formula (8) to train the regularization function and optimize the convolutional neural network by minimizing the network output and the corresponding truth value The mean square error between the two is used to train a parameterized The optimal regularizer for : （8） Among them, k represents a training data set with a total of k samples; S333. Based on the trained regularization function, a cascade neural network is used, combined with physical models and feature learning, to convert the recovered phase result into an image close to the real phase.

7. The method for dual-view quantitative phase imaging and honeycomb artifact removal based on multi-core optical fiber transmission according to claim 2, characterized in that: The HAR-CNN network consists of three convolutional layers, namely feature extraction layer, nonlinear mapping layer and reconstruction layer; The feature extraction layer is used to extract local image features, covering the sample phase , forming an initial feature map; the nonlinear mapping layer is used to map the high-dimensional feature vector into a low-dimensional feature, and realize feature filtering by capturing the complex nonlinear characteristics of the artifact; the reconstruction layer is used to generate a clear image without honeycomb artifacts.

8. The method for dual-view quantitative phase imaging and honeycomb artifact removal based on multi-core optical fiber transmission according to claim 7, characterized in that: In step S4, the HAR-CNN network is constructed to optimize the sample phase. As the input of the HAR-CNN network, the HAR-CNN network is used to remove honeycomb artifacts in the optimized phase image, including: S41. Image segmentation and feature extraction The image is divided into blocks, and features are extracted from the divided images. The rectangular area containing the honeycomb artifacts is extracted from the phase image after the blocks, and the high-dimensional data is converted into low-dimensional data to process the nonlinear features in the image. The image is convolved and the neighborhood pixels are used to output a high-resolution non-honeycomb image. S42. Network design and configuration The first and second layers of the HAR-CNN network each use activation functions as loss functions to achieve nonlinear mapping; the mean square error is used as the optimization target of the HAR-CNN network; the filter size of the first layer of the HAR-CNN network is 9 to ensure that the pattern of honeycomb artifacts is included, the convolution kernel spacing is 5-6 pixels, the filter size of the nonlinear mapping layer is set to 1, and the number of feature maps is 64 to balance computational efficiency and filtering capability; S43. Reconstruction layer configuration and model training The HAR-CNN network is configured with a reconstruction layer. In the reconstruction layer, the HAR-CNN network uses a filter size of 5 and a number of feature maps of 32 to optimize output image quality and computational efficiency. The HAR-CNN network is trained using a stochastic gradient descent optimizer to complete model optimization. S44. Use the optimized HAR-CNN network to remove honeycomb artifacts in the optimized phase image.

Images