System and method for dual-view quantitative phase imaging and honeycomb artifact removal

Abstract

The invention relates to a system and a method for dual-view quantitative phase imaging and honeycomb artifact removal based on multi-core optical fiber transmission. The system comprises a light source, a pinhole aperture, a condensing mirror, a spatial light modulator, a mirror, a multi-core optical fiber, an objective lens, a tube lens, a spectroscope, a first CMOS camera and a second CMOS camera. According to the multi-core optical fiber transmission-based system for dual-view quantitative phase imaging and honeycomb artifact removal, the phase distortion of the multi-core optical fiber is corrected through the spatial light modulator, different illumination angles are generated by using the spectroscope, and the transmission characteristics of the multi-core optical fiber are combined, so that the dual-view quantitative phase imaging and honeycomb artifact removal can be realized on the premise of not changing the position of a sample. According to the method, phase information under two different visual angles is rapidly acquired, and then the two groups of data are processed by using the following method, so that three-dimensional reconstruction is realized, and the imaging depth and accuracy are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of optical fiber imaging, and in particular to a system and method for dual-viewing angle quantitative phase imaging and honeycomb artifact removal. Background Art

[0002] Fiber optic endoscopes have a wide range of clinical application potentials in in situ diagnosis and interventional surgery in vivo due to their flexibility and compactness. They have been applied in the fields of optical biopsy of the gastrointestinal tract, urinary system and respiratory tract, as well as cancer diagnosis. However, fiber optic microscopy relies on imaging methods that only provide amplitude contrast, while quantitative phase information is required to observe "phase objects" and measure various morphological parameters of tissues (such as refractive index changes or material density). Conventional detectors can only record the amplitude of light and cannot obtain phase information. Therefore, other technical means are needed to obtain phase information.

[0003] Quantitative Phase Imaging (QPI) is a label-free technique that can provide both morphological and quantitative biophysical information in biomedicine. The Transport of Intensity Equation (TIE) uses the stacking of through-focus intensity profiles under partially coherent illumination to recover phase information. TIE phase retrieval relies on the irrotational flow assumption, which results in ambiguity of absorbing phase objects.

[0004] Multi-core fiber (MCF) enables ultra-thin probes for in vivo imaging, but images obtained through fiber bundles are affected by artifacts such as honeycomb patterns, which are caused by the different light transmittances of the core and cladding, and the presence of tiny gaps between densely packed single-filament fibers. These gaps and discontinuities appear as honeycomb-like artifacts in the reconstructed images, affecting image quality and clarity.

[0005] Therefore, there is an urgent need to realize a system and method for dual-view quantitative phase imaging and honeycomb artifact removal based on multi-core optical fiber transmission. Summary of the invention

[0006] The purpose of the present invention is to provide a system and method for dual-view quantitative phase imaging and honeycomb artifact removal, which system and method are used to perform phase recovery on biological tissue and use a neural network to suppress honeycomb artifacts in the sample phase recovered by the dual-view quantitative phase imaging system.

[0007] To achieve the above object, the present invention adopts the following technical solutions:

[0008] The present invention includes a system for dual-view quantitative phase imaging and honeycomb artifact removal based on multi-core optical fiber transmission, the system comprising: a light source, a pinhole aperture, a condensing mirror, a spatial light modulator, a mirror, a multi-core optical fiber, an objective lens, a tube lens, a spectroscope, a first CMOS camera and a second CMOS camera; a light beam emitted by the light source passes through the pinhole aperture, the condensing mirror, the spatial light modulator, the mirror, the sample, the multi-core optical fiber, the objective lens, the tube lens and the spectroscope in sequence, and the light beam is split into two beams by the beam splitter and collected by the first CMOS camera and the second CMOS camera respectively.

[0009] The light beam emitted by the light source enters a condensing mirror after the shape and range of the incident light beam are adjusted by a pinhole aperture; the condensing mirror focuses the light beam; the light beam focused by the condensing mirror enters a spatial light modulator, and the spatial light modulator corrects the phase distortion generated by the multi-core optical fiber; the mirror is used to guide the light path and adjust the direction of the light beam so that the light beam is irradiated onto the sample; the multi-core optical fiber is used to transmit the light signal, the objective lens preliminarily amplifies the light signal transmitted through the multi-core optical fiber, the tube lens further amplifies the image, and guides the light beam to a beam splitter, which divides the light path into two: one part is transmitted to a first CMOS camera to record an under-focus plane image, and the other part is transmitted to a second CMOS camera to record an over-focus plane image, the first CMOS camera and the second CMOS camera are connected to the same computer, and the computer records the image simultaneously.

[0010] The present invention also includes a method for dual-view quantitative phase imaging and honeycomb artifact removal based on multi-core optical fiber transmission. The method is implemented based on the above system and includes the following steps:

[0011] S1, using spatial light modulator to correct the phase distortion produced by multi-core optical fiber;

[0012] S2, using a CMOS camera to capture an intensity image of the sample, and converting the sample intensity image I captured by the first CMOS camera into -Z and the sample intensity image I captured by the second CMOS camera +Z , the initial phase of the sample is calculated by the TIE calculation equation

[0013] S3, build a PINN network, and convert the initial phase of the sample As the input of the PINN network, the sample phase is iteratively optimized using the PINN network.

[0014] S4, build the HAR-CNN network, and transform the optimized sample phase As the input of the HAR-CNN network, the HAR-CNN network is used to remove the honeycomb artifacts in the optimized phase image.

[0015] As a further improvement of the above technical solution, in step S1, using a spatial light modulator to correct the phase distortion generated by the multi-core optical fiber includes:

[0016] S11, spatial light modulator uses off-axis holography to measure the phase distortion of multi-core optical fiber;

[0017] S12, loading the phase conjugate of the multi-core optical fiber phase distortion onto the spatial light modulator to compensate for the distorted phase.

[0018] As a further improvement of the above technical solution, in step S2, the intensity image of the sample is captured by a CMOS camera, and the sample intensity image I captured by the first CMOS camera is -Z and the sample intensity image I captured by the second CMOS camera +Z , the initial phase of the sample is calculated by the TIE calculation equation include:

[0019] In step S2, a CMOS camera is used to capture an intensity image of the sample, and the sample intensity image I captured by the first CMOS camera (11) is converted into -Z and the sample intensity image I captured by the second CMOS camera (12) +Z , the initial phase of the sample is calculated by the TIE calculation equation include:

[0020] S21, using a CMOS camera to capture an intensity image of the sample, the image captured by the first CMOS camera (11) is the sample intensity image I -Z , the image captured by the second CMOS camera (12) is the sample intensity image I +Z .

[0021] S22, sample intensity image I of the sample recorded by two CMOS cameras +Z and the sample intensity image I -Z , use formula (1) to calculate the average intensity I(x,y) of the sample intensity image:

[0022]

[0023] S23, based on the sample intensity image I(x, y, +z) and the sample intensity image I(x, y, -z) of the sample recorded by the two CMOS cameras, calculate the intensity gradient of the sample intensity image using formula (2):

[0024]

[0025] S24, based on the average intensity and intensity gradient of the sample intensity image, calculate the initial phase of the sample using formula (3):

[0026]

[0027] In equations (1) to (3), x and y are the lateral coordinates of the z plane perpendicular to the optical axis, z is the axial distance between the camera and the imaging plane, and u and v are the spectral coordinates in the Fourier domain. and are the Fourier transform operator and the inverse Fourier transform operator respectively, k is the wave number related to the wavelength λ, and is defined as

[0028] As a further improvement of the above technical solution, the PINN network includes a data consistency layer and a CNN layer for learning network input features;

[0029] The CNN layer performs nonlinear correction on the input through learned features, capturing complex information that is difficult to express in physical models.

[0030] The data consistency layer calculates the gradient through the physical model and provides correction information that conforms to the physical constraints. The CNN layer and the data consistency layer are jointly optimized. Through back propagation, the network parameters are optimized, and the optimized sample phase is finally obtained through iterative training.

[0031] As a further improvement of the above technical solution, in step S3, the PINN network is constructed to convert the initial phase of the sample As the input of the PINN network, the sample phase is iteratively optimized using the PINN network, including:

[0032] S31, using formula (4) to calculate the initial phase Perform nonlinear optimization:

[0033]

[0034] in, is the data fidelity term, is a regularizer, and the parameter γ controls the weight balance between them.

[0035] Initial Phase The optimization is a nonlinear optimization process, which requires minimizing the objective function to complete the nonlinear optimization. Equation (4) is the objective function that defines the phase optimization problem. The goal is to optimize the predicted intensity With true strength I -Z The error is minimized, and the regularization term is added to constrain the optimization.

[0036] S32. Based on TIE phase inversion, the defocus intensity transmitted by the light field is obtained, and the phase is iteratively optimized.

[0037] S321, based on TIE phase inversion, the forward operator H is expressed as the defocus intensity I +Z , the near-field light field at the propagation distance d = +z is obtained by using the Fresnel diffraction integral, and the defocus intensity I is obtained by squaring the complex field using equation (5): +Z :

[0038]

[0039] The defocus intensity I is calculated using formula (5): +Z , which is the light field intensity when the propagation distance is +z. This is the basic step to achieve phase inversion and provides input for further optimization. Among them, * represents convolution, I -Z represents the defocus intensity, x and y are the lateral coordinates perpendicular to the optical axis z plane, z is the axial distance between the camera and the imaging plane, and k is the wave number related to the wavelength λ, which is defined as

[0040] S322, from the initial phase First, the phase profile in each iteration is calculated using formula (6) Iterate and update until the marginal gain decreases at the local minimum, and then the iteration stops:

[0041]

[0042] Formula (6) is used to update the phase of the current iteration, combined with the gradient of the loss function The phase is gradually optimized to make it close to the true phase. Among them, α is the iteration step size, H inν (I -z ,I +z ) represents the phase calculated by equations (1) to (3) is the regularization term.

[0043] S33. Use a convolutional neural network to regularize and optimize the iteratively optimized phase, and convert the recovered phase result into an image close to the real phase.

[0044] The performance of regularized optimization is heavily affected by the choice of regularizer and hyperparameters in R. Instead of using off-the-shelf regularizers as in some previous TIE retrieval algorithms, in this work, a deep learning based framework is employed to identify prior features of objects from training data.

[0045] S331, determining a regularization function based on a convolutional neural network, and using formula (7) to perform regularization constraints;

[0046]

[0047] Formula (7) introduces the convolutional neural network f on the basis of formula (6): CNN As a regularization term The present invention uses a convolutional neural network (CNN) to learn complex regularization constraints instead of traditional gradient regularization, which improves the adaptability of the model to different data distributions and features. CNN Re-expressing the regularization term To keep the structure of the cascaded neural network fixed, the total number of iterations was pre-selected to be 3.

[0048] S332. Use formula (8) to train the regularization function and optimize the convolutional neural network.

[0049] By minimizing the network output and the corresponding truth value The mean square error between is used to train an optimal regularizer parameterized as f

[0050]

[0051] Formula (8) is used to train the convolutional neural network f CNN The parameters of , ensure that the regularization constraints learned by CNN can accurately reflect the true characteristics of the data. Where k represents a training data set with a total of k samples.

[0052] S333. Based on the trained regularization function, a cascade neural network is used to combine the physical model and feature learning to convert the recovered phase result into an image close to the real phase.

[0053] As a further improvement of the above technical solution, the HAR-CNN network consists of three convolutional layers, namely a feature extraction layer, a nonlinear mapping layer and a reconstruction layer;

[0054] The feature extraction layer is used to extract local image features, covering the sample phase An initial feature map is formed; the nonlinear mapping layer is used to map the high-dimensional feature vector into a low-dimensional feature, and to achieve feature filtering by capturing the complex nonlinear characteristics of the artifacts; the reconstruction layer is used to generate a clear image without honeycomb artifacts.

[0055] As a further improvement of the above technical solution, in step S4, the HAR-CNN network is constructed to optimize the sample phase As the input of the HAR-CNN network, the HAR-CNN network is used to remove the honeycomb artifacts in the optimized phase image, including:

[0056] S41. Image segmentation and feature extraction

[0057] The image is divided into blocks, and features are extracted from the divided images. The rectangular area containing the honeycomb artifacts is extracted from the phase image after division, and the high-dimensional data is converted into low-dimensional data to process the nonlinear features in the image; the image is convolved and the neighborhood pixels are used to output a high-resolution non-cellular image.

[0058] S42. Network design and configuration

[0059] The first and second layers of the HAR-CNN network respectively use activation functions (such as ReLU) as loss functions to achieve nonlinear mapping; the mean square error (MSE) is used as the optimization target of the HAR-CNN network; the filter size of the first layer of the HAR-CNN network is 9 to ensure the pattern of containing honeycomb artifacts, the convolution kernel interval is 5-6 pixels, the nonlinear mapping layer filter size is set to 1, and the number of feature maps is 64 to balance computational efficiency and filtering capability.

[0060] S43. Reconstruction layer configuration and model training

[0061] The HAR-CNN network is configured with a reconstruction layer. In the reconstruction layer, the filter size used by the HAR-CNN network is 5, and the number of feature maps is set to 32 to optimize the output image quality and computational efficiency. The HAR-CNN network is trained using a stochastic gradient descent optimizer to complete model optimization.

[0062] S44. Use the optimized HAR-CNN network to remove honeycomb artifacts in the optimized phase image.

[0063] Compared with the prior art, the advantages of the present invention are:

[0064] The present invention can restore the phase of the sample to be tested and remove honeycomb artifacts by combining two neural network models with a dual-view quantitative phase imaging system based on multi-core optical fiber, thus overcoming the problem that the phase image generated by multi-core optical fiber in the prior art contains honeycomb artifacts. The system and method for dual-view quantitative phase imaging and honeycomb artifact removal based on multi-core optical fiber transmission described in the present invention can quickly collect phase information at two different viewing angles without changing the position of the sample through different illumination angles generated by a spatial light modulator and combined with the transmission characteristics of multi-core optical fiber, and then process the two sets of data using the method described below to achieve three-dimensional reconstruction and improve imaging depth and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1It is a schematic diagram of a system for dual-view quantitative phase imaging and honeycomb artifact removal based on multi-core optical fiber transmission in the present invention;

[0066] Figure 2 It is a flow chart of a method for dual-view quantitative phase imaging and honeycomb artifact removal based on multi-core optical fiber transmission in the present invention;

[0067] Figure 3 This is the network architecture diagram of the PINN network;

[0068] Figure 4 It is the network architecture diagram of the HAR-CNN network.

[0069] in:

[0070] 1. Light source, 2. Pinhole aperture, 3. Condenser mirror, 4. Spatial light modulator, 5. Mirror, 6. Sample, 7. Multi-core optical fiber, 8. Objective lens, 9. Tube lens, 10. Beam splitter, 11. First CMOS camera, 12. Second CMOS camera, 13. Computer. DETAILED DESCRIPTION

[0071] The present invention will be further described below in conjunction with the accompanying drawings:

[0072] like Figure 1 A system for dual-view quantitative phase imaging and honeycomb artifact removal based on multi-core optical fiber transmission is shown, the system comprising: a light source 1, a pinhole aperture 2, a condensing mirror 3, a spatial light modulator 4, a mirror 5, a multi-core optical fiber 7, an objective lens 8, a tube lens 9, a beam splitter 10, a first CMOS camera 11 and a second CMOS camera 12. The light beam emitted by the light source 1 passes through the pinhole aperture 2, the condensing mirror 3, the spatial light modulator 4, the mirror 5, the sample 6, the multi-core optical fiber 7, the objective lens 8, the tube lens 9, and the beam splitter 10 in sequence, and the beam splitter 10 splits the light beam into two beams, which are collected by the first CMOS camera 11 and the second CMOS camera 12 respectively. The light source 1 controls the shape and range of the incident light beam through the pinhole aperture 2 to ensure that the light accurately enters the condensing mirror 3; the condensing mirror 3 is used to focus the light beam and improve the light beam quality; the light beam focused by the condensing mirror 3 enters the spatial light modulator 4, and the spatial light modulator 4 is used to correct the phase distortion generated by the multi-core optical fiber 7; the mirror 5 is used to guide the light path and adjust the direction of the light beam so that the light beam is irradiated onto the sample 6; the multi-core optical fiber 7 is used to transmit the light signal, the objective lens 8 preliminarily amplifies the light signal transmitted through the multi-core optical fiber 7, the tube lens 9 further amplifies the image and guides the light beam to the beam splitter 10, the beam splitter 10 divides the light path into two: one part is transmitted to the first CMOS camera 11 to record the out-of-focus plane image, and the other part is transmitted to the second CMOS camera 12 to record the over-focus plane image. The first CMOS camera 11 and the second CMOS camera 12 are connected to the same computer 13 to record the image at the same time.

[0073] Specifically, the light source 1 provides uniform and stable illumination for the entire system, and a laser light source or LED light source with good coherence is usually selected. The pinhole aperture 2 controls the size of the light beam entering the subsequent optical element to improve the spatial resolution of the system. The condenser mirror 3 focuses the light from the light source 1 onto the sample 6 to ensure that the sample surface is evenly illuminated. The spatial light modulator 4 is used to dynamically adjust the incident light wavefront, which can achieve complex light field control, such as phase modulation. In this system, the spatial light modulator 4 can be used to generate different illumination angles to support dual-view imaging. The mirror 5 is used to change the direction of the light path and guide the light to propagate along a predetermined path. As one of the key components, the multi-core optical fiber 7 can simultaneously transmit multiple independent light channels, and each core carries information from different perspectives, which makes it possible to obtain multi-angle image data without moving the sample, thereby achieving three-dimensional reconstruction and honeycomb artifact removal. The objective lens 8 is used to magnify the sample and pass the details of the sample to the subsequent optical components. The tube lens 9 is used in conjunction with the objective lens 8 to further adjust the imaging ratio and optimize the image quality. The spectroscope 10 is used to split the light reflected from the sample into two paths, which are sent to two CMOS cameras respectively, so that images of the same area can be captured from two different perspectives. The first CMOS camera 11 and the second CMOS camera 12 are used to respectively record the images formed by the two beams of light separated by the spectroscope 10. The two cameras work synchronously to ensure that the acquired data has time consistency.

[0074] The system for dual-view quantitative phase imaging and honeycomb artifact removal based on multi-core optical fiber transmission described in the present invention can quickly collect phase information at two different viewing angles without changing the sample position through different illumination angles generated by the spatial light modulator 4 and combined with the transmission characteristics of the multi-core optical fiber 7. The two sets of data are then processed using the method described below to achieve three-dimensional reconstruction and improve imaging depth and accuracy.

[0075] like Figure 2 As shown, the present invention also includes a method for dual-view quantitative phase imaging and honeycomb artifact removal based on multi-core optical fiber transmission, the method comprising the following steps:

[0076] S1. Use the spatial light modulator 4 to correct the phase distortion generated by the multi-core optical fiber 7.

[0077] The spatial light modulator 4 is used to correct the phase distortion produced by the multi-core optical fiber 7 to eliminate the influence of the properties of the multi-core optical fiber itself on the phase of the measured sample.

[0078] Specifically, after the light beam emitted by the light source is processed by the spectral filter, the pinhole aperture, and the condensing mirror in sequence, when the light path passes through the multi-core optical fiber 7, the phase distortion caused by the multi-core optical fiber 7 can be corrected by using the digital optical phase conjugation of the spatial light modulator 4. The spatial light modulator 4 first measures the phase distortion of the multi-core optical fiber 7 using the off-axis holographic technology, and then loads the phase conjugate of the phase distortion of the multi-core optical fiber 7 onto the spatial light modulator 4 to achieve compensation for the distorted phase.

[0079] S2, using a CMOS camera to capture an intensity image of the sample, and converting the sample intensity image I captured by the first CMOS camera 11 into -Z and the sample intensity image I captured by the second CMOS camera 12 +Z , the initial phase of the sample is calculated by the TIE calculation equation

[0080] S21, using a CMOS camera to capture an intensity image of the sample, the image captured by the first CMOS camera 11 is the sample intensity image I -Z , the image captured by the second CMOS camera 12 is the sample intensity image I +Z .

[0081] S22, sample intensity image I of the sample recorded by two CMOS cameras +Z (i.e., overfocus image) and sample intensity image I -Z (out-of-focus image), the average intensity I(x,y) of the sample intensity image is calculated using formula (1):

[0082]

[0083] S23, sample intensity image I of the sample recorded by two CMOS cameras +Z (i.e., overfocus image) and sample intensity image I -Z (out-of-focus image), the intensity gradient of the sample intensity image is calculated using formula (2)

[0084]

[0085] S24, substitute the calculated average intensity and intensity gradient into formula (3) to calculate the initial phase of the sample

[0086]

[0087] In equations (1) to (3), x and y are the lateral coordinates of the z plane perpendicular to the optical axis, z is the axial distance between the camera and the imaging plane, and u and v are the spectral coordinates in the Fourier domain. and are the Fourier transform operator and the inverse Fourier transform operator respectively, k is the wave number related to the wavelength λ, and is defined as

[0088] S3, build a PINN network, and convert the initial phase of the sample As the input of the PINN network, the sample phase is iteratively optimized using the PINN network.

[0089] The PINN network includes a data consistency layer (DC) and a CNN layer for learning network input features. The CNN layer uses the learned features to perform nonlinear corrections on the input to capture complex information that is difficult to express in the physical model. The data consistency layer calculates the gradient through the physical model and provides correction information that conforms to the physical constraints. The CNN layer and the data consistency layer are jointly optimized. Through back propagation, the network parameters are optimized, and iterative training finally obtains the optimized sample phase.

[0090] The present invention uses a PINN (physics-informed neural network) network to perform iterative optimization of the phase. The PINN network architecture is as follows: Figure 3 shown.

[0091] S31, initial phase The optimization is a nonlinear optimization process, which requires minimizing the objective function To complete the nonlinear optimization. Using equation (4) to calculate the initial phase Perform nonlinear optimization:

[0092]

[0093] in, is the data fidelity term, is a regularizer, and the parameter γ controls the weight balance between them.

[0094] S32. Based on TIE phase inversion, the defocus intensity transmitted by the light field is obtained, and the phase is iteratively optimized.

[0095] S321. For phase inversion based on TIE, the present invention expresses the forward operator H as the defocused diffraction intensity I +Z First, the Fresnel diffraction integral is used to obtain the near-field light field at the propagation distance d = +z, and then the complex field is squared to obtain the defocus intensity I +Z Using formula (5), we square the complex field to get the defocus intensity I +Z :

[0096]

[0097] Among them, * represents convolution, I -Z represents the defocus intensity, x and y are the lateral coordinates perpendicular to the optical axis z plane, z is the axial distance between the camera and the imaging plane, and k is the wave number related to the wavelength λ, which is defined as

[0098] S322, The regularization optimization in can be solved by an iterative gradient descent algorithm. First, the phase profile in each iteration is calculated using formula (6) Iterate and update until the marginal gain decreases at the local minimum, and then the iteration stops:

[0099]

[0100] Formula (6) is used to update the phase of the current iteration, combined with the gradient of the loss function The phase is gradually optimized to make it close to the true phase. Among them, α is the iteration step size, H inν (I -z ,I +z ) represents the phase calculated by equations (1) to (3) is the regularization term;

[0101] S33. Use a convolutional neural network to regularize and optimize the iteratively optimized phase, and convert the recovered phase result into an image close to the real phase.

[0102] The performance of regularized optimization is heavily affected by the choice of regularizer and hyperparameters in R. Instead of using off-the-shelf regularizers as in some previous TIE retrieval algorithms, in this work, a deep learning based framework is employed to identify prior features of objects from training data.

[0103] S331, determining a regularization function based on a convolutional neural network, and using formula (7) to perform regularization constraints;

[0104]

[0105] Formula (7) introduces the convolutional neural network f on the basis of formula (6): CNN As a regularization term The approximation of CNN is to use CNN to learn complex regularization constraints instead of traditional gradient regularization, which improves the adaptability of the model to different data distributions and characteristics.

[0106] The present invention uses a convolutional neural network (CNN) CNN Re-formulate the gradient descent of R. In order to keep the structure of the cascaded neural network fixed, the total number of iterations was pre-selected to be 3.

[0107] S332, using formula (8) to train the regularization function, the convolutional neural network is optimized; by minimizing the network output and the corresponding truth value The mean square error between the two is used to train an optimal regularizer f parameterized as f CNN .

[0108]

[0109] Formula (8) is used to train the convolutional neural network f CNN The parameters of , ensure that the regularization constraints learned by CNN can accurately reflect the true characteristics of the data. Where k represents a training data set with a total of k samples.

[0110] S333. Based on the trained regularization function, a cascade neural network is used to combine the physical model and feature learning to convert the recovered phase result into an image close to the real phase.

[0111] S4, build the HAR-CNN network, and transform the optimized sample phase As the input of the HAR-CNN network, the HAR-CNN network is used to remove the honeycomb artifacts in the optimized phase image.

[0112] The HAR-CNN network consists of three convolutional layers, namely feature extraction layer, nonlinear mapping layer and reconstruction layer. The feature extraction layer is used to extract local image features, covering the sample phase The nonlinear mapping layer is used to map the high-dimensional feature vector to a low-dimensional feature, and to achieve feature filtering by capturing the complex nonlinear characteristics of the artifacts. The reconstruction layer is used to generate a clear image without honeycomb artifacts.

[0113] Since multi-core optical fibers will introduce honeycomb artifacts during the imaging process, these artifacts will significantly affect the quality and clarity of the collected images, reducing the accuracy and availability of the imaging. In order to solve this problem, the present invention adopts a deep neural network HAR-CNN (Honeycomb Artifact Removal-Convolutional Neural Network), which is specifically used to remove honeycomb artifacts and can effectively improve the clarity and fidelity of the image. By utilizing the powerful feature extraction and processing capabilities of the HAR-CNN network, the network can identify and remove artifacts caused by multi-core optical fiber transmission, thereby improving image quality without losing image details. In this way, clearer imaging results can be obtained while maintaining high-precision phase information. The HAR-CNN network structure is as follows Figure 4 shown.

[0114] S41. Image segmentation and feature extraction

[0115] The image is divided into blocks, and features are extracted from the divided images. The rectangular area containing the honeycomb artifacts is extracted from the phase image after the blocks, and the high-dimensional data is converted into low-dimensional data to process the nonlinear features in the image; the image is convolved and the neighborhood pixels are used to output a high-resolution non-cellular image;

[0116] S42. Network design and configuration

[0117] The first and second layers of the HAR-CNN network respectively use activation functions (such as ReLU) as loss functions to achieve nonlinear mapping; the mean square error (MSE) is used as the optimization target of the HAR-CNN network; the filter size of the first layer of the HAR-CNN network is 9 to ensure the pattern containing honeycomb artifacts, the convolution kernel interval is 5-6 pixels, the filter size of the nonlinear mapping layer is set to 1, and the number of feature maps is 64 to balance computational efficiency and filtering ability;

[0118] S43. Reconstruction layer configuration and model training

[0119] The HAR-CNN network is configured with a reconstruction layer. In the reconstruction layer, the filter size used by the HAR-CNN network is 5, and the number of feature maps is set to 32 to optimize the output image quality and computational efficiency; the HAR-CNN network is trained using a stochastic gradient descent optimizer to complete model optimization;

[0120] S44. Use the optimized HAR-CNN network to remove honeycomb artifacts in the optimized phase image.

[0121] The patch extraction layer creates a set of image patches around a single honeycomb region, and the nonlinear mapping layer handles the nonlinear characteristics of honeycomb artifacts by converting high-dimensional vectors into low-dimensional vectors. The reconstruction layer outputs a high-resolution honeycomb-free image through convolution of adjacent pixels. The mean square error (MSE) is used as the loss function of the HAR-CNN network. The activation functions of the first and second layers are used as rectified linear units (ReLU). The network model is trained using a stochastic gradient descent optimizer. Since the hyperparameter settings of each layer require a trade-off between speed and performance, it is necessary to choose the appropriate spatial size of the convolution kernel and the number of feature maps. For example, the filter size of the first layer is crucial for removing honeycomb artifacts. The filter size of the first layer is set to 9 to sufficiently contain the pattern of honeycomb artifacts, and the distance between two adjacent kernels is about 5-6 pixels. The filter size of the nonlinear mapping layer is set to 1 and the number of feature maps is 64, which provides sufficient nonlinear filtering while also being computationally efficient. Finally, the reconstruction layer uses a filter size of 5 and the number of feature maps is 32.

[0122] The above-described embodiments are merely descriptions of preferred implementations of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the scope of protection determined by the claims of the present invention.

Claims

1. A system for dual-view quantitative phase imaging and honeycomb artifact removal based on multi-core optical fiber transmission, characterized in that: The system comprises: a light source (1), a pinhole aperture (2), a condensing mirror (3), a spatial light modulator (4), a mirror (5), a multi-core optical fiber (7), an objective lens (8), a tube lens (9), a beam splitter (10), a first CMOS camera (11) and a second CMOS camera (12); a light beam emitted by the light source (1) passes through the pinhole aperture (2), the condensing mirror (3), the spatial light modulator (4), the mirror (5), the sample (6), the multi-core optical fiber (7), the objective lens (8), the tube lens (9) and the beam splitter (10) in sequence, and the light beam is split into two beams by the beam splitter (10) and collected by the first CMOS camera (11) and the second CMOS camera (12) respectively; The light beam emitted by the light source (1) enters a condensing mirror (3) after the shape and range of the incident light beam are adjusted by a pinhole aperture (2); the condensing mirror (3) focuses the light beam; the light beam focused by the condensing mirror (3) enters a spatial light modulator (4), and the spatial light modulator (4) corrects the phase distortion generated by the multi-core optical fiber (7); the mirror (5) is used to guide the light path and adjust the direction of the light beam so that the light beam is irradiated onto the sample (6); the multi-core optical fiber (7) is used to transmit light signals, the objective lens (8) preliminarily amplifies the light signal transmitted through the multi-core optical fiber (7), the tube lens (9) further amplifies the image, and guides the light beam to a beam splitter (10), and the beam splitter (10) divides the light path into two parts: one part is transmitted to a first CMOS camera (11) to record an under-focus plane image, and the other part is transmitted to a second CMOS camera (12) to record an over-focus plane image, the first CMOS camera (11) and the second CMOS camera (12) are connected to the same computer (13), and the computer (13) records the images simultaneously.

2. A method for dual-view quantitative phase imaging and honeycomb artifact removal based on multi-core optical fiber transmission, characterized in that: The method is implemented based on the system described in claim 1, and the method comprises the following steps: S1, using a spatial light modulator (4) to correct phase distortion generated by a multi-core optical fiber (7); S2, using a CMOS camera to capture an intensity image of the sample, and converting the sample intensity image I captured by the first CMOS camera (11) into -Z and the sample intensity image I captured by the second CMOS camera (12) +Z , the initial phase of the sample is calculated by the TIE calculation equation S3, build a PINN network, and convert the initial phase of the sample As the input of the PINN network, the sample phase is iteratively optimized using the PINN network; S4, build the HAR-CNN network, and transform the optimized sample phase As the input of the HAR-CNN network, the HAR-CNN network is used to remove the honeycomb artifacts in the optimized phase image.

3. The method for dual-view quantitative phase imaging and honeycomb artifact removal based on multi-core optical fiber transmission according to claim 2, characterized in that: In the step S1, the spatial light modulator (4) is used to correct the phase distortion generated by the multi-core optical fiber (7), comprising: S11, the spatial light modulator (4) uses off-axis holographic technology to measure the phase distortion of the multi-core optical fiber (7); S12, loading the phase conjugate of the phase distortion of the multi-core optical fiber (7) onto the spatial light modulator (4) to compensate for the distorted phase.

4. The method for dual-view quantitative phase imaging and honeycomb artifact removal based on multi-core optical fiber transmission according to claim 2, characterized in that: In step S2, a CMOS camera is used to capture an intensity image of the sample, and the sample intensity image I captured by the first CMOS camera (11) is converted into -Z and the sample intensity image I captured by the second CMOS camera (12) +Z , the initial phase of the sample is calculated by the TIE calculation equation include: S21, using a CMOS camera to capture an intensity image of the sample, the image captured by the first CMOS camera (11) is the sample intensity image I -Z , the image captured by the second CMOS camera (12) is the sample intensity image I +Z ; S22, sample intensity image I of the sample recorded by two CMOS cameras +Z and the sample intensity image I -Z , use formula (1) to calculate the average intensity I(x,y) of the sample intensity image: S23, based on the sample intensity image I(x, y, +z) and the sample intensity image I(x, y, -z) of the sample recorded by the two CMOS cameras, calculate the intensity gradient of the sample intensity image using formula (2): S24, based on the average intensity and intensity gradient of the sample intensity image, calculate the initial phase of the sample using formula (3): In equations (1) to (3), x and y are the lateral coordinates of the z plane perpendicular to the optical axis, z is the axial distance between the camera and the imaging plane, and u and v are the spectral coordinates in the Fourier domain. and are the Fourier transform operator and the inverse Fourier transform operator respectively, k is the wave number related to the wavelength λ, and is defined as 5. The method for dual-view quantitative phase imaging and honeycomb artifact removal based on multi-core optical fiber transmission according to claim 2, characterized in that: The PINN network includes a data consistency layer and a CNN layer for learning network input features; The CNN layer performs nonlinear correction on the input through learned features to capture complex information that is difficult to express in physical models; The data consistency layer calculates gradients through a physical model and provides correction information that complies with physical constraints; The CNN layer and the data consistency layer are jointly optimized. Through back propagation, the network parameters are optimized, and iterative training finally obtains the optimized sample phase.

6. The method for dual-view quantitative phase imaging and honeycomb artifact removal based on multi-core optical fiber transmission according to claim 5, characterized in that: In step S3, the PINN network is constructed to convert the initial phase of the sample As the input of the PINN network, the sample phase is iteratively optimized using the PINN network, including: S31, using formula (4) to calculate the initial phase Perform nonlinear optimization: in, is the data fidelity term, is a regularizer, and the parameter γ controls the weight balance between them; S32, based on TIE phase inversion, obtaining the defocus intensity transmitted by the light field, and iteratively optimizing the phase; S321, based on TIE phase inversion, the forward operator H is expressed as the defocus intensity I +Z , the near-field light field at the propagation distance d = +z is obtained by using the Fresnel diffraction integral, and the defocus intensity I is obtained by squaring the complex field using equation (5): +Z : Among them, * represents convolution, I -Z represents the defocus intensity, x and y are the lateral coordinates perpendicular to the optical axis z plane, z is the axial distance between the camera and the imaging plane, and k is the wave number related to the wavelength λ, which is defined as S322, from the initial phase First, the phase profile in each iteration is calculated using formula (6) Iterate and update until the marginal gain decreases at the local minimum, and then the iteration stops: Among them, α is the iteration step size, H unv (I -z ,I +z ) represents the phase calculated by equations (1) to (3) is the regularization term; S33, using a convolutional neural network to perform regularization optimization on the iteratively optimized phase, and converting the recovered phase result into an image close to the real phase; S331, determining a regularization function based on a convolutional neural network, and using formula (7) to perform regularization constraints; S332, using formula (8) to train the regularization function, optimize the convolutional neural network by minimizing the network output and the corresponding truth value The mean square error between is used to train an optimal regularizer parameterized as f Here, k represents a training data set with a total of k samples. S333. Based on the trained regularization function, a cascade neural network is used to combine the physical model and feature learning to convert the recovered phase result into an image close to the real phase.

7. The method for dual-view quantitative phase imaging and honeycomb artifact removal based on multi-core optical fiber transmission according to claim 2, characterized in that: The HAR-CNN network consists of three convolutional layers, namely, a feature extraction layer, a nonlinear mapping layer, and a reconstruction layer; The feature extraction layer is used to extract local image features, covering the sample phase An initial feature map is formed; the nonlinear mapping layer is used to map the high-dimensional feature vector into a low-dimensional feature, and to achieve feature filtering by capturing the complex nonlinear characteristics of the artifacts; the reconstruction layer is used to generate a clear image without honeycomb artifacts.

8. The method for dual-view quantitative phase imaging and honeycomb artifact removal based on multi-core optical fiber transmission according to claim 7, characterized in that: In step S4, the HAR-CNN network is constructed to optimize the sample phase As the input of the HAR-CNN network, the HAR-CNN network is used to remove the honeycomb artifacts in the optimized phase image, including: S41. Image segmentation and feature extraction The image is divided into blocks, and features are extracted from the divided images. The rectangular area containing the honeycomb artifacts is extracted from the phase image after the blocks, and the high-dimensional data is converted into low-dimensional data to process the nonlinear features in the image; the image is convolved and the neighborhood pixels are used to output a high-resolution non-cellular image; S42. Network design and configuration The first and second layers of the HAR-CNN network respectively use activation functions as loss functions to achieve nonlinear mapping; the mean square error is used as the optimization target of the HAR-CNN network; the filter size of the first layer of the HAR-CNN network is 9 to ensure the pattern containing honeycomb artifacts, the convolution kernel interval is 5-6 pixels, the filter size of the nonlinear mapping layer is set to 1, and the number of feature maps is 64 to balance computational efficiency and filtering ability; S43. Reconstruction layer configuration and model training The HAR-CNN network is configured with a reconstruction layer. In the reconstruction layer, the filter size used by the HAR-CNN network is 5, and the number of feature maps is set to 32 to optimize the output image quality and computational efficiency; the HAR-CNN network is trained using a stochastic gradient descent optimizer to complete model optimization; S44. Use the optimized HAR-CNN network to remove honeycomb artifacts in the optimized phase image.

Images