Rare earth homologous dual-emission high-resolution deep imaging method based on deep learning enhancement

Abstract

The invention discloses a rare earth homologous dual-emission high-resolution deep imaging method based on deep learning enhancement. A plurality of long-life intermediate states of lanthanide upconversion nanoparticles (UCNPs) are utilized, a single 980nm pumping source is used for exciting Tm < 3 + > / Yb < 3 + > co-doped UCNPs, and two-photon fluorescence with higher penetrability and four-photon fluorescence with higher resolution are induced at the same time. By combining a dual-mechanism CycleGAN artificial neural network based on antagonism training and cyclic consistency constraint, cross-domain mapping between two fluorescence signals is established, and the advantages of two fluorescence imaging are cooperatively utilized, so that deep penetration and high-resolution imaging are realized. According to the invention, in a self-established wide-field microscopic imaging system, the resolution of 209nm of a single particle in a 808nm fluorescence channel can be realized, and the signal-to-background ratio is doubled. According to the invention, the technical bottleneck that the resolution and the penetration depth of a traditional multi-photon microscope in deep tissue imaging are mutually restricted is effectively solved.

Description

Technical Field

[0001] This invention belongs to the field of fluorescence deep tissue microscopy imaging, specifically a high-resolution deep imaging method for rare-earth homologous dual emission based on deep learning enhancement. Background Technology

[0002] Multiphoton microscopy (MPM), based on the principle of nonlinear optics, uses long-wavelength near-infrared femtosecond lasers to excite fluorescent molecules to simultaneously absorb two or more low-energy photons, achieving electronic transitions comparable to single-photon excitation and generating fluorescence signals. This nonlinear process exhibits high spatial selectivity, occurring only in the high-intensity region at the focal point. Therefore, optical tomography can be achieved without a confocal pinhole, significantly improving the imaging capability of deep tissues. Because near-infrared light is less scattered and absorbed in biological tissues, MPM can penetrate tissues hundreds of μm thick, far exceeding the imaging depth of traditional confocal microscopes, while reducing phototoxicity and background interference. This characteristic makes it a key tool for three-dimensional imaging of deep tissues in vivo.

[0003] Upconversion nanoparticles (UCNPs) have been applied as novel probes in deep tissue imaging (MPM). Compared to traditional nonlinear fluorophores, UCNPs have longer energy level lifetimes, enabling sequential photon absorption and converting high-energy near-infrared (NIR) excitation into multiple anti-Stokes emissions. This allows for the use of more cost-effective and readily available continuous-wave (CW) NIR lasers, significantly reducing imaging costs. Furthermore, the superlinear effect of UCNPs can substantially reduce imaging power requirements, further mitigating phototoxicity. However, their application in deep tissue imaging still faces the challenge of balancing penetration depth and resolution. While short-wavelength NIR light can suppress some optical aberrations caused by tissue scattering, it cannot completely eliminate their negative impact on image quality. Conversely, using longer-wavelength NIR light to improve penetration depth and sharpness inevitably results in reduced imaging resolution due to the diffraction limit (d = λ / 2NA). This hinders the application of UCNPs in high-resolution deep tissue imaging of MPM. Summary of the Invention

[0004] This invention proposes a high-resolution deep imaging method for rare-earth homogeneous dual emission based on deep learning enhancement.

[0005] The technical solution for achieving the objective of this invention is: a high-resolution deep imaging method for rare-earth homologous dual emission based on deep learning enhancement, comprising:

[0006] Step 1: Prepare upconversion nanoparticle samples, wherein the upconversion nanoparticle samples are covered with a scattering medium;

[0007] Step 2: Achieve homologous dual-mode fluorescence emission of upconversion nanoparticles using a wide-field fluorescence system, and acquire a series of two-photon deep-penetration scattering medium images and four-photon scattering-free medium high-resolution images of the same region at different power levels.

[0008] Step 3: Input the two-photon deep-penetrating scattering medium image and the four-photon non-scattering medium high-resolution image into the pre-trained CycleGAN neural network to obtain the mapping relationship between the two-modal fluorescence.

[0009] Step 4: Utilize the established mapping relationship to reconstruct the fluorescence image of the two-photon deep-penetrating scattering medium, thereby achieving deep-penetrating high-resolution imaging.

[0010] Compared with existing technologies, the significant advantages of this invention are as follows: This invention enables multiphoton imaging under 980nm near-infrared continuous wave (CW) excitation without relying on expensive high-power femtosecond lasers; it leverages the multi-level, long-lifetime intermediate state characteristics of lanthanide ions to achieve homologous two-photon near-infrared and four-photon visible fluorescence emission; it utilizes the advantages of unsupervised CycleGAN for dual-modal fluorescence imaging, achieving 209nm single-particle resolution in an 808nm fluorescence channel, a 61% resolution improvement; it solves the problem of balancing penetration depth and resolution faced by existing multiphoton microscopes in deep tissue imaging; furthermore, by utilizing the superlinear effect of upconversion nanoparticles, it can further reduce the phototoxicity of imaging, demonstrating significant application potential in fields requiring high-resolution deep tissue imaging, such as early tumor diagnosis and neuroscience research.

[0011] The present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0012] Figure 1 The flowchart of a high-resolution deep imaging method for rare earth homogeneous dual emission based on deep learning enhancement provided by the present invention.

[0013] Figure 2 This is a diagram of a confocal wide-field fluorescence microscope imaging system.

[0014] Figure 3 This is a schematic diagram showing the structure in which the scattering medium and the sample are placed.

[0015] Figure 4 A schematic diagram of the CycleGAN neural network structure.

[0016] Figure 5 This study aims to characterize the dual-modal fluorescent single-particle PSF measured experimentally and to investigate the nonlinear changes in particle PSF under different laser intensities.

[0017] Figure 6This study analyzes particle results from input images of two-photon deep-penetrating scattering media, high-resolution ground truth images of four-photon non-scattering media, and output images from the CycleGAN network.

[0018] Figure reference numerals: 1-sample; 2-objective lens; 3-dichroic mirror; 4-tube lens; 5-flip mirror; 6-reflecting mirror; 7-bandpass filter; 8-camera; 9-single-mode fiber; 10-spectrum analyzer; 11-laser; 12-lens; Detailed Implementation

[0019] To more clearly illustrate the embodiments of the present invention or related technical solutions, the following description will be made in conjunction with the accompanying drawings. However, the scope of protection of the present invention is not limited to this example.

[0020] A high-resolution deep imaging method for rare-earth homologous dual emission based on deep learning enhancement, such as... Figure 1 As shown, the specific steps are as follows:

[0021] Step 1: Prepare upconversion nanoparticle samples and cover them with a scattering medium.

[0022] The upconversion nanoparticle sample structure is as follows: Figure 3 As shown, from bottom to top, there are glass slides, upconversion nanoparticles, and coverslips. A self-made scattering medium is added on top of the coverslips to simulate the optical environment of tissues of different depths and complexities.

[0023] The scattering medium used in this method is a turbid medium composed of TiO2, water and gelatin, with a ratio of 40 ml water: 20 mg TiO2 and 1 g gelatin particles.

[0024] The upconversion nanoparticles are 4% Tm 3+ Lanthanide upconversion nanoparticles β-NaYF4:40%Yb co-doped with 40%Yb3+ 3+ 4% Tm 3+ By adding cyclohexane for step-gradient dilution, relatively dispersed single-particle upconversion nanoparticles were obtained; these were then uniformly spread onto a glass slide to obtain single-particle samples for system characterization and subsequent experiments. 3+ Yb3+ co-doped upconversion nanoparticles, due to their different energy transfer pathways, can achieve 808nm near-infrared two-photon emission and 455nm visible four-photon dual fluorescence emission under 980nm single near-infrared light excitation. This characteristic is beneficial for simultaneously acquiring deep tissue location information and high-resolution PSF of the same particle, and the dual-infrared excitation detection characteristic has greater potential for deep tissue imaging. In addition, they also have bright, flicker-free, photobleach-resistant, and nonlinear optical properties, making them an excellent probe for multiphoton imaging.

[0025] Step 2: Achieve homologous dual-mode fluorescence emission of upconversion nanoparticles using a wide-field fluorescence system, and acquire a series of two-photon deep-penetration scattering medium images and four-photon scattering-free medium high-resolution images of the same region at different power levels.

[0026] The wide-field fluorescence system includes: an infrared wide-field excitation module, a camera acquisition module, a flip mirror 5, a dichroic mirror 3, and a spectral analysis module;

[0027] The infrared wide-field excitation module includes a laser 12, a lens 11, a dichroic mirror 3, an oil immersion objective lens 2, a tube lens 4, a flip mirror 5, a reflector 6, a bandpass filter 7, and a camera 8.

[0028] Specifically, the laser 12 is a single-mode fiber-coupled 980nm continuous semiconductor laser.

[0029] The laser 12 serves as the excitation source. The emitted laser light is focused by the lens 11 and then enters the rear aperture of the oil immersion objective lens 2 through the dichroic mirror 3. After passing through the objective lens 2, a wide-field excitation light field is generated to excite the upconversion nanoparticle sample.

[0030] The excitation light reflected by the upconversion nanoparticle sample and the fluorescence emitted by the upconversion nanoparticle sample first reach the dichroic mirror 3. The dichroic mirror 3 reflects the longer wavelength excitation light and transmits the shorter wavelength fluorescence.

[0031] The fluorescence passes through lens 4 and reaches the flip mirror 5. The flip mirror 5 controls the fluorescence to be reflected by mirror 6 to bandpass filter 7. Bandpass filter 7 filters the fluorescence to obtain the fluorescence signal of the desired wavelength band, which then reaches camera 8. Alternatively, the flip mirror 5 controls the fluorescence to enter single-mode fiber 9 and converge to the entrance of single-mode fiber 9, and finally enters spectrometer 10. The upconversion nanoparticle emission spectrum is measured by spectrometer.

[0032] Image of dual fluorescence signals generated by upconversion nanoparticles under 980nm near-infrared light excitation.

[0033] When acquiring homologous dual-mode fluorescence signals from upconversion nanoparticles, an infrared wide-field excitation module consisting of laser 12, lens 11, and oil immersion objective 2 is mainly used to excite the upconversion nanoparticles to generate homologous dual-fluorescence information. Then, fluorescence bandpass filters of different wavelengths are used to separate the two types of fluorescence signals and send them to camera 12 for image acquisition.

[0034] In a further embodiment, the specific steps for achieving homologous dual-mode fluorescence emission of upconversion nanoparticles using a wide-field fluorescence system, and acquiring a series of two-photon deep-penetration scattering medium images and four-photon high-resolution images of a non-scattering medium at different powers in the same region are as follows:

[0035] First, the sample is fixed on the sample holder, which is fixed on the triaxial closed-loop nanopositioner.

[0036] Next, a single-mode fiber-coupled 980nm continuous semiconductor laser was used as the excitation source. The laser was focused by a lens and then entered an oil immersion objective, which was used to excite the upconversion nanoparticle sample. This combination could generate wide-field excitation, causing the upconversion nanoparticles to emit fluorescence.

[0037] The excitation light reflected by sample 1 and the fluorescence emitted by sample 1 first reach dichroic mirror 3. Dichroic mirror 3 reflects the longer-wavelength infrared 980nm excitation light and transmits the shorter-wavelength fluorescence. After passing through the tubular lens 4, the fluorescence is focused into the single-mode fiber 9 by controlling the flip mirror 5 and finally enters the spectrometer 10. The Tm under 980nm excitation is measured by the spectrometer. 3+ The emission spectra of Yb3+ co-doped upconversion nanoparticles confirmed the realization of co-source emission of two-photon (808nm) and four-photon (455nm) dual fluorescence;

[0038] Secondly, the prepared scattering medium is placed in front of the sample, and a bandpass filter of the corresponding bandpass filter is matched. The camera acquisition module is used to capture the wide-field scattering image of two-photon fluorescence deep penetration.

[0039] Remove the scattering medium and replace the bandpass filter with the corresponding band to capture a four-photon fluorescence signal with higher resolution;

[0040] Finally, the laser power was switched and the image acquisition process was repeated to obtain two-photon deep-penetrating scattering medium images and four-photon scatter-free medium high-resolution images of the same sample area under different excitation intensities.

[0041] The laser powers for the series of fluorescent images are 34.33mW, 60.5mW, 86.6mW, 138.6mW, 190.6mW, 242.2mW, 291mW, and 338.2mW, respectively.

[0042] The control of each hardware component, including the laser, the three-axis closed-loop nanopositioner, the camera, the spectrometer, and the flip mirror, is achieved through external triggering. The external trigger ports of each device are connected to the input / output ports of the NI data acquisition card, and the USB port of the data acquisition card is connected to the computer. The switching of each hardware component is controlled using a LabVIEW program.

[0043] For scattering media with other characteristic thicknesses, manually place them in front of the sample in sequence and repeat the above steps to collect dual fluorescence signals under multiple sets of laser power.

[0044] Step 3: Input the two-photon deep-penetrating scattering medium image and the four-photon non-scattering medium high-resolution image into the pre-trained CycleGAN neural network to obtain the mapping relationship between the two-modal fluorescence.

[0045] In a further embodiment, the pre-training process of the CycleGAN neural network is as follows:

[0046] According to fluorescence imaging theory, the photon excitation efficiency in a two-photon scattering process is proportional to the square of the excitation light intensity, while the four-photon process exhibits a fourth-power dependence. Based on this nonlinear optical property, the Monte Carlo method is used to simulate the three-dimensional transport process of photons in biological tissues. This method simulates the interaction between photons and the medium, including key physical processes such as absorption, scattering, and energy deposition, through random sampling and statistical calculations.

[0047] Step 3.1: Random distribution of nanoparticles generated

[0048] A random distribution matrix of nanoparticles (1024×1024 pixels, resolution: 21.77nm / pixel) was generated using MATLAB.

[0049] Step 3.2: Generate X-domain two-photon deep-penetrating scattering medium image

[0050] For the generated random distribution matrix of nanoparticles, first convolve it with an 808nm Gaussian PSF, then simulate the three-dimensional transmission process of photons in biological tissues according to the Monte Carlo method, and add random Poisson noise to simulate the scattering and penetration process to generate a two-photon deep-penetrating scattering medium image.

[0051] Step 3.3: Generate a high-resolution image of the Y-domain four-photon scatter-free medium.

[0052] For the same generated random distribution matrix of nanoparticles, convolution with a 455nm Gaussian PSF yields a simulated image under diffraction limit, generating a high-resolution image of the Y-domain four-photon scatterless medium corresponding to the X-domain.

[0053] Step 3.4: Train the CycleGAN neural network

[0054] 900 pairs of X-domain and Y-domain images from steps 2 and 3 were randomly selected to form the training datasets train1 and train2, respectively. The network weights were initialized with a normal distribution and the bias was initialized to 0. The initial learning rate was set to 0.0002. The ADAM optimizer with a batch size of 1 was used for 100 rounds of training. The CycleGAN neural network was implemented based on the PyTorch framework and trained on the NVIDIA GeForce GTX 1660 SUPER GPU platform.

[0055] Step 3.5: Validate the CycleGAN neural network

[0056] During the verification phase, the network performance was evaluated using 100 pairs of images reserved in steps 2 and 3. The X-domain two-photon deep-penetrating scattering medium image was input into the trained CycleGAN network to obtain the output four-photon non-scattering medium high-resolution image. This image was then compared with the Y-domain four-photon non-scattering medium high-resolution benchmark image corresponding to the original input X-domain two-photon deep-penetrating scattering medium image. Quantitative comparisons were made with the corresponding Y-domain benchmark image using metrics such as PSNR and SSIM, and the effectiveness of the X→Y domain mapping was confirmed by visual evaluation.

[0057] The CycleGAN deep neural network includes generator G1, generator G2, discriminator D1, and discriminator D2. Generators G1 and G2 learn the conversion between the two-photon deep-penetrating scattering medium fluorescence image domain and the four-photon non-scattering medium high-resolution fluorescence image domain, while discriminators D1 and D2 are used to determine whether the input image is an image converted by the generator or a real input image.

[0058] Specifically, generator G1 is used to convert a two-photon deep-penetrating scattering medium fluorescence image into a four-photon no-scattering medium high-resolution fluorescence image; generator G2 is used to convert a four-photon no-scattering medium high-resolution fluorescence image into a two-photon deep-penetrating scattering medium fluorescence image; and discriminator D1 is used to distinguish between the four-photon no-scattering medium high-resolution fluorescence image generated by generator G1 and a real four-photon no-scattering medium high-resolution fluorescence image. Generator G1 and discriminator D1 engage in a game-like competition, and when discriminator D1 cannot distinguish the four-photon no-scattering medium high-resolution fluorescence image generated by generator G1... This indicates that generator G1 has learned the conversion from two-photon deep penetration scattering medium fluorescence image to four-photon no-scattering medium high-resolution fluorescence image; the discriminator D2 is used to distinguish between the two-photon deep penetration scattering medium fluorescence image generated by generator G2 and the real two-photon deep penetration scattering medium fluorescence image. Generator G2 and discriminator D2 play a game against each other. When discriminator D2 cannot distinguish the two-photon deep penetration scattering medium fluorescence image generated by generator G2, it indicates that generator G2 has learned the conversion from four-photon no-scattering medium high-resolution fluorescence image to two-photon deep penetration scattering medium fluorescence image.

[0059] A residual learning-based encoder-decoder architecture is adopted as the core of the generator. The encoder part achieves progressive downsampling through stride convolution, and a residual block is connected after each downsampling level to alleviate the gradient vanishing problem; the decoder part uses transposed convolution to upsample and restore spatial resolution. To maintain the consistency of multi-scale features, skip connections are introduced between corresponding layers of the encoder and decoder to form a U-Net-like structure, which effectively preserves high-frequency details and improves feature reconstruction accuracy.

[0060] The discriminator based on the PatchGAN architecture achieves image patch-level real / fake detection through a local receptive field (70×70 pixels). Its output is an N×N dimensional feature matrix, where each element corresponds to the probability distribution of a local region in the input image, allowing for more refined capture of texture and structural features compared to a global discriminator. This design achieves parameter sharing through full convolution, significantly improving the model's sensitivity to local artifacts.

[0061] The CycleGAN deep neural network employs a bidirectional cycle consistency loss function, forcing generators G1 and G2 to satisfy an invertible mapping relationship. The bidirectional cycle consistency loss function is defined as follows:

[0062]

[0063] This function constrains the reconstruction capabilities of generators G1 and G2 during bidirectional image transformation. The first term... This represents the expected error of reconstructing the X domain from a sample x in the X domain, through G1 to the Y domain, and then through G2 back to the X domain; the second term... This describes the expected error of starting from a sample y in the Y domain, transforming it to the X domain via G2, and then reconstructing it back to the Y domain via G1. In the formula, P... data(x) and P data(y) The edge data distributions of the two domains are represented respectively, and the L1 norm is used to measure the pixel-level difference between the reconstructed image and the original image. This loss function effectively solves the mode collapse problem in unsupervised cross-domain transformation by forcing the two generators to maintain an invertible mapping relationship, while ensuring the preservation of key visual features during the transformation process and improving the geometric fidelity of cross-domain transformation.

[0064] Step 4: Reconstruct the fluorescence image of the two-photon deep-penetrating scattering medium using the established mapping relationship to achieve deep-penetrating high-resolution imaging.

[0065] The simulation image with known truth value is used for verification. The simulated two-photon deep-penetrating scattering medium is input, and the high-resolution GT image of the simulated four-photon non-scattering medium is compared and evaluated with the image output by the CycleGAN neural network.

[0066] A multi-dimensional evaluation index system was established, and the structural similarity index (SSI) was used to evaluate the degree of structure preservation of the image, while the peak signal-to-noise ratio (PSNR) was used to measure the noise level of the image.

[0067] The accuracy of the mapping relationship was confirmed by simulation data. Two-photon deep-penetrating scattering medium images obtained from the experiment were input, fluorescence signals were extracted for four-photon high-resolution reconstruction, and the effectiveness of the fusion method was verified by analyzing the improvement in the resolution and signal-to-background ratio of the output image.

[0068] Example 1

[0069] Tm based on near-infrared excitation light 980nm 3+ / Yb3+ co-doped upconversion nanoparticles. The objective lens (2) used was a 1.4NA (numerical aperture), 100X oil immersion objective lens. Image acquisition was performed using a confocal wide-field system, with the focal plane of the image located at the position of the upconversion nanoparticles.

[0070] Characterization and spectral analysis of single-particle upconversion nanoparticles were performed to obtain single-particle UCNP two-photon and four-photon dual-emission peak spectra and single-particle dual-fluorescence PSF characterization. The superlinear properties of the upconversion nanoparticles were verified by changing the power.

[0071] The scattering medium was prepared according to the ratio of 40ml water, 20mg TiO2 and 1g gelatin particles. A 120μm thick gasket was used to control the thickness of the scattering medium to simulate the optical scattering environment of biological tissue with a depth of 120μm.

[0072] Simulations were performed using Matlab software. Extensive simulations were conducted to estimate the propagation paths and probability distribution of photons in the medium. Multiple sets of high-resolution simulation images of two-photon deep-penetrating scattering media and four-photon non-scattering media with different characteristic positions were simulated at various depths. A total of 1000 pairs of simulation data were generated and used for training, testing, and validation of the CycleGAN network in a 7:2:1 ratio.

[0073] A multi-dimensional evaluation index system was established to evaluate the output image of the known true image and the input two-photon deep-penetrating scattering medium test image. The structural similarity index (SSI) was used to evaluate the degree of structure preservation of the image, and the peak signal-to-noise ratio (PSNR) was used to measure the noise level of the image. The SSI reached 0.94 and the PSNR reached 32dB, which confirmed that a good mapping relationship was established.

[0074] The experiment collected real two-photon scattering fluorescence data. The prepared scattering medium was placed in front of the sample, and the corresponding bandpass filter was matched. The camera acquisition module was used to capture the scattering wide-field image of two-photon fluorescence. The scattering medium was removed and an appropriate bandpass filter was replaced to capture high-resolution four-photon fluorescence signals.

[0075] The laser intensity was switched, and the image acquisition process was repeated to obtain two-photon deep-penetrating scattering medium images and four-photon high-resolution images of the same sample area under different excitation intensities. The acquisition powers of the series of fluorescence images were 34.33mW, 60.5mW, 86.6mW, 138.6mW, 190.6mW, 242.2mW, 291mW, and 338.2mW, respectively.

[0076] Achieve high-resolution reconstruction of experimental two-photon deep-penetrating scattering medium images.

[0077] Example 2

[0078] Tm based on near-infrared excitation light 980nm 3+ / Yb3+ co-doped upconversion nanoparticles. The objective lens (2) used was a 1.4NA (numerical aperture), 100X oil immersion objective lens. Image acquisition was performed using a confocal wide-field system, with the focal plane of the image located at the position of the upconversion nanoparticles.

[0079] Characterization and spectral analysis of single-particle upconversion nanoparticles were performed to obtain single-particle UCNP two-photon and four-photon dual-emission peak spectra and single-particle dual-fluorescence PSF characterization. The superlinear properties of the upconversion nanoparticles were verified by changing the power.

[0080] The scattering medium was prepared according to the ratio of 40ml water, 20mg TiO2 and 1g gelatin particles. A 120μm thick gasket was used to control the thickness of the scattering medium to simulate the optical scattering environment of biological tissue with a depth of 240μm.

[0081] Simulations were performed using Matlab software. Extensive simulations were conducted to estimate the propagation paths and probability distribution of photons in the medium. Multiple sets of high-resolution simulation images of two-photon deep-penetrating scattering media and four-photon non-scattering media with different characteristic positions were simulated at various depths. A total of 1000 pairs of simulation data were generated and used for training, testing, and validation of the CycleGAN network in a 7:2:1 ratio.

[0082] A multi-dimensional evaluation index system was established to evaluate the output image of the known true image and the input two-photon deep-penetrating scattering medium test image. The structural similarity index (SSI) was used to evaluate the degree of structure preservation of the image, and the peak signal-to-noise ratio (PSNR) was used to measure the noise level of the image. The SSI reached 0.93 and the PSNR reached 33dB, which confirmed that a good mapping relationship was established.

[0083] The experiment collected real two-photon scattering fluorescence data. The prepared scattering medium was placed in front of the sample, and the corresponding bandpass filter was matched. The camera acquisition module was used to capture the scattering wide-field image of two-photon fluorescence. The scattering medium was removed and an appropriate bandpass filter was replaced to capture high-resolution four-photon fluorescence signals.

[0084] The laser intensity was switched, and the image acquisition process was repeated to obtain two-photon deep-penetrating scattering medium images and four-photon high-resolution images of the same sample area under different excitation intensities. The acquisition powers of the series of fluorescence images were 34.33mW, 60.5mW, 86.6mW, 138.6mW, 190.6mW, 242.2mW, 291mW, and 338.2mW, respectively.

[0085] Achieve high-resolution reconstruction of experimental two-photon deep-penetrating scattering medium images.

[0086] This invention utilizes a wide-field fluorescence system to achieve two-photon and four-photon dual-fluorescence emission of upconversion nanoparticles under single near-infrared light excitation at 980nm. By training a CycleGAN neural network to integrate the advantages of dual-modal fluorescence imaging, a single-particle resolution of 209nm is achieved under an 808nm fluorescence channel. This invention overcomes the technical bottleneck of resolution and penetration depth being mutually constrained in deep tissue imaging using traditional multiphoton microscopy, demonstrating significant application potential in fields requiring high-resolution deep tissue imaging, such as early tumor diagnosis and neuroscience research.

[0087] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A high-resolution deep imaging method for rare-earth homologous dual emission based on deep learning enhancement, characterized in that, include: Step 1: Prepare upconversion nanoparticle samples, wherein the upconversion nanoparticle samples are covered with a scattering medium; Step 2: Achieve homologous dual-mode fluorescence emission of upconversion nanoparticles using a wide-field fluorescence system, and acquire a series of two-photon deep-penetration scattering medium images and four-photon scattering-free medium high-resolution images of the same region at different power levels. Step 3: Input the two-photon deep-penetrating scattering medium image and the four-photon non-scattering medium high-resolution image into the pre-trained CycleGAN neural network to obtain the mapping relationship between the two-modal fluorescence. Step 4: Reconstruct the fluorescence image of the two-photon deep-penetrating scattering medium using the established mapping relationship to achieve deep-penetrating high-resolution imaging.

2. The high-resolution deep imaging method for rare-earth homologous dual emission based on deep learning enhancement according to claim 1, characterized in that, The upconversion nanoparticle sample consists of a glass slide, upconversion nanoparticles, and a cover glass from bottom to top.

3. The high-resolution deep imaging method for rare-earth homologous dual emission based on deep learning enhancement according to claim 1, characterized in that, The upconversion nanoparticles are 4% Tm 3+ and 40% Yb 3 + Co-doped lanthanide upconversion nanoparticles.

4. The high-resolution deep imaging method for rare-earth homologous dual emission based on deep learning enhancement according to claim 1, characterized in that, A high-resolution deep imaging method for rare-earth homologous dual emission based on deep learning enhancement is characterized in that the wide-field fluorescence system includes: a laser 12, a lens 11, a dichroic mirror 3, an oil immersion objective lens 2, a tube lens 4, a flip mirror 5, a reflecting mirror 6, a bandpass filter 7, and a camera 8. The laser 12 serves as the excitation source. The emitted laser light is focused by the lens 11 and then enters the rear aperture of the oil immersion objective lens 2 through the dichroic mirror 3. After passing through the objective lens 2, a wide-field excitation light field is generated to excite the upconversion nanoparticle sample. The excitation light reflected by the upconversion nanoparticle sample and the fluorescence emitted by the upconversion nanoparticle sample first reach the dichroic mirror 3. The dichroic mirror 3 reflects the longer wavelength excitation light and transmits the shorter wavelength fluorescence. The fluorescence passes through lens 4 and reaches the flip mirror 5. The flip mirror 5 controls the fluorescence to be reflected by mirror 6 to bandpass filter 7. Bandpass filter 7 filters the fluorescence to obtain the fluorescence signal of the desired wavelength band, which then reaches camera 8. Alternatively, the flip mirror 5 controls the fluorescence to enter single-mode fiber 9 and converge to the entrance of single-mode fiber 9, and finally enters spectrometer 10. The upconversion nanoparticle emission spectrum is measured by spectrometer.

5. The high-resolution deep imaging method for rare-earth homologous dual emission based on deep learning enhancement according to claim 4, characterized in that, The laser 12 is a single-mode fiber-coupled 980nm continuous semiconductor laser.

6. The high-resolution deep imaging method for rare-earth homologous dual emission based on deep learning enhancement according to claim 1, characterized in that, The specific steps for achieving homologous dual-mode fluorescence emission of upconversion nanoparticles using a wide-field fluorescence system, and acquiring a series of two-photon deep-penetration scattering media images at different powers in the same region, as well as high-resolution four-photon scattering-free media images, are as follows: The sample is fixed on the sample holder, which is then fixed on the triaxial closed-loop nanopositioner. A laser is used as the excitation source. The laser is focused by a lens and enters the oil immersion objective. The oil immersion objective is then used to excite the upconversion nanoparticle sample. The excitation light reflected by sample 1 and the fluorescence emitted by sample 1 first reach dichroic mirror 3. Dichroic mirror 3 reflects the longer wavelength excitation light and transmits the shorter wavelength fluorescence. After passing through the tube lens 4, the fluorescence is controlled by the flip mirror 5 to enter the single-mode fiber 9 and converge to the entrance of the single-mode fiber 9, and finally enters the spectrometer 10; the emission spectrum of the upconversion nanoparticle is measured by the spectrometer, confirming that the co-emission of two-photon and four-photon dual fluorescence has been achieved. The prepared scattering medium is placed in front of the sample, a bandpass filter of the corresponding band is matched, and a camera is used to capture two-photon deep-penetrating scattering medium images; Remove the scattering medium and replace the bandpass filter with the corresponding band to capture a high-resolution image of the four-photon scattering-free medium; Switch the laser power and repeat the image acquisition process to obtain two-photon deep-penetrating scattering medium images and four-photon high-resolution images of the same sample area under different excitation intensities.

7. The high-resolution deep imaging method for rare-earth homologous dual emission based on deep learning enhancement according to claim 1, characterized in that, The CycleGAN deep neural network includes a generator G1, a generator G2, a discriminator D1, and a discriminator D2. Generator G1 converts a two-photon deep-penetrating scattering medium fluorescence image into a four-photon non-scattering medium high-resolution fluorescence image. Generator G2 converts a four-photon non-scattering medium high-resolution fluorescence image into a two-photon deep-penetrating scattering medium fluorescence image. Discriminator D1 distinguishes between the four-photon non-scattering medium high-resolution fluorescence image generated by generator G1 and a genuine four-photon non-scattering medium high-resolution fluorescence image. Generator G1 and discriminator D1 engage in a game-like competition; when discriminator D1 cannot distinguish between generator G1 and the discriminator D2, the discriminator D1 fails to distinguish between the two images. When a high-resolution fluorescence image of a four-photon no-scattering medium is generated, it indicates that the generator G1 has learned the conversion from a two-photon deep-penetrating scattering medium fluorescence image to a four-photon no-scattering medium high-resolution fluorescence image. The discriminator D2 is used to distinguish between the two-photon deep-penetrating scattering medium fluorescence image generated by the generator G2 and the real two-photon deep-penetrating scattering medium fluorescence image. The generator G2 and the discriminator D2 play a game against each other. When the discriminator D2 cannot distinguish the two-photon deep-penetrating scattering medium fluorescence image generated by the generator G2, it indicates that the generator G2 has learned the conversion from a four-photon no-scattering medium high-resolution fluorescence image to a two-photon deep-penetrating scattering medium fluorescence image.

8. The high-resolution deep imaging method for rare-earth homologous dual emission based on deep learning enhancement according to claim 7, characterized in that, The generators G1 and G2 adopt an encoder-decoder structure based on residual learning. The encoder part achieves progressive downsampling through stride convolution, and a residual block is connected after each downsampling stage to alleviate the gradient vanishing problem. The decoder part uses transposed convolution to upsample and restore spatial resolution. Skip connections are introduced between corresponding layers of the encoder and decoder to form a U-Net-like structure.

9. The high-resolution deep imaging method for rare-earth homologous dual emission based on deep learning enhancement according to claim 7, characterized in that, The discriminators D1 and D2 adopt the PatchGAN architecture, which realizes the authenticity judgment of image blocks through local receptive fields.

10. The high-resolution deep imaging method for rare-earth homologous dual emission based on deep learning enhancement according to claim 9, characterized in that, The CycleGAN deep neural network employs a bidirectional cyclic consistency loss function, specifically: In the formula, x is a real image sample of a two-photon deep-penetrating scattering medium, y is a real image sample of a four-photon no-scattering high-resolution reference image, G1 is a generator that converts the real image sample of a two-photon deep-penetrating scattering medium into a real image sample of a four-photon no-scattering high-resolution reference image, G2 is a generator that converts the real image sample of a four-photon no-scattering high-resolution reference image into a real image sample of a two-photon deep-penetrating scattering medium, |||| is the L1 norm, used to measure the difference between the reconstructed image and the original image, and E is the expected value.

Images