Photoacoustic microscopy system and method based on Airy beam combined with sparse sampling

Abstract

The present invention discloses a photoacoustic microscopy system and method based on Airy beam combined with sparse sampling, which is suitable for scenarios with large depth of field and rapid imaging monitoring. The system comprises: a beam generation module for generating a nanosecond pulsed Gaussian beam and processing it to obtain a first standard linearly polarized Gaussian beam and a second standard linearly polarized Gaussian beam; a beam shaping module for performing polarization adjustment, phase modulation, filtering, focusing and Fourier transform processing on the first standard linearly polarized Gaussian beam to obtain an Airy beam; a system control and sparse scanning module for performing two-dimensional sparse scanning sampling of an object excited by the Airy beam using a two-dimensional triangular Lissajous trajectory upon receiving the second standard linearly polarized Gaussian beam to obtain a sparse photoacoustic signal; a signal receiving and processing module for performing maximum projection based on the sparse photoacoustic signal to obtain a sparse image; and an image reconstruction module for reconstructing the sparse image using a super-resolution reconstruction network to obtain a high-resolution photoacoustic microscopy original image.

Description

Technical Field

[0001] The present invention belongs to the technical field of optical microscopy, and in particular relates to a photoacoustic microscopy system and method based on Airy beam combined with sparse sampling. Background Art

[0002] Optical resolution photoacoustic microscopy (OR-PAM) is an emerging biomedical imaging technology that has developed rapidly in recent years. It can perform label-free in vivo anatomical and functional imaging of microvessels, lipids, and melanin in biological tissues. OR-PAM has inherent depth resolution capabilities, thanks to the time-of-flight information captured by each photoacoustic signal excited by the A-scan, which enables volumetric imaging using only two-dimensional (2D) raster scanning. In a typical OR-PAM system, a highly focused Gaussian beam is selected using an optical objective lens for optical diffraction-limited lateral resolution. However, due to the diffraction limit, the conventionally used Gaussian beam diverges rapidly along its propagation direction, resulting in a very short focal depth of photoacoustic signal excitation. The limited focal depth makes it impossible to obtain volumetric images with consistent lateral resolution along the depth direction to accurately capture the structure, function, and metabolic state of biological systems.

[0003] Propagation-invariant beam fields (such as Bessel beams) can extend the focal depth without affecting the focal size. However, Bessel beam-based methods usually obtain images with poor contrast because the lateral annular sidelobe structure produces background artifacts and reduces axial resolution. In addition, the Bessel beam has only a small amount of light energy (<20%) in its central main lobe, which affects the sensitivity of the acquired signal and the image quality. In addition, rapid screening and repeatable monitoring are inevitable requirements for clinical diagnosis of many diseases, but full sampling imaging based on traditional raster scanning is slow, especially when there is a need for large field of view imaging without sacrificing spatial resolution. Summary of the Invention

[0004] To address the aforementioned issues in related technologies, the present invention provides a photoacoustic microscopy system and method based on Airy beam combined with sparse sampling. The technical issues to be addressed by the present invention are achieved through the following technical solutions:

[0005] The present invention provides a photoacoustic microscopy system based on Airy beam combined with sparse sampling, comprising:

[0006] a beam generating module, configured to generate a nanosecond pulsed Gaussian beam, and process the nanosecond pulsed Gaussian beam to obtain a first standard linearly polarized Gaussian beam and a second standard linearly polarized Gaussian beam;

[0007] a beam shaping module, configured to perform polarization adjustment, phase modulation, filtering, focusing, and Fourier transform processing on the first standard linearly polarized Gaussian beam to obtain an Airy beam that excites the object;

[0008] a system control and sparse scanning module, configured to perform a two-dimensional sparse scanning sampling on the object excited by the Airy beam using a two-dimensional triangular Lissajous trajectory upon receiving the second standard linearly polarized Gaussian beam, to obtain a sparse photoacoustic signal;

[0009] a signal receiving and processing module, configured to receive the sparse photoacoustic signal and perform maximum projection on the sparse photoacoustic signal to obtain a sparse image;

[0010] An image reconstruction module is configured to reconstruct the sparse image using a pre-trained super-resolution reconstruction network to obtain a high-resolution photoacoustic microscopy original image of the object; wherein the super-resolution reconstruction network includes: two discriminant networks, an upsampling network, and a downsampling network; the upsampling network and one discriminant network constitute a forward GAN network, and the downsampling network and another discriminant network constitute a backward GAN network.

[0011] In some embodiments, the upsampling network includes, in sequence: a first convolutional layer, an activation layer, a residual subnetwork, a sub-pixel convolutional layer, and a second convolutional layer; the downsampling network includes, in sequence: the first convolutional layer, the activation layer, the residual subnetwork, a first downsampling convolutional layer, a second downsampling convolutional layer, and the second convolutional layer; the two discriminant networks are the discriminant networks in the super-resolution generative adversarial network.

[0012] In some embodiments, the system control and sparse scanning module includes: a photodetector, a scanning galvanometer, a data acquisition card, and a system control module;

[0013] The photoelectric detector is used to synchronously send a trigger signal to the scanning galvanometer and the data acquisition card when receiving the second standard linearly polarized Gaussian beam;

[0014] The data acquisition card and the system control module are used to generate two two-dimensional triangular waveforms at a preset frequency and send them to the two galvanometers of the scanning galvanometer respectively when receiving the trigger signal;

[0015] The scanning galvanometer is used to perform two-dimensional sparse scanning sampling on the object using two-dimensional triangular Lissajous trajectories according to the two-dimensional triangular waveforms received by the two galvanometers to obtain the sparse photoacoustic signal.

[0016] In some embodiments, the light beam generating module includes, in sequence:

[0017] Nanosecond pulse laser, used to generate nanosecond pulsed Gaussian beam;

[0018] a polarizer, configured to adjust the polarization of the nanosecond pulsed Gaussian beam to obtain a standard linearly polarized Gaussian beam;

[0019] a polarization beam splitter, configured to split the standard linearly polarized Gaussian beam into a first standard linearly polarized Gaussian beam and a second standard linearly polarized Gaussian beam according to a preset ratio, and use the second standard linearly polarized Gaussian beam as a second standard linearly polarized Gaussian beam;

[0020] an aperture, configured to optimize the first standard linearly polarized Gaussian beam to obtain an optimized first standard linearly polarized Gaussian beam;

[0021] The beam expansion lens group is used to expand the optimized first standard linearly polarized Gaussian beam to obtain a first standard linearly polarized Gaussian beam.

[0022] In some embodiments, the beam expanding lens assembly includes, in sequence: a first converging lens, a first reflector, a first pinhole, and a second converging lens;

[0023] The first aperture is used to filter out other stray light in the standard linearly polarized Gaussian light beam after passing through the first converging lens.

[0024] In some embodiments, the beam shaping module includes, in sequence: a half-wave plate, a spatial light modulator, a beam reduction lens group, a scanning array lens, an objective lens, and a three-dimensional moving platform;

[0025] The half-wave plate is used to adjust the polarization direction of the first standard linearly polarized Gaussian beam;

[0026] The spatial light modulator is used to perform phase modulation on the first standard linearly polarized Gaussian light beam after polarization adjustment according to a preset azimuth phase modulation parameter to obtain a modulated light beam;

[0027] The beam reduction lens group is used to perform beam reduction and filtering processing on the modulated light beam;

[0028] The objective lens is used to focus and Fourier transform the light beam passing through the scanning array mirror to obtain the Airy beam;

[0029] The three-dimensional moving platform is used to place the object and control the object to be located at the focus of the objective lens by moving itself in the horizontal and / or vertical directions.

[0030] In some embodiments, the spatial light modulator is loaded with a phase-type blazed grating with different grating periods, and the spatial light modulator is further used to diffract the modulated light beam to different diffraction angles to separate the zero-order light spot from the first-order light spot;

[0031] The beam reduction lens group includes, in sequence: a third converging lens, a second pinhole, and a fourth converging lens;

[0032] The second small hole is used to filter out the zero-order light spot.

[0033] In some embodiments, the beam shaping module further comprises: a third reflector and a fourth reflector; the third reflector and the fourth reflector are sequentially positioned between the beam reduction lens group and the scanning array mirror;

[0034] The third reflecting mirror is used to reflect the light beam passing through the beam reduction lens group to the fourth reflecting mirror;

[0035] The fourth reflecting mirror is used to reflect the light beam reflected by the third reflecting mirror to the scanning array mirror.

[0036] In some embodiments, the signal receiving and processing module includes:

[0037] an ultrasonic transducer, configured to receive the sparse photoacoustic signal and obtain an electrical signal based on the sparse photoacoustic signal;

[0038] The amplifier is used to amplify the electrical signal to obtain the amplified sparse photoacoustic signal.

[0039] The present invention also provides a photoacoustic microscopy method based on Airy beam combined with sparse sampling, which is applied to the above-mentioned photoacoustic microscopy system, comprising:

[0040] generating a nanosecond pulsed Gaussian beam, and processing the nanosecond pulsed Gaussian beam to obtain a first standard linearly polarized Gaussian beam and a second standard linearly polarized Gaussian beam;

[0041] performing polarization adjustment, phase modulation, filtering, focusing, and Fourier transform processing on the first standard linearly polarized Gaussian beam to obtain an Airy beam that excites the object;

[0042] When the second standard linearly polarized Gaussian beam is used, a two-dimensional triangular Lissajous trajectory is used to perform a two-dimensional sparse scanning sampling on the object excited by the Airy beam to obtain a sparse photoacoustic signal;

[0043] receiving the sparse photoacoustic signal, and performing maximum projection on the sparse photoacoustic signal to obtain a sparse image;

[0044] The sparse image is reconstructed using a pre-trained super-resolution reconstruction network to obtain a high-resolution photoacoustic microscopy original image of the object.

[0045] The present invention has the following beneficial technical effects:

[0046] In response to the current constraints between spatial resolution, depth of field, detection sensitivity, and imaging speed in photoacoustic microscopy, the present invention utilizes a beam generation module and a beam shaping module to generate a non-diffracting Airy beam with a longer focal depth and better focusing effect. Furthermore, its main lobe can carry relatively more energy (approximately 50%), significantly expanding the range of axial imaging of objects (e.g., biological tissues). This achieves photoacoustic microscopy with a large depth of field and high sensitivity, and can obtain high-resolution images with constant depth resolution. Furthermore, the present invention uses a two-dimensional triangular Lissajous trajectory to perform two-dimensional sparse scanning sampling on the object excited by the Airy beam, processes the scanned signal, and then reconstructs the image using the pre-trained super-resolution reconstruction network proposed in the present invention, meeting the imaging requirements of efficient and repeatable monitoring. Therefore, the present invention can achieve rapid imaging monitoring of objects (e.g., physiological and pathological activities) with a large depth of field and high sensitivity.

[0047] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A schematic structural diagram of a photoacoustic microscopy system based on Airy beam combined with sparse sampling provided by an embodiment of the present invention;

[0049] Figure 2 Another schematic structural diagram of an exemplary photoacoustic microscopy system based on Airy beam combined with sparse sampling provided by an embodiment of the present invention;

[0050] Figure 3A A diagram comparing the schematic structures of an exemplary traditional point-by-point raster scanning design provided in an embodiment of the present invention and the sparse scanning design adopted in the present invention;

[0051] Figure 3B A schematic diagram of an exemplary triangular wave waveform provided in an embodiment of the present invention;

[0052] Figure 4A A network structure diagram of an exemplary SR-cycleGAN network provided in an embodiment of the present invention;

[0053] Figure 4B A schematic diagram of the structure of an exemplary G_down network provided in an embodiment of the present invention;

[0054] Figure 4C A schematic diagram of the structure of an exemplary residual block provided in an embodiment of the present invention;

[0055] Figure 5 A schematic diagram of the effects of an exemplary photoacoustic microscopy system based on Airy beam combined with sparse sampling provided by an embodiment of the present invention;

[0056] Figure 6 A flow chart of a photoacoustic microscopy method based on Airy beam combined with sparse sampling provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The present invention will be further described in detail below with reference to specific examples, but the embodiments of the present invention are not limited thereto.

[0058] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0059] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification.

[0060] Although the present invention is described herein in conjunction with various embodiments, in the process of implementing the claimed invention, those skilled in the art can understand and implement other variations of the disclosed embodiments by reviewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple situations. A single processor or other unit can implement several functions listed in the claims. Certain measures are recorded in different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0061] Figure 1 FIG. 1 is a schematic structural diagram of a photoacoustic microscopy system based on Airy beam combined with sparse sampling provided by an embodiment of the present invention. Figure 1 As shown, the photoacoustic microscopy imaging system includes: a beam generating module 1 , a system control and sparse scanning module 2 , a signal receiving and processing module 3 , a signal receiving and processing module 4 and an image reconstruction module 5 .

[0062] The beam generation module 1 is used to generate a nanosecond pulsed Gaussian beam and process the nanosecond pulsed Gaussian beam to obtain a first standard linearly polarized Gaussian beam and a second standard linearly polarized Gaussian beam. The beam shaping module 2 is used to perform polarization adjustment, phase modulation, filtering, focusing, and Fourier transform on the first standard linearly polarized Gaussian beam to obtain an Airy beam that excites the object. The system control and sparse scanning module 3 is used to perform a two-dimensional sparse scanning sampling of the object excited by the Airy beam using a two-dimensional triangular Lissajous trajectory upon receiving the second standard linearly polarized Gaussian beam to obtain a sparse photoacoustic signal. The signal receiving and processing module 4 is used to receive the sparse photoacoustic signal and perform maximum projection based on the sparse photoacoustic signal to obtain a sparse image. Image reconstruction module 5 is used to reconstruct the sparse image using a pre-trained super-resolution reconstruction network to obtain a high-resolution photoacoustic microscopy original image of the object; wherein the super-resolution reconstruction network includes: two discriminant networks, an upsampling network, and a downsampling network; the upsampling network and a discriminant network constitute a forward GAN network, and the downsampling network and another discriminant network constitute a backward GAN network.

[0063] In some embodiments, as Figure 2 As shown, the beam generating module includes in sequence: a nanosecond pulse laser 11 for generating a nanosecond pulsed Gaussian beam; a polarizer 12 for adjusting the polarization of the nanosecond pulsed Gaussian beam to obtain a standard linearly polarized Gaussian beam; a polarization beam splitter 13 for dividing the standard linearly polarized Gaussian beam into a first standard linearly polarized Gaussian beam (90% beam energy) and a second standard linearly polarized Gaussian beam (10% beam energy) according to a preset ratio (for example, 9:1), and using the second standard linearly polarized Gaussian beam as a second standard linearly polarized Gaussian beam to trigger the system control and sparse scanning module 2. An aperture 14 is used to optimize the first standard linearly polarized Gaussian beam to obtain an optimized first standard linearly polarized Gaussian beam; a beam expanding lens group 15 is used to expand the optimized first standard linearly polarized Gaussian beam to obtain a first standard linearly polarized Gaussian beam. Specifically, as Figure 2 As shown, the beam expanding lens group 15 includes: a first converging lens 151, a first reflecting mirror 152, a first aperture 153 ( Figure 2 (not shown) and the second converging lens 154, and the first converging lens 151 and the second converging lens 154 are used to focus or diverge the light beam respectively. The combination of these two converging lenses can expand the spot diameter (abbreviated as beam expansion), and the first pinhole 153 is used to filter out other stray light in the standard linearly polarized Gaussian beam after passing through the first converging lens 151, so as to ultimately improve the signal-to-noise ratio of the photoacoustic signal, thereby further enhancing the clarity of the final high-resolution photoacoustic microscopy original image. In some embodiments, such as Figure 2As shown, a neutral density filter 16 is further included between the nanosecond pulse laser 11 and the polarizer 12, wherein the neutral density filter 16 is used to perform energy adjustment on the nanosecond pulsed Gaussian beam generated by the nanosecond pulsed laser 11, and the energy-adjusted nanosecond pulsed Gaussian beam is transmitted to the polarizer 12, and the polarization is adjusted by the polarizer 12.

[0064] For example, the wavelength of the nanosecond pulsed excitation light is 532 nm and the repetition frequency is 1 kHz to 50 kHz. The repetition frequency of the nanosecond pulsed excitation light can be adjusted according to actual usage.

[0065] In some embodiments, as Figure 2 As shown, the beam shaping module 2 includes: a half-wave plate 21, a spatial light modulator 22, a beam reduction lens group 23, a scanning array lens 24, an objective lens 25 and a three-dimensional moving platform 26 ( Figure 2 (not shown). The half-wave plate 21 is used to adjust the polarization direction of the first standard linearly polarized Gaussian light beam; the spatial light modulator 22 is used to perform phase modulation on the first standard linearly polarized Gaussian light beam after polarization adjustment according to the preset azimuth phase modulation parameters to obtain a modulated light beam; specifically, the spatial light modulator 22 is also loaded with a phase-type blazed grating with different grating periods on the basis of the cubic phase diagram, which is not only used to obtain a modulated light beam, but also used to diffract the modulated light beam to different diffraction angles, thereby separating the zero-order light spot and the first-order light spot. The beam reduction lens group 23 is used to perform beam reduction and filtering processing on the light beam passing through the spatial light modulator 22; specifically, as Figure 2 As shown, the beam reduction lens group 23 includes: a third converging lens 231, a second pinhole 232 and a fourth converging lens 233 in sequence, wherein the third converging lens 231 and the fourth converging lens 233 can be used in combination to reduce the diameter of the light spot, and the second pinhole 232 is used to filter out the zero-order light spot. The light beam after passing through the beam reduction lens group 23 can reach the scanning array lens 24, and the light beam after passing through the scanning array lens 24 reaches the objective lens 25. The objective lens 25 is used to focus and Fourier transform the light beam passing through the scanning array lens 24 to obtain an Airy beam. The three-dimensional moving platform 26 located at the rear end of the objective lens 25 is used to place the object 6, and controls the object 6 to be located at the focus of the objective lens 25 by moving itself in the horizontal and / or vertical directions. That is, the three-dimensional moving platform 26 can drive the object to move two-dimensionally in the horizontal direction to adjust the excitation light to focus on the object 6 to irradiate the area of interest. The three-dimensional moving platform 26 can also drive the object to move vertically up and down along the Z-axis direction so that the focus of the excitation light is accurately irradiated on the target site of the object 6. In some embodiments, as Figure 2 As shown, after the half-wave plate 21, a second reflector 27 is further included for reflecting the light beam passing through the half-wave plate 21 to the spatial light modulator 22. In some embodiments, as shown in FIG. Figure 2As shown, there are a third reflector 28 and a fourth reflector 28 between the fourth converging lens 233 and the scanning array mirror 24. The third reflector 28 is used to reflect the light beam passing through the fourth converging lens 233 to the fourth reflector 29, and the fourth reflector 29 is used to reflect the light beam reflected by the third reflector 28 to the scanning array mirror 24.

[0066] Here, the cubic phase modulation parameters (eg, fringe spacing, modulation depth, etc.) of the spatial light modulator 22 may have multiple groups for respectively modulating light beams with different focal lengths, main lobe sizes, and focused light energies.

[0067] Here, the numerical aperture parameter of the objective lens 25 can be determined according to the requirements of the object to be observed, thereby controlling the image reconstruction module 5 to perform cross-scale high-resolution imaging of the object of interest at different scales. For example, the numerical aperture (NA) of the objective lens 25 can be 0.1, and the magnification can be 4 times.

[0068] In some embodiments, as Figure 2 As shown, the system control and sparse scanning module 3 includes: a photodetector 31, a scanning galvanometer 24, a data acquisition card 32, and a system control module 33. The photodetector 31 is electrically connected to the data acquisition card 32. The photodetector 31 is configured to send a trigger signal to the scanning galvanometer 24 and the data acquisition card 32 simultaneously upon receiving the second standard linearly polarized Gaussian beam. The system control module 33 is configured to generate two two-dimensional triangular waveforms at a preset frequency upon receiving the trigger signal from the data acquisition card 32, and send them to the two galvanometers of the scanning galvanometer 24, respectively. The scanning galvanometer 24 is configured to perform a two-dimensional sparse scanning sampling of the object 6 using a two-dimensional triangular Lissajous locus based on the two-dimensional triangular waveforms received by the two galvanometers to obtain a sparse photoacoustic signal.

[0069] Here, the image reconstruction process includes: (a) the photoacoustic waves generated from each position of the absorber (e.g., biological tissue) in the depth direction propagate in sequence to the surface of the absorber and are received and collected by the signal receiving device; (b) by measuring the change in the intensity of the photoacoustic signal corresponding to the flight time of the acoustic wave, the flight time is converted into depth information, and then a one-dimensional depth-resolved image of the "A scan" is formed; (c) then, a two-dimensional scan is performed on the surface of the absorber to reconstruct a two-dimensional or three-dimensional tomographic image; (d) the xy plane photoacoustic microscopic image can be reconstructed by the maximum projection of the original 3D data. In order to ensure high spatial resolution image quality, according to the Nyquist sampling theorem, the full sampling scanning step in photoacoustic imaging needs to be less than half of the lateral resolution of the system. Taking the imaging of tumors as an example, the tumor region of interest is generally about 4mm×4mm, the scanning points are 668×668, and the scanning step is 6μm. According to this standard, it is planned to obtain a full sampling data set from 300 sets of microscopic images, such as Figure 3AAs shown in (a) in the figure. To obtain the sparse sampling data corresponding to the full sampling data, it is proposed to select the two-dimensional triangular Lissajous trajectory as the optimal sparse sampling pattern. This scanning design can provide a more uniform sampling density than the sinusoidal Lissajous trajectory, as shown in Figure 3A As shown in (b) in the figure. This sparse sampling method requires three axes to follow the triangular waveform while precisely controlling the frequency. The waveform of the spatial axis (X, Y) plane can be laser positioned by sending the triangular waveform to a pair of galvanometers (GM1, GM2). During scanning, a multi-function data acquisition card can be used to generate two waveforms at a specific selected frequency and send them to the two galvanometers to generate sparse scanning data, such as Figure 3B This is the waveform of the triangle wave output by the data acquisition card on the scanning axis.

[0070] In some embodiments, as Figure 2 As shown, the signal receiving and processing module 4 includes: an ultrasonic transducer 41, which is located between the object 6 and the objective lens 25, and is used to receive the sparse photoacoustic signal and obtain an electrical signal based on the sparse photoacoustic signal. Exemplarily, the center frequency of the ultrasonic transducer is 50 MHz and the bandwidth is 78%. The amplifier 42 is electrically connected to the ultrasonic transducer 41 and the data acquisition card 32, respectively, and is used to amplify the electrical signal sent by the ultrasonic transducer 41 to obtain an amplified sparse photoacoustic signal, and then send the sparse signal to the data acquisition card 32. The maximum value projection is performed based on the amplified and collected sparse photoacoustic signal to obtain a sparse image. Exemplarily, the amplified signal can be above 50 mV.

[0071] In some embodiments, the image reconstruction module 5 includes an imaging terminal 51, which is also electrically connected to the data acquisition card 32 and is configured to receive sparse signal data transmitted by the data acquisition card and reconstruct the sparse image using a pre-trained super-resolution reconstruction network to obtain a high-resolution photoacoustic microscopy original image of the object. It should be noted that the system control module 33 is included in the imaging terminal 51.

[0072] In some embodiments, the imaging terminal 51 is further configured to send a control signal to the nanosecond pulse laser 11 to control the nanosecond pulse laser 11 to be turned on.

[0073] Here, the network structure of the pre-trained super-resolution reconstruction network (SR-cycleGAN network) is as follows Figure 4A As shown in the figure, the upsampling network (G_up) and the discriminant network Dx form the forward GAN network, and the downsampling network (G_down) and the discriminant network Dy form the reverse GAN network. The G_up network and the G_down network are used to generate high-resolution images and low-resolution images respectively. The discriminant network Dx and the discriminant network Dy are the two discriminant networks in the SRGAN network.

[0074] Specifically, the G_up network includes: the first convolutional layer, the activation layer, the residual sub-network, the sub-pixel convolutional layer and the second convolutional layer. The G_down network includes: the first convolutional layer, the activation layer, the residual sub-network, the first downsampling convolutional layer, the second downsampling convolutional layer and the second convolutional layer. The two discriminant networks are the discriminant networks in the super-resolution generative adversarial network. For example, Figure 4B As shown, the parameters of the first convolutional layer are k9n64s1, which means the convolution kernel is 9*9, the dimension is 64, the step size is 1, the activation layer is the PRelu layer, and the residual subnetwork can include 16 residual blocks. The structure of each residual block is as follows Figure 4C As shown in the figure, the parameters of the first and second downsampling convolutional layers are both k2n64s2, and the parameters of the second convolutional layer are k2n3s1. The G_up network is an SRResNet network, and the structure of G_down is obtained by replacing the sub-pixel convolutional layer in G_up with the first and second downsampling convolutional layers.

[0075] Specifically, the forward GAN network realizes the mapping from the X image domain to the Y image domain, and the reverse GAN network realizes the mapping from the Y image domain to the X image domain. Since the two are inverse mapping relationships, the forward cycle consistency loss can be used to realize x→G_up(x)→G_down(G_up(x))≈x, and the reverse consistency loss can be used to realize y→G_down(y)→G_up(G_down(y))≈y. Based on the above analysis, the adversarial loss can be applied to these two mapping functions. For the mapping function G_up:X→Y and the discriminant network D Y , the objective function can be expressed as formula (1):

[0076]

[0077] D Y (y) represents the discriminant network D Y The discriminant value of the real sample image y, represents the expected value, D Y (G_up(x)) represents the discriminant network D Y The discriminant value of the generated image G_up(x), G_up(x) is the image generated by the G_up network. The goal of the G_up network is to try to generate an image G_up(x) similar to the image in the Y domain, while D Y The purpose is to distinguish G_up(x) from the real sample y. For the mapping function G_down:Y→X and its discriminant network D X A similar adversarial loss is also introduced, namely L GAN (G_down,D X,X,Y). Since it is difficult to ensure that a single input can output the expected result with a simple adversarial loss constraint, a cycle consistency loss is added to ensure the cycle consistency of the mapping relationship, that is, the following formula (2):

[0078]

[0079] G_down(y) is the image generated by the G_down network, G_up(G_down(y)) represents the image generated by the G_up network based on the input image G_down(y), and similarly, G_down(G_up(x)) represents the image generated by the G_down network based on the input image G_up(x). Therefore, the objective function of the entire network is formula (3):

[0080]

[0081] Among them, L GAN (G_up,D Y ,X,Y) is the objective function of the forward GAN network, L GAN (G_down,D X ,X,Y) is the objective function of the reverse GAN network, λ is used to control the relative importance of the two objective functions, and λ is a preset value. The task of the entire network is classified as deoptimization, as shown in formula (4):

[0082]

[0083] Here, when training the SR-cycleGAN network, 300 sets of original data can be selected and split into training, validation, and test sets in a ratio of 0.8:0.1:0.1. Sparsely sampled images are used as input, and fully sampled images (668×668 pixels) are used as output (i.e., ground truth). The Adam optimization algorithm is used to update the network parameters during training, and the initial learning rate is set to 0.0001.

[0084] Here, the above network is used for image reconstruction, which can quickly and continuously image the region of interest, while improving the acquisition speed of the photoacoustic imaging system and maintaining a certain spatial resolution.

[0085] Conventional imaging methods are based on scanning and imaging the sample using a Gaussian beam. Compared to the Airy beam, the linearly polarized Gaussian light can be directly transmitted to the scanning array lens 24 and the objective lens 25 by turning off the spatial light modulator 22 and not modulating it. Due to the small focal depth of the Gaussian beam, the three-dimensional mobile platform 26 carrying the object needs to be moved along the Z-axis to find the focal point for two-dimensional sparse synchronous scanning and acquisition. Due to the limited depth of field of the Gaussian beam, the imaging sensitivity is poor during this imaging process. In the present invention, the Airy beam can be generated by phase modulation of the linearly polarized Gaussian light by the spatial light modulator 22 and Fourier transform by the lens 25. The Airy beam is used to sparsely sample the object, obtaining sparse image data for super-resolution reconstruction to restore the original photoacoustic microscopy image. Compared to the traditional Gaussian beam, the Airy beam has a longer focal length and a smaller focal size. Therefore, high-resolution images with a large depth of field can be acquired through two-dimensional synchronous scanning and acquisition and reconstruction, maintaining a lateral resolution better than 4 μm within a depth of field of 926 μm. Moreover, since the energy of the main lobe of the Airy beam can reach more than 50%, it can also improve the imaging sensitivity and contrast compared to the Bessel beam which is also non-diffraction.

[0086] Figure 5 Schematic diagram of the effect of the photoacoustic microscopy imaging system based on Airy beam combined with sparse sampling provided by an embodiment of the present invention. In order to characterize the optical properties of the modulated light beam, a CMOS camera (Thorlabs) is placed on the rear focal plane where the excitation light beam is focused by the objective lens, and is fixed on the lower end face of the three-dimensional mobile platform. The three-dimensional mobile platform is moved at a uniform speed along the Z axis. Each time it moves, the CMOS camera captures a layer of XY plane spot image, and the captured multiple layers of XY images are superimposed to obtain a three-dimensional stacked image, which is projected on the XZ plane as the final depth direction spot image. The spot image contains the spot shape of the current excitation light, and the Airy spot images with different modulation parameters modulated by the spatial light modulator are obtained and compared with the Gaussian spot images that are not modulated by the spatial light modulator. The comparison results are shown as follows: Figure 5 As shown in (d), (e), and (f) in the figure. Specifically, when the lateral scale factors of the Gaussian beam and the Airy beam with different cubic phase parameters are 3.5 and 1.8, respectively, they are marked as Gaussian light, Airy light #1, and Airy light #2. In the absence of phase modulation, the measured spot size (full width at half maximum) of the Gaussian beam captured by the objective lens at its focal plane is about 11μm. When the cubic phase parameter is loaded onto the spatial light modulator, a finite energy Airy beam can be captured on the Fourier focal plane, as shown in Figure 5As shown in (e), (f), (h), (i), and (k) in the figure, the intensity distribution is multi-lobed, with the main lobe containing the majority of the total beam energy, which is consistent with the relevant simulation results. Both Airy beams of finite energy exhibit low diffraction at different transverse scale factors. In the phase diagrams of Airy beam #1 and Airy beam #2, the main lobe sizes of the Airy beams measured at the focus are approximately 5μm and 3μm, respectively. The attenuation factor of Airy beams #1 and #2 is 0.0011, which determines the attenuation rate of the lateral sidelobes and beam propagation direction. The light spot is obtained by using the threshold of the light intensity in the image greater than 1 / e of the maximum light intensity. The length of the Gaussian light spot in the Z-axis direction is approximately 58μm. The focal length of Airy beam #1 is approximately 11 times that of a Gaussian beam of the same wavelength, and the focal length of Airy beam #2 is approximately 16 times that of a Gaussian beam of the same wavelength. It can be seen from this that the focal length of the Airy beam in the Z-axis direction is much larger than that of the Gaussian beam. The main lobe size of Airy beam #2 is smaller than that of Airy beam #1, but the energy it carries is greater than that of Airy beam #1, which are 66% and 42% respectively.

[0087] The present invention also provides a photoacoustic microscopy method based on Airy beam combined with sparse sampling, which is applied to the above-mentioned photoacoustic microscopy system, such as Figure 6 As shown, the method includes:

[0088] S101 , generating a nanosecond pulsed Gaussian beam, and processing the nanosecond pulsed Gaussian beam to obtain a first standard linearly polarized Gaussian beam and a second standard linearly polarized Gaussian beam.

[0089] S102 , performing polarization adjustment, phase modulation, filtering, focusing, and Fourier transform processing on the first standard linearly polarized Gaussian beam to obtain an Airy beam that excites the object.

[0090] S103 , performing two-dimensional sparse scanning sampling on the object excited by the Airy beam using a two-dimensional triangular Lissajous trajectory according to the second standard linearly polarized Gaussian beam to obtain a sparse photoacoustic signal.

[0091] S104: Receive a sparse photoacoustic signal, and perform maximum projection on the sparse photoacoustic signal to obtain a sparse image.

[0092] S105. Use a pre-trained super-resolution reconstruction network to reconstruct the sparse image to obtain a high-resolution photoacoustic microscopy original image of the object.

[0093] In response to the current constraints between spatial resolution, depth of field, detection sensitivity, and imaging speed in photoacoustic microscopy, the present invention utilizes a beam generation module and a beam shaping module to generate a non-diffracting Airy beam with a longer focal depth and better focusing effect. Furthermore, its main lobe can carry relatively more energy (approximately 50%), significantly expanding the range of axial imaging of objects (e.g., biological tissues). This achieves photoacoustic microscopy with a large depth of field and high sensitivity, and can obtain high-resolution images with constant depth resolution. Furthermore, the present invention uses a two-dimensional triangular Lissajous trajectory to perform two-dimensional sparse scanning sampling on the object excited by the Airy beam, processes the scanned signal, and then reconstructs the image using the pre-trained super-resolution reconstruction network proposed in the present invention, meeting the imaging requirements of efficient and repeatable monitoring. Therefore, the present invention can achieve rapid imaging monitoring of objects (e.g., physiological and pathological activities) with a large depth of field and high sensitivity.

[0094] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art of the present invention, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should be considered to fall within the scope of protection of the present invention.

Claims

1. A photoacoustic microscopy system based on Airy beam combined with sparse sampling, characterized in that: include: a beam generating module, configured to generate a nanosecond pulsed Gaussian beam, and process the nanosecond pulsed Gaussian beam to obtain a first standard linearly polarized Gaussian beam and a second standard linearly polarized Gaussian beam; a beam shaping module, configured to perform polarization adjustment, phase modulation, filtering, focusing, and Fourier transform processing on the first standard linearly polarized Gaussian beam to obtain an Airy beam that excites the object; a system control and sparse scanning module, configured to perform a two-dimensional sparse scanning sampling on the object excited by the Airy beam using a two-dimensional triangular Lissajous trajectory upon receiving the second standard linearly polarized Gaussian beam, to obtain a sparse photoacoustic signal; a signal receiving and processing module, configured to receive the sparse photoacoustic signal and perform maximum projection on the sparse photoacoustic signal to obtain a sparse image; An image reconstruction module is configured to reconstruct the sparse image using a pre-trained super-resolution reconstruction network to obtain a high-resolution photoacoustic microscopy original image of the object; wherein the super-resolution reconstruction network includes: two discriminant networks, an upsampling network, and a downsampling network; the upsampling network and a discriminant network constitute a forward GAN network, and the downsampling network and another discriminant network constitute a backward GAN network; The beam generation module includes: a nanosecond pulse laser, a polarizer, a polarization beam splitter, an aperture and a beam expansion lens group; the beam shaping module includes: a half-wave plate, a spatial light modulator, a beam reduction lens group, a scanning array lens, an objective lens and a three-dimensional moving platform; The system control and sparse scanning module includes: a photoelectric detector, a scanning galvanometer, a data acquisition card and a system control module; The photoelectric detector is used to synchronously send a trigger signal to the scanning galvanometer and the data acquisition card when receiving the second standard linearly polarized Gaussian beam; The data acquisition card and the system control module are used to generate two two-dimensional triangular waveforms at a preset frequency and send them to the two galvanometers of the scanning galvanometer respectively when receiving the trigger signal; The scanning galvanometer is used to perform two-dimensional sparse scanning sampling on the object using two-dimensional triangular Lissajous trajectories according to the two-dimensional triangular waveforms received by the two galvanometers to obtain the sparse photoacoustic signal.

2. The photoacoustic microscopy system based on Airy beam combined with sparse sampling according to claim 1, characterized in that: The upsampling network includes, in sequence: a first convolutional layer, an activation layer, a residual subnetwork, a sub-pixel convolutional layer, and a second convolutional layer; the downsampling network includes, in sequence: the first convolutional layer, the activation layer, the residual subnetwork, a first downsampling convolutional layer, a second downsampling convolutional layer, and the second convolutional layer; the two discriminant networks are the discriminant networks in the super-resolution generative adversarial network.

3. The photoacoustic microscopy system based on Airy beam combined with sparse sampling according to claim 1, characterized in that: The nanosecond pulse laser is used to generate a nanosecond pulsed Gaussian beam; The polarizer is used to adjust the polarization of the nanosecond pulsed Gaussian beam to obtain a standard linearly polarized Gaussian beam; The polarization beam splitter is used to split the standard linearly polarized Gaussian beam into a first standard linearly polarized Gaussian beam and a second standard linearly polarized Gaussian beam according to a preset ratio, and use the second standard linearly polarized Gaussian beam as a second standard linearly polarized Gaussian beam; The aperture is used to optimize the first standard linearly polarized Gaussian beam to obtain an optimized first standard linearly polarized Gaussian beam; The beam expansion lens group is used to expand the optimized first standard linearly polarized Gaussian beam to obtain a first standard linearly polarized Gaussian beam.

4. The photoacoustic microscopy system based on Airy beam combined with sparse sampling according to claim 3, characterized in that: The beam expanding lens group includes, in sequence: a first converging lens, a first reflecting mirror, a first pinhole, and a second converging lens; The first aperture is used to filter out other stray light in the standard linearly polarized Gaussian light beam after passing through the first converging lens.

5. The photoacoustic microscopy system based on Airy beam combined with sparse sampling according to claim 1, characterized in that: The half-wave plate is used to adjust the polarization direction of the first standard linearly polarized Gaussian beam; The spatial light modulator is used to perform phase modulation on the first standard linearly polarized Gaussian light beam after polarization adjustment according to a preset azimuth phase modulation parameter to obtain a modulated light beam; The beam reduction lens group is used to perform beam reduction and filtering processing on the modulated light beam; The objective lens is used to focus and Fourier transform the light beam passing through the scanning array mirror to obtain the Airy beam; The three-dimensional moving platform is used to place the object and control the object to be located at the focus of the objective lens by moving itself in the horizontal and / or vertical directions.

6. The photoacoustic microscopy system based on Airy beam combined with sparse sampling according to claim 5, characterized in that: The spatial light modulator is loaded with a phase-type blazed grating with different grating periods. The spatial light modulator is further used to diffract the modulated light beam to different diffraction angles to separate the zero-order light spot from the first-order light spot; The beam reduction lens group includes, in sequence: a third converging lens, a second pinhole, and a fourth converging lens; The second small hole is used to filter out the zero-order light spot.

7. The photoacoustic microscopy system based on Airy beam combined with sparse sampling according to claim 1, characterized in that: The beam shaping module further comprises: a third reflector and a fourth reflector; the third reflector and the fourth reflector are sequentially positioned between the beam reduction lens group and the scanning array mirror; The third reflecting mirror is used to reflect the light beam passing through the beam reduction lens group to the fourth reflecting mirror; The fourth reflecting mirror is used to reflect the light beam reflected by the third reflecting mirror to the scanning array mirror.

8. The photoacoustic microscopy system based on Airy beam combined with sparse sampling according to claim 1, characterized in that: The signal receiving and processing module includes: an ultrasonic transducer, configured to receive the sparse photoacoustic signal and convert the sparse photoacoustic signal into an electrical signal; The amplifier is used to amplify the electrical signal to obtain the amplified sparse photoacoustic signal.

9. A photoacoustic microscopy method based on Airy beam combined with sparse sampling, characterized in that: The photoacoustic microscopy system according to any one of claims 1 to 8 comprises: generating a nanosecond pulsed Gaussian beam, and processing the nanosecond pulsed Gaussian beam to obtain a first standard linearly polarized Gaussian beam and a second standard linearly polarized Gaussian beam; performing polarization adjustment, phase modulation, filtering, focusing, and Fourier transform processing on the first standard linearly polarized Gaussian beam to obtain an Airy beam that excites the object; performing a two-dimensional sparse scanning sampling of the object excited by the Airy beam using a two-dimensional triangular Lissajous trajectory according to the second standard linearly polarized Gaussian beam to obtain a sparse photoacoustic signal; receiving the sparse photoacoustic signal, and performing maximum projection on the sparse photoacoustic signal to obtain a sparse image; The sparse image is reconstructed using a pre-trained super-resolution reconstruction network to obtain a high-resolution photoacoustic microscopy original image of the object.

Images