Adaptive optical optimization multi-modal imaging method based on photoacoustic signal assistance

Abstract

The invention relates to a multi-mode imaging method based on photoacoustic signal-assisted adaptive optical optimization, which comprises the following steps of: collecting imaging results of photoacoustic imaging modes, calculating by combining a machine learning algorithm to obtain photoacoustic signal phase change, and feeding back the photoacoustic signal phase change to a photoacoustic adaptive optical module in a system to carry out imaging phase adjustment of an optical system; due to the fact that the photoacoustic microscope imaging mode, the multi-photon microscopic imaging mode, the multiple harmonic microscopic imaging mode and the optical coherence imaging mode are designed in a common optical path mode, the resolution ratio of photoacoustic imaging is improved. The resolution of photoacoustic microscopic imaging is improved, and the penetration depth and the imaging resolution of two imaging modes, namely a multi-photon microscopic imaging mode and a multiple harmonic microscopic imaging mode, can be synchronously improved; according to the method, the depth penetration function and the rapid imaging advantage of the photoacoustic microscope are utilized, the full-view and high-speed regulation and control capacity is achieved, the multi-modal imaging effect can be remarkably improved, and application of a multi-modal system in disease detection and scientific research is promoted.

Description

Technical Field

[0001] The present invention relates to the field of microscopic imaging technology, and in particular, to a method for multi-modal imaging based on photoacoustic signal-assisted adaptive optics optimization. Background Art

[0002] Existing microscopic imaging technologies often involve multiple different imaging modes, such as three-photon microscopes, third-harmonic microscopes, optical coherence tomography microscopes, and photoacoustic microscopes. Each imaging mode has different imaging characteristics and advantages, but there are also certain limitations, especially in terms of image resolution and penetration depth. To address these issues, in recent years, Adaptive Optics (AO) technology has been widely applied to microscopic imaging systems to dynamically adjust the imaging performance of the optical system.

[0003] However, traditional adaptive optics control methods usually only optimize the imaging effect of a single modality, ignoring the mutual influence and collaborative optimization between multi-modal systems. Therefore, how to improve the overall performance of a multi-modal microscopic imaging system through intelligent means remains an urgent problem to be solved.

[0004] Study the method of multi-modal imaging function integration. After realizing different modality imaging using a single light source, how to synchronously optimize the imaging effects of multiple modalities. After realizing the basic function, adopt the hardware adaptive optics method. Using the imaging effect diagram of the photoacoustic imaging microscope as a benchmark, calculate the image resolution of different regions in blocks by combining machine learning methods, and obtain the optical distortion phase that needs to be compensated, and feedback it to the adaptive optics module for compensation, so as to realize the synchronous improvement of the resolution and imaging depth of a multi-modal microscopic imaging system. Summary of the Invention

[0005] The object of the present invention is to propose a method for optimizing a multimodal imaging system based on photoacoustic signal-assisted adaptive optics for the problems existing in the prior art. This method simultaneously integrates multimodalities including multi-photon microscopy (MPM), multi-harmonic microscopy (MHM), photoacoustic microscopy modality or photoacoustic computed tomography modality [including photoacoustic microscopy (PAM) or photoacoustic computed tomography (PACT)], and optical coherence imaging modality [including optical coherence microscopy (OCM) or optical coherence tomography (OCT)], and combines corresponding optical and ultrasonic detectors to effectively detect the imaging signals of each modality; and by collecting the imaging results of the photoacoustic imaging modality and calculating the phase change of the photoacoustic signal through a machine learning algorithm, and feeding it back to the photoacoustic adaptive optics module in the system for imaging phase adjustment of the optical system, the resolution of photoacoustic imaging is improved. Since the photoacoustic microscopy imaging modality shares the optical path with the multi-photon microscopy modality, multi-harmonic microscopy modality, and optical coherence imaging modality, while improving the resolution of photoacoustic microscopy imaging, the penetration depth and imaging resolution of the multi-photon microscopy modality and multi-harmonic microscopy modality can be synchronously improved, so that optical coherence imaging images, optimized photoacoustic microscopy images, optimized fluorescence images, and optimized harmonic images can be synchronously output.

[0006] The method for optimizing a multimodal imaging system based on photoacoustic signal-assisted adaptive optics provided by the embodiments of the present invention utilizes the deep penetration function and fast imaging advantage of a photoacoustic microscope, has a full-field and high-speed regulation ability, can significantly improve the effect of multimodal imaging, and promotes the application of multimodal systems in disease detection and scientific research.

[0007] To achieve the above object, in a first aspect, the embodiments of the present invention provide a method for optimizing multimodal imaging based on photoacoustic signal-assisted adaptive optics. The system for executing the method includes: a laser module, a laser broadening module, an optical coherence imaging module, a first optical path conduction module, a photoacoustic adaptive optics module, a second optical path conduction module, a beam scanning module, a third optical path conduction module, a sample excitation module, a photoacoustic signal acquisition module, a photoacoustic image calculation module, and a detection module; the photoacoustic adaptive optics module includes a spatial light modulator.

[0008] The method includes:

[0009] Step S1, start the system. The laser module emits a first laser beam and a second laser beam. The laser broadening module converts the first laser beam into a broadened optical signal. The optical coherence imaging module splits the second laser beam into a compensation optical signal and a test optical signal. The compensation optical signal forms a reference arm signal after multiple reflections. The broadened optical signal and the test optical signal sequentially pass through the first optical path conduction module, the photoacoustic adaptive optical module, the second optical path conduction module, the beam scanning module, and the third optical path conduction module and enter the sample excitation module to excite the sample to be measured, generating photoacoustic signals, optical coherence imaging signals, multi-photon signals, and multiple harmonic signals. The optical coherence imaging signal is used to form an optical coherence imaging image. After the detection module detects the multi-photon signal and the multiple harmonic signal, they are respectively output as a fluorescence image and a harmonic image.

[0010] Step S2, the photoacoustic signal acquisition module acquires the photoacoustic electrical signal and converts the photoacoustic electrical signal into a photoacoustic microscopy electrical signal and sends it to the photoacoustic image calculation module.

[0011] Step S3, the photoacoustic image calculation module collects the photoacoustic microscopy electrical signal, performs preprocessing, generates an optimized photoacoustic microscopy image, and uses a machine learning algorithm to perform phase inversion calculation on the optimized photoacoustic microscopy image to generate a photoacoustic phase adjustment signal for adjusting the spatial light modulator.

[0012] Step S4, the adjusted spatial light modulator performs phase adjustment on the broadened optical signal and the test optical signal that re-enter the photoacoustic adaptive optical module.

[0013] Step S5, the phase-adjusted broadened optical signal and test optical signal sequentially pass through the second optical path conduction module, the beam scanning module, and the third optical path conduction module and enter the sample excitation module to re-excite the sample to be measured, respectively generating optimized multi-photon signals, multiple harmonic signals, and photoacoustic signals, and further outputting an optimized fluorescence image and an optimized harmonic image.

[0014] Preferably, step S1 specifically includes:

[0015] Start the system. The laser module emits a first laser beam and a second laser beam.

[0016] The laser broadening module obtains the first laser beam, performs broadening processing, and forms a broadened optical signal.

[0017] The optical coherence imaging module obtains the second laser beam, performs beam splitting processing, generates a compensation optical signal and a test optical signal, and further performs multiple reflections on the compensation light to form a reference arm signal.

[0018] The broadened optical signal and the test optical signal sequentially pass through the first optical path conduction module, the photoacoustic adaptive optical module, the second optical path conduction module, the beam scanning module, and the third optical path conduction module to respectively form an output broadened optical signal and an output test optical signal, and enter the sample excitation module;

[0019] The output broadened optical signal excites the sample to be measured in the sample excitation module to generate a photoacoustic signal, and converts the photoacoustic signal into a photoacoustic electrical signal for acquisition by the photoacoustic signal acquisition module;

[0020] The output test optical signal excites the sample to be measured to generate an excitation optical signal;

[0021] The excitation optical signal is separated to generate a sample excitation optical signal and an optical coherence imaging signal;

[0022] The optical coherence imaging signal sequentially passes through the third optical path conduction module, the beam scanning module, the second optical path conduction module, the photoacoustic adaptive optical module, and the first optical path conduction module and returns to enter the optical coherence imaging module, and performs beam combination interference processing with the reference arm signal to form an interference optical signal, and the interference optical signal is converted into an optical coherence imaging electrical signal for outputting an optical coherence imaging image;

[0023] The detection module acquires the sample excitation optical signal, and after focusing and beam splitting processing, forms a multi-photon signal and a multi-harmonic signal, and converts the multi-photon signal and the multi-harmonic signal into a multi-photon electrical signal and a multi-harmonic imaging electrical signal respectively for outputting a fluorescence image and a harmonic image.

[0024] Further preferably, the laser light source module includes: a chirped amplifier laser, a first half-wave plate, a first polarization beam splitter prism, a first mirror, and an optical parametric amplifier laser;

[0025] The laser module emits a first laser beam and a second laser beam. Specifically, the laser beam emitted by the chirped amplifier laser sequentially undergoes phase processing by the first half-wave plate and beam splitting processing by the first polarization beam splitter prism to generate a first laser beam and a laser signal to be amplified; the first laser beam is reflected by the first mirror and enters the laser broadening module; the optical parametric amplifier laser acquires the laser signal to be amplified, undergoes chirped amplification processing, generates a second laser beam, and sends it to the optical coherence imaging module;

[0026] The broadening process includes: grating dispersion pulse broadening process or material dispersion pulse broadening process; among them, when the grating dispersion pulse broadening process is adopted, the laser broadening module includes any one of a Martinez stretcher, an Offner stretcher, a Treacy stretcher, and a Martinez stretcher based on a 4F system; when material dispersion pulse broadening is adopted, the broadening medium of the laser broadening module includes: quartz material or zinc selenide material.

[0027] Further preferably, the optical coherence imaging module includes: a first beam splitting prism, an optical path compensation component, a second half-wave plate, a second polarization beam splitting prism, a quarter-wave plate, a silicon window plate, a beam combining prism, and an imaging component; among them, the optical path compensation component includes: a retroreflecting prism and a plurality of plane mirrors; the retroreflecting prism moves according to the position of the sample to be measured; the imaging component includes: a first lens, an optical fiber, and a spectrometer; one end of the optical fiber is connected to the first lens, and the other end of the optical fiber is connected to the spectrometer;

[0028] The first beam splitting prism acquires the second laser beam, performs beam splitting processing to form a compensation optical signal and a test optical signal; among them, the beam energy ratio of the compensation optical signal to the test optical signal is 1:99;

[0029] The plurality of plane mirrors acquire the compensation optical signal, and after multiple reflection processes, form a first reflected optical signal; the retroreflecting prism acquires the first reflected optical signal, and after reflection processing, forms a reference arm signal;

[0030] The test optical signal sequentially passes through the second half-wave plate, the second polarization beam splitting prism, the quarter-wave plate, and then after being reflected by the silicon window plate, enters the first optical path conduction module; the silicon window plate is placed at the Brewster angle to perform dispersion compensation on the second amplified optical signal;

[0031] The beam combining prism combines and interferes the reference arm signal and the optical coherence imaging signal to form an interference optical signal;

[0032] The first lens acquires the interference optical signal, and after focusing processing, forms a focused imaging optical signal;

[0033] The spectrometer acquires the focused imaging optical signal through the optical fiber, and after analysis processing, generates an optical coherence imaging electrical signal for an external display device to process the optical coherence imaging electrical signal and output an optical coherence imaging image.

[0034] Further preferably, the first optical path conduction module includes: a first dichroic mirror;

[0035] The photoacoustic adaptive optical module further includes: a second lens, a right-angle reflector, an achromatic focusing lens, and a third lens; the spatial light modulator includes any one of a deformable mirror, a digital micromirror array, or a liquid crystal spatial light modulator;

[0036] The second optical path conduction module includes: a second reflector;

[0037] The broadened optical signal and the test optical signal sequentially pass through the first optical path conduction module, the photoacoustic adaptive optical module, and the second optical path conduction module and enter the beam scanning module, specifically including:

[0038] The broadened optical signal sequentially passes through the reflection of the first dichroic mirror, the focusing of the second lens, the first reflection of the right-angle reflector, the first transmission of the achromatic focusing lens, and the reflection of the spatial light modulator to generate a first parallel light; the first parallel light sequentially passes through the second transmission of the achromatic focusing lens, the second reflection of the right-angle reflector, the focusing of the third lens, and the reflection of the second reflector, and then forms an incident broadened optical signal to enter the beam scanning module;

[0039] The test optical signal sequentially passes through the transmission of the first dichroic mirror, the focusing of the second lens, the first reflection of the right-angle reflector, the first transmission of the achromatic focusing lens, and the reflection of the spatial light modulator to generate a second parallel light; the second parallel light sequentially passes through the second transmission of the achromatic focusing lens, the second reflection of the right-angle reflector, the focusing of the third lens, and the reflection of the second reflector, and forms an incident test optical signal to enter the beam scanning module;

[0040] In step S4, phase adjustment is performed on the broadened optical signal and the test optical signal that re-enter the photoacoustic adaptive optical module, specifically including:

[0041] The broadened optical signal that re-enters the photoacoustic adaptive optical module sequentially passes through the focusing of the second lens, the first reflection of the right-angle reflector, the first transmission of the achromatic focusing lens, and the phase adjustment of the spatial light modulator to generate a phase-adjusted broadened optical signal; the phase-adjusted broadened optical signal sequentially passes through the second transmission of the achromatic focusing lens, the second reflection of the right-angle reflector, the focusing of the third lens, and the reflection of the second reflector, and forms an optimized incident broadened optical signal;

[0042] The test optical signal that re - enters the photoacoustic adaptive optical module sequentially undergoes the focusing of the second lens, the first reflection of the right - angle reflector, the first transmission of the achromatic focusing lens, and the phase adjustment of the spatial light modulator to generate a phase - adjusted test optical signal; the phase - adjusted test optical signal sequentially undergoes the second transmission of the achromatic focusing lens, the second reflection of the right - angle reflector, the focusing of the third lens, and the reflection of the second reflector to generate an optimized incident test optical signal.

[0043] Further preferably, the beam scanning module includes: a third reflector, a scanning device, and a galvanometer;

[0044] The third optical path conduction module includes: a fourth lens, a fourth reflector, a fifth lens, and a fifth reflector;

[0045] The third reflector adjusts the incident angle of the incident broadened optical signal and the incident test optical signal, or adjusts the incident angle of the optimized incident broadened optical signal and the optimized incident test optical signal;

[0046] The scanning device performs a scanning process on the incident broadened optical signal and the incident test optical signal, or the optimized incident broadened optical signal and the optimized incident test optical signal in the X - axis direction, and the galvanometer performs a scanning process on the incident broadened optical signal and the incident test optical signal, or the optimized incident broadened optical signal and the optimized incident test optical signal in the Y - axis direction to generate an outgoing broadened optical signal and an outgoing test optical signal, or generate an optimized outgoing broadened optical signal and an optimized outgoing test optical signal; the X - axis is perpendicular to the Y - axis;

[0047] The outgoing broadened optical signal and the outgoing test optical signal, or the optimized outgoing broadened optical signal and the optimized outgoing test optical signal, enter the sample excitation module after passing through the transmission of the fourth lens, the reflection of the third reflector, the transmission of the fifth lens, and the reflection of the fourth reflector.

[0048] Further preferably, the sample excitation module includes: a second dichroic mirror, an objective lens, an ultrasonic transducer, and a water tank; wherein, the second dichroic mirror is placed above the objective lens; the ultrasonic transducer and the water tank are integrated on the objective lens; the ultrasonic transducer is annular and is placed on the lower surface of the objective lens or surrounds the side surface of the objective lens; the lower surface or the side surface of the objective lens and the ultrasonic transducer are immersed together in the upper part of the water tank; the sample to be measured is placed in the lower part of the water tank;

[0049] The outgoing broadened optical signal and the outgoing test optical signal pass through the second dichroic mirror and enter the objective lens, and then act on the sample to be measured.

[0050] The emitted and broadened optical signal excites the sample to be measured to generate a photoacoustic signal, and the ultrasonic transducer converts the photoacoustic signal into a photoacoustic electrical signal and sends it into the photoacoustic signal acquisition module; the photoacoustic signal acquisition module acquires the photoacoustic electrical signal, generates a photoacoustic microscopy imaging electrical signal through analysis and processing, and sends the photoacoustic microscopy imaging electrical signal to the photoacoustic image calculation module;

[0051] The emitted test optical signal excites the sample to be measured to generate an excitation optical signal, and the second dichroic mirror separates the excitation optical signal to form a sample excitation optical signal and an optical coherence imaging signal, reflects the sample excitation optical signal into the detection module, and transmits the optical coherence imaging signal into the third optical path conduction module.

[0052] Further preferably, the detection module includes: a sixth lens, a third dichroic mirror, a fluorescence detection device and a harmonic detection device; wherein, the fluorescence detection device includes a first photomultiplier tube; the harmonic detection device includes a second photomultiplier tube;

[0053] In the step S1, the sample excitation optical signal is separated to generate a multi-photon signal and a multi-harmonic signal, specifically including; the sample excitation optical signal is sequentially subjected to the focusing process of the sixth lens and the beam splitting process of the third dichroic mirror to form a multi-photon signal and a multi-harmonic signal;

[0054] The first photomultiplier tube detects and analyzes the multi-photon signal, generates a multi-photon electrical signal, and sends the multi-photon electrical signal to an external display device connected to the first photomultiplier tube for the external display device to process the multi-photon electrical signal and output a fluorescence image;

[0055] The second photomultiplier tube detects and analyzes the multi-harmonic signal, generates a multi-harmonic imaging electrical signal, and sends the multi-harmonic imaging electrical signal to an external display device connected to the second photomultiplier tube for the external display device to process the multi-harmonic imaging electrical signal and output a harmonic image.

[0056] Preferably, the photoacoustic image calculation module includes: a photoacoustic image calculation unit and an adaptive signal processing unit;

[0057] The specific steps of step S3 are as follows: The photoacoustic image calculation unit collects the photoacoustic microscopy electrical signals, preprocesses the photoacoustic microscopy electrical signals to generate an optimized photoacoustic microscopy image, and transmits the optimized photoacoustic microscopy image to the adaptive signal processing unit; The adaptive signal processing unit uses a machine learning algorithm to perform phase inversion calculation on the photoacoustic microscopy image, identifies and analyzes the low-resolution regions in the optimized photoacoustic microscopy image, generates a photoacoustic phase adjustment signal, and sends the photoacoustic phase adjustment signal to the photoacoustic adaptive optical module;

[0058] Among them, the preprocessing includes: the photoacoustic image calculation unit performs one or more of signal denoising, dynamic range adjustment, and spectral filtering on the photoacoustic electrical signals;

[0059] The machine learning algorithm includes: a deep neural network;

[0060] In step S5, outputting the optimized fluorescence image and harmonic image specifically includes:

[0061] The detection module acquires the optimized multi-photon signal, and after detection, converts the optimized multi-photon signal into a multi-photon electrical signal for an external display device to process the multi-photon electrical signal and output the optimized fluorescence image;

[0062] The detection module acquires the optimized multiple harmonic signal, and after detection, converts the optimized multiple harmonic signal into a multiple harmonic imaging electrical signal for an external display device to process the multiple harmonic imaging electrical signal and output the optimized harmonic image.

[0063] In a second aspect, an embodiment of the present invention provides a system for performing the method for photoacoustic signal-assisted adaptive optical optimization of multimodal imaging described in the first aspect above. The system includes: a laser module, a laser broadening module, an optical coherence imaging module, a first optical path conduction module, a photoacoustic adaptive optical module, a second optical path conduction module, a beam scanning module, a third optical path conduction module, a sample excitation module, a photoacoustic signal acquisition module, a photoacoustic image calculation module, and a detection module; The photoacoustic adaptive optical module includes a spatial light modulator;

[0064] The laser module emits a first laser beam and a second laser beam;

[0065] The laser broadening module broadens the first laser beam to form a broadened optical signal;

[0066] The optical coherence imaging module splits the second laser beam to generate a compensation optical signal and a test optical signal, and further reflects the compensation light multiple times to form a reference arm signal;

[0067] The broadened optical signal and the test optical signal sequentially pass through the first optical path conduction module, the photoacoustic adaptive optical module, the second optical path conduction module, the beam scanning module, and the third optical path conduction module, and respectively generate an output broadened optical signal and an output test optical signal to enter the sample excitation module;

[0068] The output test optical signal excites the sample to be measured to generate an excitation optical signal; the sample excitation module separates the excitation optical signal to generate a sample excitation optical signal and an optical coherence imaging signal;

[0069] The detection module separates the sample excitation optical signal to form a multi-photon signal and a multi-harmonic signal, detects the fluorescence signal to generate a multi-photon imaging electrical signal, and then sends the multi-photon imaging electrical signal to an external display device connected to the detection module, and detects the harmonic signal to generate a multi-harmonic imaging electrical signal, and sends the multi-harmonic imaging electrical signal to an external display device connected to the detection module;

[0070] The optical coherence imaging module performs beam combination interference processing on the optical coherence imaging signal and the reference arm signal to generate an interference optical signal, then performs photoelectric conversion processing on the interference optical signal to generate an optical coherence imaging electrical signal, and sends the optical coherence imaging electrical signal to an external display device connected to the optical coherence imaging module;

[0071] The output broadened optical signal excites the sample to be measured in the sample excitation module to generate a photoacoustic signal; the photoacoustic signal acquisition module acquires the photoacoustic signal and converts the photoacoustic signal into a photoacoustic electrical signal and sends it to the photoacoustic image calculation module;

[0072] The photoacoustic image calculation module preprocesses the photoacoustic electrical signal to generate a photoacoustic microscopic image, performs phase inversion calculation to generate a photoacoustic phase adjustment signal, and sends the photoacoustic phase adjustment signal to the photoacoustic adaptive optical module to adjust the spatial light modulator to perform phase adjustment on the broadened optical signal and the test optical signal that enter the photoacoustic adaptive optical module again.

[0073] The method for optimizing multimodal imaging based on photoacoustic signal-assisted adaptive optics provided by the embodiments of the present invention simultaneously integrates multimodal imaging modalities such as multiphoton microscopy imaging modality, multiple harmonic microscopy imaging modality, photoacoustic microscopy imaging modality or photoacoustic computed tomography imaging modality, and optical coherence imaging modality, and combines corresponding optical components and ultrasonic transducers to effectively detect imaging signals of each modality. Moreover, in the multimodal imaging system provided by the embodiments of the present invention, a photoacoustic image calculation module and a photoacoustic adaptive optics module are adopted. By combining machine learning algorithms and adaptive optics regulation technologies, the imaging resolution is improved. Specifically, the machine learning algorithm is used to identify and analyze low-resolution regions in the photoacoustic microscopy image, and the photoacoustic adaptive optics module is used to adjust the phase of the optical system in real time, significantly improving the imaging resolution of photoacoustic microscopy imaging.

[0074] The method for optimizing multimodal imaging based on photoacoustic signal-assisted adaptive optics provided by the embodiments of the present invention can effectively reduce image blurring and distortion and provide clearer and more detailed imaging results. Moreover, on the basis of increasing the resolution, the penetration depth of other modalities is enhanced. Since the photoacoustic microscopy imaging modality shares the same optical path with the other three modalities, namely the multiphoton microscopy imaging modality, the multiple harmonic microscopy imaging modality, and the optical coherence imaging modality, and since the same sample barriers (such as the skull, skin, etc.) are passed through, the aberration caused by the phase misalignment after all excitation lights penetrate the sample barrier can be synchronously improved by optimizing the optical phase to compensate the optical phase.

[0075] The method for optimizing multimodal imaging based on photoacoustic signal-assisted adaptive optics provided by the embodiments of the present invention can synchronously improve the penetration depth of the multiphoton microscopy imaging modality and the multiple harmonic microscopy imaging modality.

[0076] In the system for executing the method for optimizing multimodal imaging based on photoacoustic signal-assisted adaptive optics provided by the embodiments of the present invention, a photoacoustic adaptive optics module and a photoacoustic image calculation module are set up, realizing an automated workflow for adjusting the phase of the laser signal. The machine learning algorithm automatically identifies low-resolution regions in the photoacoustic image, automatically derives an adjustment scheme for optimizing the optical phase, and feeds it back to the adaptive optics module for multiple cyclic optimization processes. The entire process requires no manual intervention, realizing the automated optimization of the imaging system.

[0077] Moreover, compared with other adaptive optics modes, such as using methods like optical coherence tomography scanning modality phase calculation, the present invention uses photoacoustic signal-assisted adaptive optics technology to collect photoacoustic images, which has the advantage of high speed. It can realize reconstruction while scanning during the scanning process of the photoacoustic image by the photoacoustic image calculation module, and transmit the phase adjustment signal to the photoacoustic adaptive optics module during the reconstruction process, reducing the transmission waiting time and improving the efficiency and speed. Description of the Drawings

[0078] Figure 1 This is a structural block diagram of System 1 for implementing the method of multi-modal imaging with photoacoustic signal-assisted adaptive optical optimization provided by an embodiment of the present invention.

[0079] Figure 2 This is a flowchart of the method of multi-modal imaging with photoacoustic signal-assisted adaptive optical optimization provided by an embodiment of the present invention.

[0080] Figure 3 This is a schematic structural diagram of System 2 for implementing the method of multi-modal imaging with photoacoustic signal-assisted adaptive optical optimization provided by Embodiment 1 of the present invention. Detailed implementation manners

[0081] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0082] The technical solutions of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0083] An embodiment of the present invention provides a method of multi-modal imaging with photoacoustic signal-assisted adaptive optical optimization. For ease of understanding, the modules of the system for implementing this method will be introduced first.

[0084] Figure 1 This is a structural block diagram of System 1 for multi-modal imaging with photoacoustic signal-assisted adaptive optical optimization for implementing the above method provided by an embodiment of the present invention.

[0085] As Figure 1 shown, the multi-modal imaging system 1 with photoacoustic signal-assisted adaptive optical optimization includes:

[0086] Laser module 11, laser broadening module 12, optical coherence imaging module 13, first optical path conduction module 14, photoacoustic adaptive optical module 15, second optical path conduction module 16, beam scanning module 17, third optical path conduction module 18, sample excitation module 19, photoacoustic signal acquisition module 20, photoacoustic image calculation module 21, and detection module 22.

[0087] Among them, the laser module 11 includes: a chirped pulse amplification laser (CPA), an optical parametric amplification laser (OPA), a first half-wave plate (not shown in the figure), a first polarization beam splitter prism (not shown in the figure), and a first mirror (not shown in the figure). The laser emitted by the chirped pulse amplification laser (CPA) sequentially undergoes the phase processing of the first half-wave plate and the beam splitting processing of the first polarization beam splitter prism to generate a first path of laser and a laser signal to be amplified; the first path of laser enters the laser broadening module after being reflected by the first mirror; the optical parametric amplification laser acquires the laser signal to be amplified, undergoes chirped pulse amplification processing, generates a second path of laser, and sends it to the optical coherence imaging module; among them, the wavelength of the first path of laser is 800 nm to 1064 nm; the wavelength of the second path of laser is 1300 nm to 1700 nm, which is picosecond laser.

[0088] In the present invention, the parameters of the chirped pulse amplification laser (CPA) are not fixed and can be replaced according to different experimental requirements and imaging effect requirements. For example, the output parameters of the CPA can be set as: a laser with a central wavelength of 1030 nm, a full width at half maximum of 70 nm, and a pulse width of 200 fs; and then the laser emitted by the CPA is broadened into a picosecond pulse light source with a central wavelength of 1030 nm and a pulse width of 20 ps through the photoacoustic light source broadening module.

[0089] The laser broadening module 12 broadens the first path of laser to form a broadened optical signal. Preferably, the broadened optical signal is nanosecond laser; the broadening process of the laser broadening module 12 extends the time width of the laser pulse through different optical technologies, so that its spectral distribution is wider. The broadening methods of the present invention include: grating dispersion pulse broadening or material dispersion pulse broadening; when grating dispersion pulse broadening is adopted, the laser broadening module 12 includes but is not limited to: any one of a Martinez stretcher, an Offner stretcher, a Treacy stretcher, and a Martinez stretcher based on a 4F system; when material dispersion pulse broadening is adopted, the broadening medium of the laser broadening module 12 includes but is not limited to: quartz material or zinc selenide material. When adopting grating dispersion pulse broadening, the laser broadening module 12 includes but is not limited to: any one of a Martinez stretcher, an Offner stretcher, a Treacy stretcher, and a Martinez stretcher based on a 4F system; when adopting material dispersion pulse broadening, the broadening medium of the laser broadening module 12 includes but is not limited to: quartz material or zinc selenide material.

[0090] The optical coherence imaging module 13 includes: a first beam splitting prism (not shown in the figure), an optical path compensation component 131, a second half-wave plate (not shown in the figure), a second polarization beam splitting prism (not shown in the figure), a quarter-wave plate (not shown in the figure), a silicon window (Silicon Window, not shown in the figure), a beam combining prism (not shown in the figure), and an imaging component 132; wherein, the optical path compensation component 131 includes: a retroreflective prism and a plurality of plane mirrors; the retroreflective prism moves according to the position of the sample to be measured; the imaging component 132 includes: a first lens (not shown in the figure), an optical fiber (not shown in the figure), and a spectrometer (not shown in the figure); one end of the optical fiber is connected to the first lens, and the other end of the optical fiber is connected to the spectrometer. The spectrometer includes but is not limited to a CCD spectrometer.

[0091] The first optical path conduction module 14 includes: a first dichroic mirror (not shown in the figure); optionally, optical devices such as lenses and mirrors can be added to the first optical path conduction module 14 according to the needs of laser conduction.

[0092] The photoacoustic adaptive optical module 15 includes: a second lens (not shown in the figure), a right-angle mirror (not shown in the figure), an achromatic focusing lens (not shown in the figure), a spatial light modulator (not shown in the figure), and a third lens (not shown in the figure). Among them, the spatial light modulator includes but is not limited to any one of a deformable mirror, a liquid crystal spatial light modulator, and a digital micromirror array.

[0093] The second optical path conduction module 16 includes: a second mirror (not shown in the figure); optionally, optical devices such as lenses and mirrors can be added to the second optical path conduction module 16 according to the needs of laser conduction.

[0094] The beam scanning module 17 includes: a third mirror, a scanning device 171, and a galvanometer 172; the scanning device 161 includes one or more of a polygon scanning mirror, a MEMS scanning mirror, and an AOM scanning mirror.

[0095] The third optical path conduction module 18 includes: a fourth lens (not shown in the figure), a fourth mirror (not shown in the figure), a fifth lens (not shown in the figure), and a fifth mirror (not shown in the figure). Optionally, optical devices such as lenses and mirrors can be added to the third optical path conduction module 18 according to the needs of laser conduction.

[0096] The sample excitation module 19 includes: a second dichroic mirror (not shown in the figure), an objective lens (not shown in the figure), an ultrasonic transducer (not shown in the figure), and a water tank (not shown in the figure); the second dichroic mirror is placed above the objective lens; the ultrasonic transducer and the water tank are integrated on the objective lens; the ultrasonic transducer is annular and placed on the lower surface of the objective lens or surrounds the side surface of the objective lens; the lower surface or the side surface of the objective lens and the ultrasonic transducer are immersed together in the upper part of the water tank; the sample to be measured is placed in the lower part of the water tank; wherein, the medium contained in the water tank can be water or a coupling agent for ultrasonic detection; the ultrasonic transducer includes but is not limited to any one of a 64-array piezoelectric transducer, a 128-array piezoelectric transducer, and a 256-array piezoelectric transducer, and preferably a 256-array piezoelectric transducer is used.

[0097] The photoacoustic signal acquisition module 20 is used to acquire the photoacoustic signals generated by the excitation of the sample to be measured in the sample excitation module 19, and convert the photoacoustic signals into photoacoustic electrical signals and send them to the photoacoustic image calculation module 21.

[0098] The photoacoustic image calculation module 21 includes: a photoacoustic image calculation unit 211 and an adaptive signal processing unit 212.

[0099] The detection module 22 includes: a sixth lens (not shown in the figure), a third dichroic mirror (not shown in the figure), a fluorescence detection device 221, and a harmonic detection device 222; wherein, the fluorescence detection device includes a first photomultiplier tube (not shown in the figure); the harmonic detection device includes a second photomultiplier tube (not shown in the figure). In an alternative embodiment, the fluorescence detection device 221 further includes a first band-pass filter (not shown in the figure), and the first band-pass filter is disposed between the third dichroic mirror and the first photomultiplier tube; the harmonic detection device 222 includes a second band-pass filter (not shown in the figure), and the second band-pass filter is disposed between the third dichroic mirror and the second photomultiplier tube.

[0100] In an alternative embodiment, the above-mentioned photoacoustic signal-assisted adaptive optical optimization multimodal imaging system 1 further includes a main frame (not shown in the figure); wherein the laser module 11, the laser broadening module 12, the optical coherence imaging module 13, the first optical path conduction module 14, the photoacoustic adaptive optical module 15, the second optical path conduction module 16, the beam scanning module 17, the third optical path conduction module 18, the sample excitation module 19, the photoacoustic signal acquisition module 20, the photoacoustic image calculation module 21, and the detection module 22 are arranged inside the main frame; there is an opening outside the main frame for conducting signal lines and power supply lines to the outside world.

[0101] The embodiment of the present invention provides a method for photoacoustic signal-assisted adaptive optical optimization multimodal imaging, as Figure 2 shown, in combination with Figure 1The system 1 will illustrate this method. The method for optimizing multimodal imaging based on photoacoustic signal-assisted adaptive optics specifically includes the following steps:

[0102] Step 210, start system 1. The emitted laser excites the sample to be measured, generating interference light signals, photoacoustic signals, optical coherence imaging signals, multiphoton signals, and multiple harmonic signals;

[0103] Specifically, it includes:

[0104] (1) The laser module 11 emits the first laser beam and the second laser beam. Specifically, it includes: the laser emitted by the chirped pulse amplification laser (CPA) passes through the phase processing of the first half-wave plate and the beam splitting processing of the first polarization beam splitter prism to generate the first laser beam and the laser signal to be amplified; the first laser beam enters the laser stretching module 12 after being reflected by the first mirror; the optical parametric amplification laser (OPA) obtains the laser signal to be amplified, undergoes chirped pulse amplification processing, generates the second laser beam, and sends it to the optical coherence imaging module 13. Among them, the wavelength of the first laser beam is 800nm - 1064nm; the wavelength of the chirped pulse amplified laser signal and the second laser beam is 1300nm - 1700nm.

[0105] (2) The laser stretching module 12 obtains the first laser beam and forms a stretched optical signal after stretching processing. Among them, the stretching processing methods include: grating dispersion pulse stretching or material dispersion pulse stretching; when using grating dispersion pulse stretching, the laser stretching module 12 includes but is not limited to: any one of the Martinez stretcher, the Offner stretcher, the Treacy stretcher, and the Martinez stretcher based on the 4F system; when using material dispersion pulse stretching, the stretching medium of the laser stretching module 12 includes but is not limited to: quartz material or zinc selenide material. When using material dispersion pulse stretching, the stretching medium of the laser stretching module 12 includes but is not limited to: quartz material or zinc selenide material.

[0106] (3) The optical coherence imaging module 13 obtains the second laser beam, undergoes beam splitting processing to generate a compensation optical signal and a test optical signal, and further reflects the compensation light multiple times to form a reference arm signal. Specifically, it includes: the first beam splitter prism obtains the second laser beam and undergoes beam splitting processing to form a compensation optical signal and a test optical signal; multiple plane mirrors in the optical path compensation component 131 obtain the compensation optical signal and form a first reflected optical signal after multiple reflection processing; the retroreflective prism obtains the first reflected optical signal and forms a reference arm signal after reflection processing. Among them, the beam energy ratio of the compensation optical signal to the test optical signal is 1:99.

[0107] (4) The stretched optical signal and the test optical signal sequentially pass through the first optical path conduction module 14, the photoacoustic adaptive optics module 15, and the second optical path conduction module 16 and enter the beam scanning module 17. Specifically, it includes:

[0108] The broadened optical signal sequentially undergoes reflection by the first dichroic mirror, focusing by the second lens, the first reflection by the right-angle reflector, the first transmission by the achromatic focusing lens, and reflection by the spatial light modulator to generate a first parallel light; the first parallel light sequentially undergoes the second transmission by the achromatic focusing lens, the second reflection by the right-angle reflector, focusing by the third lens, and reflection by the second reflector, and then forms an incident broadened optical signal to enter the beam scanning module 17;

[0109] The test optical signal sequentially undergoes transmission by the first dichroic mirror, focusing by the second lens, the first reflection by the right-angle reflector, the first transmission by the achromatic focusing lens, and reflection by the spatial light modulator to generate a second parallel light; the second parallel light sequentially undergoes the second transmission by the achromatic focusing lens, the second reflection by the right-angle reflector, focusing by the third lens, and reflection by the second reflector, and forms an incident test optical signal to enter the beam scanning module 17.

[0110] (5) The third reflector of the beam scanning module 17 adjusts the incident angles of the incident broadened optical signal and the incident test optical signal. The scanning device 171 performs scanning processing on the incident broadened optical signal and the incident test optical signal in the X-axis direction, and the galvanometer 172 performs scanning processing on the incident broadened optical signal and the incident test optical signal in the Y-axis direction to generate an outgoing broadened optical signal and an outgoing test optical signal; the X-axis is perpendicular to the Y-axis.

[0111] The outgoing broadened optical signal and the outgoing test optical signal enter the sample excitation module through the third optical path conduction module 18. Specifically, it includes: the outgoing broadened optical signal and the outgoing test optical signal sequentially undergo transmission by the fourth lens, reflection by the fourth reflector, transmission by the fifth lens, and reflection by the fifth reflector to enter the sample excitation module 19.

[0112] (6) The sample excitation module 19 acquires the outgoing broadened optical signal and the outgoing test optical signal. After the outgoing broadened optical signal and the outgoing test optical signal pass through the second dichroic mirror and enter the objective lens, they act on the sample to be measured;

[0113] The outgoing broadened optical signal excites the sample to be measured to generate a photoacoustic signal, and the ultrasonic transducer converts the photoacoustic signal into a photoacoustic electrical signal and sends it to the photoacoustic signal acquisition module 20; the photoacoustic signal acquisition module 20 collects the photoacoustic signal, generates a photoacoustic microscopy imaging electrical signal through analysis and processing, and sends the photoacoustic microscopy imaging electrical signal to the photoacoustic image calculation module 21;

[0114] The emitted test optical signal excites the sample under test to generate an excitation optical signal. The second dichroic mirror separates the excitation optical signal to form a sample excitation optical signal and an optical coherence imaging signal. The sample excitation optical signal enters the detection module 22 after being reflected by the second dichroic mirror. The optical coherence imaging signal enters the third optical path conduction module 18 after being transmitted by the second dichroic mirror. After the sixth lens of the detection module 22 focuses the excitation optical signal, the excitation optical signal is separated into a multi-photon signal and a multi-harmonic signal through the beam splitting process of the third dichroic mirror.

[0115] The optical coherence imaging signal successively passes through the third optical path conduction module 18, the beam scanning module 17, the second optical path conduction module 16, the photoacoustic adaptive optical module 15, and the first optical path conduction module 14 and returns to enter the optical coherence imaging module 13. The beam combining prism in the optical coherence imaging module 13 performs beam combining interference processing on the reference arm signal and the optical coherence imaging signal to form an interference optical signal. The first lens acquires the interference optical signal and, after focusing processing, forms a focused imaging optical signal. The spectrometer of the imaging component 132 acquires the focused imaging optical signal through an optical fiber, and after analysis and processing, generates an optical coherence imaging electrical signal for an external display device (not shown in the figure) to process the optical coherence imaging electrical signal and output an optical coherence imaging image.

[0116] The first photomultiplier tube of the detection module 22 detects and analyzes the multi-photon signal, generates a multi-photon electrical signal, and sends the multi-photon electrical signal to an external display device (not shown in the figure) connected to the first photomultiplier tube for the external display device to process the multi-photon electrical signal and output a fluorescence image.

[0117] The second photomultiplier tube of the detection module 22 detects and analyzes the multi-harmonic signal, generates a multi-harmonic imaging electrical signal, and sends the multi-harmonic imaging electrical signal to an external display device (not shown in the figure) connected to the second photomultiplier tube for the external display device to process the multi-harmonic imaging electrical signal and output a harmonic image.

[0118] Step 220, the photoacoustic signal acquisition module 20 acquires a photoacoustic electrical signal and converts the photoacoustic signal into a photoacoustic microscopy imaging electrical signal and sends it to the photoacoustic image calculation module 21.

[0119] Step 230, the photoacoustic image calculation module 21 collects the photoacoustic microscopy imaging electrical signals, performs preprocessing, generates an optimized photoacoustic microscopy image, performs phase inversion calculation on the optimized photoacoustic microscopy image using a machine learning algorithm, generates a photoacoustic phase adjustment signal, and sends the photoacoustic phase adjustment signal to the photoacoustic adaptive optical module 15 to adjust the spatial light modulator.

[0120] Specifically, the photoacoustic image calculation unit 211 collects photoacoustic microscopy electrical signals, preprocesses the photoacoustic microscopy electrical signals to obtain an optimized photoacoustic microscopy image, and conveys the optimized photoacoustic microscopy image to the adaptive signal processing unit 212; the adaptive signal processing unit 212 performs phase inversion calculation on the optimized photoacoustic microscopy image using a machine learning algorithm, identifies and analyzes low-resolution regions in the optimized photoacoustic microscopy image, generates a photoacoustic phase adjustment signal, and sends the photoacoustic phase adjustment signal to the photoacoustic adaptive optics module 15 for adjusting the spatial light modulator.

[0121] Among them, the preprocessing includes: the photoacoustic image calculation unit 211 performs one or more of signal denoising, dynamic range adjustment, and spectral filtering on the photoacoustic electrical signal.

[0122] The machine learning algorithm includes: a deep neural network. The machine learning algorithm analyzes the collected photoacoustic microscope images to identify low-resolution regions in the images. These low-resolution regions are usually related to image detail loss or insufficient imaging depth. The model of the machine learning algorithm can infer the optical phase changes required for optimization based on image features and previous learning experiences. According to the results inferred by the machine learning, the required photoacoustic phase adjustment signal is fed back to the optical adaptive optics module 15 in the system.

[0123] Step 240, the adjusted spatial light modulator performs phase adjustment on the broadened light signal and the test light signal that enter the photoacoustic adaptive optics module 15 again.

[0124] The photoacoustic adaptive optics module 15 changes the morphology of the spatial light modulator (including but not limited to any one of a deformable mirror, a liquid crystal spatial light modulator, and a digital micromirror array) in real time according to the adjustment information of the input photoacoustic phase adjustment signal, so as to precisely adjust the optical performance of the multimodal imaging system and improve the resolution of photoacoustic microscopy imaging.

[0125] This method can realize the dynamic adjustment and optimization of the adaptive optics system through a multiple iteration feedback mechanism. After each optimization, the system will re-collect imaging data and further adjust the optical settings through the machine learning algorithm until the expected imaging effect is achieved.

[0126] Specifically, the broadened light signal that enters the photoacoustic adaptive optics module 15 again is successively reflected by the first dichroic mirror, focused by the second lens, first reflected by the right-angle reflector, first transmitted by the achromatic focusing lens, and phase-adjusted by the spatial light modulator to generate a phase-adjusted broadened light signal.

[0127] The test optical signal that re - enters the photoacoustic adaptive optical module 15 sequentially passes through the transmission of the first dichroic mirror, the focusing of the second lens, the first reflection of the right - angle reflector, the first transmission of the achromatic focusing lens, and the phase adjustment of the spatial light modulator, generating a phase - adjusted test optical signal;

[0128] The phase - adjusted broadened optical signal and the phase - adjusted test optical signal sequentially pass through the second transmission of the achromatic focusing lens, the second reflection of the right - angle reflector, the focusing of the third lens, and the reflection of the second reflector, respectively forming an optimized incident broadened optical signal and an optimized incident test optical signal.

[0129] Step 250, the phase - adjusted broadened optical signal and test optical signal sequentially pass through the second optical path conduction module 16, the beam scanning module 17, and the third optical path conduction module 18 and enter the sample excitation module 19, re - exciting the sample to be measured, respectively generating optimized multi - photon signals, multiple harmonic signals, and photoacoustic signals;

[0130] Specifically, in this step, the phase - adjusted broadened optical signal and test optical signal in the photoacoustic adaptive optical module 15, that is, the optimized incident broadened optical signal and the optimized incident test optical signal, repeat the process of (4) - (6) in step S1. The phase - adjusted broadened optical signal and test optical signal sequentially pass through the second optical path conduction module 16, the beam scanning module 17, and the third optical path conduction module 18 and enter the sample excitation module 19, re - exciting the sample to be measured, respectively generating optimized multi - photon signals, multiple harmonic signals, and photoacoustic signals;

[0131] The photoacoustic signal acquisition module 20 continues to collect the optimized photoacoustic signal and converts it into an optimized photoacoustic electrical signal and sends it to the photoacoustic image calculation module 21. The photoacoustic image calculation unit 211 continues to pre - process the optimized photoacoustic electrical signal to form a new optimized photoacoustic microscopic image. The adaptive signal processing unit 212 then performs phase inversion calculation on the new optimized photoacoustic microscopic image, generates a photoacoustic phase adjustment signal, and sends the photoacoustic phase adjustment signal to the photoacoustic adaptive optical module 15, so as to cycle and iterate for optimization

[0132] Step 260, output the optimized fluorescence image and harmonic image;

[0133] Specifically: The detection module 19 obtains the optimized multi - photon signal. After detection, the optimized multi - photon signal is converted into a multi - photon electrical signal for an external display device to process the multi - photon electrical signal and output the optimized fluorescence image;

[0134] The detection module 19 acquires the optimized multiple harmonic signals. After detection, the optimized multiple harmonic signals are converted into multiple harmonic imaging electrical signals for an external display device (not shown in the figure) to process the multiple harmonic imaging electrical signals and output an optimized harmonic image.

[0135] The photoacoustic signal acquisition module 20 collects the optimized photoacoustic signals and sends them to the photoacoustic image calculation module 21. The optimized photoacoustic image formed after preprocessing of the photoacoustic signals can be stored and backed up in the photoacoustic image calculation module as the signal source for the phase retrieval technique, thereby performing adaptive control. When the image result of this photoacoustic modality needs to be called, the photoacoustic image information can be extracted from the photoacoustic image calculation module 21 and transmitted to an external display device (not shown in the figure) to output an optimized photoacoustic microscopic image.

[0136] In the present invention, the photoacoustic image calculation module 21 is composed of a photoacoustic image calculation unit 211 and an adaptive signal processing unit 212. The photoacoustic image calculation unit acquires the photoacoustic signals from the photoacoustic signal acquisition module 20 in the multimodal imaging system, converts them into image data, and performs certain preprocessing. The preprocessing process includes signal denoising, dynamic range adjustment, spectral filtering, etc. to ensure signal quality and processing accuracy. This process can effectively remove background noise and instrument errors and improve the reliability of subsequent calculation processes. The photoacoustic image acquisition module obtains the photoacoustic signals in the sample through a photoacoustic microscope. The photoacoustic image calculation unit processes these signals to generate corresponding photoacoustic images (ultrasound images). These are input into a machine learning algorithm for further analysis. Specifically, the machine learning models used may include deep convolutional neural networks (CNNs) or other image processing and recognition algorithms. The model learns from a large amount of labeled data to extract the feature information in the images. The algorithm analyzes the spatial distribution and signal intensity of the images, identifies the low-resolution regions therein, infers the optical phase changes that need to be adjusted, and transmits this information to the adaptive signal processing unit.

[0137] The effects of the adaptive optical technology adopted in the present invention on three-photon and third-harmonic are as follows:

[0138] For the resolution of the three-photon imaging signal and the third-harmonic imaging signal, it is positively correlated with the number of photons emitted by the sample. Without photobleaching, the number of photons emitted by the sample is proportional to its absorbed photon number: where N abs is the number of photons absorbed by the sample, C is the sample concentration, σ is the sample absorption cross-section, I is the laser intensity, and the last term S integral is the three-dimensional intensity spatial distribution at the laser focus.

[0139] The adaptive optics adopted in the present invention affects the S integral term By using adaptive optics to adjust the laser wavefront phase, the scattering caused by the system can be reduced, thereby improving the point spread function (PSF) of the system, focusing more laser energy at the intersection, increasing the S integral, and thus improving signal quality.

[0140] In order to better understand the technical solution provided by the present invention, the following specific examples are used to illustrate the system for executing the method of optimizing multimodal imaging based on photoacoustic signal-assisted adaptive optics of the present invention.

[0141] Example 1

[0142] The embodiment of the present invention provides a system 2 for performing a method for optimizing multimodal imaging based on photoacoustic signal-assisted adaptive optics, such as Figure 3 The structural diagram is shown in the figure. The modules and components of system 2 are introduced as follows.

[0143] System 2 includes: a laser module A, a laser broadening module B, an optical coherence imaging module C, a first optical path conduction module D, a photoacoustic adaptive optics module E, a second optical path conduction module F, a beam scanning module G, a third optical path conduction module H, a sample excitation module M, a photoacoustic signal acquisition module N, a photoacoustic image calculation module S and a detection module T.

[0144] The laser module A includes: a chirped amplified laser assembly (not shown) and an optical parametric amplifier laser (OPA); the chirped amplified laser assembly includes: a chirped amplified laser (CPA), a first half-wave plate 30, a first polarization beam splitter prism 31 and a first reflector 32. The wavelength of the first laser is 800nm ​​to 1064nm; the wavelength of the second laser of the chirped amplified laser signal is 1300nm to 1700nm.

[0145] The laser broadening module B uses different optical technologies to extend the time width of the laser pulse, thereby making its spectrum distribution wider. The broadening treatment methods of the present invention include: grating dispersion pulse broadening or material dispersion pulse broadening; when grating dispersion pulse broadening is used, the laser broadening module B includes but is not limited to: Martinez stretcher, Oefler The stretcher can be any one of a quartz stretcher, a Treacy stretcher, and a Martinez stretcher based on a 4F system; when material dispersion pulse stretching is adopted, the stretching medium of the laser stretching module 12 includes: a quartz material or a zinc selenide material.

[0146] The optical coherence imaging module C includes: a first beam splitting prism 33, a plurality of plane mirrors 34, a retroreflecting prism 35, a second half-wave plate 36, a second polarization beam splitting prism 37, a quarter-wave plate 38, a silicon window plate 39, a beam combining prism 40, a first lens 41, an optical fiber 42, and a CCD spectrometer 43; one end of the optical fiber 42 is connected to the first lens 41, and the other end of the optical fiber 42 is connected to the CCD spectrometer 43.

[0147] The first optical path conduction module D includes: a first dichroic mirror 44.

[0148] The photoacoustic adaptive optical module E includes: a second lens 45, a right-angle mirror 46, an achromatic focusing lens 47, a deformable mirror 48, and a third lens 49.

[0149] The second optical path conduction module F includes: a second mirror 50.

[0150] The beam scanning module G includes: a third mirror 51, a scanning device 52, and a galvanometer 53; wherein, the scanning device 52 uses a polygonal scanning mirror.

[0151] The third optical path conduction module H includes: a fourth lens 54, a fourth mirror 55, a fifth lens 56, and a fifth mirror 57.

[0152] The sample excitation module M includes: a second dichroic mirror 58, an objective lens 59, an ultrasonic transducer 60, and a water tank 61; the second dichroic mirror 58 is placed above the objective lens 59; the ultrasonic transducer 60 and the water tank 61 are integrated on the objective lens; the ultrasonic transducer 60 is annular and surrounds the side surface of the objective lens 59; the lower surface of the objective lens 59 is immersed in the upper part of the water tank 61; the sample to be measured is placed in the lower part of the water tank 61. In this embodiment, the ultrasonic transducer 59 uses a 256-array piezoelectric transducer; the medium carried in the water tank 61 is water.

[0153] The detection module T includes: a sixth lens 62, a third dichroic mirror 63, a first photomultiplier tube 64, and a second photomultiplier tube 65.

[0154] The photoacoustic image calculation module S includes: a photoacoustic image calculation unit 66 and an adaptive signal processing unit 67.

[0155] An embodiment of the present invention provides an application system 2 to execute a method for multi-modal imaging based on photoacoustic signal-assisted adaptive optical optimization, combined with Figure 3 The specific steps of this method are introduced as follows.

[0156] Step 310, start the system 2, and the emitted laser excites the sample to be measured, generating an interference light signal, a photoacoustic signal, an optical coherence imaging signal, a multi-photon signal, and a multiple harmonic signal;

[0157] Specifically, it includes:

[0158] (1) The laser module A emits the first laser beam and the second laser beam, specifically including: the laser beam emitted by the chirped pulse amplification (CPA) laser, which successively undergoes the phase processing by the first half-wave plate 30 and the beam splitting processing by the first polarization beam splitter prism 31 to generate the first laser beam and the laser signal to be amplified; the first laser beam enters the laser broadening module B after being reflected by the first mirror 32; the optical parametric amplification (OPA) laser obtains the laser signal to be amplified, undergoes chirped pulse amplification processing, generates the second laser beam, and sends it to the optical coherence imaging module C. In this embodiment, the parameters of the output laser of the CPA are: a laser beam with a central wavelength of 1030 nm, a full width at half maximum (FWHM) of 70 nm, and a pulse width of 200 fs; and then the laser beam emitted by the CPA is broadened by the laser broadening module B into a picosecond pulse laser with a central wavelength of 1030 nm and a pulse width of 20 ps.

[0159] (2) The laser broadening module B obtains the first laser beam and forms a broadened optical signal after broadening processing; among them, the broadening processing method in this embodiment uses a pulse stretcher based on material dispersion, and quartz is selected as the broadening material. As Figure 3 shown in the laser broadening module B, the incident first laser beam passes through the quartz material back and forth multiple times through multiple groups of mirrors, so as to achieve the effect of broadening using material dispersion.

[0160] (3) The optical coherence imaging module C obtains the second laser beam, undergoes beam splitting processing to generate a compensation optical signal and a test optical signal, and further reflects the compensation light multiple times to form a reference arm signal, specifically including: the first beam splitter prism 33 obtains the second laser beam and performs beam splitting processing to form a compensation optical signal and a test optical signal; multiple plane mirrors 34 obtain the compensation optical signal and form a first reflected optical signal after multiple reflection processes; the retroreflective prism 35 obtains the first reflected optical signal and forms a reference arm signal after reflection processing; among them, the beam energy ratio of the compensation optical signal to the test optical signal is 1:99.

[0161] (4) The broadened optical signal and the test optical signal successively pass through the first optical path conduction module D, the photoacoustic adaptive optical module E, and the second optical path conduction module F and enter the beam scanning module G, specifically including:

[0162] The broadened optical signal successively undergoes reflection by the first dichroic mirror 44, focusing by the second lens 45, the first reflection by the right-angle mirror 46, the first transmission by the achromatic focusing lens 47, and reflection by the deformable mirror 48 to generate a first parallel light; the first parallel light successively undergoes the second transmission by the achromatic focusing lens 47, the second reflection by the right-angle mirror 46, focusing by the third lens 49, and reflection by the second mirror 50, and then forms an incident broadened optical signal and enters the beam scanning module G;

[0163] The test optical signal sequentially passes through the transmission of the first dichroic mirror 44, the focusing of the second lens 45, the first reflection of the right-angle mirror 46, the first transmission of the achromatic focusing lens 47, and the reflection of the deformable mirror 48 to generate a second parallel light; the second parallel light sequentially passes through the second transmission of the achromatic focusing lens 47, the second reflection of the right-angle mirror 46, the focusing of the third lens 49, and the reflection of the second mirror 50 to form an incident test optical signal and enter the beam scanning module G;

[0164] The incident broadened optical signal and the incident test optical signal share the same optical path and are physically almost coincident.

[0165] (5) The third reflecting mirror 51 of the beam scanning module G adjusts the incident angles of the incident broadened optical signal and the incident test optical signal. The scanning device 52 performs scanning processing on the incident broadened optical signal and the incident test optical signal in the X-axis direction, and the galvanometer 53 performs scanning processing on the incident broadened optical signal and the incident test optical signal in the Y-axis direction to generate an output broadened optical signal and an output test optical signal; the X-axis is perpendicular to the Y-axis.

[0166] The output broadened optical signal and the output test optical signal pass through the third optical path conduction module H and enter the sample excitation module M. Specifically, the output broadened optical signal and the output test optical signal sequentially pass through the transmission of the fourth lens 54, the reflection of the fourth mirror 55, the transmission of the fifth lens 56, and the reflection of the fifth mirror 57 to enter the sample excitation module M.

[0167] (6) The sample excitation module M acquires the output broadened optical signal and the output test optical signal. After the output broadened optical signal and the output test optical signal pass through the second dichroic mirror 58 and enter the objective lens 59, they act on the sample to be measured;

[0168] The output broadened optical signal excites the sample to be measured to generate a photoacoustic signal, and the ultrasonic transducer 60 converts the photoacoustic signal into a photoacoustic electrical signal and sends it to the photoacoustic signal acquisition module N; the photoacoustic signal acquisition module N collects the photoacoustic electrical signal, generates a photoacoustic microscopy imaging electrical signal through analysis and processing, and sends the photoacoustic microscopy imaging electrical signal to the photoacoustic image calculation module S;

[0169] The output test optical signal excites the sample to be measured to generate an excitation optical signal. The second dichroic mirror 58 separates the excitation optical signal to form a sample excitation optical signal and an optical coherence imaging signal; the sample excitation optical signal enters the detection module T through the reflection of the second dichroic mirror 58; the optical coherence imaging signal enters the third optical path conduction module H through the transmission of the second dichroic mirror 58; the sixth lens 62 of the detection module T focuses the sample excitation optical signal and then separates it through the third dichroic mirror 63 to form a multi-photon signal and a multiple harmonic signal;

[0170] The optical coherence imaging signal sequentially passes through the third optical path conduction module H, the beam scanning module G, the second optical path conduction module F, the photoacoustic adaptive optical module E, and the first optical path conduction module D and then returns to enter the optical coherence imaging module C; the beam combining prism 40 in the optical coherence imaging module C performs beam combining interference processing on the reference arm signal and the optical coherence imaging signal to form an interference optical signal; the first lens 41 acquires the interference optical signal, performs focusing processing to form a focused imaging optical signal; the CCD spectrometer 43 acquires the focused imaging optical signal through the optical fiber 42, performs analysis processing to generate an optical coherence imaging electrical signal, and uses an external display device (not shown in the figure) to process the optical coherence imaging electrical signal and output an optical coherence imaging image;

[0171] The first photomultiplier tube 64 of the detection module T detects and analyzes the multi-photon signal, generates a multi-photon electrical signal, and sends the multi-photon electrical signal to an external display device (not shown in the figure) connected to the first photomultiplier tube 64, and uses the external display device to process the multi-photon electrical signal and output a fluorescence image.

[0172] The second photomultiplier tube 65 of the detection module T detects and analyzes the multiple harmonic signal, generates a multiple harmonic imaging electrical signal, and sends the multiple harmonic imaging electrical signal to an external display device (not shown in the figure) connected to the second photomultiplier tube 65, and uses the external display device to process the multiple harmonic imaging electrical signal and output a harmonic image.

[0173] Step 320, the photoacoustic signal acquisition module 20 acquires a photoacoustic electrical signal, converts the photoacoustic signal into a photoacoustic microscopy imaging electrical signal, and sends it to the photoacoustic image calculation module S.

[0174] Step 330, the photoacoustic image calculation unit 66 of the photoacoustic image calculation module S collects the photoacoustic microscopy imaging electrical signal, performs preprocessing such as signal denoising, dynamic range adjustment, and spectral filtering on the photoacoustic microscopy imaging electrical signal to obtain an optimized photoacoustic microscopy image, and transports the optimized photoacoustic microscopy image to the adaptive signal processing unit 67; the adaptive signal processing unit 67 uses a deep neural network to perform phase inversion calculation on the optimized photoacoustic microscopy image, identifies and analyzes the low-resolution regions in the photoacoustic microscopy image, generates a photoacoustic phase adjustment signal, and sends the photoacoustic phase adjustment signal to the photoacoustic adaptive optical module E to adjust the deformable mirror 48.

[0175] Step 340, the adjusted deformable mirror 48 performs phase adjustment on the broadened optical signal and the test optical signal that re-enter the photoacoustic adaptive optical module E;

[0176] Specifically, the broadened optical signal and the test optical signal that re-enter the photoacoustic adaptive optical module E are successively focused by the second lens 45, first reflected by the right-angle mirror 46, first transmitted by the achromatic focusing lens 47, and phase-adjusted by the deformable mirror 48, respectively generating a phase-adjusted broadened optical signal and a phase-adjusted test optical signal;

[0177] The phase-adjusted broadened optical signal and the phase-adjusted test optical signal successively pass through the second transmission of the achromatic focusing lens 47, the second reflection of the right-angle mirror 46, the focusing of the third lens 49, and the reflection of the second mirror 60, respectively forming an optimized incident broadened optical signal and an optimized incident test optical signal;

[0178] In this step, according to the adjustment information of the photoacoustic phase adjustment signal input, the photoacoustic adaptive optical module E can perform multiple iterative feedbacks to dynamically change the shape of the deformable mirror 48 in real time, thereby dynamically and precisely adjusting the optical performance in the system and improving the resolution of photoacoustic microscopy imaging.

[0179] Step 350, the broadened optical signal and the test optical signal that have undergone phase adjustment in the photoacoustic adaptive optical module E, that is, the optimized incident broadened optical signal and the optimized incident test optical signal, repeat the processes of (4) to (6) in step S1. The phase-adjusted broadened optical signal and the test optical signal successively pass through the second optical path conduction module F, the beam scanning module G, and the third optical path conduction module H and enter the sample excitation module M to re-excite the sample to be measured, respectively generating optimized multi-photon signals, multiple harmonic signals, and photoacoustic signals.

[0180] Step 360, output optimized fluorescence images and harmonic images;

[0181] Specifically: The specific process of generating optimized fluorescence images and harmonic images according to the optimized multi-photon signals and multiple harmonic signals in this step is the same as the imaging operation process in step 310(6) and will not be elaborated here; the optimized photoacoustic image can extract photoacoustic image information through the photoacoustic image calculation module S and transmit it to an external display device (not shown in the figure), thereby outputting the optimized photoacoustic image.

[0182] Those skilled in the art should also further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described according to their functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of the present invention.

[0183] The steps of the methods or algorithms described in connection with the embodiments disclosed herein may be implemented by hardware, software modules executed by a processor, or a combination of both. The software modules may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium well known in the art.

[0184] The specific embodiments described above further elaborate on the objectives, technical solutions, and beneficial effects of the present invention. It should be understood that the above description is only for the specific embodiments of the present invention and is not intended to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for optimizing multimodal imaging based on photoacoustic signal-assisted adaptive optics, characterized in that: The system for executing the method comprises: a laser module, a laser broadening module, an optical coherence imaging module, a first light path transmission module, a photoacoustic adaptive optics module, a second light path transmission module, a light beam scanning module, a third light path transmission module, a sample excitation module, a photoacoustic signal acquisition module, a photoacoustic image calculation module and a detection module; the photoacoustic adaptive optics module comprises a spatial light modulator; The method comprises: Step S1, start the system, the laser module emits a first laser and a second laser, the laser broadening module converts the first laser into a broadened light signal; the optical coherence imaging module splits the second laser into a compensation light signal and a test light signal, and the compensation light signal forms a reference arm signal after multiple reflections; the broadened light signal and the test light signal sequentially pass through the first optical path conduction module, the photoacoustic adaptive optics module, the second optical path conduction module, the light beam scanning module, and the third optical path conduction module to enter the sample excitation module, excite the sample to be tested, and generate a photoacoustic signal, an optical coherence imaging signal, a multi-photon signal, and a multiple harmonic signal; the optical coherence imaging signal is used to form an optical coherence imaging image; the detection module detects the multi-photon signal and the multiple harmonic signal, and outputs them as a fluorescence image and a harmonic image respectively; Step S2, the photoacoustic signal acquisition module acquires the photoacoustic electrical signal, converts the photoacoustic electrical signal into a photoacoustic microscopy imaging electrical signal and sends it to the photoacoustic image calculation module; Step S3, the photoacoustic image calculation module collects the photoacoustic microscopy imaging electrical signal, generates an optimized photoacoustic microscopy image after preprocessing, performs phase inversion calculation on the optimized photoacoustic microscopy image using a machine learning algorithm, and generates a photoacoustic phase adjustment signal for adjusting the spatial light modulator; Step S4, the adjusted spatial light modulator performs phase adjustment on the stretching light signal and the test light signal that re-enter the photoacoustic adaptive optics module; Step S5, the phase-adjusted broadened light signal and the test light signal pass through the second light path conduction module, the light beam scanning module, and the third light path conduction module in sequence and enter the sample excitation module, and excite the sample to be tested again, and generate optimized multi-photon signals, multiple harmonic signals, and photoacoustic signals, respectively, and then output optimized fluorescence images and optimized harmonic images.

2. The method according to claim 1, characterized in that The step S1 specifically includes: Start the system, and the laser module emits a first laser and a second laser; The laser broadening module acquires the first laser beam and forms a broadened optical signal through broadening processing; The optical coherence imaging module acquires the second laser beam, generates a compensation light signal and a test light signal through beam splitting processing, and further reflects the compensation light multiple times to form a reference arm signal; The stretched optical signal and the test optical signal sequentially pass through the first optical path transmission module, the photoacoustic adaptive optics module, the second optical path transmission module, the light beam scanning module, and the third optical path transmission module to form an outgoing stretched optical signal and an outgoing test optical signal, respectively, and enter the sample excitation module; The emitted broadened light signal excites the sample to be tested in the sample excitation module to generate a photoacoustic signal, and the photoacoustic signal is converted into a photoacoustic electrical signal for collection by the photoacoustic signal collection module; The emitted test light signal excites the sample to be tested to generate an excitation light signal; The excitation light signal is separated to generate a sample excitation light signal and an optical coherence imaging signal; The optical coherence imaging signal sequentially passes through the third optical path transmission module, the light beam scanning module, the second optical path transmission module, the photoacoustic adaptive optics module, and the first optical path transmission module, and then returns to the optical coherence imaging module, and performs beam combining interference processing with the reference arm signal to form an interference light signal, and the interference light signal is converted into an optical coherence imaging electrical signal to output an optical coherence imaging image; The detection module acquires the sample excitation light signal, forms a multi-photon signal and a multiple harmonic signal through focusing and beam splitting processing, and converts the multi-photon signal and the multiple harmonic signal into a multi-photon electrical signal and a multiple harmonic imaging electrical signal respectively, so as to output a fluorescence image and a harmonic image.

3. The method according to claim 2, characterized in that The laser light source module comprises: a chirped amplifying laser, a first half-wave plate, a first polarizing beam splitter prism, a first reflecting mirror and an optical parametric amplifying laser; The laser module emits a first laser and a second laser, specifically comprising: the laser emitted by the chirped amplified laser is sequentially subjected to phase processing by the first half-wave plate and beam splitting processing by the first polarization beam splitter prism to generate a first laser and a laser signal to be amplified; the first laser enters the laser broadening module after being reflected by the first reflector; the optical parametric amplifier laser acquires the laser signal to be amplified, undergoes chirped amplification processing, generates a second laser, and sends it to the optical coherent imaging module; The stretching process includes: grating dispersion pulse stretching process or material dispersion pulse stretching process; wherein, when the grating dispersion pulse stretching process is adopted, the laser stretching module includes: any one of a Martinez stretcher, an Oeffler stretcher, a Trichy stretcher, and a Martinez stretcher based on a 4F system; when material dispersion pulse stretching is adopted, the stretching medium of the laser stretching module includes: quartz material or zinc selenide material.

4. The method according to claim 2, characterized in that: The optical coherence imaging module comprises: a first beam splitter prism, an optical path compensation component, a second half-wave plate, a second polarization beam splitter prism, a quarter-wave plate, a silicon window plate, a beam combining prism and an imaging component; wherein the optical path compensation component comprises: a retroreflective prism and a plurality of plane reflectors; the retroreflective prism moves according to the position of the sample to be measured; the imaging component comprises: a first lens, an optical fiber and a spectrometer; one end of the optical fiber is connected to the first lens, and the other end of the optical fiber is connected to the spectrometer; The first beam splitter prism acquires the second laser beam and performs beam splitting processing to form a compensation light signal and a test light signal; wherein the beam energy ratio of the compensation light signal to the test light signal is 1:99; The multiple plane reflectors acquire the compensation light signal, and after multiple reflections, form a first reflected light signal; the retroreflective prism acquires the first reflected light signal, and after reflections, forms a reference arm signal; The test optical signal passes through the second half-wave plate, the second polarization beam splitter, and the quarter-wave plate in sequence, and then enters the first optical path transmission module after being reflected by the silicon window; the silicon window is placed according to the Brewster angle to perform dispersion compensation on the second amplified optical signal; The beam combining prism performs beam combining interference processing on the reference arm signal and the optical coherent imaging signal to form an interference light signal; The first lens acquires the interference light signal and forms a focused imaging light signal after focusing processing; The spectrometer acquires the focused imaging light signal through the optical fiber, and generates an optical coherence imaging electrical signal after analysis and processing, so that an external display device processes the optical coherence imaging electrical signal and outputs an optical coherence imaging image.

5. The method according to claim 1 or 2, characterized in that: The first light path transmission module comprises: a first dichroic mirror; The photoacoustic adaptive optics module further includes: a second lens, a right-angle reflector, an achromatic focusing mirror and a third lens; the spatial light modulator includes: any one of a deformable mirror, a digital micromirror array or a liquid crystal spatial light modulator; The second light path conducting module comprises: a second reflector; The broadening optical signal and the test optical signal sequentially pass through the first optical path transmission module, the photoacoustic adaptive optics module, and the second optical path transmission module and enter the optical beam scanning module, specifically including: The stretched light signal is sequentially reflected by the first dichroic mirror, focused by the second lens, first reflected by the right-angle reflector, first transmitted by the achromatic focusing mirror, and reflected by the spatial light modulator to generate a first parallel light; the first parallel light is sequentially transmitted by the achromatic focusing mirror for a second time, reflected by the right-angle reflector for a second time, focused by the third lens, and reflected by the second reflector to form an incident stretched light signal that enters the light beam scanning module; The test light signal is sequentially transmitted through the first dichroic mirror, focused through the second lens, reflected through the right-angle mirror for the first time, transmitted through the achromatic focusing mirror for the first time, and reflected through the spatial light modulator to generate a second parallel light; the second parallel light is sequentially transmitted through the achromatic focusing mirror for the second time, reflected through the right-angle mirror for the second time, focused through the third lens, and reflected through the second reflecting mirror to form an incident test light signal that enters the light beam scanning module; In the step S4, phase adjustment is performed on the stretched optical signal and the test optical signal which re-enter the photoacoustic adaptive optics module, specifically including: The stretched light signal that enters the photoacoustic adaptive optics module again is sequentially focused by the second lens, reflected by the right-angle reflector for the first time, transmitted by the achromatic focusing mirror for the first time, and phase-adjusted by the spatial light modulator to generate a phase-adjusted stretched light signal; the phase-adjusted stretched light signal is sequentially transmitted by the achromatic focusing mirror for the second time, reflected by the right-angle reflector for the second time, focused by the third lens, and reflected by the second reflector to form an optimized incident stretched light signal; The test light signal that enters the photoacoustic adaptive optics module again is sequentially focused by the second lens, reflected by the right-angle mirror for the first time, transmitted by the achromatic focusing mirror for the first time, and phase-adjusted by the spatial light modulator to generate a phase-adjusted test light signal; the phase-adjusted test light signal is sequentially transmitted by the achromatic focusing mirror for the second time, reflected by the right-angle mirror for the second time, focused by the third lens, and reflected by the second mirror to generate an optimized incident test light signal.

6. The method according to claim 5, characterized in that The beam scanning module comprises: a third reflector, a scanning device and a galvanometer; The third light path transmission module comprises: a fourth lens, a fourth reflector, a fifth lens and a fifth reflector; The third reflector adjusts the incident angles of the incident stretched optical signal and the incident test optical signal, or adjusts the incident angles of the optimized incident stretched optical signal and the optimized incident test optical signal; The scanning device performs scanning processing on the incident stretched light signal and the incident test light signal, or the optimized incident stretched light signal and the optimized incident test light signal in the X-axis direction, and the galvanometer performs scanning processing on the incident stretched light signal and the incident test light signal, or the optimized incident stretched light signal and the optimized incident test light signal in the Y-axis direction to generate an outgoing stretched light signal and an outgoing test light signal, or generate an optimized outgoing stretched light signal and an optimized outgoing test light signal; the X-axis is perpendicular to the Y-axis; The outgoing broadened light signal and the outgoing test light signal, or the optimized outgoing broadened light signal and the optimized outgoing test light signal, enter the sample excitation module after being transmitted by the fourth lens, reflected by the third reflector, transmitted by the fifth lens and reflected by the fourth reflector.

7. The method according to claim 2, characterized in that: The sample excitation module comprises: a second dichroic mirror, an objective lens, an ultrasonic transducer and a water tank; wherein the second dichroic mirror is placed above the objective lens; the ultrasonic transducer and the water tank are integrated on the objective lens; the ultrasonic transducer is annular and placed on the lower surface of the objective lens or surrounds the side surface of the objective lens; the lower surface or the side surface of the objective lens is immersed in the upper part of the water tank together with the ultrasonic transducer; the sample to be tested is placed in the lower part of the water tank; After the outgoing broadening light signal and the outgoing test light signal pass through the second dichroic mirror and enter the objective lens, they act on the sample to be tested; The emitted broadened light signal excites the sample to be tested to generate a photoacoustic signal, and the photoacoustic signal is converted into a photoacoustic electrical signal through the ultrasonic transducer and sent to the photoacoustic signal acquisition module; the photoacoustic signal acquisition module collects the photoacoustic electrical signal, generates a photoacoustic microscopic imaging electrical signal after analysis and processing, and sends the photoacoustic microscopic imaging electrical signal to the photoacoustic image calculation module; The outgoing test light signal excites the sample to be tested to generate an excitation light signal, and the second dichroic mirror separates the excitation light signal to form a sample excitation light signal and an optical coherent imaging signal, so that the sample excitation light signal is reflected into the detection module, and the optical coherent imaging signal is transmitted into the third optical path conduction module.

8. The method according to claim 2, characterized in that: The detection module includes: a sixth lens, a third dichroic mirror, a fluorescence detection device and a harmonic detection device; wherein the fluorescence detection device includes a first photomultiplier tube; the harmonic detection device includes a second photomultiplier tube; In the step S1, the sample excitation light signal is separated to generate a multi-photon signal and a multiple harmonic signal, specifically comprising: the sample excitation light signal is sequentially subjected to focusing processing by the sixth lens and beam splitting processing by the third dichroic mirror to form a multi-photon signal and a multiple harmonic signal; The first photomultiplier tube detects and analyzes the multiphoton signal to generate a multiphoton electrical signal, and sends the multiphoton electrical signal to an external display device connected to the first photomultiplier tube, so that the external display device processes the multiphoton electrical signal and outputs a fluorescent image; The second photomultiplier tube detects and analyzes the multiple harmonic signals to generate multiple harmonic imaging electrical signals, and sends the multiple harmonic imaging electrical signals to an external display device connected to the second photomultiplier tube, so that the external display device processes the multiple harmonic imaging electrical signals and outputs a harmonic image.

9. The method according to claim 1, characterized in that: The photoacoustic image calculation module includes: a photoacoustic image calculation unit and an adaptive signal processing unit; The step S3 specifically includes: the photoacoustic image calculation unit collects the photoacoustic microscopy imaging electrical signal, pre-processes the photoacoustic microscopy imaging electrical signal, generates an optimized photoacoustic microscopy image, and transmits the optimized photoacoustic microscopy image to the adaptive signal processing unit; the adaptive signal processing unit uses a machine learning algorithm to perform phase inversion calculation on the photoacoustic microscopy image, identifies and analyzes the low-resolution area in the optimized photoacoustic microscopy image, generates a photoacoustic phase adjustment signal, and sends the photoacoustic phase adjustment signal to the photoacoustic adaptive optics module; Wherein, the preprocessing includes: the photoacoustic image calculation unit performs one or more of signal denoising, dynamic range adjustment, and spectrum filtering on the photoacoustic electrical signal; The machine learning algorithm includes: a deep neural network; In the step S5, outputting the optimized fluorescence image and harmonic wave image specifically includes: The detection module acquires the optimized multi-photon signal, and after detection, converts the optimized multi-photon signal into a multi-photon electrical signal, so that an external display device processes the multi-photon electrical signal and outputs the optimized fluorescence image; The detection module acquires the optimized multiple harmonic signals, and after detection, converts the optimized multiple harmonic signals into multiple harmonic imaging electrical signals, so that an external display device processes the multiple harmonic imaging electrical signals and outputs the optimized harmonic images.

10. A system for executing the method for optimizing multimodal imaging based on photoacoustic signal-assisted adaptive optics as described in any one of claims 1 to 9, characterized in that: The system comprises: a laser module, a laser broadening module, an optical coherence imaging module, a first light path transmission module, a photoacoustic adaptive optics module, a second light path transmission module, a light beam scanning module, a third light path transmission module, a sample excitation module, a photoacoustic signal acquisition module, a photoacoustic image calculation module and a detection module; the photoacoustic adaptive optics module comprises a spatial light modulator; The laser module emits a first laser beam and a second laser beam; The laser stretching module performs stretching processing on the first laser to form a stretched optical signal; The optical coherence imaging module performs beam splitting processing on the second laser to generate a compensation light signal and a test light signal, and further performs multiple reflections on the compensation light to form a reference arm signal; The stretched optical signal and the test optical signal sequentially pass through the first optical path transmission module, the photoacoustic adaptive optics module, the second optical path transmission module, the light beam scanning module, and the third optical path transmission module to generate an outgoing stretched optical signal and an outgoing test optical signal, respectively, and enter the sample excitation module; The emitted test light signal excites the sample to be tested to generate an excitation light signal; the sample excitation module separates the excitation light signal to generate a sample excitation light signal and an optical coherent imaging signal; The detection module separates the sample excitation light signal to form a multi-photon signal and a multiple harmonic signal, detects the fluorescence signal to generate a multi-photon imaging electrical signal, and then sends the multi-photon imaging electrical signal to an external display device connected to the detection module, and detects the harmonic signal to generate a multiple harmonic imaging electrical signal, and sends the multiple harmonic imaging electrical signal to an external display device connected to the detection module; The optical coherence imaging module performs beam combining interference processing on the optical coherence imaging signal and the reference arm signal to generate an interference light signal, then performs photoelectric conversion processing on the interference light signal to generate an optical coherence imaging electrical signal, and sends the optical coherence imaging electrical signal to an external display device connected to the optical coherence imaging module; The emitted broadened light signal excites the sample to be tested in the sample excitation module to generate a photoacoustic signal; the photoacoustic signal acquisition module acquires the photoacoustic signal, and converts the photoacoustic signal into a photoacoustic electrical signal and sends it to the photoacoustic image calculation module; The photoacoustic image calculation module preprocesses the photoacoustic electrical signal to generate a photoacoustic microscopic image, generates a photoacoustic phase adjustment signal through phase inversion calculation, and sends the photoacoustic phase adjustment signal to the photoacoustic adaptive optics module, adjusts the spatial light modulator, and performs phase adjustment on the broadened light signal and the test light signal that re-enter the photoacoustic adaptive optics module.

Images