Photoacoustic endoscopic imaging method and system based on composite scanning to realize optical interpolation

Abstract

The application provides a photoacoustic endoscopy imaging method and system based on composite scanning for optical interpolation, and relates to the technical field of photoacoustic endoscopy imaging. The method comprises the following steps: under the composite scanning mechanism, when a multimode optical fiber is moved to each preset position of mechanical scanning, M*N array independent focal points are generated according to a pre-stored array focal point binary image corresponding to the preset position, and photoacoustic signals corresponding to a plurality of sub-pixel sampling points are collected in parallel through the array independent focal points at the preset position; a global coordinate grid is established according to preset track coordinates of the mechanical scanning, the spatial positions of the sampling points are determined in combination with a fixed offset of the sub-pixel sampling points, after the multi-channel photoacoustic signals are time-domain aligned, amplitude corrected and preprocessed, a light absorption distribution image is reconstructed, and finally a high-resolution photoacoustic endoscopy image is generated. The application realizes efficient and high-resolution scanning of the global imaging area.

Description

Technical Field

[0001] This invention relates to the field of photoacoustic endoscopic imaging technology, and in particular to a photoacoustic endoscopic imaging method and system based on composite scanning to achieve optical interpolation. Background Technology

[0002] Existing photoacoustic endoscopic imaging scanning technologies are mainly divided into two categories: mechanical scanning and optical scanning. Mechanical scanning technology typically uses driving elements such as miniature motors or piezoelectric ceramics to physically move optical fibers or probes, achieving point-by-point scanning of the area to be examined through a preset trajectory. The advantage of this technology is its large scanning range, covering a wide imaging field of view, suitable for preliminary exploration of larger tissue areas. However, mechanical scanning is limited by the mechanical precision and movement speed of the driving elements, making it difficult to further reduce the scanning step size. This results in a large sampling point spacing and limited pixel precision, making it difficult to achieve high-resolution imaging of small lesions or delicate tissue structures. In addition, in pure mechanical scanning, reducing the scanning step size to achieve high-resolution imaging would significantly prolong the scanning time, making it difficult to meet the imaging speed requirements of clinical endoscopic environments.

[0003] Optical scanning technology achieves non-mechanical focus deflection or switching by manipulating the incident light field. Typical schemes include using optical elements such as galvanometers, spatial light modulators, or digital micromirrors to manipulate the beam. Among these, the light field manipulation method based on multimode fiber and wavefront shaping technology has received widespread attention in recent years. By suppressing mode crosstalk in multimode fiber, it can generate a stable and controllable array of independent focusing points at the fiber output end, achieving non-mechanical fine scanning. This technology can achieve sub-micron level focus positioning accuracy, high imaging resolution, and fast scanning speed. However, the scanning range achieved solely by light field manipulation is limited by the numerical aperture of the multimode fiber and the field of view of the imaging objective, usually making it difficult to cover a large imaging area, thus facing limitations in large-field-of-view imaging requirements. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a photoacoustic endoscopic imaging method and system based on composite scanning to achieve optical interpolation, which realizes efficient and high-resolution scanning of the entire imaging area.

[0005] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0006] A first aspect is a photoacoustic endoscopic imaging method based on composite scanning to achieve optical interpolation, the method comprising:

[0007] Step 1 involves complex amplitude modulation of the light field incident on the multimode fiber to suppress mode crosstalk and generate a stable and controllable array of M×N independent focusing points at the output end of the multimode fiber to achieve non-mechanical fine scanning. The digital micromirror device pre-stores an array of binary focus maps corresponding one-to-one with the movement positions of the multimode fiber during mechanical scanning, and the linkage between light field modulation and mechanical scanning is realized based on the binary focus maps.

[0008] Step 2: Based on the completion of non-mechanical fine scanning, the multimode optical fiber is driven by a driving module composed of XY two-dimensional micro piezoelectric ceramics to perform a large-scale mechanical scanning along a preset trajectory, so that the multimode optical fiber switches between different preset positions, forming a composite scanning mechanism that combines non-mechanical scanning and mechanical scanning.

[0009] Step 3: Under the composite scanning mechanism, when the multimode fiber is moved to each preset position of the mechanical scan, M×N array-type independent focal points are generated according to the pre-stored array-type focal binary map corresponding to the preset position, and the photoacoustic signals corresponding to multiple sub-pixel sampling points are collected in parallel at the preset position through the array-type independent focal points.

[0010] Step 4: Establish a global coordinate grid based on the preset trajectory coordinates of the mechanical scan, determine the spatial position of each sampling point by combining the fixed offset of the sub-pixel sampling points, and reconstruct the light absorption distribution image after performing temporal alignment, amplitude correction and preprocessing on the multi-channel photoacoustic signals, and finally generate a high-resolution photoacoustic endoscopic image.

[0011] Secondly, a photoacoustic endoscopic imaging system based on composite scanning to achieve optical interpolation, the system implementing the method includes: a field control module, a driving module and an image reconstruction module;

[0012] The field control module regulates the light field incident on the multimode fiber, generating a stable and controllable M×N array of independent focal points at the end of the multimode fiber; the driving module drives the multimode fiber to perform a large-area scan along a preset trajectory, constructing a global coordinate grid for the imaging area and achieving large-area coverage; the image reconstruction module collects and processes the photoacoustic signals excited by each array focal point, and combines the scanning trajectory of the multimode fiber with the sub-pixel sampling data obtained by optical interpolation to complete the photoacoustic endoscopic image reconstruction.

[0013] Thirdly, a computing device, comprising:

[0014] One or more processors;

[0015] A storage device for storing one or more programs that, when executed by one or more processors, cause the one or more processors to perform the above-described method.

[0016] Fourthly, a computer-readable storage medium storing a program that, when executed by a processor, implements the above-described method.

[0017] The above-described solution of the present invention has at least the following beneficial effects:

[0018] This invention achieves synergistic optimization of wide field-of-view coverage and high pixel accuracy by constructing a composite scanning mechanism that combines non-mechanical scanning with mechanical scanning. Specifically, a multimode fiber optical field control system with digital micromirror devices at its core generates M×N arrayed independent focusing points to achieve non-mechanical fine scanning, obtaining sub-micron level focus positioning accuracy. Building upon this, an XY two-dimensional micro piezoelectric ceramic drives the multimode fiber to perform a large-scale mechanical scan along a preset trajectory, allowing the multimode fiber to switch between different preset positions. This organically combines the local high precision of non-mechanical fine scanning with the wide-area coverage capability of mechanical scanning, overcoming the inherent defects of insufficient pixel accuracy in pure mechanical scanning and the limited field of view in pure optical scanning, thus achieving efficient and high-resolution coverage of the entire imaging area.

[0019] This invention achieves precise linkage between optical field control and mechanical scanning by pre-storing array-type focal binary maps that correspond one-to-one with the mechanical scanning positions. When the drive module moves the multimode fiber to each preset position of the mechanical scanning, the digital micromirror device can instantly call up the pre-storing array-type focal binary map corresponding to that position to generate M×N array-type independent focal points. This ensures that the position of each focal point is precisely matched with the mechanical coordinates during the mechanical scanning process, providing a reliable spatial position reference for subsequent optical interpolation and image reconstruction, and significantly improving the scanning accuracy and imaging quality of the system.

[0020] This invention achieves sub-pixel-level sampling through optical interpolation technology. At each preset position of the mechanical scan, photoacoustic signals corresponding to multiple sub-pixel sampling points are acquired in parallel using an array of M×N independent focal points. Each sub-pixel sampling point has a fixed offset relative to the center coordinates of the preset position of the mechanical scan, which is equivalent to inserting multiple fine sampling points between adjacent sampling points of the mechanical scan, significantly improving the spatial sampling density. Compared with simply relying on reducing the mechanical scan step size to improve resolution, this invention achieves a higher effective sampling rate without increasing the mechanical scan time, balancing imaging speed and resolution. Attached Figure Description

[0021] Figure 1 A schematic diagram of the optical path structure of the field control module of the photoacoustic endoscopic imaging system based on composite scanning to achieve optical interpolation, provided for an embodiment of the present invention.

[0022] Figure 2The flowchart illustrates the field control module of a photoacoustic endoscopic imaging system based on composite scanning for optical interpolation, as provided in an embodiment of the present invention.

[0023] Figure 3 This is a schematic diagram of a non-mechanically scanned DMD projection binary pattern in an embodiment of the present invention (taking a 2×2 focal point as an example).

[0024] Figure 4 This is a schematic diagram of a front-view example structure of a probe containing a transparent photoacoustic transducer array in an embodiment of the present invention (taking 2×2 transducers as an example).

[0025] Figure 5 This is a schematic diagram of a probe with a transparent photoacoustic transducer array in an embodiment of the present invention (taking 2×2 transducers as an example).

[0026] Figure 6 A schematic diagram of a photoacoustic endoscopic imaging system based on composite scanning to achieve optical interpolation, provided for an embodiment of the present invention (taking non-mechanical scanning to generate a 1×4 array focus as an example).

[0027] Figure 7 A schematic diagram of a photoacoustic endoscopic imaging system based on composite scanning to achieve optical interpolation, provided for an embodiment of the present invention (taking non-mechanical scanning to generate a 2×2 array focus as an example).

[0028] Explanation of reference numerals: 1-Pulsed laser, 2-First optical lens assembly, 3-Digital micromirror device, 4-Second optical lens assembly, 5-Aperture, 6-Third optical lens assembly, 7-Fiber optic coupler, 8-Multimode fiber, 9-Objective lens, 10-Reflector, 11-Fourth optical lens, 12-Camera, 13-Transparent photoacoustic transducer array, 14-Sample, 15-Controller, 16-Amplifier, 17-Filter, 18-Acquisition card, 19-Computer, 20-DMD binary detection map, 21-Multimode fiber output binary speckle map, 22-Probe housing, 23-Transparent 3D printed part, 24-XY two-dimensional micro piezoelectric ceramic, 25-First reserved slot, 26-Second reserved slot. Detailed Implementation

[0029] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0030] The photoacoustic endoscopic imaging system used in this embodiment includes a field control module, a driving module, an image reconstruction module, and an FPGA core controller. The hardware parameters and connection relationships of each module are as follows:

[0031] The field control module is a multimode fiber optical field control system centered on a digital micromirror device (DMD). Specifically, it includes: a 532nm wavelength pulsed laser (1kHz repetition rate, 10μJ single-pulse energy), a first optical lens assembly (two plano-convex lenses, beam expansion ratio 10:1), a 1920×1080 resolution DMD (modulation speed 40kHz), a 4f optical system (a second optical lens assembly, a circular low-pass aperture, and a third optical lens assembly coaxially arranged), a 0.22 numerical aperture multimode fiber coupler, a 105μm core diameter / 0.22 numerical aperture step-index multimode fiber, and a calibration unit consisting of an objective lens, a mirror, optical lenses, and a CCD camera. The drive module is an XY two-dimensional micro piezoelectric ceramic with a displacement range of ±2.5mm and a displacement resolution of 10nm, connected to the multimode fiber via a transparent 3D-printed component.

[0032] Image Reconstruction Module: A 2×2 array of transparent photoacoustic transducers (made of LiNbO3 piezoelectric material + ITO transparent electrodes, transmittance ≥85%, center frequency 15MHz, bandwidth 60%), a four-channel low-noise preamplifier (60dB gain), a four-channel bandpass filter (passband 1MHz-20MHz), a four-channel high-speed acquisition card (sampling rate 100MS / s, 12-bit resolution), and a computer with built-in image reconstruction algorithms. Integrated Probe: Multimode fiber, transparent photoacoustic transducer array, and XY two-dimensional micro piezoelectric ceramics are integrated and packaged in a cylindrical probe shell with a diameter of 3mm and a length of 15mm. It is compatible with medical endoscope clamp channels. The transducers are axially symmetrically distributed around the optical axis, with a 1mm diameter laser transmission channel reserved in the center. Each of the four transducers corresponds to one of the four focal points. The FPGA controller is electrically connected to the DMD, pulsed laser, drive module, and acquisition card to achieve synchronous trigger control throughout the entire process.

[0033] The complete implementation steps of the imaging method are as follows:

[0034] Step S100, Digital Micromirror Device Pre-calibration:

[0035] Before formal imaging, the transmission matrix of the DMD and multimode fiber is calibrated to ensure stable generation of the array focus point. This includes:

[0036] Superpixel encoding configuration: Each group of 4×4 pixels on the DMD is used as a superpixel unit, and the amplitude and phase of the incident light field are synchronously modulated through micromirror state encoding; K=1024 pre-generated random binary probe images are projected on the DMD and converted into probe light fields with random phase and amplitude distribution.

[0037] Speckle pattern acquisition: The pulsed laser emitted by the pulsed laser is modulated by the DMD of the projected binary probe image and coupled into the multimode fiber, forming a binary speckle pattern at the output end of the multimode fiber. The light intensity information of the speckle pattern is acquired by a CCD camera.

[0038] Transmission matrix reconstruction: Based on the collected light intensity information, the transmission matrix of the multimode fiber is reconstructed iteratively using the generalized Gerchberg-Saxton algorithm. During the iteration initialization phase, an initial estimate of the transmission matrix is ​​generated. During the iteration process, the estimated transmission matrix is ​​updated by inputting the pseudo-inverse matrix of the probe matrix. The iteration termination condition is: the correlation between adjacent iteration results is ≥99.9999%, or the number of iterations reaches the upper limit of 2000. The row elements that converge during the iteration are directly output to reduce the computational dimensionality.

[0039] Binary image generation and calibration: Based on the reconstructed transmission matrix, the transmission matrix row corresponding to the predetermined 2×2 position on the target plane is selected, and the corresponding conjugate wavefront is generated. After superposition, a 2×2 array-type focal binary modulation speckle map is generated through superpixel encoding and projected onto the DMD. Four high-contrast independent focal points are generated at the output end of the multimode fiber. The focal position is calibrated by feedback control to ensure that the focal spot diameter is ≤5μm and the focal spacing is 20μm, ensuring no crosstalk between focal points. Finally, the array-type focal binary map corresponding to each position of the mechanical scan is pre-stored in the DMD.

[0040] Step S200, non-mechanical fine scanning light field modulation is as follows:

[0041] The optical field incident on the multimode fiber is complexly amplitude-modulated by the calibrated DMD to suppress mode crosstalk in the multimode fiber and generate a stable and controllable 2×2 array of independent focal points at the output end of the multimode fiber to achieve non-mechanical fine scanning. Based on the pre-stored focal binary map, a linkage mechanism between optical field modulation and subsequent mechanical scanning is established.

[0042] Step S300, the mechanical large-area scanning and composite scanning mechanism is constructed as follows:

[0043] A multimode optical fiber and an integrated probe are driven by a two-dimensional XY micro piezoelectric ceramic to perform a large-area mechanical scan along a pre-set equidistant elliptical trajectory. The specific parameters and implementation are as follows:

[0044] Elliptical trajectory calibration is performed by establishing the scan coordinates using the elliptical trajectory formula:

[0045] ;

[0046] Among them, the semi-major axis of the X-axis A=2mm, the semi-minor axis of the Y-axis B=2mm, the linear offset speed coefficient v=0.1mm / s, the angular frequency ω=20π rad / s, the direction control number N=4, and the coordinate reference for accurately calibrating the preset position of each mechanical scan.

[0047] Scanning parameter control involves applying a driving voltage to the piezoelectric ceramic via an FPGA controller to adjust displacement parameters. A scanning step size of 20 μm and a scanning cycle of 0.5 s are set to achieve full coverage scanning of a circular imaging area with a diameter of 4 mm. A global coordinate grid is simultaneously established to provide a positional reference for optical interpolation. A composite scanning mechanism involves mechanical scanning driving the multimode fiber to switch between different preset positions. Each position is synchronously matched with the array focus point of the non-mechanical scanning, forming a composite scanning mechanism that combines large-scale mechanical coverage with localized non-mechanical fine sampling.

[0048] Step S400, the composite scanning parallel sub-pixel sampling is as follows:

[0049] At each preset position of the mechanical scan, perform the following operations:

[0050] The DMD instantly retrieves a pre-stored 2×2 array of focal binary images corresponding to the preset position, and simultaneously generates a 2×2 array of independent focal points at the multimode fiber output end; the center coordinates of the 2×2 sub-pixel sampling points are compared with the coordinates of the current mechanical scan preset position. Consistent, the fixed offset of each sub-pixel sampling point relative to the center coordinate is The specific coordinates of the four sub-pixel sampling points are determined as follows: , , , And substitute it into the global coordinate system to complete the coordinate calibration of optical interpolation.

[0051] Synchronous acquisition is achieved through a 2×2 transparent photoacoustic transducer array, with four transducers respectively collecting photoacoustic signals from the four focal areas, realizing four-channel synchronous parallel acquisition; the FPGA controller controls the DMD binary image switching, pulsed laser emission, and acquisition card sampling synchronous triggering to ensure the spatiotemporal consistency of light field modulation, laser excitation, and signal acquisition.

[0052] Step S500, high-resolution image reconstruction is as follows:

[0053] The acquired four-channel photoacoustic signals are sequentially amplified by a 60dB gain amplifier and then bandpass filtered from 1MHz to 20MHz to remove low-frequency drift and high-frequency electromagnetic interference, resulting in preprocessed photoacoustic signals. The preprocessed multi-channel photoacoustic signals are then time-domain aligned and amplitude-corrected to eliminate time delay and gain deviations between channels. Combining the spatial coordinate data of each sampling point, a delay-stack algorithm is used to reconstruct the light absorption distribution image, and three-dimensional volume data is generated using the following formula:

[0054] ;

[0055] in, It is a time-domain sound pressure signal. For the speed of sound, For time delay, For imaging depth, The data is in three-dimensional volume. Finally, based on a global coordinate grid, the reconstructed images from all preset positions of the mechanical scan are fused to obtain a complete high-resolution photoacoustic endoscopic image. In this embodiment, the effective sampling density is increased by 4 times through optical interpolation, and the spatial resolution is increased from 20μm in pure mechanical scanning to 5μm. The imaging field of view remains at a diameter of 4mm, and the scanning time is consistent with that of pure mechanical scanning, without additional time loss. At the same time, it takes into account both large field of view coverage and high pixel accuracy, meeting the imaging needs of clinical endoscopy scenarios.

[0056] Example 2

[0057] The core of this embodiment is a composite scanning mode that combines large-area mechanical scanning, single-position multi-focus parallel raster scanning, and deep learning image fusion. The system hardware is the same as in Embodiment 1, the difference lies in the scanning strategy and image reconstruction method, and the specific steps are as follows:

[0058] Step S100, DMD pre-calibration and binary map pre-storage are as follows:

[0059] The transmission matrix calibration of the DMD and multimode fiber is completed in the same manner as in Example 1. After calibration, multiple sets of 2×2 array-type focal binary images are pre-stored in the DMD. Each set of binary images corresponds one-to-one with a preset position of the multimode fiber during mechanical scanning, and each set of binary images contains 16 consecutive sub-images. Each sub-image corresponds to a set of 2×2 array-type independent focal point generation parameters at a specific spatial position, which can drive the focal point to complete a 4×4 step grating scan in the corresponding sub-region. The scanning step size is 5μm, and the single focal scan range is 20μm×20μm.

[0060] Step S200, large-area mechanical scanning is as follows:

[0061] Consistent with Example 1, an XY two-dimensional micro piezoelectric ceramic is used to drive a multimode optical fiber to complete the scanning of an imaging area with a diameter of 4mm along an equidistant elliptical trajectory. The scanning step size is 20μm, a global coordinate grid is established, and the coordinate reference of each preset position is calibrated.

[0062] Step S300: Composite scanning and parallel fine scanning are as follows:

[0063] When the XY two-dimensional micro piezoelectric ceramic drives the multimode optical fiber to any preset mechanical scanning position, the following operations are performed:

[0064] The DMD instantly invokes and loads a set of 2×2 array-type focal binary images matching the location, sequentially playing 16 sub-images within the set at a switching speed of 40kHz. This drives four independent focal points in the 2×2 array, simultaneously completing a 4×4 step raster scan within their respective 20μm×20μm sub-regions. Each focal point corresponds to an independent sub-sampling region, with the four focal points covering four adjacent sub-regions in parallel, eliminating scanning blind spots. The photoacoustic signals excited by each focal point are simultaneously acquired through the four channels of the 2×2 transparent photoacoustic transducer array, completing 16 scans and signal acquisitions within a single mechanical position to obtain fine-grained sampling data for the four local sub-regions corresponding to that location.

[0065] Step S400, image reconstruction and stitching based on deep learning is as follows:

[0066] Consistent with Example 1, the acquired four-channel photoacoustic signals are amplified and filtered to eliminate noise interference.

[0067] Parallel backprojection was performed on 16 sets of signals at each scanning position to obtain four initial reconstructed sub-images with 10% overlap in the field of view. These four initial reconstructed sub-images were then input into a pre-trained U-Net deep convolutional neural network. The network extracted features from each sub-image at three different scales, and after feature alignment, generated a pixel-level adaptive weight map. This weight map was then used to perform weighted fusion of the four sub-images, eliminating stitching boundary artifacts and completing high-resolution local image reconstruction at the current preset position of the mechanical scan. For all trajectory positions of the mechanical scan, the same method was used to complete local image reconstruction. All locally reconstructed images were then input into the same deep convolutional neural network again. Through the same feature extraction, alignment, and weighted fusion algorithms, seamless stitching of the entire image was achieved, ultimately yielding a complete photoacoustic endoscopic image.

[0068] In this embodiment, the local imaging area of ​​a single mechanical position is increased by 4 times compared to that in Embodiment 1, enabling high-resolution imaging with a larger field of view within the same scanning time. At the same time, deep learning image fusion effectively suppresses boundary artifacts from multi-region stitching, improving the imaging signal-to-noise ratio by more than 15dB, further optimizing the imaging quality under a large field of view.

[0069] Example 3

[0070] This embodiment is a photoacoustic endoscopic imaging system based on composite scanning to achieve optical interpolation, used to implement the imaging method described in Embodiment 1 or Embodiment 2. Specifically, it includes a field control module, a driving module, an image reconstruction module, and an FPGA core controller electrically connected to each module. The detailed structure and function of each module are as follows:

[0071] Field control module: Its core function is to achieve non-mechanical fine scanning, control the light field incident on the multimode fiber, and generate a stable and controllable M×N array of independent focal points at the end of the multimode fiber.

[0072] The field control module is centered around a digital micromirror device (DMD) and includes a pulsed laser, a first optical lens assembly, a 4f optical system, an optical fiber coupler, a multimode fiber, and a calibration unit connected to the DMD. The 4f optical system consists of a second optical lens assembly, an aperture, and a third optical lens assembly arranged coaxially in sequence. It achieves precise frequency domain control of the optical field through Fourier transform, frequency domain filtering, and inverse Fourier transform. The calibration unit consists of an objective lens, a mirror, an optical lens, and a CCD camera. It is used to calibrate the transmission matrix between the DMD and the multimode fiber to ensure the accuracy of the focus point generation.

[0073] Drive module: Its core function is to realize large-area mechanical scanning, drive the multimode fiber to scan along a preset trajectory, and construct a global coordinate grid for the imaging area. The drive module adopts XY two-dimensional micro piezoelectric ceramics, which are connected to the multimode fiber through a transparent 3D printed part. By applying a driving voltage, the position of the multimode fiber is changed. The displacement parameters are controlled by the FPGA controller, which can accurately limit the scanning speed, scanning cycle and scanning trajectory of the multimode fiber, and provide a position reference for optical interpolation and image reconstruction.

[0074] Image Reconstruction Module: The core function is to acquire and process the photoacoustic signals excited by each array focal point, and combine the scanning trajectory and sub-pixel sampling data to complete the photoacoustic endoscopic image reconstruction. The image reconstruction module includes a transparent photoacoustic transducer array, amplifiers, filters, an acquisition card, and a computer. The transparent photoacoustic transducer array consists of multiple transparent photoacoustic transducers symmetrically distributed around the optical axis. The number of transducers precisely matches the number of M×N independent focal points generated by non-mechanical scanning. A laser transmission channel is reserved at the center of the array to synchronously acquire the photoacoustic signals excited by each focal point. The amplifier and filter sequentially amplify and filter the photoacoustic signals. The acquisition card and controller are synchronously triggered to complete the synchronous acquisition of multi-channel signals. The computer has a built-in photoacoustic image reconstruction algorithm and a deep learning fusion model to output the final high-resolution photoacoustic endoscopic image. The FPGA core controller is electrically connected to the DMD, pulsed laser, driver module, and acquisition card to achieve synchronous control of the entire process of DMD binary image switching, laser pulse emission, piezoelectric ceramic scanning, and photoacoustic signal acquisition, ensuring the spatiotemporal consistency of all parts of the system.

[0075] See Figure 1 , Figure 1 The diagram shows a schematic of the system structure of the field control module of a composite photoacoustic scanning endoscope imaging system based on mechanical and field control in one embodiment of the present invention. The field control module is a multimode fiber optical field control system with a digital micromirror device 3 (DMD) as the core, and also includes a pulsed laser 1, a first optical lens assembly 2, a 4f optical system, an optical fiber coupler 7, and a multimode fiber 8.

[0076] Understandably, the digital micromirror device (DMD) is a microelectromechanical system (MEMS) chip, essentially a high-speed, programmable, and digital optical switch array. It is worth noting that the DMD possesses significant technological advantages, with current modulation speeds reaching 40kHz, far exceeding those of spatial light modulators (SLMs). Furthermore, utilizing superpixel modulation methods, the DMD can achieve complex amplitude modulation of the incident light field, further enhancing the accuracy and flexibility of light field control. In addition, the DMD's high modulation speed efficiently supports rapid switching and stable operation of non-mechanical scanning multifocal points, matching the speed requirements of the XY two-dimensional micro piezoelectric ceramic driving the multimode fiber for trajectory scanning in the drive module, providing reliable support for efficient, high-resolution imaging across the entire domain.

[0077] The pulsed laser emitted by the pulsed laser 1 can be amplified by the first optical lens assembly 2 to sufficiently cover the micromirror range used by the DMD, and then wavefront modulated by the digital micromirror device (DMD). The 4f optical system consists of a second optical lens assembly 4, an aperture 5, and a third optical lens assembly 6, used to filter the pulsed laser beam after wavefront modulation by the digital micromirror device (DMD), and after filtering, the DMD-modulated pulsed laser beam is coupled into the multimode fiber 8 via the fiber coupler 7. The second optical lens assembly 4 is used to convert the binary spatial light field distribution emitted from the DMD, determined by the "on / off" state of the micromirrors, into a frequency-domain light field distribution, and form a clear spatial spectrum on its rear focal plane (i.e., the plane where the aperture 5 is set, also known as the Fourier plane). The aperture 5, located on the Fourier plane, adopts a circular structure with a diameter matching the system and performs a low-pass filtering operation, allowing only the low-frequency components at the center of the spectrum to pass through. After filtering out some high-frequency components, the spatial occupancy range of each sub-pixel expands to the periphery and overlaps with each other, blurring the outline. Based on the overlapping effect between sub-pixels, light of different phases interferes, thereby realizing the vector superposition process of superpixels. The third optical lens assembly 6 is used to restore the frequency-domain light field distribution filtered by the aperture to the filtered spatial light field distribution, and form the final light field pattern on its rear focal plane. This 4f optical filtering system achieves precise frequency-domain control of the light field through a collaborative workflow of "Fourier transform → frequency-domain filtering → inverse Fourier transform", outputting a spatial light field that meets the requirements of the superpixel mode. Preferably, the pulsed laser 1 is a high-repetition-rate pulsed laser with a wavelength of 532 nm.

[0078] In one specific embodiment, the field modulation module further includes a calibration unit connected to the digital micromirror device (DMM). The calibration unit calibrates the DMM using a pre-generated binary detection map. The pulsed laser emitted by the pulsed laser is wavefront modulated by the calibrated DMM, forming multiple independent focal points at the output end of the multimode fiber. The process includes the following steps:

[0079] S1, Project a pre-generated binary detection pattern onto the digital micromirror device, and calibrate the digital micromirror device according to the pre-generated binary detection pattern; wherein, the binary detection pattern is a pattern of a probe light field with random phase and amplitude distribution converted by superpixel encoding;

[0080] Specifically, step S1 includes:

[0081] S12, project a pre-generated binary detection map onto the digital micromirror device so that the pulsed laser beam emitted by the pulsed laser is transmitted through the multimode fiber and forms a binary speckle pattern at the output end of the multimode fiber.

[0082] Specifically, the digital micromirror device (DMD) is first calibrated by projecting a series of pre-generated binary probe maps onto it. These pre-generated binary probe maps are then converted into a probe light field with random phase and amplitude distributions through superpixel encoding. In essence, superpixel encoding treats each 4×4 pixel group on the DMD as a superpixel unit, and by encoding the micromirror state within each superpixel unit, it achieves simultaneous modulation of the amplitude and phase of the incident light field. The pulsed laser beam emitted by the laser passes through the probe light field formed by the digital micromirror device projected with a series of binary probe maps, and after being transmitted through a multimode fiber, forms a binary speckle pattern at the output end.

[0083] S14, acquire the light intensity information of the binary speckle pattern, and reconstruct the transmission matrix based on the light intensity information of the binary speckle pattern;

[0084] Specifically, a pre-generated binary probe image is projected onto a digital micromirror device. After transmission through a multimode fiber, a binary speckle pattern is formed at the output end of the multimode fiber. A camera is used to acquire the light intensity information of the binary speckle pattern formed at the output end of the multimode fiber. The camera can be an avalanche photodiode or a CCD. Once the light intensity information of the speckle pattern is acquired, the transmission matrix of the binary probe image is non-holographically phase-recovered using the generalized Gerchberg-Saxton algorithm, thus obtaining the iteratively reconstructed transmission matrix.

[0085] Specifically, the generalized Gerchberg-Saxton algorithm is an iterative phase retrieval algorithm. Its core advantage is that it can recover phase information by measuring only the pure intensity value without directly measuring phase information.

[0086] Regarding the technical solution of this invention, K independent and random detection patterns are first projected using a digital micromirror device (DMD). Each detection pattern undergoes superpixel conversion processing to form a detection light field with phase and amplitude distribution characteristics. To simplify subsequent mathematical calculations, the above K field modes can be integrated into a single input detection matrix P, whose matrix element expression is as follows:

[0087] ;

[0088] In the formula, the subscript and superscript correspond to the index identifier of the superpixel in the DMD plane and the label of the field mode, respectively, so as to clarify the spatial and mode belonging relationship of the matrix elements.

[0089] After the probe light field is transmitted through a multimode fiber, a speckle pattern is formed at the output end. At this point, the output light field E can be expressed as...

[0090] ;

[0091] in, The multimode fiber transmission matrix to be solved directly reflects the transmission characteristics of the light field within the fiber. In practical engineering applications, an avalanche photodiode (APD) or a camera (CCD) can be used as the detection element to collect the light intensity signal of the output planar speckle pattern, and then the amplitude distribution parameters of the light field can be obtained by inversion. The number of pixels of the detection device or the number of spatially independent intensity measurements directly determines the number of rows M of the transmission matrix, i.e., the vertical dimension of the matrix.

[0092] In the iterative initialization phase, an initial estimate of the transfer matrix is ​​first generated. This provides an initial reference for subsequent iterative calculations. After entering the f-th iteration, the approximate optical field of the multimode fiber output plane can be constructed as follows:

[0093] ;

[0094] In the above formula, This indicates that elements are multiplied one by one (i.e., the product of elements at corresponding positions). To solve for the principal argument of the complex signal and ensure effective extraction and computation of phase information, the estimated value of the transmission matrix is ​​updated based on the aforementioned approximate optical field. The update expression is as follows:

[0095] ;

[0096] in, The pseudo-inverse matrix representing the input probe matrix P is used to achieve stability and rationality in matrix solving through pseudo-inverse operations. The termination condition of the iterative process is set to one of the following two cases: First, The correlation between the two solutions reaches a preset threshold of 99.9999% or higher; secondly, the number of iterations reaches a preset upper limit of 2000 to avoid infinite looping in the iteration process. Given the periodic oscillations that occur during the numerical solution evolution process, the correlation index is defined as... The correlation between them, rather than The correlation between them is improved, thereby enhancing the accuracy of phase recovery results.

[0097] During the iterative execution process, if the estimated value of the transfer matrix... If a row of elements in the matrix reaches the convergence condition, that row will be directly output as the final solution. By iteratively eliminating convergent row elements, the dimensionality of the transmission matrix can be effectively reduced, the computational cost of iteration can be lessened, and the transmission matrix of multimode fiber can be efficiently reconstructed.

[0098] S16, select transmission matrix rows corresponding to multiple predetermined positions on the target plane according to the reconstructed transmission matrix, generate corresponding conjugate wavefronts, and superimpose each conjugate field to generate a corresponding binary modulation speckle pattern, projecting it onto the digital micromirror device to achieve calibration of the digital micromirror device; wherein, multiple high-contrast focal points are formed in the binary modulation speckle pattern.

[0099] Specifically, after reconstructing the transmission matrix TM, rows of the transmission matrix corresponding to multiple predetermined positions on the target plane are selected to generate corresponding conjugate wavefronts. These conjugate fields are then superimposed and superimposed using superpixel encoding technology to generate a corresponding binary modulation speckle pattern. Projecting this pattern onto the DMD allows for the formation of multiple high-contrast focal points at the output end of the multimode fiber. This invention uses feedback control to calibrate the focal point positions, ensuring uniform distribution of each focal point on the imaging plane. Specifically, the binary modulation speckle pattern of the DMD determines the spacing of each focal point based on the divergence angle of the transparent photoacoustic transducer designed under endoscopic conditions, ensuring the independence of each focal point. Furthermore, the binary modulation speckle pattern of the DMD can be dynamically adjusted based on the multi-focal point position information fed back from the image reconstruction module to guarantee the uniformity and independence of each focal point. Understandably, the binary modulation speckle pattern of the DMD is dynamically programmed according to the required number, position, and energy distribution of focal points.

[0100] Specifically, the calibration unit includes an objective lens 9, a reflector 10, a fourth optical lens 11, and a camera 12. The objective lens 9 projects the light field emitted from the multimode fiber 8 onto the reflector 10, and the reflector 10 changes the beam transmission direction to the fourth optical lens 11. The fourth optical lens 11 is used to perform beam shrinking processing on the reflected beam, and the beam after beam shrinking is acquired by the CCD camera 12 to obtain a binary speckle image. Specifically, in this embodiment, the pulsed laser beam emitted by the pulsed laser 1 is uniformly irradiated onto the micromirror region of the digital micromirror device 3 (DMD) via the first optical lens assembly 2, projecting a pre-generated binary detection map onto the DMD. The pulsed laser is coupled to a multimode fiber 8 using an optical fiber coupler 7, and the output optical field of the multimode fiber 8 is captured by a camera 12. The transmission matrix TM of the binary speckle pattern output by the multimode fiber 8 is measured. The DMD is then calibrated by solving for the DMD binary modulation speckle pattern corresponding to the multifocal points based on the transmission matrix TM of the binary speckle pattern output by the multimode fiber 8, ensuring uniform distribution on the imaging plane.

[0101] Further reading Figure 3 As shown, the above steps are used to obtain a binary image of a 2×2 array of independent focused light spots at each position during the movement of the multimode fiber in the non-mechanical fine scanning step, and the image is pre-stored in the DMD.

[0102] Further reading Figure 4As shown, the multimode fiber 8, the transparent 3D printed part 23, the XY two-dimensional micro piezoelectric ceramic 24, and the transparent photoacoustic transducer array 13 are integrated and packaged in a housing, together forming a forward photoacoustic probe.

[0103] The transparent photoacoustic transducer array 13 consists of multiple transparent photoacoustic transducers symmetrically distributed around the optical axis, with the central region allowing laser transmission. Understandably, the transparent photoacoustic transducer array structure is specifically designed to meet the multifocal scanning imaging requirements of this invention under endoscopic conditions. The number of transparent photoacoustic transducers corresponds to the number of focal points to achieve synchronous acquisition. Specifically, in this embodiment, the transparent photoacoustic transducer array 13 divides the exit face of the multimode fiber 8 into four regions, each region corresponding to an independent focal point, and the scanning range of each focal point is limited to its corresponding region, thereby enabling simultaneous acquisition of photoacoustic signals generated by each focal point under endoscopic conditions. Preferably, the transparent photoacoustic transducer can be fabricated using transparent electrodes made of piezoelectric materials such as lithium niobate or PVDF film (LiNbO3), indium tin oxide (Indium Tin Oxide), or silver nanowires, thereby forming a piezoelectric receiving device with high light transmittance to receive the photoacoustic signal generated by the sample being excited by pulsed light; the first reserved slot 25 refers to the reserved signal line channel in the 3D printed part for placing the transducer signal line; the second reserved slot 26 refers to the reserved multimode fiber channel in the 3D printed part for placing the multimode fiber.

[0104] Further reading Figure 5 As shown, the outer shell 22 of the forward photoacoustic probe has a cylindrical structure and adopts a two-part design. The upper and lower shells are fitted together by precision-machined positioning grooves, fixing the XY two-dimensional micro piezoelectric ceramic 24 in the support groove inside the lower shell. A transparent 3D printed part 23 connects four transparent transducers to the multimode optical fiber and the XY two-dimensional micro piezoelectric ceramic. The transparent 3D printed part adopts a spliced ​​structure, with a disc-shaped top and a hollowed-out center for placing the four transparent transducers. The end near the XY two-dimensional micro piezoelectric ceramic is a flexible rod, with its center for placing the multimode optical fiber and transducer signal lines. The multimode optical fiber and transducer signal lines are bonded to the rod-shaped end of the 3D printed part, and the multimode optical fiber moves with the transparent 3D printed part via the XY two-dimensional micro piezoelectric ceramic.

[0105] The amplifier 16 and filter 17 are used to amplify and filter the photoacoustic signal converted by the transparent photoacoustic transducer in sequence. The parameters and models of the amplifier and filter can be designed according to the response sensitivity, main frequency, and bandwidth of the transparent photoacoustic transducer; this embodiment does not impose any limitations. The FPGA is used to control the switching of the binary speckle pattern of the DMD corresponding to the array focal point, the synchronous emission of the pulsed laser, and the synchronous acquisition of the photoacoustic signal.

[0106] The image reconstruction module includes a data acquisition card 18 and a computer 19. The data acquisition card 18 is connected to both a controller 15 and the computer 19. The controller 15 controls the data acquisition card 18 to synchronously acquire photoacoustic signals processed by an amplifier 16 and a filter 17. The data acquisition card 18 sends the acquired photoacoustic signals to the computer 19 for storage. Based on the acquired photoacoustic signals, a deep learning algorithm is used to complete photoacoustic image reconstruction and fusion. The data acquisition card 18 is directly connected to each partition unit of the transparent photoacoustic transducer array 13 to synchronously acquire photoacoustic signals from all regions. The electrical signals from the four transparent photoacoustic transducers are simultaneously acquired by the data acquisition card to ensure spatiotemporal consistency between signals. The signal processing module of the computer 19 uses parallel computing technology to perform temporal alignment, amplitude correction, and reconstruction on the signals from the four channels and multiple focal points, ultimately obtaining a high-resolution photoacoustic image.

[0107] In a specific embodiment of the first technical solution of this invention, the mechanical scanning employs an equidistant elliptical trajectory, and the trajectory coordinates are established using an elliptical trajectory formula. The elliptical trajectory formula is as follows:

[0108] ;

[0109] In the above formula, The function representing the change of the scanning device's position on the X and Y axes with time t. The x-axis represents the semi-major axis of the ellipse's basic amplitude. This represents the semi-minor Y-axis in the basic amplitude of the ellipse. Indicates the linear offset velocity coefficient. ω represents angular frequency. This represents the direction control number, and the coordinate reference for each preset position of the mechanical scan can be accurately calibrated using this formula.

[0110] At each preset position of the mechanical scan At this location, 2×2 sub-pixel sampling points are generated through non-mechanical scanning. The center coordinates of these 2×2 sub-pixel sampling points are consistent with the preset position coordinates of the current mechanical scan. Each sub-pixel sampling point has a fixed offset relative to the center coordinates. Therefore, the specific coordinates of the 2×2 sub-pixel sampling points can be determined as follows:

[0111] ;

[0112] Substituting the coordinates of all subpixel sampling points and the preset mechanical scanning positions into the global coordinate system, the coordinate calibration of the optical interpolation is completed. During image reconstruction, the acquired photoacoustic signals are first preprocessed: after the transparent photoacoustic transducer array acquires the photoacoustic signals generated by each subpixel sampling point, the signals are transmitted to the amplifier and filter of the image reconstruction module for filtering and amplification. The preprocessed photoacoustic signals, combined with their corresponding coordinate data, are used to reconstruct the light absorption distribution image using a time-reversal algorithm or a delay-superposition algorithm, and three-dimensional volume data is generated using the following formula:

[0113] ;

[0114] In the above formula, Represents the time-domain sound pressure signal. Indicates the speed of sound. Indicates a time delay. Indicates the imaging depth. This represents the three-dimensional volume data, where N represents the directional control number and t represents time. The image is reconstructed using an algorithm, resulting in a high-resolution photoacoustic endoscopic image.

[0115] According to the second technical solution of the present invention, the digital micromirror device (DMD) pre-stores 2×2 array-type focal binary images. Each set of the binary images corresponds one-to-one with each moving position of the multimode fiber during mechanical scanning, and each single image in each set of binary images corresponds to a set of 2×2 array-type focal points at a specific spatial position.

[0116] During the mechanical scanning process, the XY two-dimensional micro piezoelectric ceramic drives the multimode optical fiber to any preset position. The DMD instantly calls and loads a set of 2×2 array-type focal binary images that match the preset position, and plays each image in the set of binary images in sequence to drive the raster scanning of the 2×2 array-type focal. This scanning method is equivalent to dividing the original non-mechanical scanning area into 4 sub-regions, and each sub-region is scanned by a single focal point in the 2×2 array-type focal.

[0117] Synchronizing the scanning process described above, the four zones of the ultrasonic transducer array simultaneously acquire photoacoustic signals excited by four focal points, perform preprocessing, and then perform parallel back-projection on the preprocessed photoacoustic signals to obtain a series of initial reconstructed images with partially overlapping fields of view. Each of these initial reconstructed images is input into a trained deep convolutional neural network, which extracts features from each initial reconstructed image at different scales and performs feature alignment to generate a pixel-level adaptive weight map. The pixel-level adaptive weight map is then used to perform weighted fusion of the initial reconstructed images to determine the preset position of the current mechanical scanning trajectory. Image reconstruction.

[0118] Similarly, for each trajectory position of the mechanical scan After reconstructing the photoacoustic image corresponding to each trajectory point using the same image reconstruction method described above, the reconstructed photoacoustic image of each trajectory point is input into the deep convolutional neural network again. The image stitching of each trajectory point is completed through the feature extraction, alignment and weighted fusion algorithms described above, resulting in a complete photoacoustic reconstruction image with a large field of view and high pixel count, thus realizing multi-region parallel fine sampling of the detection area.

[0119] Further reading Figure 6 , Figure 7 As shown, the first technical solution of the present invention is a special example of this figure. During the scanning process, the XY two-dimensional micro piezoelectric ceramic drives the multimode optical fiber to perform elliptical scanning motion. The black dots in the figure represent the sampling points collected when only mechanical scanning is used, while the 2×2 white dots near the black dots represent the sub-pixel sampling points collected at the mechanical scanning position in conjunction with the aforementioned light field modulation technology. The two work together to achieve large-scale, high-precision sampling.

[0120] It should be noted that this system is a system corresponding to the above method. All implementation methods in the above method embodiments are applicable to this embodiment and can achieve the same technical effect.

[0121] Embodiments of the present invention also provide a computing device, including: a processor and a memory storing a computer program, wherein the computer program, when executed by the processor, performs the method described above. All implementations in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0122] Embodiments of the present invention also provide a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the method described above. All implementations in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0123] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A photoacoustic endoscopic imaging method based on composite scanning and optical interpolation, characterized in that, The method includes the following steps: Step 1 involves complex amplitude modulation of the optical field incident on the multimode fiber to suppress mode crosstalk and generate a stable and controllable array of M×N independent focal points at the output end of the multimode fiber, thereby achieving non-mechanical fine scanning. The digital micromirror device (DMM) pre-stores an array of focal point binary images corresponding one-to-one with the movement positions of the multimode fiber during mechanical scanning. The optical field modulation and mechanical scanning are linked based on these focal point binary images. Before complex amplitude modulation of the optical field incident on the multimode fiber, a calibration step of the DMM is also included, specifically: A pre-generated binary probe map is projected onto the digital micromirror device (DMM), causing the pulsed laser beam emitted by the pulsed laser to form a binary speckle pattern at the output end of the multimode fiber after transmission through the multimode fiber. The intensity information of the binary speckle pattern is acquired, and the transmission matrix is ​​iteratively reconstructed based on the intensity information of the binary speckle pattern. Based on the reconstructed transmission matrix, multiple rows of the transmission matrix corresponding to predetermined positions on the target plane are selected to generate corresponding conjugate wavefronts. The conjugate fields are superimposed to generate corresponding binary modulation speckle patterns, which are then projected onto the DMM to achieve calibration of the DMM. Step 2: Based on the completion of non-mechanical fine scanning, the multimode optical fiber is driven by a driving module composed of XY two-dimensional micro piezoelectric ceramics to perform a large-scale mechanical scanning along a preset trajectory, so that the multimode optical fiber switches between different preset positions, forming a composite scanning mechanism that combines non-mechanical scanning and mechanical scanning. Step 3: Under the composite scanning mechanism, when the multimode fiber is moved to each preset position of the mechanical scan, M×N array-type independent focal points are generated based on the pre-stored array-type focal point binary image corresponding to the preset position. At the preset position, the photoacoustic signals corresponding to multiple sub-pixel sampling points are collected in parallel through the array-type independent focal points. When the photoacoustic signals corresponding to multiple sub-pixel sampling points are collected in parallel through the array-type independent focal points at the preset position, the photoacoustic signals excited by each focal point are collected synchronously through a transparent photoacoustic transducer array. The transparent photoacoustic transducer array consists of multiple transparent photoacoustic transducers that are symmetrically distributed around the optical axis. The number of transducers is precisely matched with the number of M×N independent focal points generated by the non-mechanical scan. A laser transmission channel is reserved in the central area of ​​the array. Each transparent photoacoustic transducer collects the photoacoustic signals excited by the independent focal points in its own area, realizing multi-channel parallel acquisition. Step 4: Establish a global coordinate grid based on the preset trajectory coordinates of the mechanical scan, determine the spatial position of each sampling point by combining the fixed offset of the sub-pixel sampling points, and reconstruct the light absorption distribution image after performing temporal alignment, amplitude correction and preprocessing on the multi-channel photoacoustic signals, and finally generate a high-resolution photoacoustic endoscopic image.

2. The photoacoustic endoscopic imaging method based on composite scanning for optical interpolation according to claim 1, characterized in that, In step 2, when the multimode fiber is driven by the driving module composed of XY two-dimensional micro piezoelectric ceramics to perform a large-scale mechanical scan along a preset trajectory, the preset trajectory is specifically an equidistant elliptical trajectory. The equidistant elliptical trajectory establishes and precisely calibrates the coordinate reference of each preset position of the mechanical scan. The driving module changes the position of the multimode fiber by applying voltage, and its displacement parameters are controlled by the control system, thereby limiting the scanning speed, scanning cycle and scanning trajectory of the multimode fiber.

3. The photoacoustic endoscopic imaging method based on composite scanning for optical interpolation according to claim 2, characterized in that, Step 4, which involves temporal alignment, amplitude correction, and preprocessing of the multi-channel photoacoustic signals before reconstructing the light absorption distribution image, specifically includes the following steps: The acquired photoacoustic signals are amplified and filtered sequentially to obtain preprocessed photoacoustic signals; By combining the preprocessed photoacoustic signal with its corresponding spatial coordinate data, the light absorption distribution image is reconstructed and three-dimensional volume data is generated. A global coordinate grid is established based on the preset trajectory coordinates of the mechanical scan and the fixed offset of the sub-pixel sampling points. The images reconstructed at each preset position are then fused to obtain a complete photoacoustic reconstructed image.

4. The photoacoustic endoscopic imaging method based on composite scanning for optical interpolation according to claim 3, characterized in that, In step 3, when the XY two-dimensional micro piezoelectric ceramic drives the multimode fiber to each preset position of the mechanical scan, the digital micromirror device directly calls the pre-stored array-type focal binary map corresponding to the preset position, and generates M×N array-type independent focal points at the preset position. The center coordinates of the M×N sub-pixel sampling points are consistent with the preset position coordinates of the current mechanical scan. Each sub-pixel sampling point has a fixed offset relative to the center coordinates. The specific coordinates of each sub-pixel sampling point are determined by the offset and substituted into the global coordinate system to complete the coordinate calibration of optical interpolation. In step 4, the photoacoustic signals generated by each sub-pixel sampling point are preprocessed and combined with their coordinate data to reconstruct the light absorption distribution image, ultimately generating a high-resolution photoacoustic endoscopic image.

5. The photoacoustic endoscopic imaging method based on composite scanning for optical interpolation according to claim 4, characterized in that, In step 3, the digital micromirror device pre-stores multiple sets of array-type focal binary images. Each set of binary images corresponds one-to-one with each moving position of the multimode fiber during the mechanical scanning process, and each single image in each set of binary images corresponds to a set of M×N array-type independent focal point generation parameters at a specific spatial position. When the XY two-dimensional micro piezoelectric ceramic drives the multimode optical fiber to any preset position, the digital micromirror device instantly calls up and loads a set of array-type focal binary images that match the preset position, and sequentially plays each image in the set of binary images, driving M×N array-type independent focal points to synchronously complete grating scanning in their respective sub-regions. In step 4, the photoacoustic signals excited by each focal point are synchronously acquired by the transparent photoacoustic transducer array and preprocessed to obtain a series of initial reconstructed images with partially overlapping fields of view. Then, each initial reconstructed image is input into the trained deep convolutional neural network for feature extraction, feature alignment and weighted fusion to complete the image reconstruction of the current mechanical scanning preset position. After reconstructing the photoacoustic image corresponding to each trajectory point using the same image reconstruction method for each trajectory position of the mechanical scanning, the reconstructed photoacoustic image of each trajectory point is input into a deep convolutional neural network again. The image stitching of each trajectory point is completed through feature extraction, alignment and weighted fusion algorithms, and finally a complete photoacoustic reconstructed image is obtained.

6. A photoacoustic endoscopic imaging system based on composite scanning for optical interpolation, wherein the system implements the method as described in any one of claims 1 to 5, characterized in that, It includes: a field control module, a driving module, and an image reconstruction module; The field control module regulates the light field incident on the multimode fiber, generating a stable and controllable M×N array of independent focal points at the end of the multimode fiber; the driving module drives the multimode fiber to perform a large-area scan along a preset trajectory, constructing a global coordinate grid for the imaging area and achieving large-area coverage; the image reconstruction module collects and processes the photoacoustic signals excited by each array focal point, and combines the scanning trajectory of the multimode fiber with the sub-pixel sampling data obtained by optical interpolation to complete the photoacoustic endoscopic image reconstruction.

7. A computing device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program that, when executed by a processor, implements the method as described in any one of claims 1-5.

Images