Real-time fluorescent microscopy imaging method based on image scanning microscopy, ism

Abstract

This invention discloses a real-time fluorescence microscopy imaging method based on image scanning microscopy (ISM), belonging to the field of optical microscopy imaging technology. The method comprises the following steps: First, a multifocal illumination array is generated using illumination light to excite the sample in parallel, producing fluorescence signals. Then, within a single frame exposure time of the image sensor, the multifocal illumination array is controlled to perform high-speed sparse scanning on the sample plane to acquire fluorescence signals. Finally, the fluorescence signals acquired by the image sensor are processed in real time, and the intensity of multiple sampling signals from the same position on the sample plane within adjacent exposure frames is compared and synthesized to output a synthesized fluorescence image. This invention proposes a real-time fluorescence microscopy imaging method based on image scanning microscopy (ISM), achieving high-speed, high-resolution, low-optical-damage, and low-data-redundancy real-time fluorescence imaging, meeting the needs of long-term dynamic observation of complex biological samples.

Description

Technical Field

[0001] This invention relates to the field of optical microscopy imaging technology, and in particular to a real-time fluorescence microscopy imaging method based on image scanning microscopy (ISM). Background Technology

[0002] Fluorescence microscopy, with its high molecular specificity, excellent imaging contrast, and high spatial resolution, has become an important tool in modern life science research and medical diagnosis. Existing wide-field fluorescence microscopes rely on high-sensitivity cameras, achieving imaging speeds on the order of kilohertz, and are widely used in biomedical imaging at various timescales. However, when the optically imaged object experiences strong scattering, the resolution and contrast of traditional wide-field microscopes decrease significantly, leading to blurred images and difficulty in accurately acquiring information about deep or large-scale tissue structures. Furthermore, prolonged continuous illumination of whole samples can easily cause photobleaching and phototoxic effects, thus limiting the observation of long-term dynamic processes in organisms.

[0003] To reduce scattering and photobleaching, researchers have proposed various scanning-based fluorescence microscopy methods, such as confocal microscopy, two-photon microscopy, and stimulated emission depletion microscopy. These methods can achieve high-resolution imaging within the diffraction limit, and even surpass the optical diffraction limit. Image scanning microscopy (ISM) is a fluorescence microscopy method that combines point-scan illumination with array detector acquisition and improves spatial resolution through pixel redistribution algorithms. It can further improve resolution within the diffraction limit, even achieving super-resolution imaging. However, because it is essentially based on a single-point scanning mechanism, its imaging speed is limited, making it difficult to capture fast-moving biological processes. The single-point scanning method of ISM limits imaging speed, making it unsuitable for capturing fast-moving biological processes. To improve imaging speed, existing studies have attempted to utilize fast scanning components such as galvanometers, microelectromechanical systems (MEMS), and acousto-optic deflectors, or employ parallel acquisition methods such as rotating disk confocal microscopy and structured illumination microscopy. However, existing methods typically only achieve high-resolution images within a relatively small field of view. When attempting to achieve centimeter-scale large field-of-view imaging, they inevitably face challenges such as reduced speed, decreased resolution, or increased optical complexity. While multifocal illumination fluorescence microscopy can achieve high spatiotemporal resolution imaging, its scanning speed is severely limited by the camera frame rate, and the massive amount of data poses challenges for subsequent image storage and processing.

[0004] In summary, existing fluorescence microscopy techniques generally suffer from the following shortcomings: (1) the imaging quality deteriorates significantly in scattering media, making it difficult to achieve clear imaging of intact tissues; (2) while achieving high resolution, the imaging speed is limited, making it difficult to meet the dynamic observation needs of rapid biological processes; (3) existing data processing methods for high-speed imaging techniques are generally performed at the back end, resulting in a large amount of data transmission, extremely high requirements for data transmission and storage, and a large amount of data redundancy; (4) prolonged illumination can easily cause photobleaching and phototoxicity, limiting long-term observation of in vivo samples. Therefore, there is an urgent need for a new fluorescence microscopy technique that can achieve high-speed, high-resolution, low-damage imaging with low data redundancy within a centimeter-scale large field of view. Summary of the Invention

[0005] The purpose of this invention is to propose a real-time fluorescence microscopy imaging method based on image scanning microscopy (ISM) within the framework of image scanning microscopy. This method involves parallel excitation of sample fluorescence using a multi-focal illumination array, signal acquisition via high-speed sparse scanning within a single exposure frame, physical pixel redistribution using a specially designed objective lens, and real-time maximum value projection synthesis of multi-frame sampled signals at the same location using an FPGA pipeline architecture. This enables high-speed, high-resolution, low-optical-damage, and low-data-redundancy real-time fluorescence microscopy imaging with a centimeter-scale large field of view, making it particularly suitable for dynamic observation and functional imaging of complex biological samples.

[0006] To achieve the above objectives, this invention proposes a real-time fluorescence microscopy imaging method based on image scanning microscopy (ISM), comprising the following steps: Step S1: Use illumination light to generate a multi-focal illumination array and excite the sample in parallel to generate fluorescence signals. In this process, each illumination focal point and its corresponding camera photosensitive area together constitute a tiny parallelized ISM unit. Each illumination focal point is equivalent to an independent ISM illumination point source, and the fluorescence information it excites is collected by multiple pixels (sub-pixels) in the detector array. Step S2: During the single-frame exposure time of the image sensor, control the multi-focus illumination array to perform high-speed sparse scanning on the sample plane to prevent crosstalk caused by signals being too close together, and to achieve fine scanning of the image in adjacent frames, that is, the scanning point of the next frame is finely shifted compared to the previous frame, to obtain image information at different offset distances between the center of the diffusion function of the probe point and the center of the diffusion function of the excitation point.

[0007] Step S3: During the acquisition process, the fluorescence signal obtained by the image sensor is processed in real time. The intensity of multiple sampling signals from the same position on the sample plane within several consecutive frames is compared and synthesized, and the synthesized fluorescence image is output. The complex digital redistribution process in traditional ISM is simplified to real-time screening of the signal with the most consistent point spread function (PSF) coincidence center, which significantly reduces data redundancy and subsequent processing burden.

[0008] Preferably, in step S1, the illumination light is a laser, and the specific operation of generating a multifocal illumination array is as follows: the incident laser is split into multiple microbeams by a beam splitting element, and a multifocal illumination array is formed on the sample plane by a focusing element.

[0009] Preferably, the beam splitter is any one of a microlens array, a diffraction grating, a digital micromirror, an optical metasurface, and a spatial light modulator.

[0010] Preferably, the multifocal illumination array is one or more of a rectangular dot matrix, a hexagonal close-packed dot matrix, and a custom sparse dot matrix.

[0011] Preferably, in step S2, the step of controlling the multifocal illumination array to perform high-speed scanning is as follows: the beam is deflected by any one of an acousto-optic deflector, an electro-optic deflector, and an electromagnetically driven scanning mirror, thereby realizing high-speed scanning of the multifocal illumination array.

[0012] Preferably, the fluorescence signal is collected by using an imaging objective with a receiving numerical aperture twice that of the excitation numerical aperture, thereby achieving a precise displacement of the fluorescence signal collected by each detection pixel to the corresponding excitation center at a distance of 0.5 times the pixel spacing.

[0013] Preferably, the scanning parameters within a single frame exposure time satisfy the following conditions: In a periodic lattice formed by a beam splitter grating, the interval between two adjacent points in one direction is defined as the total scan length. S The scanning step size is , M This represents the total number of scans, corresponding to M One point; exist x In direction, with scan step size t Perform equal-interval scanning. M Once; within a single frame exposure time N Second scan, and M Can be N Divisible; exist y In direction, with scan step size t Perform equal-interval scanning. M Once; within a single frame exposure time N Second scan; exist x In direction, M The points are sequentially numbered from 1 to... M For different composite frames in an image, in the first... During the second exposure, the set of location numbers collected is: ;in m , n It is an integer and satisfies ; ，Z It is an integer; exist y In direction, M The points are sequentially numbered from 1 to... M In the n During the second exposure, the set of location numbers collected is: ; The final image point acquired in a single-frame exposure is determined by the current exposure. x The set of points for directional data acquisition and y The location of the points collected in the direction is determined by arranging and combining them; Through continuous A collection of exposure scans across multiple frames, providing complete coverage. x direction and y All directions M The scanning area formed by the combination of individual points is synthesized into a complete image.

[0014] Preferably, in step S3, the real-time image processing method is as follows: The expression for the raw signal captured by the camera is as follows: (1); in, For the camera in position The excitation center acquired by the pixel is Fluorescent signal, In order to be in Excitation fluorescence signal at the location, As the excitation center of the lattice light source, For the stimulation center The fluorescence signal source at the location The excitation point diffusion function at that location, For located The received pixel is at position The spread function at the probe point, Detecting pixels for the camera The sum of all detectable lighting points; In sparse scanning scenarios, the distance between adjacent excitation points is much greater than the spread function of the excitation point and the spread function of the detector point. Therefore, for each camera detector unit, the number of excitation centers to be considered is less than or equal to 1, and formula (1) can be simplified to: (2); in, It is a constant; To simplify calculations and obtain analytical solutions, it is assumed that both the excitation and detection PSFs are Gaussian functions, and that the excitation and emission wavelengths are similar (i.e., the same width). According to ISM theory, pixel redistribution maps the signal of each pixel to the correct spatial location, thereby improving the spatial resolution of the image. When the excitation point spread function... With probe point spread function When they are equal, for the scan position Camera pixels The optimal pixel redistribution position for the acquired signal is the midpoint between these two points. The formula is as follows: (3); The image can now be represented as: (4); in, This represents the signal assigned to different pixels after pixel redistribution; the signal is only accumulated when the redistributed position is exactly the same as the target position, achieving accurate position mapping. The equivalent point spread function can then be expressed as: (5); in, The standard deviation of the diffusion function at the excitation point; According to Gauss's formula, the full width at half maximum (FWHM) of a Gaussian distribution is proportional to its standard deviation. At this point, the corresponding standard deviation is 1 / 3 of the original standard deviation. This achieves approximately This invention achieves a resolution improvement of several times. Compared to traditional ISM which uses digital pixel redistribution in back-end processing, this invention uses an objective lens with a numerical aperture twice that of the incident scanning lens. This process is equivalent to the classical ISM digital domain performing a 0.5 times pixel spacing displacement on each pixel signal towards the excitation center and then summing the results. Although the excitation point spread function and the detector point spread function are not equal at this point, it only affects the optimal displacement distance and still achieves high-resolution imaging. Furthermore, this method is completed in real time during photon propagation without any computational delay, thus achieving a resolution improvement.

[0015] Furthermore, existing multifocal ISM technologies typically employ pinholes of fixed size to achieve defocused light and scattering. However, a fixed pinhole size cannot adapt to the PSF distortion of strongly scattering biological samples, making it difficult to achieve an optimal balance between signal preservation and background suppression. This invention also provides a simple back-end processing method that uses maximum value summation in post-processing. This method is essentially equivalent to spatial domain aperture screening of the pinhole, and its projection method is shown in the following formula: (6); in, For different exposure times The light intensity at the location, for Maximum light intensity at the location.

[0016] Preferably, the image processing unit is one or more programmable FPGAs or other digital image processing devices. By extracting the maximum value and projecting and synthesizing multiple frames of images within the FPGA, microsecond-level processing latency is achieved. The system adopts a pipelined parallel architecture, which supports real-time processing during image data stream transmission.

[0017] Therefore, this invention proposes a real-time fluorescence microscopy imaging method based on image scanning microscopy (ISM), and demonstrates it through a related system. Its beneficial effects are as follows: (1) Imaging speed and resolution are improved simultaneously: By using multi-focal parallel illumination and sparse scanning strategies, while maintaining high spatial resolution, the effective imaging speed is increased to tens to hundreds of times that of traditional point scanning systems, which is suitable for observing rapid biodynamic processes.

[0018] (2) Scattering suppression and image quality enhancement: Sparse scanning effectively reduces scattering crosstalk between adjacent illumination points. Combined with real-time signal fusion algorithm, it significantly improves imaging contrast and signal-to-noise ratio in scattering media.

[0019] (3) Great improvement in data efficiency: Real-time data processing avoids the transmission and storage of all raw data, and only outputs the final image, reducing the system's storage and communication burden.

[0020] (4) Significantly reduced light damage: Illumination time and light dose are dispersed through sparse scanning, and the light radiation received per unit area is greatly reduced, which is conducive to extending the survival time and observation window of live samples.

[0021] (5) High system integration and easy implementation: Each subsystem is integrated through standardized optical and electronic interfaces, supporting modular configuration and functional expansion, and can flexibly adapt to imaging needs of different scales and resolutions.

[0022] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0023] Figure 1 This is a flowchart of a real-time fluorescence microscopy imaging method based on image scanning microscopy (ISM) according to the present invention. Figure 2 Here is a flowchart of the image preprocessing method; Figure 3 This is a timing diagram of the control and synchronization module in this invention; where set represents the set of scan points in a single exposure, and p represents the position of a single scan point in a single exposure; Figure 4 This is a schematic diagram of a square multifocal illumination mode in an embodiment of the present invention; wherein, (a) is a schematic diagram of a multifocal illumination pattern, and (b) is a schematic diagram of an image after several consecutive frames of images are synthesized; Figure 5 This is a schematic diagram of the scanning points of a single point in a square multifocal illumination according to an embodiment of the present invention; wherein, the same number represents the scanning points within the same exposure time, and 81 points are scanned in 9 exposures and combined into one image. Detailed Implementation

[0024] To make the technical solutions, advantages, and objectives of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below. The described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention without creative effort are within the protection scope of the present invention.

[0025] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains.

[0026] like Figure 1 As shown, this invention provides a real-time fluorescence microscopy imaging method based on image scanning microscopy (ISM), with the following specific steps: Step S1: Generate a multifocal illumination array using illumination light and excite the sample in parallel to generate fluorescence signals; wherein, the illumination light is a laser, and the specific operation for generating the multifocal illumination array is as follows: the incident laser is split into multiple microbeams by a beam splitter and a multifocal illumination array is formed on the sample plane by a focusing element; the beam splitter can be any one of a microlens array, a diffraction grating, and a spatial light modulator, and the multifocal illumination array can be one or more of a rectangular dot array, a hexagonal close-packed dot array, and a custom sparse dot array.

[0027] Step S2: During the single-frame exposure time of the image sensor, control the multi-focal illumination array to perform high-speed sparse scanning on the sample plane and acquire fluorescence signals; The steps for controlling a multifocal illumination array to perform high-speed scanning are as follows: the beam is deflected by any one of an acousto-optic deflector, an electro-optic deflector, and an electromagnetically driven scanning mirror, thereby achieving high-speed scanning of the multifocal illumination array.

[0028] The scanning parameters within a single frame exposure time must satisfy the following conditions: In a periodic lattice formed by a beam splitter grating, the interval between two adjacent points in one direction is defined as the total scan length. S The scanning step size is , M This represents the total number of scans, corresponding to M One point; exist x In direction, with scan step size t Perform equal-interval scanning. M Once; within a single frame exposure time N Second scan, and M Can be N Divisible; exist y In direction, with scan step size t Perform equal-interval scanning. M Once; within a single frame exposure time N Second scan; exist x In direction, M The points are sequentially numbered from 1 to... M For different composite frames in an image, in the first... During the second exposure, the set of location numbers collected is: ;in m , n It is an integer and satisfies ; ，Z It is an integer; exist y In direction, M The points are sequentially numbered from 1 to... M In the n During the second exposure, the set of location numbers collected is: ; The final image point acquired in a single-frame exposure is determined by the current exposure. x The set of points for directional data acquisition and y The location of the points collected in the direction is determined by arranging and combining them; Through continuous A collection of exposure scans across multiple frames, providing complete coverage. x direction and y All directions MThe scanning area formed by the combination of individual points is synthesized into a complete image.

[0029] Step S3: During the acquisition process, the fluorescence signal obtained by the image sensor is processed in real time. Intensity comparison and synthesis are performed on multiple sampling signals from the same location on the sample plane in several consecutive frames, and the synthesized fluorescence image is output. The specific steps of the real-time processing are as follows: The maximum intensity value of each sampled signal from the same spatial location on the sample plane is extracted from several consecutive frames of images, and the maximum value is projected to synthesize the image. The formula is as follows: ; in, For different exposure times The light intensity at the location, for Maximum light intensity at the location.

[0030] like Figure 2 As shown, to implement the above method in actual imaging, this invention also proposes a specific implementation method of ISM based on FPGA, which mainly consists of three steps: signal input, signal processing, and signal output, as detailed below: For signal input, the high-speed camera output signal is connected to the FPGA board via a standard industrial interface (Camera Link, CoaXPress, or GigE Vision), and the LVDS signal is received using the FPGA's dedicated differential input pins. For intensity signal input, the number of pins is allocated according to the camera's bit depth. For example, an 8-bit image corresponds to 8 value receiving pins per pixel. The number of onboard channels on the FPGA determines the maximum number of parallel pixels imported: maximum number of parallel pixels = number of pins ÷ camera bit depth. After data input, the fluorescence signal intensity value is retrieved using an onboard chip and added to the intensity register. For position information input, pixels are imported sequentially using a raster scan method. A control signal channel is set up for position calibration, determining the xy position of the point. Each time a single-point fluorescence signal intensity collection is completed, a count is performed, and the position value of the onboard register is incremented by one. The inversion calculation is performed using the storage address = Y coordinate × image width + X coordinate. The position register is reset to zero after collecting fluorescence signal intensity equivalent to the number of pixels in an image. After a data reception is completed, a pixel data packet with a uniform format is generated: {Y coordinate, X coordinate, fluorescence signal intensity} and stored in Block RAM. At this point, the signal input is complete.

[0031] During signal processing, a comparison operation is performed for each signal input. The fluorescence signal intensity value of the input data is compared with the fluorescence signal intensity value at the same coordinate in the Block RAM. If the new value is greater than the original value, a replacement operation is performed; otherwise, no operation is performed. This process continues until the input signal intensity value is reached. After a full-field pixel comparison during the first exposure, the image is output, and the fluorescence signal intensity values ​​at all locations are reset to zero (initialization). The synthesized projected image data undergoes format reconstruction and protocol encapsulation to reassemble the processed pixel stream into a standard image format for output, which is then transmitted to a computer via USB 3.0 or Ethernet interface. This method achieves real-time, online processing during image sensor data stream transmission, avoiding the data transmission bottlenecks and processing delays of traditional software solutions.

[0032] The present invention also provides a real-time fluorescence microscopy imaging system for parallel scanning, which demonstrates a real-time fluorescence microscopy imaging method based on image scanning microscopy (ISM) technology, comprising: a laser generation module, a two-dimensional scanning module, a beam splitting module, a focusing module, an imaging camera, an image processing module, and a control and synchronization module.

[0033] The laser generating module is used to generate illumination laser, which is either continuous light or high-frequency pulsed laser with a repetition frequency of not less than 100kHz, and the wavelength range covers the visible light to near-infrared band. The two-dimensional scanning module is connected to the output optical path of the laser generator module and is used to perform high-speed, precise two-dimensional deflection scanning of the incident laser beam; the two-dimensional scanning module is an acousto-optic deflector. The beam splitting module is located after the two-dimensional scanning module and is used to split the scanned laser beam into multiple parallel microbeams that are spatially distributed regularly. The focusing module is located downstream of the beam splitting module. It converges and precisely focuses the multiple microbeams after beam splitting onto the surface of the sample to be tested, forming a high-resolution structured light illumination pattern and exciting the sample to generate fluorescence signals in parallel. At the same time, the beam splitting module and the focusing module together constitute the structured light illumination subsystem. The output light spot distribution of the structured light illumination subsystem is one or more of the following: rectangular dot matrix, hexagonal close-packed dot matrix, and custom sparse dot matrix.

[0034] The imaging camera is used to synchronously acquire fluorescence signals excited by multifocal illumination. It adopts a high frame rate and high sensitivity scientific-grade CMOS camera and an external image processing module, which can perform real-time noise reduction, fusion and image reconstruction of fluorescence signals during the acquisition process. like Figure 3As shown, the control and synchronization module is electrically connected to the 2D scanning module and the imaging camera, respectively, to coordinate the timing relationship between the modules and control the synchronization of the scanning path, exposure time, and signal acquisition; where set is the set of scan points in a single exposure, p is the position of a single scan point in a single exposure, and N is scanned in each exposure. 2 Different sparse points, continuous Z 2 The reconstruction of the entire image is achieved through exposures at different point sets and maximum density projection (MIP) technology. It employs one or more of the following: FPGA programmable logic chip, embedded microcontroller, and dedicated motion control board. This supports programmable scan trajectories, multi-mode imaging switching, and real-time feedback control. The control and synchronization module also features a user interface, supporting real-time adjustment of imaging parameters and selection of scan areas.

[0035] The fluorescence spectrometer is positioned between the sample and the imaging camera to separate the excitation light and fluorescence signal, thereby improving the image signal-to-noise ratio.

[0036] like Figure 4 As shown in (a) of this embodiment, the continuous laser emitted from the laser source is split into multiple microbeams by a grating device in the beam splitting module, forming a 17×17 two-dimensional dot array. This dot array is focused onto the sample surface by a focusing lens group, forming a multifocal illumination array, which excites the sample to generate fluorescence signals in parallel. The excitation light and fluorescence signals are spectrally separated by a dichroic mirror, and the fluorescence signal is transmitted and then collected by a scientific-grade camera.

[0037] After the excitation light passes through the beam splitter, it is coordinated by the scanning module (acousto-optic deflector) and the synchronization module (NI acquisition card PCIe-6535B) to perform multi-point two-dimensional high-speed scanning within the single-frame exposure time of the camera. This controls the synchronization of camera exposure and scanning, driving the illumination array to complete the two-dimensional high-speed scan within the single-frame exposure time window. The scanning step size is configured as 1 / M of the distance between adjacent illumination points generated by the beam splitter; in this embodiment, M=15. During one exposure, the system controls the illumination points to sample in the X and Y directions at every Z step size, where Z=M / N; in this embodiment, N=5, thus sampling occurs every three step sizes, achieving [the desired result] within a single-frame exposure. N 2 There are 25 distinct structured light illumination sites, and a single-frame exposure image is as follows: Figure 4 As shown in (b) above, a complete super-resolution image can be reconstructed by acquiring multiple consecutive frames of images and performing computational synthesis.

[0038] In this embodiment, only continuous Z-series data collection is required. 2 Nine frames (Z=3, i.e., 9 frames) of original images can be synthesized into a final output image. Compared to the M frames required by traditional structured light illumination methods... 2(i.e., 225) frames of images. This invention significantly reduces the number of images acquired and greatly improves the effective utilization of temporal resolution and camera frame rate.

[0039] To illustrate the scanning control method of the present invention in more detail, this embodiment combines... Figure 5 The scanning trajectory of a single lighting point is described.

[0040] like Figure 5 The diagram illustrates the scanning path of an illumination focal point on the sample plane in a simplified embodiment where parameters M, N, and Z are all 3. Points marked with the same number indicate the positions sequentially scanned by that focal point within the same camera exposure time. This scanning trajectory reveals how a real-time fluorescence microscopy system, through precise timing control, rapidly and systematically moves a single structured light spot to a series of predetermined positions within a single exposure frame, thereby efficiently acquiring illumination information from multiple phases and laying the foundation for subsequent image super-resolution reconstruction.

[0041] Therefore, this invention provides a real-time fluorescence microscopy imaging method based on image scanning microscopy (ISM). By synthesizing multiple frames, particle motion vector parameters can be decoded, reducing the dependence on high-speed hardware, improving temporal resolution and measurement accuracy, and simplifying the data processing flow. This method meets the needs of low-cost, high-precision, and high-efficiency particle motion measurement in various transient and steady-state scenarios.

[0042] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A real-time fluorescence microscopy imaging method based on image scanning microscopy (ISM), characterized in that, Includes the following steps: Step S1: Use illumination light to generate a multifocal illumination array and excite the sample in parallel to generate fluorescence signals; Step S2: During the single-frame exposure time of the image sensor, control the multi-focal illumination array to perform high-speed sparse scanning on the sample plane and acquire fluorescence signals; Step S3: During the acquisition process, the fluorescence signal obtained by the image sensor is processed in real time by the image processing unit. The intensity of multiple sampling signals from the same position on the sample plane within several consecutive exposure frames is compared and synthesized, and the synthesized fluorescence image is output.

2. The real-time fluorescence microscopy imaging method based on image scanning microscopy (ISM) according to claim 1, characterized in that, In step S1, the illumination light is a laser. The specific operation for generating a multifocal illumination array is as follows: the incident laser is split into multiple microbeams by a beam splitting element, and a multifocal illumination array is formed on the sample plane by a focusing element.

3. The real-time fluorescence microscopy imaging method based on image scanning microscopy (ISM) according to claim 2, characterized in that, The beam splitter can be any one of a microlens array, a diffraction grating, a digital micromirror, an optical metasurface, and a spatial light modulator.

4. The real-time fluorescence microscopy imaging method based on image scanning microscopy (ISM) according to claim 3, characterized in that, The multifocal illumination array is one or more of the following: rectangular dot matrix, hexagonal close-packed dot matrix, and custom sparse dot matrix.

5. The real-time fluorescence microscopy imaging method based on image scanning microscopy (ISM) according to claim 4, characterized in that, In step S2, the step of controlling the multifocal illumination array to perform high-speed scanning is as follows: the beam is deflected by any one of the acousto-optic deflector, electro-optic deflector and electromagnetic drive scanning mirror, thereby realizing high-speed scanning of the multifocal illumination array.

6. The real-time fluorescence microscopy imaging method based on image scanning microscopy (ISM) according to claim 5, characterized in that, By using an imaging objective with a receiving numerical aperture twice that of the excitation numerical aperture to collect the fluorescence signal, the fluorescence signal collected by each detection pixel is precisely shifted to the corresponding excitation center at a distance of 0.5 times the pixel spacing.

7. The real-time fluorescence microscopy imaging method based on image scanning microscopy (ISM) according to claim 6, characterized in that, The scanning parameters within a single frame exposure time must satisfy the following conditions: In a periodic lattice formed by a beam splitter grating, the interval between two adjacent points in one direction is defined as the total scan length. S The scanning step size is , M This represents the total number of scans, corresponding to M One point; exist x In direction, with scan step size t Perform equal-interval scanning. M Second-rate; Performed within a single frame exposure time N Second scan, and M Can be N Divisible; exist y In direction, with scan step size t Perform equal-interval scanning. M Second-rate; Performed within a single frame exposure time N Second scan; exist x In direction, M The points are sequentially numbered from 1 to... M For different composite frames in an image, in the first... During the second exposure, the set of location numbers collected is: ;in m , n It is an integer and satisfies ; ，Z It is an integer; exist y In direction, M The points are sequentially numbered from 1 to... M In the n During the second exposure, the set of location numbers collected is: ; The final image point acquired in a single-frame exposure is determined by the current exposure. x The set of points for directional data acquisition and y The location of the points collected in the direction is determined by arranging and combining them; Through continuous A collection of exposure scans across multiple frames, providing complete coverage. x direction and y All directions M The scanning area formed by the combination of individual points is synthesized into a complete image.

8. A real-time fluorescence microscopy imaging method based on image scanning microscopy (ISM) according to claim 7, characterized in that, In step S3, the real-time image processing method is as follows: The maximum intensity value of each sampled signal from the same spatial location on the sample plane is extracted from the consecutive frames of images, and the maximum value is projected to synthesize the image, as shown in the following formula: ； in, For different exposure times The light intensity at the location, for Maximum light intensity at the location.

9. A real-time fluorescence microscopy imaging method based on image scanning microscopy (ISM) according to claim 8, characterized in that, The image processing unit is one or more programmable FPGAs or other digital image processing devices. It achieves microsecond-level processing latency by extracting the maximum value of multiple frames of images and projecting and synthesizing them within the FPGA. The system adopts a pipelined parallel architecture, which supports real-time processing during image data stream transmission.

Images