A high-fidelity three-dimensional random optical fluctuation super-resolution imaging method, device, electronic equipment, storage medium and program product

Abstract

The present application relates to the technical field of three-dimensional super-resolution fluorescence microscopic imaging, and discloses a high-fidelity three-dimensional random optical fluctuation super-resolution imaging method, device, electronic equipment, storage medium and program product, wherein an effective flicker band spectrum mean is used to construct a defocus evaluation benchmark, multi-plane time sequence registration is completed relying on fluorescent bead calibration, weight fusion is combined with axial geometric distance and spectral response, and defocus background interference is suppressed from a time sequence source. A virtual axial layer is generated by original time sequence weighting before SOFI accumulation calculation, an axial interpolation factor is decoupled from an imaging order, and axial sampling can be encrypted without improving high-order accumulation. After SOFI accumulation statistics contraction and three-dimensional post-processing, axial crosstalk is effectively reduced, three-dimensional structure continuity, axial positioning accuracy and weak fluorescence structure fidelity are improved, and the present application is suitable for high-fidelity three-dimensional super-resolution imaging of living cells and thick samples.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional super-resolution fluorescence microscopy, specifically to a high-fidelity three-dimensional random optical fluctuation super-resolution imaging method, apparatus, electronic device, storage medium, and program product. Background Technology

[0002] Super-resolution optical fluctuation imaging (SOFI) is a super-resolution imaging method based on the statistical characteristics of the random scintillation time of fluorescent molecules. SOFI calculates the higher-order cumulants or cross-cumulants of fluorescence intensity temporal fluctuations, causing the effective point spread function to shrink statistically, thus achieving spatial resolution exceeding the traditional diffraction limit. Multiplanar 3D SOFI typically acquires fluorescence scintillation time series from multiple axial physical planes simultaneously, utilizing the temporal correlations and higher-order statistical relationships between adjacent physical planes to generate 3D SOFI reconstruction results. Compared to single-planar SOFI, multiplanar 3D SOFI can acquire 3D volumetric data in a shorter time, making it suitable for observing live cells, thick samples, and dynamic 3D structures.

[0003] However, existing multi-planar 3D SOFI still has significant shortcomings. First, significant defocus backgrounds often exist in thick samples or multi-layered structures. The defocus background may still maintain high brightness in the average intensity map and may appear in multiple physical planes, leading to axial crosstalk, virtual layer aliasing, and weak structure submersion.

[0004] Secondly, axial fusion in traditional 3D SOFI typically relies on molecular fluorescence intensity, which cannot effectively distinguish between in-focus effective scintillation signals and out-of-focus background. For SOFI, what truly participates in super-resolution reconstruction is the random temporal fluctuation of fluorescent molecules, rather than the molecular fluorescence intensity itself.

[0005] Furthermore, the number of virtual axis sampling layers in existing 3D cross-cumulant SOFI is usually coupled with the SOFI order. If it is desired to increase the number of virtual axis sampling layers, it is often necessary to increase the SOFI order. However, higher-order SOFI will bring problems such as decreased statistical stability, noise amplification, enhanced brightness nonlinearity, and increased frame rate requirements.

[0006] Furthermore, multi-plane 3D SOFI data is large in scale, typically containing multiple physical planes, thousands of time series frames, and multiple SOFI orders. Repeatedly performing registration, spectrum calculation, weight calculation, and cumulant calculation in each subsequence, each virtual layer, and each order would result in high memory consumption and computation time. Summary of the Invention

[0007] This invention provides a high-fidelity three-dimensional random optical fluctuation super-resolution imaging method, device, electronic device, storage medium, and program product to solve the problem that existing technologies cause high memory consumption and computation time if registration, spectrum calculation, weight calculation, and cumulant calculation are repeatedly performed in each subsequence, each virtual layer, and each order.

[0008] In a first aspect, the present invention provides a high-fidelity three-dimensional random optical fluctuation super-resolution imaging method, the method comprising: Obtain fluorescence scintillation time series of multiple axial physical planes and perform coordinate system registration; The mean spectral value within the effective scintillation frequency band of the fluorescence scintillation time series of each physical plane is extracted to determine the discrete spectral response of each physical plane. Based on the preset geometric distance from the virtual axial layer to the two adjacent physical planes, and combined with the discrete spectral responses corresponding to the two adjacent physical planes, the local axial fusion weights are obtained. The fluorescence scintillation time series of two adjacent physical planes are weighted and fused with the corresponding local axial fusion weights to obtain the fluorescence scintillation time series of the virtual axial layer. The cumulative amount of random optical fluctuation super-resolution imaging of the fluorescence scintillation time series of each virtual axial layer is calculated layer by layer and three-dimensional post-processing is performed to generate three-dimensional random optical fluctuation super-resolution imaging reconstruction results.

[0009] This invention constructs a defocus evaluation benchmark using the average spectral value of the effective scintillation band, completes multi-plane temporal registration based on fluorescent bead calibration, and integrates weighted synchronous axial geometric distance and spectral response to suppress defocus background interference at the temporal source. The virtual axial layer is generated from the original temporal weights before SOFI cumulant calculation, decoupling the axial interpolation factor from the imaging order, thus enabling encrypted axial sampling without increasing higher-order cumulants. Through SOFI cumulant statistical shrinkage and 3D post-processing, axial crosstalk is effectively reduced, improving 3D structural continuity, axial positioning accuracy, and the fidelity of weak fluorescence structures, making it suitable for high-fidelity 3D super-resolution imaging of live cells and thick samples.

[0010] In one optional implementation, the step of acquiring the fluorescence scintillation time series of multiple axial physical planes and performing coordinate system registration includes: The fluorescence scintillation time series of multiple axial physical planes were obtained using a prism-based multi-plane synchronous imaging system. The fluorescence scintillation time series of each of the aforementioned axial physical planes are registered to a unified reference coordinate system.

[0011] This invention employs a prism-based multi-plane synchronous imaging system to simultaneously acquire multiple axial fluorescence scintillation time sequences, resulting in higher acquisition efficiency. By utilizing fluorescence bead calibration data to complete multi-plane coordinate system registration, it corrects the planar spatial offset and misalignment issues caused by the beam splitting optical path, ensuring that signals from different axial planes have a unified spatial reference. This guarantees accurate matching of fluorescence fluctuation signals in adjacent planes during subsequent spectrum analysis and axial weighted fusion, laying a data foundation for suppressing defocus background and constructing reliable virtual axial time sequences.

[0012] In one optional implementation, the step of extracting the spectral mean within the effective scintillation frequency band of the fluorescence scintillation time series of each physical plane after processing, and determining the discrete spectral response of each physical plane, includes: The processed fluorescence scintillation time series of each physical plane is divided into multiple time subsequences; Each of the aforementioned time subsequences is subjected to time spectrum analysis along the time dimension and preprocessed. The mean spectral amplitude within the effective scintillation band is calculated for each time subsequence after preprocessing, and the mean spectral amplitude corresponding to multiple time subsequences for each physical plane is averaged to obtain the axial defocus evaluation function. The axial defocus evaluation function is normalized, spatially smoothed, and subjected to non-negative constraints to obtain the discrete spectral response of each physical plane.

[0013] This invention segments the time sequence into multiple sub-sequences, performs spectral analysis on each sub-sequence, and then fuses the mean values ​​to reduce evaluation bias caused by temporal bleaching and random noise. Only the mean amplitude of the effective flicker frequency band is extracted to construct a defocus evaluation function, accurately distinguishing effective fluctuations within the focus area from high-brightness defocused backgrounds. After normalization, smoothing, and non-negativity constraint optimization, a discrete spectral response is obtained, weakening pixel noise interference and providing a reliable and quantifiable effective signal weight basis for subsequent axial fusion.

[0014] In one optional implementation, the step of obtaining the local axial fusion weight based on the preset geometric distance from the virtual axial layer to two adjacent physical planes, combined with the discrete spectral responses corresponding to the two adjacent physical planes, includes: Based on the number of physical planes and the preset axial interpolation factor, multiple virtual axial layers are constructed; Calculate the geometric distance between any virtual axial layer and two adjacent physical planes, and generate the first geometric distance and the second geometric distance; The local axial fusion weights of the virtual axial layer are obtained by calculating the multiplication values ​​between the first geometric distance, the second geometric distance and the discrete spectral response of the corresponding physical plane.

[0015] This invention constructs a dense virtual axial layer based on an axial interpolation factor independent of the SOFI order, enabling encrypted axial sampling without the need for high-order cumulants. The fusion weights integrate axial geometric distance with discrete spectral responses characterizing effective fluorescence fluctuations, automatically reducing the planar contribution of high-proportion defocused background. During the temporal fusion stage, axial crosstalk is weakened, ensuring the virtual layer retains the random fluctuation characteristics of fluorescent molecules, effectively improving the axial positioning accuracy and structural continuity of 3D imaging.

[0016] In one optional implementation, the step of calculating the cumulative amount of random optical fluctuation super-resolution imaging of the fluorescence scintillation time series of each of the virtual axial layers layer by layer and performing three-dimensional post-processing to generate a three-dimensional random optical fluctuation super-resolution imaging reconstruction result includes: Calculate the super-resolution imaging accumulation of multi-order random optical fluctuations in the fluorescence scintillation time series of each of the virtual axial layers; The multi-order random optical fluctuation super-resolution imaging cumulative quantities of multiple virtual axial layers are stacked along the axial direction to generate three-dimensional random optical fluctuation super-resolution imaging cumulative quantity data. The accumulated data of the three-dimensional random optical fluctuation super-resolution imaging are subjected to three-dimensional post-processing to generate the reconstruction result of the three-dimensional random optical fluctuation super-resolution imaging.

[0017] This invention calculates the cumulative amount of SOFI in each virtual layer by shrinking the system point diffusion function based on the statistical characteristics of fluorescence fluctuations, thus breaking the diffraction limit; axial stacking yields complete three-dimensional cumulative amount data, fully preserving the multi-layer axial structure information of the sample.

[0018] In one optional implementation, the step of performing three-dimensional post-processing on the three-dimensional random optical fluctuation super-resolution imaging cumulative data to generate a three-dimensional random optical fluctuation super-resolution imaging reconstruction result includes: The three-dimensional random optical fluctuation super-resolution imaging cumulative data is subjected to three-dimensional deconvolution processing; The cumulative data of 3D random optical fluctuation super-resolution imaging after 3D deconvolution processing is linearized to generate the reconstruction result of 3D random optical fluctuation super-resolution imaging.

[0019] This invention employs a three-dimensional deconvolution with a matching SOFI order to counteract optical diffraction blur and remove residual axial crosstalk and defocused background from the cumulant data. Subsequent linearization corrects the nonlinear brightness distortion caused by higher-order cumulants, highlighting subtle subcellular structures. The combination of these two methods optimizes the grayscale distribution and spatial sharpness of the three-dimensional volumetric data, significantly improving imaging axial positioning accuracy, structural fidelity, and dynamic viewing range.

[0020] In a second aspect, the present invention provides a high-fidelity three-dimensional random optical fluctuation super-resolution imaging device, the device comprising: The acquisition module is used to acquire fluorescence scintillation time series of multiple axial physical planes and perform coordinate system registration processing; The extraction module is used to extract the mean spectral value within the effective scintillation frequency band of the fluorescence scintillation time series of each physical plane after processing, and to determine the discrete spectral response of each physical plane. The module is used to obtain local axial fusion weights based on the preset geometric distance from the virtual axial layer to two adjacent physical planes and the discrete spectral responses corresponding to the two adjacent physical planes. The weighted fusion processing module is used to perform weighted fusion processing on the fluorescence scintillation time series of two adjacent physical planes and the corresponding local axial fusion weights to obtain the fluorescence scintillation time series of the virtual axial layer. The calculation module is used to calculate the cumulative amount of random optical fluctuation super-resolution imaging of the fluorescence scintillation time series of each of the virtual axial layers layer by layer and perform three-dimensional post-processing to generate three-dimensional random optical fluctuation super-resolution imaging reconstruction results.

[0021] Thirdly, the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the high-fidelity three-dimensional random optical fluctuation super-resolution imaging method of the first aspect or any corresponding embodiment described above.

[0022] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the high-fidelity three-dimensional random optical fluctuation super-resolution imaging method of the first aspect or any corresponding embodiment thereof.

[0023] Fifthly, the present invention provides a computer program product, including computer instructions for causing a computer to execute the high-fidelity three-dimensional random optical fluctuation super-resolution imaging method of the first aspect or any corresponding embodiment thereof. Attached Figure Description

[0024] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0025] Figure 1 This is a schematic diagram of the first process of a high-fidelity three-dimensional random optical fluctuation super-resolution imaging method according to an embodiment of the present invention; Figure 2This is a schematic diagram of the second process of the high-fidelity three-dimensional random optical fluctuation super-resolution imaging method according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the third process of the high-fidelity three-dimensional random optical fluctuation super-resolution imaging method according to an embodiment of the present invention; Figure 4 This is a schematic diagram showing the comparison results of defocus evaluation functions based on simulated three-dimensional multi-plane imaging data according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the original experimental data of multi-plane imaging and the third-order time-spectrum 3D-SOFI reconstruction results according to an embodiment of the present invention; Figure 6 This is a schematic diagram comparing the axial color-coded three-dimensional projection results of temporal spectrum 3D-SOFI and traditional 3D-SOFI according to an embodiment of the present invention. Figure 7 This is a structural block diagram of a high-fidelity three-dimensional random optical fluctuation super-resolution imaging device according to an embodiment of the present invention; Figure 8 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0028] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0029] This invention provides a high-fidelity three-dimensional random optical fluctuation super-resolution imaging method. It constructs a defocus evaluation benchmark using the average spectral value of an effective scintillation band, completes multi-plane temporal registration based on fluorescent bead calibration, and integrates weighted synchronous axial geometric distance and spectral response to suppress defocus background interference at the temporal source. The virtual axial layer is generated from the original temporal weights before SOFI cumulant calculation, decoupling the axial interpolation factor from the imaging order, thus enabling encrypted axial sampling without increasing higher-order cumulants. Through SOFI cumulant statistical shrinkage and three-dimensional post-processing, axial crosstalk is effectively reduced, improving three-dimensional structural continuity, axial positioning accuracy, and the fidelity of weak fluorescence structures, making it suitable for high-fidelity three-dimensional super-resolution imaging of live cells and thick samples.

[0030] This embodiment provides a high-fidelity three-dimensional random optical fluctuation super-resolution imaging method. Figure 1 This is a flowchart of a high-fidelity three-dimensional random optical fluctuation super-resolution imaging method according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps: Step S101: Obtain the fluorescence scintillation time series of multiple axial physical planes and perform coordinate system registration processing.

[0031] It should be noted that the axial physical plane refers to the single-layer imaging focal planes at different Z-axis depths simultaneously acquired after the prism beam splitting multi-plane imaging system, with multiple planes distributed along the sample axis. The fluorescence scintillation time series refers to the stack of time-series images recording the random brightness and dimming changes of fluorescent molecules under continuous multi-frame acquisition; it is the raw signal upon which SOFI technology relies to calculate statistical super-resolution. Coordinate system registration refers to the correction operation that uses the fluorescent bead calibration data as a reference to correct translations, offsets, and distortions between the physical planes along each axis, uniformly mapping all planes to the same standard spatial coordinate system.

[0032] In this embodiment of the invention, a prism-based multi-plane synchronous imaging system is used to simultaneously acquire fluorescence scintillation time series of multiple axial physical planes corresponding to different depths of the sample. Then, the spatial transformation matrix is ​​solved using the multi-plane calibration data obtained by axial scanning calibration of fluorescent beads. Based on this matrix, translation and affine registration operations are performed on the fluorescence scintillation time series of all axial physical planes to uniformly map all planes to the same standard reference coordinate system, eliminating the spatial offset and misalignment of the planes caused by the beam splitting optical path, and providing a unified spatial reference for subsequent spectral analysis of the time series of each plane and weighted fusion of axial time series.

[0033] Step S102: Extract the mean spectral value within the effective scintillation frequency band of the fluorescence scintillation time series of each physical plane after processing, and determine the discrete spectral response of each physical plane.

[0034] It should be noted that the effective scintillation band refers to a specific frequency range defined by matching the scintillation dynamics of fluorescent molecules and the camera frame rate. The spectrum of this band can accurately reflect the effective fluorescence fluctuations that can participate in SOFI reconstruction. The spectral mean refers to the average value calculated by extracting all spectral amplitudes within the effective scintillation band after performing a Fourier transform on the time series. The discrete spectral response refers to the quantized value obtained after normalization, spatial smoothing, and non-negativity constraints on the axial defocus evaluation function.

[0035] In this embodiment of the invention, the time series of fluorescence scintillation of each physical plane after coordinate system registration is divided into multiple time subsequences. Fourier transform is performed on each subsequence and the effective scintillation frequency band matching the fluorescence scintillation characteristics is extracted. The mean spectral amplitude within the frequency band is calculated to obtain the axial defocus evaluation function. Then, the evaluation function is subjected to normalization, spatial smoothing and non-negativity constraint processing in sequence to finally obtain the discrete spectral response that can characterize the intensity of effective fluorescence fluctuations of each pixel, providing a quantitative basis for subsequent calculation of axial fusion weight.

[0036] Step S103: Based on the preset geometric distance from the virtual axial layer to the two adjacent physical planes, and combined with the discrete spectral responses corresponding to the two adjacent physical planes, the local axial fusion weights are obtained.

[0037] It should be noted that the virtual axial layer refers to a dense virtual imaging layer generated based on interpolation of finite physical planes. Geometric distance refers to the axial spatial interval between the virtual axial layer and two adjacent real axial physical planes along the Z-axis. Local axial fusion weight refers to the normalized coefficient calculated by combining the axial geometric distance and the discrete spectral response.

[0038] In this embodiment of the invention, a virtual axial layer is constructed based on the number of physical planes and the axial interpolation factor. The axial geometric distance between any virtual axial layer and two adjacent physical planes is calculated. The two geometric distances are multiplied by the discrete spectral response of the corresponding plane to obtain two sets of intermediate coefficients. The two sets of intermediate coefficients are then normalized to obtain a local axial fusion weight that sums to 1. This automatically reduces the signal contribution of the plane with a high proportion of out-of-focus signals in the fusion.

[0039] Step S104: The fluorescence scintillation time series of two adjacent physical planes are weighted and fused with the corresponding local axial fusion weights to obtain the fluorescence scintillation time series of the virtual axial layer.

[0040] It should be noted that weighted fusion processing refers to the linear weighted superposition of the original fluorescence time series of two adjacent planes using local axial fusion weights to generate a virtual layer time series with complete fluorescence fluctuation statistical characteristics.

[0041] In this embodiment of the invention, the local axial fusion weights are multiplied by the corresponding values ​​of two adjacent physical plane fluorescence scintillation time sequences that have completed registration preprocessing. The two sets of weighted time sequence signals are then added together to complete linear fusion. The statistical characteristics of synchronous random fluctuations of fluorescent molecules are preserved throughout the process, and finally a virtual axial layer fluorescence scintillation time sequence that is adapted to the current axial depth and suppresses defocus background interference is generated.

[0042] Step S105: Calculate the cumulative amount of random optical fluctuation super-resolution imaging of fluorescence scintillation time series of each virtual axial layer layer and perform three-dimensional post-processing to generate three-dimensional random optical fluctuation super-resolution imaging reconstruction results.

[0043] It should be noted that the stochastic optical fluctuation super-resolution imaging (SOFI) cumulant refers to a high-order statistical quantity calculated from the fluorescence scintillation time series, which can achieve statistical contraction of the system's point spread function, breaking through the optical diffraction limit to achieve super-resolution. 3D post-processing refers to optimization operations such as 3D deconvolution and linearization correction performed on the 3D SOFI cumulant volume data to eliminate axial blurring and brightness nonlinear distortion. The 3D stochastic optical fluctuation super-resolution imaging reconstruction result refers to the high-fidelity 3D super-resolution volume imaging data output after cumulant calculation and 3D post-processing, clearly presenting the fine subcellular structure at various depths of the sample.

[0044] In this embodiment of the invention, the cumulative amount of multi-order random optical fluctuation super-resolution imaging is calculated for the fluorescence scintillation time series of each virtual axial layer obtained by fusion. Then, the cumulative amounts corresponding to all virtual axial layers are stacked along the axis to form a complete three-dimensional cumulative volume data. Subsequently, three-dimensional deconvolution matching the imaging order and three-dimensional post-processing with power linearization correction are performed on the volume data to eliminate axial crosstalk and brightness nonlinear distortion, and finally, a high-fidelity three-dimensional random optical fluctuation super-resolution imaging reconstruction result is output.

[0045] This embodiment provides a high-fidelity three-dimensional random optical fluctuation super-resolution imaging method. Figure 2 This is a flowchart of a high-fidelity three-dimensional random optical fluctuation super-resolution imaging method according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps: Step S201: Obtain the fluorescence scintillation time series of multiple axial physical planes and perform coordinate system registration processing.

[0046] Specifically, step S201 includes: Step S2011: Obtain the fluorescence scintillation time series of multiple axial physical planes using a prism-based multi-plane synchronous imaging system.

[0047] In embodiments of the present invention, such as Figure 3As shown, a prism-based multi-plane synchronous imaging system is used to simultaneously expose and photograph the sample. The light path is split by the prism to simultaneously separate multiple imaging focal planes with different axial depths. Multiple sets of fluorescence scintillation time series of axial physical planes that synchronously record the brightness fluctuations of fluorescent molecules are acquired at one time. The axial sampling step size between adjacent physical planes is controlled to be close to or less than half the width of the axial point spread function of the microscopic system, so that there is axial response overlap between adjacent physical planes, providing basic data for subsequent multi-plane time registration and fusion.

[0048] Specifically, a prism-based multi-plane microscopy system is used to simultaneously acquire fluorescence scintillation time series across multiple axial physical planes. These multiple physical planes can be obtained using dual cameras, multiple cameras, multi-plane spectral paths, light field microscopy systems, or other multi-plane imaging devices. The prism-based multi-plane microscopy system employs a multi-plane fluorescence scintillation imaging physical model, the calculation formula of which is: Ip(x,y,t)=sum_i[ei×si(t)×hp(x-xi,y-yi,zi)]+Bp(x,y,t)+Np(x,y,t)(1) In the formula, Ip(x,y,t) represents the fluorescence time series of the p-th physical plane; ei represents the brightness of the i-th fluorescent molecule; si(t) represents the normalized random scintillation time process of the i-th fluorescent molecule; hp(x-xi,y-yi,zi) represents the point spread function response of the p-th physical plane; Bp(x,y,t) represents the background term; and Np(x,y,t) represents the noise term.

[0049] Step S2012: Register the fluorescence scintillation time series of each axial physical plane to a unified reference coordinate system.

[0050] In this embodiment of the invention, after obtaining the original multi-plane time series, each physical plane is registered to a unified reference coordinate system based on the registration matrix obtained from the axial scanning calibration sample. The registration method includes one or more of translation, rotation, scaling, affine transformation, or non-rigid transformation.

[0051] The registered time series data for each physical plane can be written to the registered preprocessing cache. Subsequent spectrum analysis and SOFI reconstruction directly read the cached data, avoiding repeated data reading, plane rearrangement, and registration operations.

[0052] Step S202: Extract the mean spectral value within the effective scintillation frequency band of the fluorescence scintillation time series of each physical plane after processing, and determine the discrete spectral response of each physical plane.

[0053] Specifically, step S201 includes: Step S2021: Divide the processed fluorescence scintillation time series of each physical plane into multiple time subsequences.

[0054] In this embodiment of the invention, the fluorescence scintillation time series is divided into multiple time subsequences on each physical plane.

[0055] Step S2022: Perform time spectrum analysis and preprocessing on each time subsequence along the time dimension.

[0056] In this embodiment of the invention, time mean subtraction and time spectrum analysis are performed on each time subsequence to extract the average spectral amplitude within the effective scintillation band. The effective fluorescence scintillation band is determined based on parameters such as the scintillation dynamics characteristics of the fluorescent molecules, camera frame rate, exposure time, bleaching speed, and spectral peak value.

[0057] It should be noted that the temporal spectrum is used for defocus background identification, defocus background suppression, axial defocus evaluation, and axial fusion weight construction in 3D SOFI. Specifically, temporal spectrum analysis includes Fourier transform, filtering analysis, and spectral amplitude analysis.

[0058] Before calculating the axial defocus evaluation function, the fluorescence scintillation time series of each axial physical plane are sequentially preprocessed with time mean subtraction, background correction, bleaching correction, drift correction, intensity normalization, and lightweight pre-deconvolution.

[0059] Step S2023: Calculate the mean spectral amplitude within the effective scintillation frequency band in each preprocessed time subsequence, and average the mean spectral amplitude corresponding to multiple time subsequences of each physical plane to obtain the axial defocus evaluation function.

[0060] It should be noted that the axial defocus evaluation function refers to a two-dimensional mapping constructed from the average of the effective scintillation frequency band spectrum. The higher the value, the stronger the effective fluorescence signal in the focus of the pixel and the lower the background interference in the defocus. It is used to characterize the effective fluorescence fluctuation response that can be utilized by SOFI in each physical plane and to distinguish between the effective scintillation structure in the focus and the background in the defocus.

[0061] In this embodiment of the invention, time-spectral analysis is performed along the time dimension on the fluorescence scintillation time series of each physical plane. The mean spectral value within the effective fluorescence scintillation frequency band is extracted as the axial defocus evaluation function. That is, the spectral amplitude maps obtained from multiple time sub-sequences are averaged, weighted averaged, fused by median, or robustly fused to obtain the time-spectral defocus evaluation function (i.e., the axial defocus evaluation function) for that physical plane. The time-spectral defocus evaluation function is as follows: Dp(x,y)=mean_{f=fl~fh}|FFT_t[Cp(x,y,t)-mean_t(Cp(x,y,t))](f)|(2) In the formula, Dp(x,y) represents the temporal spectral defocus evaluation function of the p-th physical plane; Cp(x,y,t) represents the preprocessed time series of the p-th physical plane after registration, background correction, bleaching correction, intensity normalization, and lightweight pre-deconvolution; fl and fh represent the lower and upper limits of the effective scintillation band. This function is used to reflect the effective scintillation response that can be utilized by SOFI in the physical plane and to reduce the contribution of defocus background.

[0062] It is worth noting that while the out-of-focus background may appear brighter in the average intensity map, its effective blinking band response and local temporal fluctuation modulation characteristics are usually weaker than those of the in-focus structure. Time-spectrum-based evaluation of the out-of-focus background can more directly reflect the effective fluctuation signals available for SOFI, and therefore is more suitable than average intensity as a basis for 3D SOFI axial fusion.

[0063] In a specific embodiment, such as Figure 4 As shown, the results of the defocus evaluation function are compared based on simulated three-dimensional multi-plane imaging data. Figure 4 In the diagram, (a) represents the spatial coordinate distribution of the simulated three-dimensional luminescent body. Figure 4 (b) in the figure represents the evaluation response maps of different defocus evaluation functions on multiple physical planes, including temporal spectrum, average intensity, gradient sharpness, and high-frequency sharpness evaluation functions. Traditional defocus evaluation functions are mainly constructed based on the intensity, gradient, Laplacian response, or spatial high-frequency energy of a single frame image or a temporally averaged image, with the evaluation target being image spatial sharpness. In contrast, the temporal spectrum defocus evaluation function directly extracts the effective fluctuation response, which is more in line with the physical mechanism of SOFI's statistical reconstruction of fluorescence random fluctuation signals. Simulation results show that the temporal spectrum defocus evaluation function has a more concentrated axial focus response, a lower background response, and a smoother transition between adjacent physical planes, which is more in line with the physical principles of three-dimensional multi-plane imaging. Therefore, it can be used for axial defocus evaluation and fusion weight construction in multi-plane 3D-SOFI.

[0064] Step S2024: Normalize, spatially smooth, and non-negatively constrain the axial defocus evaluation function to obtain the discrete spectral response of each physical plane.

[0065] In this embodiment of the invention, after obtaining the temporal spectral defocus evaluation function for each physical plane, it is normalized to reduce the overall response differences caused by different camera channels, different optical paths, and different axial beam splitting efficiencies. The normalization can be selected from plane maximum value normalization, robust percentile normalization, and mean normalization.

[0066] The spectral response map is then smoothed in two dimensions to reduce pixel-level noise, and the minimum or lower percentile value along the physical plane at each horizontal pixel location is taken as the axial common background baseline. This baseline reflects the stable background, out-of-focus background, and non-specific spectral response that coexist on multiple physical planes.

[0067] The axial common background baseline is subtracted from the spectral response of each physical plane, and a non-negativity constraint is applied to obtain the discrete spectral response of the physical plane. This process can reduce the influence of the defocused background on the weight estimation before axial fusion; wherein, the discrete spectral response: Ap(x,y)=Dp(x,y) / (max_xy[Dp(x,y)])(3) In the formula, Ap(x,y) represents the discrete spectral response of the p-th physical plane.

[0068] Step S203: Based on the preset geometric distance from the virtual axial layer to the two adjacent physical planes, and combined with the discrete spectral responses corresponding to the two adjacent physical planes, the local axial fusion weights are obtained.

[0069] Specifically, step S203 includes: Step S2031: Construct multiple virtual axial layers based on the number of physical planes and the preset axial interpolation factor.

[0070] In this embodiment of the invention, a denser virtual axial layer is constructed based on the number of physical planes P and the axial interpolation factor a, and the number of layers is: Nz=(P-1)×a+1(4) In the formula, Nz represents the number of virtual axial layers, P represents the number of axial physical planes, and a represents the axial interpolation factor. The axial interpolation factor a is independent of the SOFI cumulant order.

[0071] The axial interpolation factor a is decoupled from the SOFI order, thus allowing for denser virtual axial sampling at lower SOFI orders.

[0072] It should be noted that the axial interpolation factor 'a' mainly improves the axial sampling density and the positioning accuracy of the virtual layer; the improvement in the true axial resolution mainly comes from the statistical shrinkage of the effective point spread function of the SOFI cumulative amount and the subsequent three-dimensional deconvolution processing.

[0073] Step S2032: Calculate the geometric distance between any virtual axial layer and two adjacent physical planes to generate the first geometric distance and the second geometric distance.

[0074] In this embodiment of the invention, for any virtual axial position, two adjacent physical planes are selected for local fusion. The physical basis for this method is that the axial sampling step size of adjacent physical planes in a multi-plane system is preferably close to or less than half the width of the axial point spread function, thus ensuring sufficient axial response overlap between adjacent physical planes.

[0075] Calculate the geometric distances from any virtual axial position of a certain virtual axial layer to two adjacent physical planes, and denote them as the first geometric distance and the second geometric distance.

[0076] Step S2033: Calculate the multiplication between the first geometric distance and the second geometric distance and the discrete spectral response of the corresponding physical plane to obtain the local axial fusion weight of the virtual axial layer.

[0077] In this embodiment of the invention, for any virtual axial position, the geometric distances from two adjacent physical planes to the virtual plane (i.e., the first geometric distance and the second geometric distance) are multiplied by the corresponding discrete spectral responses to obtain two sets of intermediate coefficients [beta0, beta1]. These two sets of intermediate coefficients are then normalized to obtain a local axial fusion weight [w0, w1] that sums to 1. The local axial fusion weight [w0, w1] is as follows: w0(x,y,zq)=beta0(x,y) / [beta0(x,y)+beta1(x,y)](5) w1(x,y,zq)=beta1(x,y) / [beta0(x,y)+beta1(x,y)](6) In the formula, beta0 and beta1 are jointly determined by the geometric distance from the virtual axial position to the adjacent physical plane multiplied by the discrete spectral response of the adjacent physical plane. beta0(x,y) is the first geometric distance multiplied by the corresponding discrete spectral response Ap(x,y); beta1(x,y) is the second geometric distance multiplied by the corresponding discrete spectral response Ap(x,y).

[0078] It is worth mentioning that the local axis fusion weight is determined by both geometric distance and discrete spectral response. When a physical plane contributes significantly to the current virtual axis position and has a strong effective scintillation spectral response, its weight increases; when a physical plane mainly provides the defocus background and has a weak effective scintillation response, its weight decreases.

[0079] Specifically, the local axial fusion weights include a first physical plane weight w0(x,y,zq) and a second physical plane weight w1(x,y,zq). The two weights are determined by multiplying the geometric interpolation coefficients with the discrete spectral response of the corresponding physical plane and are then normalized.

[0080] When the spectral response of the adjacent physical plane corresponding to the virtual axial position is lower than a preset threshold, the local axial fusion weight degenerates into a linear interpolation weight determined only by the geometric distance from the virtual axial position to the adjacent physical plane.

[0081] Step S204: The fluorescence scintillation time series of two adjacent physical planes are weighted and fused with the corresponding local axial fusion weights to obtain the fluorescence scintillation time series of the virtual axial layer.

[0082] In this embodiment of the invention, local axial fusion weights are used to weight and fuse the fluorescence scintillation time series of adjacent physical planes to form a virtual axial layer fluorescence scintillation time series, so as to suppress the contribution of the defocus background to the virtual axial layer.

[0083] Cw(x,y,t;zq)=w0(x,y,zq)×Cp0(x,y,t)+w1(x,y,zq)×Cp1(x,y,t)(7) In the formula, Cp0(x,y,t) is the time series of the first adjacent physical plane; Cp1(x,y,t) is the time series of the second adjacent physical plane.

[0084] The above equation represents the construction of a virtual axial layer fluorescence scintillation time series before calculating the SOFI cumulative amount. Because w0 and w1 are constrained by the defocus evaluation function, the contribution of physical planes with strong defocus background and weak effective scintillation response in the fusion is reduced. Since the multi-plane system synchronously acquires the scintillation sequence of the same fluorescent molecule, fluctuations in adjacent physical planes are time-dependent. Therefore, the fused Cw(x,y,t;zq) still retains the random fluctuation statistical characteristics required for SOFI.

[0085] Step S205: Calculate the cumulative amount of random optical fluctuation super-resolution imaging of fluorescence scintillation time series of each virtual axial layer layer and perform three-dimensional post-processing to generate three-dimensional random optical fluctuation super-resolution imaging reconstruction results.

[0086] Specifically, step S205 includes: Step S2051: Calculate the cumulative amount of multi-order random optical fluctuations in super-resolution imaging of the fluorescence scintillation time series of each virtual axial layer.

[0087] In this embodiment of the invention, the virtual axial layer is not obtained by simple axial interpolation of the reconstructed SOFI image, but by spectrally guided local weighted fusion of the fluorescence scintillation time series of adjacent physical planes before the calculation of SOFI accumulation (i.e., multi-order random optical fluctuation super-resolution imaging accumulation).

[0088] Because multi-plane systems synchronously acquire the blinking time process of the same fluorescent molecule, fluorescence fluctuations in adjacent physical planes exhibit temporal correlation, and their intensity differences are mainly determined by the axial point diffusion function response and the degree of defocusing. Therefore, the fused virtual axial layer time series still retains the stochastic fluctuation statistical characteristics required for SOFI.

[0089] The nth-order SOFI cumulant is calculated for the weighted fused virtual axial layer time series, which includes a weighted combination of the self-cumulants and cross-cumulants of synchronous fluorescence fluctuation signals from adjacent physical planes. This process, like the traditional three-dimensional cross-cumulant SOFI, utilizes the temporal correlation of multi-plane synchronous fluctuations.

[0090] Calculating the SOFI accumulation for the fusion time series of each virtual axial layer can statistically shrink the effective point spread function and suppress stable background, uncorrelated noise, and defocused ineffective fluctuation signals.

[0091] SOFI accumulators include second-order accumulators, third-order accumulators, fourth-order accumulators, higher-order accumulators, self-accumulators, cross-accumulators, zero-latency accumulators, non-zero-latency accumulators, or combinations thereof.

[0092] Step S2052: The multi-order random optical fluctuation super-resolution imaging cumulative quantities of multiple virtual axial layers are stacked along the axial direction to generate three-dimensional random optical fluctuation super-resolution imaging cumulative quantity data.

[0093] In this embodiment of the invention, the SOFI cumulative values ​​of multiple virtual axial layers are stacked along the axial direction to obtain three-dimensional SOFI cumulative volume data. The three-dimensional SOFI cumulative volume data is as follows: Kn(x,y,zq)=cum_n[Cw(x,y,t;zq)](8) In the formula, Kn(x,y,zq) represents the nth-order SOFI cumulative result at the virtual axial position zq.

[0094] Step S2053: Perform three-dimensional post-processing on the three-dimensional random optical fluctuation super-resolution imaging cumulative data to generate the three-dimensional random optical fluctuation super-resolution imaging reconstruction result.

[0095] In some optional implementations, step S2053 above includes: Step a1: Perform three-dimensional deconvolution processing on the cumulative data of three-dimensional random optical fluctuation super-resolution imaging.

[0096] In this embodiment of the invention, after obtaining the three-dimensional SOFI cumulative volume data, flattening correction can be performed to reduce the dynamic range imbalance caused by molecular brightness, scintillation probability and local intensity differences in high-order SOFI.

[0097] Subsequently, three-dimensional deconvolution is performed on the three-dimensional SOFI cumulative volume data. The point spread function used for deconvolution can be constructed based on system calibration, theoretical PSF, or effective PSF corresponding to the SOFI order.

[0098] It is worth mentioning that the 3D deconvolution uses an effective point spread function corresponding to the SOFI order, and performs 1 / n power linearization on the nth-order SOFI result after deconvolution to reduce the brightness nonlinearity of higher-order SOFI.

[0099] Step a2 involves linearizing the cumulative data of the three-dimensional random optical fluctuation super-resolution imaging after three-dimensional deconvolution processing to generate the reconstruction result of the three-dimensional random optical fluctuation super-resolution imaging.

[0100] In this embodiment of the invention, after deconvolution, the nth-order SOFI result is linearized to the power of 1 / n to reduce the brightness nonlinearity of higher-order SOFI and enhance the visualization of weak structures.

[0101] The final output includes 3D SOFI super-resolution volume data, z-stack, maximum intensity projection, Z-colorMIP, xz section, yz section, 3D rendering results, and quantitative analysis results.

[0102] Specifically, 3D SOFI super-resolution: Let the equivalent signal of the i-th fluorescent molecule in the virtual axial layer be expressed as: Cw,i(x,y,t;zq)=ei×si(t)×hw(x-xi,y-yi,zi;zq)(9) In the formula, hw(x,y,z;zq)=w0hp0(x,y,z)+w1hp1(x,y,z) represents the equivalent point diffusion function response of the molecule at the virtual axial position zq.

[0103] Therefore, the virtual axial layer fluorescence scintillation time series can be expressed as: Cw(x,y,t;zq)=sum_i[ei×si(t)×hw(x-xi,y-yi,zi;zq)]+Bw(x,y,t)+Nw(x,y,t) (10) Where Bw(x,y,t) represents the fused background term; Nw(x,y,t) represents the fused noise term.

[0104] Under the ideal condition that the random scintillation processes of different fluorescent molecules are independent of each other, when calculating the cumulative amount of SOFI at the nth order using equation (10), the cross terms between different molecules tend to disappear, while the cumulative amount terms of the same molecule are retained. Therefore: Kn(x,y,zq)=sum_i[ei^n×k_n(si)×hw(x-xi,y-yi,zi;zq)^n](11) In the formula, k_n(si) represents the nth-order cumulative quantity of the random scintillation process si(t) of the i-th fluorescent molecule. This formula shows that the effective point spread function of the nth-order virtual layer SOFI is equivalent to the nth power of the virtual layer point spread function, that is, the statistical contraction of the three-dimensional effective point spread function is achieved.

[0105] In a specific embodiment, such as Figure 5 The time-averaged intensity map of the wide-field fluorescence scintillation sequence of the first physical plane shown in (a) illustrates the sample fluorescence distribution under the original wide-field imaging. Figure 5 (b) shows the third-order SOFI reconstruction results of the first to third virtual axial layers obtained by axial weighted fusion using the time-spectrum defocus evaluation function.

[0106] The results show that, compared with the original wide-field average intensity map, the third-order temporal spectrum 3D-SOFI reconstructed image has higher lateral resolution and contrast. Simultaneously, different virtual axial layers exhibit discernible structural differences, indicating that this method can achieve finer axial layer reconstruction between adjacent physical planes. These results demonstrate that temporal spectrum 3D-SOFI can simultaneously improve lateral resolution and axial structural resolution, making it suitable for multi-planar three-dimensional super-resolution fluorescence imaging.

[0107] like Figure 6 As shown, a comparison of the axial color-coded 3D projection results of temporal spectrum 3D-SOFI and traditional 3D-SOFI is presented.

[0108] In the figure, different colors represent different axial depth positions, used to display the three-dimensional distribution and axial positioning results of the reconstructed structure. Traditional 3D-SOFI is easily affected by defocus background, axial crosstalk, and ineffective fluctuation signals during axial reconstruction, resulting in a diffuse axial color distribution, obvious background residue, and unclear local structural layers. In contrast, the temporal spectrum 3D-SOFI of this invention constructs axial fusion weights by extracting the temporal spectrum response within the effective scintillation band, thereby enhancing the effective fluorescence fluctuation signal within the focus range while suppressing defocus background and ineffective fluctuation signals. The results show that temporal spectrum 3D-SOFI has more accurate axial positioning, stronger defocus background suppression capability, and higher three-dimensional structure fidelity.

[0109] In application examples, this invention employs a preprocessing cache registration, concentrating raw data reading, planar rearrangement, registration, and lightweight preprocessing into the cache construction phase. It utilizes block-based spectral analysis, dividing the long-term series into multiple time blocks, calculating the spectral response for each block, and averaging or robustly fusing the results from multiple time blocks. It employs pre-calculation of fusion weights, pre-calculating adjacent physical plane indices and local fusion weights for all virtual axis positions. It uses multi-order SOFI for fast computation, calculating the cumulative values ​​of multiple orders for the same virtual layer time series at once. It employs single-precision floating-point storage, memory-mapped files, GPU parallel computing, or block matrix operations to reduce memory usage and computation time. It avoids positive stability constants involving division by zero.

[0110] This invention can be applied to multi-planar fluorescence microscopy systems, dual-camera synchronous microscopy systems, and three-dimensional imaging platforms for multi-camera synchronous microscopy systems.

[0111] This invention is applicable to super-resolution imaging of microtubules, actin, cytoskeleton, neuronal processes, mitochondria, endoplasmic reticulum, nuclear structures, organelle structures, and other three-dimensional subcellular structures.

[0112] This invention is particularly suitable for low-phototoxicity live cell three-dimensional imaging, large field-of-view three-dimensional super-resolution imaging, thick sample defocus background suppression, high background sample imaging, and high-throughput three-dimensional cell phenotypic analysis.

[0113] The purpose of this invention is to use time-spectrum modulation for defocus background evaluation and suppression. By extracting the spectral response within the effective scintillation band, an axial defocus evaluation function is constructed for each physical plane. This evaluation function is then used for virtual axial layer weighted fusion, thereby weakening the defocus background before it enters the SOFI cumulative calculation.

[0114] The beneficial effects of this invention are as follows: First, this invention uses time-frequency response as the basis for evaluating the defocus background of three-dimensional SOFI, which can more accurately distinguish between in-focus effective scintillation signals and defocus background than fluorescence intensity.

[0115] Second, before calculating the SOFI cumulative amount, the present invention uses spectrum-guided weighting to fuse the time series of adjacent physical planes, so that the defocus background is suppressed before entering the cumulative amount calculation, thereby improving the axial positioning reliability of three-dimensional SOFI.

[0116] Third, this invention decouples the construction of the axial virtual layer from the SOFI order, which can increase the virtual axial sampling density without increasing the SOFI order, thereby improving the continuity of three-dimensional volume data and reducing the dependence on high-order SOFI.

[0117] Fourth, this invention does not perform simple axial interpolation on the reconstructed SOFI image, but instead constructs a virtual axial layer time series with statistical significance of time fluctuations before calculating the SOFI cumulative amount, which is more in line with the physical nature of SOFI imaging.

[0118] Fifth, this invention reduces the computational load and memory consumption of multi-plane 3D SOFI by registering a preprocessing cache, pre-calculating the spectral response, and pre-calculating the axial fusion weights.

[0119] This embodiment also provides a high-fidelity three-dimensional random optical fluctuation super-resolution imaging device, which is used to implement the above embodiments and preferred embodiments, and will not be repeated as already described. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0120] This embodiment provides a high-fidelity three-dimensional random optical fluctuation super-resolution imaging device, such as... Figure 7 As shown, it includes: The acquisition module 301 is used to acquire fluorescence scintillation time series of multiple axial physical planes and perform coordinate system registration processing; Extraction module 302 is used to extract the mean spectral value within the effective scintillation frequency band of the fluorescence scintillation time series of each physical plane after processing, and to determine the discrete spectral response of each physical plane; Combined with module 303, it is used to obtain local axial fusion weights based on the preset geometric distance from the virtual axial layer to two adjacent physical planes and the discrete spectral responses corresponding to the two adjacent physical planes. The weighted fusion processing module 304 is used to perform weighted fusion processing on the fluorescence scintillation time series of two adjacent physical planes and the corresponding local axial fusion weights to obtain the fluorescence scintillation time series of the virtual axial layer. The calculation module 305 is used to calculate the cumulative amount of random optical fluctuation super-resolution imaging of fluorescence scintillation time series of each virtual axial layer layer by layer and perform three-dimensional post-processing to generate three-dimensional random optical fluctuation super-resolution imaging reconstruction results.

[0121] In some optional implementations, the acquisition module 301 includes: The acquisition unit is used to acquire fluorescence scintillation time series of multiple axial physical planes through a prism-splitting multi-plane synchronous imaging system. The registration unit is used to register the fluorescence scintillation time series of physical planes along each axis to a unified reference coordinate system.

[0122] In some alternative implementations, the extraction module 302 includes: The partitioning unit is used to divide the processed fluorescence scintillation time series of each physical plane into multiple time subsequences; The preprocessing unit is used to perform time spectrum analysis and preprocess each time subsequence along the time dimension; The averaging unit is used to calculate the average spectral amplitude within the effective scintillation frequency band in each time subsequence after preprocessing, and to average the average spectral amplitude corresponding to multiple time subsequences of each physical plane to obtain the axial defocus evaluation function. The processing unit is used to normalize, spatially smooth, and perform non-negative constraint processing on the axial defocus evaluation function to obtain the discrete spectral response of each physical plane.

[0123] In some alternative implementations, the incorporation module 303 includes: The building unit is used to construct multiple virtual axial layers based on the number of physical planes and a preset axial interpolation factor; The first calculation unit is used to calculate the geometric distance between any virtual axial layer and two adjacent physical planes, and generate the first geometric distance and the second geometric distance; The second calculation unit is used to calculate the multiplication between the first geometric distance and the second geometric distance and the discrete spectral response of the corresponding physical plane, respectively, to obtain the local axial fusion weight of the virtual axial layer.

[0124] In some alternative implementations, the computing module 305 includes: The cumulant calculation unit is used to calculate the multi-order random optical fluctuation super-resolution imaging cumulant of the fluorescence scintillation time series for each virtual axial layer; The stacking processing unit is used to stack and process the multi-order random optical fluctuation super-resolution imaging cumulative quantities of multiple virtual axial layers along the axis to generate three-dimensional random optical fluctuation super-resolution imaging cumulative quantity data. The three-dimensional post-processing unit is used to perform three-dimensional post-processing on the cumulative data of three-dimensional random optical fluctuation super-resolution imaging to generate the reconstruction results of three-dimensional random optical fluctuation super-resolution imaging.

[0125] In some alternative implementations, the 3D post-processing unit includes: The three-dimensional deconvolution processing subunit is used to perform three-dimensional deconvolution processing on the cumulative data of three-dimensional random optical fluctuation super-resolution imaging. The linearization processing subunit is used to linearize the cumulative data of 3D random optical fluctuation super-resolution imaging after 3D deconvolution processing, and generate the 3D random optical fluctuation super-resolution imaging reconstruction result.

[0126] The high-fidelity three-dimensional random optical fluctuation super-resolution imaging device provided in this embodiment of the invention can execute the high-fidelity three-dimensional random optical fluctuation super-resolution imaging method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the above modules and units are the same as in the corresponding embodiments described above, and will not be repeated here.

[0127] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0128] The following is a detailed reference. Figure 8 This diagram illustrates a structural schematic suitable for implementing an electronic device according to embodiments of the present invention. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 401, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 402 or a program loaded from memory 408 into random access memory (RAM) 403. The RAM 403 also stores various programs and data required for the operation of the electronic device. The processor 401, ROM 402, and RAM 403 are interconnected via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0129] Typically, the following devices can be connected to I / O interface 405: input devices 406 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 407 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 408 including, for example, magnetic tapes, hard disks, etc.; and communication devices 409. Communication device 409 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 8 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0130] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 409, or installed from a memory 408, or installed from a ROM 402. When the computer program is executed by the processor 401, it performs the functions defined in the high-fidelity three-dimensional random optical fluctuation super-resolution imaging method of the embodiments of the present invention.

[0131] Figure 8The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0132] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the high-fidelity three-dimensional random optical fluctuation super-resolution imaging method shown in the above embodiments is implemented.

[0133] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0134] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A high-fidelity three-dimensional random optical fluctuation super-resolution imaging method, characterized in that, The method includes: Obtain fluorescence scintillation time series of multiple axial physical planes and perform coordinate system registration; The mean spectral value within the effective scintillation frequency band of the fluorescence scintillation time series of each physical plane is extracted to determine the discrete spectral response of each physical plane. Based on the preset geometric distance from the virtual axial layer to the two adjacent physical planes, and combined with the discrete spectral responses corresponding to the two adjacent physical planes, the local axial fusion weights are obtained. The fluorescence scintillation time series of two adjacent physical planes are weighted and fused with the corresponding local axial fusion weights to obtain the fluorescence scintillation time series of the virtual axial layer. The cumulative amount of random optical fluctuation super-resolution imaging of the fluorescence scintillation time series of each virtual axial layer is calculated layer by layer and three-dimensional post-processing is performed to generate three-dimensional random optical fluctuation super-resolution imaging reconstruction results.

2. The method according to claim 1, characterized in that, The process of acquiring fluorescence scintillation time series of multiple axial physical planes and performing coordinate system registration includes: The fluorescence scintillation time series of multiple axial physical planes were obtained using a prism-based multi-plane synchronous imaging system. The fluorescence scintillation time series of each of the aforementioned axial physical planes are registered to a unified reference coordinate system.

3. The method according to claim 1, characterized in that, The extraction and processing of the average spectral value within the effective scintillation frequency band of the fluorescence scintillation time series of each physical plane, and the determination of the discrete spectral response of each physical plane, include: The processed fluorescence scintillation time series of each physical plane is divided into multiple time subsequences; Each of the aforementioned time subsequences is subjected to time spectrum analysis along the time dimension and preprocessed. The mean spectral amplitude within the effective scintillation band is calculated for each time subsequence after preprocessing, and the mean spectral amplitude corresponding to multiple time subsequences for each physical plane is averaged to obtain the axial defocus evaluation function. The axial defocus evaluation function is normalized, spatially smoothed, and subjected to non-negative constraints to obtain the discrete spectral response of each physical plane.

4. The method according to claim 1, characterized in that, The step of obtaining local axial fusion weights based on the preset geometric distances from the virtual axial layer to two adjacent physical planes, combined with the discrete spectral responses corresponding to the two adjacent physical planes, includes: Based on the number of physical planes and the preset axial interpolation factor, multiple virtual axial layers are constructed; Calculate the geometric distance between any virtual axial layer and two adjacent physical planes, and generate the first geometric distance and the second geometric distance; The local axial fusion weights of the virtual axial layer are obtained by calculating the multiplication values ​​between the first geometric distance, the second geometric distance and the discrete spectral response of the corresponding physical plane.

5. The method according to claim 1, characterized in that, The process of calculating the cumulative amount of random optical fluctuation super-resolution imaging of the fluorescence scintillation time series of each of the virtual axial layers layer by layer and performing three-dimensional post-processing to generate a three-dimensional random optical fluctuation super-resolution imaging reconstruction result includes: Calculate the super-resolution imaging accumulation of multi-order random optical fluctuations in the fluorescence scintillation time series of each of the virtual axial layers; The multi-order random optical fluctuation super-resolution imaging cumulative quantities of multiple virtual axial layers are stacked along the axial direction to generate three-dimensional random optical fluctuation super-resolution imaging cumulative quantity data. The accumulated data of the three-dimensional random optical fluctuation super-resolution imaging are subjected to three-dimensional post-processing to generate the reconstruction result of the three-dimensional random optical fluctuation super-resolution imaging.

6. The method according to claim 5, characterized in that, The step of performing three-dimensional post-processing on the accumulated data of the three-dimensional random optical fluctuation super-resolution imaging to generate the three-dimensional random optical fluctuation super-resolution imaging reconstruction result includes: The three-dimensional random optical fluctuation super-resolution imaging cumulative data is subjected to three-dimensional deconvolution processing; The cumulative data of 3D random optical fluctuation super-resolution imaging after 3D deconvolution processing is linearized to generate the reconstruction result of 3D random optical fluctuation super-resolution imaging.

7. A high-fidelity three-dimensional random optical fluctuation super-resolution imaging device, characterized in that, The device includes: The acquisition module is used to acquire fluorescence scintillation time series of multiple axial physical planes and perform coordinate system registration processing; The extraction module is used to extract the mean spectral value within the effective scintillation frequency band of the fluorescence scintillation time series of each physical plane after processing, and to determine the discrete spectral response of each physical plane. The module is used to obtain local axial fusion weights based on the preset geometric distance from the virtual axial layer to two adjacent physical planes and the discrete spectral responses corresponding to the two adjacent physical planes. The weighted fusion processing module is used to perform weighted fusion processing on the fluorescence scintillation time series of two adjacent physical planes and the corresponding local axial fusion weights to obtain the fluorescence scintillation time series of the virtual axial layer. The calculation module is used to calculate the cumulative amount of random optical fluctuation super-resolution imaging of the fluorescence scintillation time series of each of the virtual axial layers layer by layer and perform three-dimensional post-processing to generate three-dimensional random optical fluctuation super-resolution imaging reconstruction results.

8. An electronic device, characterized in that, include: A memory and a processor are interconnected, the memory stores computer instructions, and the processor executes the computer instructions to perform the high-fidelity three-dimensional random optical fluctuation super-resolution imaging method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the high-fidelity three-dimensional random optical fluctuation super-resolution imaging method according to any one of claims 1 to 6.

10. A computer program product, characterized in that, Includes computer instructions for causing a computer to perform the high-fidelity three-dimensional random optical fluctuation super-resolution imaging method according to any one of claims 1 to 6.

Images