A method and system for long-term drift-free super-resolution fluorescence imaging of live cells

Abstract

The application belongs to the technical field of biomedical imaging, and provides a long-term live cell super-resolution fluorescence imaging method and system without drift, comprising: collecting live cell single molecule positioning imaging frame sequence, exogenous fluorescence reference marker displacement trajectory and objective stage feedback displacement data, and constructing double-channel space-time displacement data; based on the rigid body translation assumption between frames, performing incremental difference operation on the double-channel space-time displacement data set, and constructing drift deviation evolution representation; taking the drift deviation evolution representation as a unified criterion, dividing the imaging frame segment into rigid body co-moving candidate frame segment and non-rigid body disturbance distortion frame segment; combining the drift path constraint interpolation of adjacent stable frame segments, reconstructing the drift reference trajectory; correcting the imaging frame space coordinates according to the drift reference trajectory, updating the drift deviation evolution representation and the frame segment determination boundary.

Description

Technical Field

[0001] This invention belongs to the field of biomedical imaging technology, specifically a method and system for long-term drift-free super-resolution fluorescence imaging of live cells. Background Technology

[0002] In long-term super-resolution fluorescence imaging of live cells, for the observation of single-molecule localization of membrane proteins in the dendritic synaptic region of adherent neurons, existing techniques generally adopt the method of introducing exogenous fluorescent microspheres as inherent reference markers. The stage drift is corrected by tracking the position change of the microspheres in real time, so as to compensate for the overall displacement error of the imaging system and maintain nanometer-level spatial positioning accuracy.

[0003] However, because the synaptic region is a special imaging micro-region with limited local space, its cell membrane exhibits dynamic fluctuations and its microenvironment is constantly changing. Fluorescent microspheres attached to or near the membrane are prone to superimposed subpixel-level local positional fluctuations independent of the rigid translation of the system. During long-term continuous imaging, the biological perturbations of these microspheres are coupled with the actual physical drift signal of the imaging system, making it difficult for traditional drift correction algorithms to effectively identify them. In actual registration operations, these positional fluctuations are easily misjudged as system drift and reversed for compensation. Moreover, this error cannot be automatically eliminated in long sequences and will gradually accumulate with the number of imaging frames, eventually causing a persistent slight misalignment of the spatial structure of the reconstructed image, which seriously restricts the reconstruction accuracy and reliability of single-molecule localization of synaptic membrane proteins. Summary of the Invention

[0004] In order to overcome the shortcomings of the prior art, at least one technical problem raised in the background art is solved.

[0005] The technical solution adopted by this invention to solve its technical problem is: to acquire live cell single-molecule localization imaging frame sequence, external fluorescent reference marker displacement trajectory and stage feedback displacement data, and to perform temporal registration of the reference marker displacement trajectory and stage feedback displacement data based on time axis consistency constraints to construct dual-channel spatiotemporal displacement data; Based on the assumption of rigid body translation between frames, incremental difference operation is performed on the dual-channel spatiotemporal displacement dataset to calculate the offset residual data of the fluorescent reference marker relative to the stage frame by frame, and the time dimension is recursively accumulated to construct a drift deviation evolution characterization quantity that quantifies the degree of rigid body separation of the reference marker. Using the drift deviation evolution characterization quantity as a unified criterion, the continuous state of residual propagation within the imaging window is determined. Based on the residual stability or abrupt divergence characteristics, the imaging frame segment is divided into rigid body co-motion candidate frame segment and non-rigid body perturbation distortion frame segment. The candidate frames of rigid body co-motion are time-aligned and stitched together, and a global coordinate regression mapping relationship is constructed based on the stage feedback displacement. The global aligned time displacement trajectory is output after unified coordinate system transformation. The failed reference marker displacement is removed from the non-rigid body disturbance distortion frame segments. Combined with the drift path constraint interpolation of adjacent stable frame segments, the missing segments of the global aligned time displacement trajectory in the distortion interval are completed, and a continuous drift reference trajectory covering the entire time series is generated. The spatial coordinates of the imaging frames are corrected frame by frame based on the drift reference trajectory, and the correction results are fed back to the residual recursive calculation process to update the drift deviation evolution characterization and frame segment determination boundary.

[0006] As a further aspect of the present invention: The process of constructing dual-channel spatiotemporal displacement data is as follows: The frame-by-frame timestamp information of the live cell single-molecule localization imaging frame sequence was obtained, and the sampling time sequence of the displacement trajectory of the exogenous fluorescent reference marker and the stage feedback displacement data was extracted simultaneously. Time alignment processing is performed on data from different sources based on timestamps to construct a multi-source displacement mapping relationship under a unified time benchmark; Calculate the consistency error between the displacement trajectory of the reference marker and the displacement data fed back by the stage at the corresponding time node, and correct the timing alignment relationship based on the principle of minimizing the error; The aligned reference marker displacement data and the stage feedback displacement data are fused and stored according to a frame-by-frame correspondence to form dual-channel spatiotemporal displacement data for drift analysis.

[0007] The process of calculating the offset residual data of the fluorescence reference mark relative to the stage frame by frame is as follows: Read the reference marker displacement and stage displacement information corresponding to adjacent imaging frames in the dual-channel spatiotemporal displacement data; The theoretically consistent displacement increment is calculated based on the inter-frame rigid body translation constraint and used as the reference datum increment. The frame-by-frame difference between the actual observed displacement increment and the theoretical reference increment is calculated to obtain the offset residual of the reference mark relative to the stage; The offset residuals are updated using a sliding frame window to form continuous time series residual data.

[0008] The process of constructing a drift deviation evolution characterization of the degree of rigid body detachment from the quantified reference marker is as follows: Based on frame-by-frame offset residual data, and using the initial imaging frame as the recursive starting boundary condition, the residual recursive initial state domain is constructed. The residual of the current frame is weighted and recursively calculated with the cumulative residual of the previous frame to form a time-continuous residual evolution sequence; The drift deviation evolution characterization is constructed based on the residual growth rate and the trend of cumulative amplitude change.

[0009] The process of determining the continuous state of residual propagation within the imaging window is as follows: Extract the drift deviation evolution characteristics within a continuous time window, analyze the continuous change characteristics of the evolution quantity within different time windows, and calculate the local fluctuation consistency index. When the evolution quantity maintains stable growth or stable convergence in the time dimension, it is determined that the residual propagation is in a continuous state. When the evolution quantity exhibits discontinuous jumps or non-stationary fluctuations, it is determined that the residual propagation has entered an abnormal disturbance state.

[0010] The process of dividing imaging frames into rigid body motion candidate frames and non-rigid body perturbation distortion frames is as follows: The threshold interval for frame segmentation is set based on the slope and fluctuation amplitude of the drift deviation evolution characterization quantity. Sliding window analysis is performed on continuous imaging frames to identify the set of frames where the residual changes are in a stable range; When the residual change satisfies the stability constraint, the corresponding frame segment is marked as a candidate frame segment for rigid body co-motion; When the residual change exceeds the threshold and exhibits abrupt divergence characteristics, the corresponding frame segment is marked as a non-rigid body perturbation distortion frame segment.

[0011] The process of outputting the globally aligned temporal displacement trajectory after a unified coordinate system transformation is as follows: The displacements of the reference markers and the stage within the candidate frames of rigid body co-motion are sorted according to their temporal consistency. Multiple frames of stable displacement data are continuously stitched together in chronological order to form a global time-series trajectory; A global coordinate regression mapping relationship is constructed based on the global temporal trajectory and the stage feedback displacement, and the frame-by-frame coordinate regression mapping parameters are calculated. By using regression mapping parameters to perform a unified coordinate system transformation on the spliced ​​frame segments, a globally aligned temporal displacement trajectory is output.

[0012] The process of generating a continuous drift reference trajectory covering the entire time series is as follows: The reference marker displacement data in the non-rigid body disturbance distortion frame segment is filtered for validity, and abnormal displacement observation points with offset residuals exceeding the preset stability threshold are removed to form an effective observation sequence of the distortion interval. Based on the drift trajectories corresponding to the candidate rigid body co-motion segments before and after the distortion interval, the spatial displacement state quantities and temporal continuity constraint parameters of the boundary frames are extracted to construct the set of drift constraint boundary conditions for the distortion interval. Using the continuous change trend of the stage feedback displacement as a global prior constraint, and integrating effective observation sequence information, a piecewise continuity constraint function is constructed under the time axis consistency constraint. Trajectory reconstruction interpolation is performed on the distortion interval to generate a complete displacement sequence that satisfies the rigid body translation consistency and displacement smoothness constraints. The completed displacement sequence is spliced ​​and fused with the global aligned temporal displacement trajectory of the preceding and following stable frames in time index order to form a continuous and unbroken global temporal drift trajectory. A drift consistency check is performed on the stitched continuous trajectory. The check is based on the continuity constraint of the displacement change rate of adjacent frames and the displacement gradient matching error of the boundary frames. The displacement change trend of the complete interval is made to maintain a continuous transition with the adjacent stable frame segments, thereby generating a continuous drift reference trajectory covering the entire time series.

[0013] The process of updating the drift deviation evolutionary representation and frame determination boundary is as follows: Based on the frame-by-frame spatial displacement correction parameters corresponding to the drift reference trajectory, the imaging frames are transformed frame by frame. The coordinate transformation result is used as a recursive input to the residual recursive calculation process and participates in the frame-by-frame offset residual update calculation. The drift deviation evolution characterization and frame segmentation rules are adjusted based on the updated residual sequence.

[0014] A live-cell long-term drift-free super-resolution fluorescence imaging system includes: Dual-source registration module: Performs time-axis consistency constraint registration on imaging frame sequence, reference marker displacement and stage feedback displacement to construct dual-channel spatiotemporal displacement data; The residual recursion module performs incremental difference and frame-by-frame recursive calculations on the dual-channel displacement data based on the inter-frame rigid body translation assumption to generate the offset residual of the reference mark relative to the stage. Drift determination module: Performs time-dimensional recursive cumulative calculation on the residual sequence to form a drift deviation evolution characterization quantity, and performs rigid body co-motion and non-rigid body perturbation frame segment determination based on continuity and abrupt change characteristics; Trajectory Reconstruction Module: Performs global coordinate regression stitching on rigid body co-motion frames and performs path constraint interpolation to complete distorted frames, generating a continuous drift reference trajectory; Correction and update module: Based on the drift reference trajectory, the imaging spatial coordinates are corrected frame by frame, and the correction results are fed back to the residual recursion process to update the drift deviation evolution characterization and frame segment determination boundary.

[0015] The beneficial effects of this invention are as follows: By constructing a dual-channel spatiotemporal consistency modeling system based on single-molecule localization imaging frame sequences, external fluorescent reference marker displacement, and stage feedback displacement, a unified representation and high-precision temporal alignment of multi-source displacement information were achieved, effectively improving the resolvability and traceability of drift information during long-term live-cell imaging. Furthermore, by introducing an incremental difference and recursive accumulation mechanism based on the inter-frame rigid body assumption, a continuous quantitative expression of the reference marker's offset relative to the stage was formed, thus constructing a drift deviation evolutionary characterization quantity. This enables the system to amplify and structurally characterize minute drift perturbations, improving the sensitivity and stability of drift identification. Simultaneously, based on this evolutionary quantity, the continuity and abrupt changes of the residual propagation state were discriminated, thereby adaptively dividing the imaging frame sequence into rigid body co-motion frame segments. By separating non-rigid perturbation frames, local distortion areas in complex imaging processes are effectively isolated, avoiding cumulative errors caused by erroneous drift information in the overall reconstruction. On this basis, by integrating the global displacement constraint of the stage with the drift path interpolation mechanism of the neighborhood stable frames, the failed reference marker information is eliminated and compensated, achieving highly consistent reconstruction of the drift reference trajectory, thereby significantly improving the spatial continuity and trajectory integrity of long-term imaging. Finally, by feeding back the reconstructed drift reference trajectory to the frame-by-frame coordinate correction and residual recursive closed-loop update process, the drift error is dynamically suppressed and continuously corrected, enabling the imaging system to maintain nanometer-level spatial stability and super-resolution imaging accuracy under long-term observation conditions, thus improving the reliability and reconstruction accuracy of live cell dynamic observation. Attached Figure Description

[0016] The invention will now be further described with reference to the accompanying drawings.

[0017] Figure 1 This is a flowchart of the steps of a long-term drift-free super-resolution fluorescence imaging method for live cells as described in Embodiment 1 of the present invention; Figure 2 This is a flowchart of a live-cell long-term drift-free super-resolution fluorescence imaging system as described in Embodiment 2 of the present invention. Detailed Implementation

[0018] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with specific embodiments.

[0019] Example 1: Please refer to Figure 1 As shown in the embodiment of the present invention, a method for long-term drift-free super-resolution fluorescence imaging of live cells includes the following steps: Step S10: Collect live cell single-molecule localization imaging frame sequences, exogenous fluorescent reference marker displacement trajectories, and stage feedback displacement data. Based on time axis consistency constraints, perform temporal registration of the reference marker displacement trajectory and stage feedback displacement data to construct dual-channel spatiotemporal displacement data.

[0020] In step S10, the process of constructing dual-channel spatiotemporal displacement data is as follows: The system acquires frame-by-frame timestamp information of live-cell single-molecule localization imaging frame sequences and simultaneously extracts the sampling time series of the displacement trajectory of the exogenous fluorescence reference marker and the stage feedback displacement data. The single-molecule localization imaging frame sequences are marked frame-by-frame using the frame trigger signal of the imaging system, recording the acquisition timestamp corresponding to each frame to form a time index sequence. Simultaneously, the spatial displacement trajectory of the reference marker during continuous observation is obtained using an exogenous fluorescence reference marker tracking algorithm, and the feedback displacement data output by the stage control system is read synchronously and recorded as time series data according to the sampling period. A unified clock source, such as the system clock or a hardware synchronization trigger signal, is used to unify the time reference for the three types of data, ensuring that different data sources have comparable time identifiers at the acquisition level, providing a basic data structure for subsequent time series alignment.

[0021] Based on timestamps, time alignment processing is performed on data from different sources to construct a multi-source displacement mapping relationship under a unified time reference. The frame-by-frame timestamps of the single-molecule localization imaging frame sequence are used as the main time axis reference, and the displacement trajectory of the reference marker and the stage feedback displacement data are mapped to this time axis respectively. For cases where the sampling frequency is inconsistent or the time points are not completely matched, linear interpolation or spline interpolation methods are used to resample the displacement of the reference marker and the stage, so that they have corresponding displacement values ​​at each imaging frame time node. By establishing a ternary mapping relationship of "timestamp - reference marker displacement - stage displacement", the aligned expression of multi-source data under a unified time reference is realized.

[0022] The offset consistency error between the reference marker displacement trajectory and the stage feedback displacement data at corresponding time nodes is calculated, and the temporal alignment relationship is corrected based on the principle of minimizing error. After completing the initial time alignment, the difference vector between the reference marker displacement and the stage feedback displacement is calculated frame by frame, and Euclidean distance or per-axis component error is used as the offset consistency error index. Sliding statistical analysis is performed on the error sequence within the continuous time window to identify the intervals of sudden error increase or drift offset accumulation. The time alignment relationship is further optimized and adjusted by least squares fitting or dynamic time warping methods to minimize the overall error of the correspondence between the reference marker trajectory and the stage displacement in the time dimension, thereby correcting the alignment deviation caused by sampling delay or time drift.

[0023] The aligned reference marker displacement data and the stage feedback displacement data are fused and stored according to a frame-by-frame correspondence to form dual-channel spatiotemporal displacement data for drift analysis. The reference marker displacement and stage feedback displacement, after time alignment and error correction, are bound one-to-one according to frame number to construct a frame-by-frame dual-channel data structure, where each frame contains the reference marker spatial displacement vector and the stage feedback displacement vector. The dual-channel data is stored using a structured data storage method, such as matrix form or time series database, and the frame timestamp is retained as an index field to support subsequent drift residual calculation and time-series recursive analysis. Finally, a dual-channel spatiotemporal displacement dataset is formed that can be directly used for drift deviation modeling and rigid body co-motion determination under a unified time axis constraint.

[0024] Step S20: Based on the assumption of rigid body translation between frames, perform incremental difference operation on the dual-channel spatiotemporal displacement dataset, calculate the offset residual data of the fluorescent reference marker relative to the stage frame by frame, and perform time-dimensional recursive cumulative calculation to construct a drift deviation evolution characterization quantity that quantifies the degree of rigid body separation of the reference marker.

[0025] In step S20, the process of calculating the offset residual data of the fluorescence reference mark relative to the stage frame by frame is as follows: The system reads the reference marker displacement and stage displacement information corresponding to adjacent imaging frames from the dual-channel spatiotemporal displacement data; extracts the reference marker displacement information and stage feedback displacement information corresponding to adjacent imaging frames from the dual-channel spatiotemporal displacement data in chronological order, and establishes a frame-by-frame pairing relationship using the frame order as an index; for cases where there are inconsistent sampling times or missing local frames, linear interpolation based on adjacent time nodes is used to fill in the displacement data, so that the reference marker displacement and stage displacement form a corresponding relationship at the same imaging frame time node, thereby obtaining continuous and consistent dual-channel input data.

[0026] The theoretically consistent displacement increment is calculated based on the inter-frame rigid body translation constraint and used as the reference increment. The displacement change of the stage between adjacent imaging frames is used as the overall rigid body motion input. The stage displacement is mapped to the spatial coordinate system of the reference mark through the spatial calibration relationship, and this is used as the theoretical displacement change that the reference mark should have under the ideal rigid body co-motion condition. The spatial calibration relationship is obtained by pre-calibration of the imaging system and is used to unify the correspondence between the stage coordinate system and the imaging spatial coordinate system, thereby ensuring that the theoretical displacement increment is expressed under the same spatial reference.

[0027] The frame-by-frame difference between the actual observed displacement increment and the theoretical reference increment is calculated to obtain the offset residual of the reference marker relative to the stage. The actual displacement change of the reference marker between adjacent imaging frames is calculated frame by frame and compared frame by frame with the theoretical displacement change derived from the stage and converted by spatial calibration. The difference value is calculated in each spatial direction and the difference is uniformly represented in the form of Euclidean distance to obtain the offset residual value corresponding to each frame, which is used to characterize the degree of deviation of the reference marker motion from the rigid body translation model.

[0028] The offset residuals are updated using a sliding frame window to form continuous time-series residual data. The frame-by-frame offset residuals are segmented using a fixed number of frames as the time window. Within each time window, the residuals are filtered using a linear smoothing method to reduce the impact of single-frame positioning errors and random fluctuations caused by fluorescence flicker. At the same time, the time window is updated frame by frame along the imaging frame sequence, and the residual at the end of the previous window is retained as the initial state of the current window to achieve continuous connection in time, thereby forming offset residual time-series data with continuity and stability.

[0029] In step S20, the process of constructing the drift deviation evolution characterization quantity of the quantification reference marker rigid body detachment degree is as follows: Based on frame-by-frame offset residual data, and using the initial imaging frame as the recursive starting boundary condition, an initial state domain for residual recursion is constructed. The residual data corresponding to the first frame is read from the offset residual sequence in chronological order as the initial state reference, and is used as the starting boundary condition for subsequent recursive calculations. At the same time, an initial state data structure containing time index, residual amplitude, and spatial direction components is established to record the distribution characteristics of the residual under the initial imaging frame. In the implementation process, the initial frame is regarded as the reference starting point of the ideal state of rigid body co-motion of the system, so that all subsequent residual evolutions are based on this state for cumulative deviation analysis, thereby constructing an initial state domain for residual recursion with time starting point constraints.

[0030] The residuals of the current frame and the cumulative residuals of the previous frame are weighted and recursively calculated to form a time-continuous residual evolution sequence. A time recursion mechanism is introduced in the frame-by-frame calculation process to fuse the offset residuals of the current frame and the cumulative residuals of the previous frame. The historical residuals are given decreasing weights, while the current frame residuals are given higher weights to reflect the dominant role of recent changes in the system state during the time evolution process. Through the frame-by-frame recursive update method, the residuals form a continuous cumulative evolution process on the time axis, and short-term fluctuations are smoothed and constrained, thereby obtaining a residual evolution sequence that can reflect the gradual deviation process of the rigid body relationship between the reference mark and the stage.

[0031] A drift deviation evolution characterization quantity is constructed based on the residual growth rate and cumulative amplitude change trend. The residual evolution sequence is analyzed in the time dimension to extract the residual change amplitude between adjacent time frames to characterize the residual growth rate, and the cumulative amplitude change trend of the residual within a certain time window is statistically analyzed. By jointly analyzing the growth rate and cumulative amplitude, the drift deviation characterization weight is increased when the residual shows a continuous upward or accelerated expansion trend, and the weight is reduced when the residual tends to stabilize or the fluctuation weakens. Thus, a drift deviation evolution characterization quantity that can quantitatively describe the degree of separation of the reference mark from the rigid body co-motion state is constructed, providing a unified judgment basis for subsequent frame segmentation and abnormal disturbance identification.

[0032] Step S30: Using the drift deviation evolution characterization quantity as a unified criterion, determine the continuous state of residual propagation within the imaging window. Based on the residual stability or abrupt divergence characteristics, divide the imaging frame segment into rigid body co-motion candidate frame segment and non-rigid body perturbation distortion frame segment.

[0033] In step S30, the process of determining the continuous state of residual propagation within the imaging window is as follows: The drift deviation evolution characteristics within continuous time windows are extracted, and the continuous change characteristics of the evolution characteristics within different time windows are analyzed. A local fluctuation consistency index is calculated. Within a sliding time window, a drift deviation evolution characteristics sequence is extracted at a fixed frame length, and the evolution characteristics between adjacent time windows are aligned and compared. By calculating the consistency of change amplitude, change direction, and fluctuation amplitude distribution characteristics between adjacent frames, a local fluctuation consistency index is constructed to characterize the stability of the evolution characteristics within a short time scale. When the evolution trend between different time windows maintains the same change direction and the fluctuation amplitude is within a preset stable range, the residual propagation within that window is considered to have a continuous basis.

[0034] When the evolution quantity maintains a stable increase or stable convergence in the time dimension, the residual propagation is determined to be in a continuous state. Trend analysis is performed on the drift deviation evolution characteristic quantity within the continuous time window. By comparing the evolution level changes between adjacent windows, it is determined whether it shows a stable change pattern of continuous slow increase or gradual convergence. When the change of the evolution quantity maintains a unidirectional consistent change trend in multiple continuous windows, and the local fluctuations do not show significant reversal or abnormal jitter, this state is determined to be a continuous state of residual propagation, indicating that the rigid body co-motion relationship between the reference mark and the stage is still in a stable evolution process.

[0035] When the evolution quantity exhibits discontinuous jumps or non-stationary fluctuations, the residual propagation is determined to have entered an abnormal perturbation state. The drift deviation evolution characteristic quantity is continuously monitored window by window. When a sudden increase, sudden decrease, or directional reversal is detected between adjacent windows, it is determined to be a discontinuous jump. At the same time, if there is an increase in local high-frequency fluctuations, a significant increase in amplitude distribution dispersion, or a continuous decrease in stability index within the window, it is determined to be a non-stationary fluctuation state. When any of the above abnormal features meets the preset judgment conditions, the corresponding time window is marked as an abnormal perturbation state for subsequent non-rigid frame segment identification and removal processing.

[0036] In step S30, the process of dividing the imaging frame segment into rigid body co-motion candidate frame segments and non-rigid body perturbation distortion frame segments is as follows: The frame segmentation threshold interval is set based on the change slope and fluctuation amplitude of the drift deviation evolution characterization quantity; time series analysis is performed on the drift deviation evolution characterization quantity, and the change slope characteristics are obtained by calculating the change amplitude between adjacent imaging frames, and a stability description index is constructed by combining the fluctuation amplitude statistical results within a certain time window; the threshold interval is obtained by statistical analysis of historical stable imaging data and disturbed imaging data, and is used to distinguish the slow change range under normal rigid body co-motion state and the rapid fluctuation range under non-rigid body disturbance state, thereby forming a dynamic discrimination boundary for frame segmentation.

[0037] A sliding window analysis is performed on continuous imaging frames to identify the set of frames where the residual changes are in a stable range. The continuous imaging frames are processed segment by segment with a fixed frame length as the time analysis window. The local change trend of the drift deviation evolution characteristic quantity is extracted in each window, and the stability of the data in the window is judged. When the data change in the window shows low amplitude fluctuation, consistent change direction and no abrupt change characteristics, the set of frames corresponding to the window is marked as a candidate stable frame set, and its continuous distribution range on the time axis is recorded to provide basic data units for subsequent rigid body co-motion frame segment identification.

[0038] When the residual change satisfies the stability constraint, the corresponding frame segment is marked as a candidate frame segment for rigid body co-motion. The set of stable frames identified by the sliding window is continuously merged. When there is no obvious discontinuity between adjacent stable windows and their drift deviation evolution characterization is always maintained within the preset stable range, the entire continuous frame interval is marked as a candidate frame segment for rigid body co-motion. The stability constraint includes the change amplitude not exceeding the upper limit of the threshold interval, no sudden jump in local fluctuations, and consistent trend direction, so as to ensure that the divided frame segment corresponds to the time interval in which the reference mark and the stage are in a consistent motion relationship.

[0039] When the residual change exceeds the threshold and exhibits abrupt divergence, the corresponding frame segment is marked as a non-rigid perturbation distortion frame segment. The drift deviation evolution characterization quantity within the continuous time window is monitored in real time. When its change amplitude exceeds the upper limit of the preset threshold range and exhibits abrupt characteristics such as rapid rise, fall, or direction reversal in a short period of time, the frame segment corresponding to the time window is determined to be a non-rigid perturbation distortion frame segment. At the same time, combined with the significant increase in fluctuation amplitude and the continuity interruption characteristics within the window, the frame segment is further confirmed and marked for subsequent failure reference mark removal and drift baseline trajectory reconstruction.

[0040] Step S40: Perform temporal alignment and stitching on the candidate rigid body co-motion frames, and construct a global coordinate regression mapping relationship based on the stage feedback displacement. Output the global aligned temporal displacement trajectory after unified coordinate system transformation. Remove the failed reference marker displacement from the non-rigid body disturbance distortion frames. Combine the drift path constraint interpolation of adjacent stable frames to complete the missing segments of the global aligned temporal displacement trajectory in the distortion interval, and generate a continuous drift reference trajectory covering the entire temporal sequence.

[0041] In step S40, the process of outputting the globally aligned temporal displacement trajectory after unified coordinate system transformation is as follows: The displacements of the reference marker and the stage within the candidate frames of rigid body co-motion are sorted for temporal consistency. The displacement sequences of the reference marker and the feedback displacement sequences of the stage are extracted from the dual-channel spatiotemporal displacement data corresponding to the candidate frames of rigid body co-motion, and rearranged according to their respective acquisition timestamps to align and sort the data from different sources according to the same time reference. For cases with inconsistent temporal sampling densities or local missing data, temporal completion is performed by linear interpolation based on adjacent time nodes to ensure that the displacements of the reference marker and the stage form a one-to-one correspondence at each imaging frame time point, thereby obtaining the basic displacement data set after temporal consistency sorting.

[0042] Multi-frame stable displacement data are continuously stitched together in chronological order to form a global temporal trajectory. After completing the time-consistent sorting, the multi-frame stable displacement data in the rigid body co-motion state are connected frame by frame in chronological order, and the spatial continuity between adjacent frames is preserved during the stitching process to avoid trajectory breakage or jumps. For the small displacement deviations that may exist between different frame segments, a smooth connection is achieved by transition alignment based on the end of the adjacent frame and the starting frame, thereby forming a global temporal trajectory that is continuous in the time dimension and has smooth spatial changes.

[0043] A global coordinate regression mapping relationship is constructed based on the global temporal trajectory and the stage feedback displacement, and frame-by-frame coordinate regression mapping parameters are calculated. The stitched global temporal trajectory is used as the observation motion trajectory, and the stage feedback displacement is used as the system rigid body motion reference. By establishing the spatial correspondence between the two, a regression mapping model is constructed to describe the transformation relationship between the imaging coordinate system and the stage coordinate system. In the implementation process, the spatial offset relationship is obtained frame by frame by performing pair analysis on the corresponding displacement points under different time frames, and the coordinate regression mapping parameters that change with time are generated accordingly to describe the spatial transformation relationship at different time points.

[0044] The stitched frame segments are transformed to a unified coordinate system using regression mapping parameters, and a globally aligned temporal displacement trajectory is output. The coordinate regression mapping parameters obtained frame by frame are applied to the stitched global temporal trajectory, and the spatial coordinate transformation is performed on the displacement data of each frame to map it uniformly to a global coordinate system based on the stage. During the transformation, the temporal order is kept unchanged, and only the spatial position is corrected for consistency, thereby eliminating the coordinate offset caused by system drift or mechanical error between different frames. Finally, a globally aligned temporal displacement trajectory with temporal continuity and spatial consistency under a unified coordinate system is output.

[0045] In step S40, the process of generating a continuous drift reference trajectory covering the entire time series is as follows: The reference marker displacement data within the non-rigid body perturbation distortion frame segment is filtered for validity, and abnormal displacement observation points with offset residuals exceeding a preset stability threshold are removed to form a valid observation sequence for the distortion interval. When filtering the reference marker displacement data within the non-rigid body perturbation distortion frame segment, the stage feedback displacement sequence is first used as the reference trajectory. The difference between the observed displacement of the reference marker under the same time index and the reference displacement is calculated to obtain a frame-by-frame offset residual sequence. Then, the local mean and variance of the residual sequence are statistically analyzed using a sliding window method, and each observation point is judged according to the preset stability threshold. When the residual value of a frame continuously exceeds the threshold range or abrupt transitions occur, the displacement observation point corresponding to that frame is marked as an anomaly and removed. The remaining observation data after removing anomalies is time-reordered and continuity-completed to form a valid observation sequence containing only stable components, providing reliable input for subsequent distortion interval trajectory reconstruction.

[0046] Based on the drift trajectories corresponding to adjacent rigid body co-motion candidate frames before and after the distortion interval, the spatial displacement state variables and temporal continuity constraint parameters of the boundary frames are extracted to construct a set of drift constraint boundary conditions for the distortion interval. Based on the adjacent rigid body co-motion candidate frames before and after the distortion interval, their globally aligned drift trajectories are extracted respectively, and the starting and ending boundary frames of the distortion interval are selected as constraint anchor points to construct a set of boundary state variables, including the two-dimensional or three-dimensional spatial displacement vector, instantaneous displacement change rate, and time interval parameters between adjacent frames of the boundary frames. At the same time, the consistency of the local drift direction of the boundary frames in the stable frames is statistically analyzed to constrain the interpolation direction of the distortion interval. The above boundary information is uniformly encoded into a set of drift constraint boundary conditions to ensure that the subsequent reconstructed trajectory maintains continuous connection with the real stable trajectory at the start and end points.

[0047] Using the continuous trend of the stage's feedback displacement as a global prior constraint, and integrating effective observation sequence information, a piecewise continuity constraint function is constructed under the time axis consistency constraint. Trajectory reconstruction interpolation is performed on the distorted interval to generate a complete displacement sequence that satisfies the rigid body translation consistency and displacement smoothness constraints. Using the continuous trend sequence of the stage's feedback displacement as a global prior input, it is low-pass smoothed to eliminate the influence of high-frequency mechanical jitter, resulting in a global baseline motion trend curve. Based on this, the effective observation sequence is mapped to a unified time axis, and a set of boundary constraint conditions is introduced to construct a piecewise continuity constraint function for the distorted interval. This function uses the time index as the independent variable and the spatial displacement as the dependent variable, while simultaneously constraining the continuity of its first derivative and minimizing the boundary matching error. Subsequently, piecewise spline interpolation or optimization methods are used to reconstruct the missing trajectory in the distorted interval, generating a complete displacement sequence that satisfies the rigid body translation consistency, trend consistency, and smooth transition constraints.

[0048] The completed displacement sequence is spliced ​​and fused with the globally aligned temporal displacement trajectories of the preceding and following stable frames in chronological order to form a continuous and unbroken global temporal drift trajectory. The completed displacement sequence is spliced ​​with the globally aligned temporal displacement trajectories of the preceding and following stable frames according to the original time index. During the splicing process, the time axis is monotonically increasing as a constraint to ensure that there are no time jumps or repeated indices. At the seam, a weighted fusion method is used, that is, several frames are taken before and after the boundary and the overlapping intervals are fused. The displacement is smoothly transitioned through linear weight decay, thereby eliminating the trajectory discontinuity problem caused by splicing breakpoints, and finally forming a continuous and unbroken global temporal drift trajectory covering the entire time range.

[0049] A drift consistency check is performed on the stitched continuous trajectory. This check is based on the continuity constraints of the displacement change rate between adjacent frames and the displacement gradient matching error of the boundary frames. The displacement change trend of the completed interval maintains a continuous transition with adjacent stable frames, thus generating a continuous drift reference trajectory covering the entire time series. When performing a consistency check on the stitched global temporal drift trajectory, firstly, the displacement change rate sequence between adjacent frames is calculated and subjected to continuity constraint analysis to determine if there are any abnormal abrupt changes. Secondly, the displacement gradient matching error at the boundary between the completed interval and the preceding and following stable frames is calculated, including direction error and amplitude error. When the error exceeds a set threshold, the completed interval is re-interpolated locally. When the error meets the constraint conditions, the trajectory is confirmed to meet the requirements of continuous transition and physical consistency on the time axis. The final output serves as a continuous drift reference trajectory covering the entire time series and is used for unified spatial coordinate correction of subsequent imaging frames.

[0050] Step S50: Correct the spatial coordinates of the imaging frame frame by frame according to the drift reference trajectory, and feed the correction results back to the residual recursive calculation process to update the drift deviation evolution characterization quantity and the frame segment determination boundary.

[0051] In step S50, the process of updating the drift deviation evolutionary characterization and frame segment determination boundary is as follows: Based on the frame-by-frame spatial displacement correction parameters corresponding to the drift reference trajectory, the imaging frames are transformed frame by frame. The spatial displacement correction parameters corresponding to the time points of each imaging frame are extracted from the drift reference trajectory and loaded into the imaging data processing flow frame by frame in the order of frame sequence. By correcting the spatial position of the reference marker displacement data of each frame in a unified coordinate system, the cumulative offset caused by stage drift and system mechanical errors is eliminated, thereby obtaining the frame-by-frame corrected spatial displacement data sequence, which provides a consistent input basis for subsequent residual updates.

[0052] The coordinate transformation results are used as recursive inputs to the residual recursive calculation process and participate in the frame-by-frame offset residual update calculation. The frame-by-frame displacement results after spatial coordinate transformation are used as new observation inputs and fused with the residual recursive state of the previous time step. By introducing the corrected displacement change of the current frame into the historical residual accumulation structure, dynamic correction of the residual evolution process is achieved. The residual accumulation state is updated synchronously during the recursive process so that it can reflect the true offset trend after correction, thereby reducing the drift deviation caused by error accumulation.

[0053] The drift deviation evolution characterization and frame segmentation rules are adjusted based on the updated residual sequence. The temporal variation characteristics of the drift deviation evolution characterization, including the variation amplitude, growth trend, and local fluctuation state, are recalculated based on the recursively updated residual sequence, and compared with the preset stable interval. When the updated residual sequence shows higher consistency or lower volatility, the weight parameters of the drift deviation evolution characterization are corrected, and the frame segmentation threshold interval is dynamically adjusted so that the boundary between rigid body co-motion frames and non-rigid body disturbance distortion frames can be adaptively updated with the system correction results, thereby improving the stability and accuracy of the overall drift determination.

[0054] Example 2: Please refer to Figure 2 As shown, a live-cell long-term drift-free super-resolution fluorescence imaging system includes: Dual-source registration module: Performs time-axis consistency constraint registration on imaging frame sequence, reference marker displacement and stage feedback displacement to construct dual-channel spatiotemporal displacement data; The residual recursion module performs incremental difference and frame-by-frame recursive calculations on the dual-channel displacement data based on the inter-frame rigid body translation assumption to generate the offset residual of the reference mark relative to the stage. Drift determination module: Performs time-dimensional recursive cumulative calculation on the residual sequence to form a drift deviation evolution characterization quantity, and performs rigid body co-motion and non-rigid body perturbation frame segment determination based on continuity and abrupt change characteristics; Trajectory Reconstruction Module: Performs global coordinate regression stitching on rigid body co-motion frames and performs path constraint interpolation to complete distorted frames, generating a continuous drift reference trajectory; Correction and update module: Based on the drift reference trajectory, the imaging spatial coordinates are corrected frame by frame, and the correction results are fed back to the residual recursion process to update the drift deviation evolution characterization and frame segment determination boundary.

[0055] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A long-term drift-free super-resolution fluorescence imaging method for live cells, characterized in that: Includes the following steps: Live cell single-molecule localization imaging frame sequence, external fluorescent reference marker displacement trajectory and stage feedback displacement data were acquired. Based on the time axis consistency constraint, the reference marker displacement trajectory and stage feedback displacement data were time-registered to construct dual-channel spatiotemporal displacement data. Based on the assumption of rigid body translation between frames, incremental difference operation is performed on the dual-channel spatiotemporal displacement dataset to calculate the offset residual data of the fluorescent reference marker relative to the stage frame by frame, and the time dimension is recursively accumulated to construct a drift deviation evolution characterization quantity that quantifies the degree of rigid body separation of the reference marker. Using the drift deviation evolution characterization quantity as a unified criterion, the continuous state of residual propagation within the imaging window is determined. Based on the residual stability or abrupt divergence characteristics, the imaging frame segment is divided into rigid body co-motion candidate frame segment and non-rigid body perturbation distortion frame segment. The candidate frames of rigid body co-motion are time-aligned and stitched together, and a global coordinate regression mapping relationship is constructed based on the stage feedback displacement. The global aligned time displacement trajectory is output after unified coordinate system transformation. The failed reference marker displacement is removed from the non-rigid body disturbance distortion frame segments. Combined with the drift path constraint interpolation of adjacent stable frame segments, the missing segments of the global aligned time displacement trajectory in the distortion interval are completed, and a continuous drift reference trajectory covering the entire time series is generated. The spatial coordinates of the imaging frames are corrected frame by frame based on the drift reference trajectory, and the correction results are fed back to the residual recursive calculation process to update the drift deviation evolution characterization and frame segment determination boundary.

2. The method for long-term drift-free super-resolution fluorescence imaging of live cells according to claim 1, characterized in that, The process of constructing dual-channel spatiotemporal displacement data is as follows: The frame-by-frame timestamp information of the live cell single-molecule localization imaging frame sequence was obtained, and the sampling time sequence of the displacement trajectory of the exogenous fluorescent reference marker and the stage feedback displacement data was extracted simultaneously. Time alignment processing is performed on data from different sources based on timestamps to construct a multi-source displacement mapping relationship under a unified time benchmark; Calculate the consistency error between the displacement trajectory of the reference marker and the displacement data fed back by the stage at the corresponding time node, and correct the timing alignment relationship based on the principle of minimizing the error; The aligned reference marker displacement data and the stage feedback displacement data are fused and stored according to a frame-by-frame correspondence to form dual-channel spatiotemporal displacement data for drift analysis.

3. The method for long-term drift-free super-resolution fluorescence imaging of live cells according to claim 1, characterized in that, The process of calculating the offset residual data of the fluorescence reference mark relative to the stage frame by frame is as follows: Read the reference marker displacement and stage displacement information corresponding to adjacent imaging frames in the dual-channel spatiotemporal displacement data; The theoretically consistent displacement increment is calculated based on the inter-frame rigid body translation constraint and used as the reference datum increment. The frame-by-frame difference between the actual observed displacement increment and the theoretical reference increment is calculated to obtain the offset residual of the reference mark relative to the stage; The offset residuals are updated using a sliding frame window to form continuous time series residual data.

4. The method for long-term drift-free super-resolution fluorescence imaging of live cells according to claim 1, characterized in that, The process of constructing a drift deviation evolution characterization of the degree of rigid body detachment from the quantified reference marker is as follows: Based on frame-by-frame offset residual data, and using the initial imaging frame as the recursive starting boundary condition, the residual recursive initial state domain is constructed. The residual of the current frame is weighted and recursively calculated with the cumulative residual of the previous frame to form a time-continuous residual evolution sequence; The drift deviation evolution characterization is constructed based on the residual growth rate and the trend of cumulative amplitude change.

5. The method for long-term drift-free super-resolution fluorescence imaging of live cells according to claim 1, characterized in that, The process of determining the continuous state of residual propagation within the imaging window is as follows: Extract the drift deviation evolution characteristics within a continuous time window, analyze the continuous change characteristics of the evolution quantity within different time windows, and calculate the local fluctuation consistency index. When the evolution quantity maintains stable growth or stable convergence in the time dimension, it is determined that the residual propagation is in a continuous state. When the evolution quantity exhibits discontinuous jumps or non-stationary fluctuations, it is determined that the residual propagation has entered an abnormal disturbance state.

6. The method for long-term drift-free super-resolution fluorescence imaging of live cells according to claim 1, characterized in that, The process of dividing imaging frames into rigid body co-motion candidate frames and non-rigid body perturbation distortion frames is as follows: The threshold interval for frame segmentation is set based on the slope and fluctuation amplitude of the drift deviation evolution characterization quantity. Sliding window analysis is performed on continuous imaging frames to identify the set of frames where the residual changes are in a stable range; When the residual change satisfies the stability constraint, the corresponding frame segment is marked as a candidate frame segment for rigid body co-motion; When the residual change exceeds the threshold and exhibits abrupt divergence characteristics, the corresponding frame segment is marked as a non-rigid body perturbation distortion frame segment.

7. The method for long-term drift-free super-resolution fluorescence imaging of live cells according to claim 1, characterized in that, The process of outputting the globally aligned temporal displacement trajectory after a unified coordinate system transformation is as follows: The displacements of the reference markers and the stage within the candidate frames of rigid body co-motion are sorted according to their temporal consistency. Multiple frames of stable displacement data are continuously stitched together in chronological order to form a global time-series trajectory; A global coordinate regression mapping relationship is constructed based on the global temporal trajectory and the stage feedback displacement, and the frame-by-frame coordinate regression mapping parameters are calculated. By using regression mapping parameters to perform a unified coordinate system transformation on the spliced ​​frame segments, a globally aligned temporal displacement trajectory is output.

8. The method for long-term drift-free super-resolution fluorescence imaging of live cells according to claim 1, characterized in that, The process of generating a continuous drift reference trajectory covering the entire time series is as follows: The reference marker displacement data in the non-rigid body disturbance distortion frame segment is filtered for validity, and abnormal displacement observation points with offset residuals exceeding the preset stability threshold are removed to form an effective observation sequence of the distortion interval. Based on the drift trajectories corresponding to the candidate rigid body co-motion segments before and after the distortion interval, the spatial displacement state quantities and temporal continuity constraint parameters of the boundary frames are extracted to construct the set of drift constraint boundary conditions for the distortion interval. Using the continuous change trend of the stage feedback displacement as a global prior constraint, and integrating effective observation sequence information, a piecewise continuity constraint function is constructed under the time axis consistency constraint. Trajectory reconstruction interpolation is performed on the distortion interval to generate a complete displacement sequence that satisfies the rigid body translation consistency and displacement smoothness constraints. The completed displacement sequence is spliced ​​and fused with the global aligned temporal displacement trajectory of the preceding and following stable frames in time index order to form a continuous and unbroken global temporal drift trajectory. A drift consistency check is performed on the stitched continuous trajectory. The check is based on the continuity constraint of the displacement change rate of adjacent frames and the displacement gradient matching error of the boundary frames. The displacement change trend of the complete interval is made to maintain a continuous transition with the adjacent stable frame segments, thereby generating a continuous drift reference trajectory covering the entire time series.

9. The method for long-term drift-free super-resolution fluorescence imaging of live cells according to claim 1, characterized in that, The process of updating the drift deviation evolutionary representation and frame determination boundary is as follows: Based on the frame-by-frame spatial displacement correction parameters corresponding to the drift reference trajectory, the imaging frames are transformed frame by frame. The coordinate transformation result is used as a recursive input to the residual recursive calculation process and participates in the frame-by-frame offset residual update calculation. The drift deviation evolution characterization and frame segmentation rules are adjusted based on the updated residual sequence.

10. A live-cell long-term drift-free super-resolution fluorescence imaging system, applied to the method described in any one of claims 1-9, characterized in that, include: Dual-source registration module: Performs time-axis consistency constraint registration on imaging frame sequence, reference marker displacement and stage feedback displacement to construct dual-channel spatiotemporal displacement data; The residual recursion module performs incremental difference and frame-by-frame recursive calculations on the dual-channel displacement data based on the inter-frame rigid body translation assumption to generate the offset residual of the reference mark relative to the stage. Drift determination module: Performs time-dimensional recursive cumulative calculation on the residual sequence to form a drift deviation evolution characterization quantity, and performs rigid body co-motion and non-rigid body perturbation frame segment determination based on continuity and abrupt change characteristics; Trajectory Reconstruction Module: Performs global coordinate regression stitching on rigid body co-motion frames and performs path constraint interpolation to complete distorted frames, generating a continuous drift reference trajectory; Correction and update module: Based on the drift reference trajectory, the imaging spatial coordinates are corrected frame by frame, and the correction results are fed back to the residual recursion process to update the drift deviation evolution characterization and frame segment determination boundary.

Images