Space-time calibration system and method for whole-board high-flux photoelectric joint detection

Abstract

The invention relates to the technical field of photoelectric measurement and calibration, in particular to a space-time calibration system and method for whole-board high-flux photoelectric joint detection. The system comprises a time alignment module used for accurately measuring and aligning an exposure window; the space alignment module is used for multi-level dynamic focal plane compensation; the self-adaptive light source shaping system is used for exciting light pixel-level uniformity; and the electrode artifact positioning and correcting system is used for eliminating the electrode optical artifacts. The invention provides a space-time calibration system and a space-time calibration method for whole-board high-flux photoelectric joint detection, which solve or at least alleviate the problems of time synchronization, focal plane drift, illumination uniformity, electrode artifacts and the like of whole-board high-flux photoelectric detection in the prior art, and realize comprehensive focal plane compensation and signal fidelity. Space-time precision and uniformity challenges are effectively solved, and an efficient and accurate technical basis is provided for biological medicine detection.

Description

Technical Field

[0001] This invention relates to the field of photoelectric measurement and calibration technology, and in particular to a spatiotemporal calibration system and method for high-throughput photoelectric joint detection of a whole board. Background Technology

[0002] With the increasing demand for high-throughput photoelectric co-detection technology in drug screening, cell physiology, and neuroscience from research and development in the biomedical field, the demand for whole-plate multi-well imaging analysis is also growing rapidly, especially for high-fidelity, high-throughput functional analysis. High-throughput photoelectric co-detection technology enables simultaneous high-throughput drug screening and analyses in cell physiology and neuroscience on the same platform. The demand for whole-plate multi-well imaging analysis, in particular, is constantly increasing as the biomedical field's demand for high-throughput photoelectric co-detection technology in drug screening, cell physiology, and neuroscience continues to rise. It simultaneously acquires and correlates information on cell populations at both the optical (e.g., fluorescence intensity and morphological changes) and electrophysiological (e.g., membrane potential and ion pathways) levels, thereby providing insights into complex cellular activities.

[0003] To address the demands of high-throughput detection, numerous solutions have been disclosed and implemented. See patent CN104142317B, which discloses a scanning system with interchangeable optical boxes for fluorescence measurement. This system includes a main unit and a stage interchangeable with the multi-channel optical box. The stage scans a multi-well plate row-by-row or column-by-column. The main unit reads and uses calibration information associated with the multi-channel optical box to correct the optical signal obtained by the box. This solution utilizes pre-stored calibration information for preliminary optical signal correction, solving the problems of flexibility and basic signal calibration in early fluorescence measurements, and supporting the acquisition of localized fluorescence signals. See patent CN111366720B, which discloses a calibration card for a fluorescence immunoassay analyzer and its preparation method. Through multi-band fluorescence signal calibration of a fluorescent composition, it achieves long-term stability and a low coefficient of variation in the fluorescence signal. This solution stabilizes the fluorescence signal itself through multi-band fluorescence signal calibration, improving the measurement accuracy and repeatability, and is suitable for applications requiring stable quantitative fluorescence signals.

[0004] However, as biological research advances to deeper and broader scales, requiring high-throughput, global monitoring of the dynamic processes of cell populations, certain inherent and insurmountable characteristics in the principles of the aforementioned existing technologies inevitably present insurmountable obstacles in addressing these needs. While the row-by-row or column-by-column scanning method in CN104142317B improves the detection efficiency of local areas to some extent, it also determines that it is a sequential data acquisition method—a non-parallel, time-division imaging mechanism. This prevents the simultaneous recording of the entire plate in a single step. Consequently, images acquired at different times and in different areas have segmented imaging time intervals, which cannot match rapid cellular physiological activities. This leads to unavoidable temporal inconsistencies between electrical and optical signals, introducing non-negligible errors during photoelectric coupling analysis and affecting the attribution analysis of the causal relationship between rapid electrophysiological events and their corresponding optical signal responses. In addition, the above scheme does not adequately consider the problem of dynamic focal plane drift caused by manufacturing tolerances, changes in ambient temperature, or long-term experiments in porous plates. The statically stored calibration information is difficult to match the real-time changing focal plane, resulting in a decrease in image quality and affecting quantitative fluorescence analysis, especially in large field-of-view imaging.

[0005] While CN111366720B makes progress in fluorescence signal calibration and stability, its focus is primarily on fluorescence signal calibration. It fails to explore the synchronous acquisition and analysis mechanism of electrophysiological signals, essential for photoelectric co-detection. Therefore, it cannot fundamentally solve the time synchronization problem between photoelectric signals. Furthermore, the above schemes, when considering fluorescence signal stability, do not address the illumination uniformity issue of the porous plate under large field-of-view imaging. The difficulty of large field-of-view illumination lies in achieving pixel-level and sub-pixel-level illumination uniformity of the entire detection area by a large area of ​​excitation light. Simply calibrating the fluorescence stability of a single point cannot resolve the deviation of the fluorescence signal system in the region caused by uneven illumination. More importantly, the above schemes do not address the optical artifact problem faced by transparent electrodes (such as indium tin oxide, ITO) in photoelectric co-detection. Transparent electrodes may exhibit absorption, scattering, or interference effects at specific excitation wavelengths and emitted fluorescence, manifesting as "ghosting" or local signal attenuation in the imaging data, thus severely affecting the acquisition of true biological signals. Current fluorescence calibration methods are unable to locate, quantify, and eliminate complex optical artifacts, which greatly reduces the reliability and comparability of the detection data.

[0006] Due to the shortcomings of the aforementioned prior art, the technical problem to be solved is to provide a systematic solution for a high-fidelity and high-efficiency spatiotemporal calibration system and method for high-throughput photoelectric joint detection of the entire board. Specifically, it is to solve the technical problems in the aforementioned prior art, such as low time synchronization accuracy, dynamic drift of the spatial focal plane, non-uniformity of large-area illumination, and optical artifacts introduced by transparent electrodes. The applicant believes that there are many technical problems in the aforementioned prior art. However, if the applicant were to solve all of the aforementioned technical problems, it would inevitably make the proposed spatiotemporal calibration system and method too large and complex, and difficult to implement. Therefore, the aforementioned prior art does not provide a highly integrated, collaborative system architecture with multi-level dynamic calibration capabilities to simultaneously solve the aforementioned technical problems. Therefore, in pursuing the efficiency of whole-panel imaging and the accuracy of photoelectric joint analysis, the refinement of time synchronization, the multi-scale dynamic compensation of the spatial focal plane, the pixel-level uniformity of large-area illumination, and the elimination of transparent electrode artifacts are not just simple technical issues, but key bottlenecks that are mutually coupled and closely related. If the improvement of accuracy in any one aspect is not coordinated with other aspects, it will be difficult to meet the stringent requirements of modern drug screening and biological research for efficient and accurate detection technologies. Summary of the Invention

[0007] The purpose of this invention is to provide a spatiotemporal calibration system and method for whole-plate high-throughput photoelectric co-detection, which can solve or at least alleviate the problems of insufficient time synchronization accuracy, dynamic drift of the spatial focal plane, poor uniformity of large-area illumination, and optical artifacts introduced by transparent electrodes in existing technologies for whole-plate high-throughput photoelectric co-detection. This invention provides a highly integrated, collaborative system architecture with multi-level dynamic calibration capabilities to acquire and analyze whole-plate cell photoelectric co-detection data with high fidelity and high efficiency.

[0008] To achieve the above objectives, the present invention provides the following technical solution: a spatiotemporal calibration system for whole-board high-throughput photoelectric joint detection, the system comprising: The time alignment module is used to accurately determine the image exposure time window and calculate a high-precision timestamp to achieve precise synchronization between image data and continuous electrophysiological signals. The spatial alignment module is used to achieve multi-level focal plane compensation through initial physical compensation, macroscopic dynamic drift monitoring and microscopic real-time dynamic tracking, so as to ensure that the cell growth layer of the multi-well plate and the focal plane of the imaging objective lens are always coplanar. An adaptive light source shaping system is used to employ multiple independently controlled LED arrays. Through a self-calibration algorithm and a 3D topographic map of the surface, a lighting model is established, and the optimal combination of driving currents is determined to achieve uniform excitation light. An electrode artifact localization and correction system is used to generate an electrode digital mask using multi-angle auxiliary illumination and to generate an artifact correction map using a special calibration plate to quantitatively recover optical measurement errors introduced by transparent electrodes.

[0009] To further realize the present invention, the following technical solutions may be preferred: Preferably, the time alignment module includes: a main control device for generating a synchronization trigger pulse and monitoring the camera exposure status signal; an image acquisition camera for acquiring images and outputting the exposure status signal; and a software unit for calculating a high-precision timestamp based on the exposure status signal and binding and storing the image data with the timestamp.

[0010] Preferably, the spatial alignment module includes: a stage platform for supporting the porous plate; multiple piezoelectric nanopositioners for submicron-level tilting and Z-axis height adjustment of the stage platform; a non-contact high-precision displacement sensor for measuring the Z-axis height of the porous plate surface; a stepper motor Z-axis platform for large-range coarse adjustment; and a computing unit for performing image processing and deep learning inference.

[0011] Preferably, the spatial alignment module is configured to perform an initial static focal plane calibration, which includes: Partitioned Image Coarse Adjustment and Preliminary Terrain Construction: The system software divides the camera's field of view into a grid, the stepper motor Z-axis platform drives the stage to scan, the camera acquires images, the calculation unit evaluates the sharpness and constructs the preliminary three-dimensional coordinates of the perforated plate. Fine-grained mapping and model correction using displacement sensors: The displacement sensors are activated, the platform is moved to key mapping points, and the sensors record the Z-axis height value. The software unit uses this data to correct the initial terrain model and generate the final 3D terrain map of the slab. Initial active tilt and warp compensation: The software unit acquires the topographic map and fits the spatial mathematical model, calculates the extension and retraction distance of the piezoelectric nanopositioner, drives its coordinated action to complete the precise adjustment and locking of the physical posture.

[0012] Preferably, the spatial alignment module is configured to perform dynamic focal plane drift compensation, which includes: Long-term macroscopic dynamic deformation real-time adaptive compensation: After initial static calibration, the system initiates background dynamic drift monitoring; the system periodically performs rapid sharpness assessment or blur detection on the preset monitoring area of ​​the perforated plate; if the focal plane deviates beyond the threshold, it is determined to be dynamic deformation drift, and the software unit recalculates and performs background physical attitude fine-tuning based on the trend to restore the stage platform to the optimal focal plane; and High-speed real-time microscopic dynamic focal plane tracking: During high-speed image data acquisition, a pre-trained deep learning model is initialized, which infers the focal plane offset from image features; the computing unit processes the image data in real time and evaluates the focal plane offset of multiple sub-regions in the camera's field of view in parallel through the deep learning model; the software unit integrates the offset estimates of all sub-regions and calculates the overall micro-tilt and Z-axis drift of the porous plate; the real-time drift is fed back to the piezoelectric nanopositioner controller, driving it to continuously and with high bandwidth fine-tuning to compensate for the focal plane drift in real time and ensure that the focal plane is always coplanar with the cell growth layer.

[0013] Preferably, the adaptive light source shaping system includes: a matrix array light source composed of multiple independently controlled light-emitting diode units; an independent current driver for controlling each light-emitting diode unit; and a software unit for acquiring a topographic map and a light intensity contribution map of the board surface, establishing a mathematical model of illumination, and solving in reverse the optimal combination of driving currents to achieve illumination uniformity across the entire board.

[0014] Preferably, the electrode artifact localization and correction system includes: multiple auxiliary illumination sources distributed at specific geometric angles; a specially made glass calibration plate with a stable fluorescent dye uniformly coated on the bottom; and a software unit for generating an accurate electrode mask through image difference operations and morphological operations; and generating an artifact correction spectrum using the electrode mask and the specially made calibration plate, and performing pixel-by-pixel multiplication of the original fluorescence image with this spectrum for correction in subsequent experiments.

[0015] A spatiotemporal calibration method for whole-panel high-throughput photoelectric joint detection, the method comprising the following steps: Step S1: Establish a unified master-slave clock architecture: accurately determine the start and end times of the image exposure window, calculate its arithmetic midpoint as a high-precision timestamp of the image, and bind the image data with the timestamp to achieve precise synchronization between the image data and continuous electrophysiological signals. Step S2, Multi-level active whole-plate dynamic focal plane compensation: Perform initial static focal plane calibration, long-term macroscopic dynamic deformation real-time adaptive compensation, and high-speed real-time microscopic dynamic focal plane tracking on the porous plate to ensure that the cell growth layer of the porous plate and the focal plane of the imaging objective lens are always coplanar. Step S3, Adaptive light source shaping: Based on the partition contribution model, the light-emitting diode array light source is self-calibrated. Combined with the three-dimensional terrain map generated in step S2, a mathematical model of illumination is established, and the optimal combination of driving currents is solved to achieve pixel-level uniformity of excitation light. Step S4, Electrode Artifact Localization and Correction: Identify and quantitatively correct optical artifacts introduced by transparent electrodes. The correction includes generating an electrode mask using multi-angle auxiliary illumination, generating an artifact correction spectrum using a special calibration plate, and then applying the spectrum to the subsequently acquired raw fluorescence image.

[0016] Preferably, step S1 specifically includes: Step S101, establish a unified master-slave clock architecture: during system initialization, the master control device generates a trigger pulse sequence synchronized with the electrophysiological signal sampling clock; Step S102, Precise capture of exposure window status: The main control device listens to the camera's exposure status signal and records the precise moments when the exposure begins and ends; Step S103, Timestamp Centroid Calculation and Data Binding: The software unit obtains the start and end times of the exposure window, calculates its arithmetic midpoint as a high-precision timestamp, and stores the image data and timestamp in a strong binding manner.

[0017] Preferably, step S2 specifically includes: S201. Initial Static Focal Plane Calibration: Coarse adjustment of partitioned images and preliminary terrain construction: The system software divides the camera's field of view into a virtual grid; the stepper motor drives the stage to scan along the Z-axis, and the camera acquires images; the software calculates the sharpness of each partition in parallel, determines the optimal focal plane height, and constructs the preliminary three-dimensional coordinates of the perforated plate. Fine mapping and model correction using displacement sensors: When the displacement sensors are activated, the platform is moved to key mapping points, and the sensors record the Z-axis height; high-precision data is used to correct the initial terrain model and generate the final 3D terrain map of the slab. Initial active tilt and warp compensation: The software acquires a 3D topographic map and fits a spatial mathematical model; the extension and retraction distance of the piezoelectric nanopositioner is calculated based on the model, and its coordinated action is driven to complete the precise adjustment and locking of the physical attitude of the stage platform. S202. Dynamic focal plane drift compensation: Long-term macroscopic dynamic deformation real-time adaptive compensation: After initial static calibration, the system initiates background dynamic drift monitoring; periodically performs rapid sharpness assessment or blur detection on the preset monitoring area; if the focal plane deviates beyond the threshold, it is determined to be dynamic deformation, and background physical attitude fine-tuning is recalculated and performed according to the trend to restore the panel to the optimal focal plane; and High-speed real-time microscopic dynamic focal plane tracking: During high-speed image data acquisition, a pre-trained deep learning model is initialized; the computing unit processes the image data in real time, extracts cell features through the model, and evaluates the focal plane offset of multiple sub-regions of the camera's field of view in parallel; the offset estimation is combined to calculate the overall micro-tilt and Z-axis drift of the porous plate; the real-time drift is fed back to the piezoelectric nanopositioner controller, driving it to continuously and with high bandwidth fine-tuning to compensate for the focal plane drift in real time.

[0018] The beneficial effects of this invention are: This invention establishes a complete and automated signal fidelity system. Its key lies in multi-level, comprehensive focal plane compensation, from initial static to real-time dynamic, ensuring that every node in signal transmission and conversion is calibrated and corrected. The system and method collaboratively solve the complex challenges of interconnected and dynamically changing spatiotemporal accuracy, signal fidelity, and illumination uniformity faced in whole-panel high-throughput photoelectric joint detection, thus laying the foundation for efficient and accurate detection in the field of biomedical research. Attached Figure Description

[0019] Figure 1 This is an overall block diagram of the spatiotemporal calibration system of the present invention.

[0020] Figure 2 This is a schematic diagram illustrating the principle of the time alignment module of the present invention.

[0021] Figure 3 This is a schematic diagram of the spatial alignment module of the present invention.

[0022] Figure 4 This is a flowchart of the spatiotemporal calibration method of the present invention.

[0023] Figure 5 This is a system configuration diagram for Embodiment 3 of the present invention. Detailed Implementation

[0024] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Example 1

[0026] This embodiment discloses a spatiotemporal calibration system for whole-board high-throughput photoelectric joint detection, referring to... Figures 1-3 It mainly comprises four core functional modules: a time alignment module, a spatial alignment module, an adaptive light source shaping system, and an electrode artifact localization and correction system. These modules are tightly integrated, enabling automated and high-precision calibration across the entire chain from data acquisition to processing and analysis.

[0027] Specifically, the time alignment module is a key component for achieving sub-microsecond time synchronization between optical and electrical signals. Currently, commonly used high-throughput photoelectric joint detection systems use the same trigger pulse moment as the timestamp for all images across all cameras; that is, the trigger pulse moment is treated as the timestamp for all images, ignoring the length of the camera's exposure window. In high-throughput electrophysiological sampling, the actual exposure time of an image may fall within multiple consecutive electrophysiological sampling cycles. Therefore, simply using this single trigger pulse moment as the image timestamp leads to significant errors between timestamps. The time alignment module of this invention adopts a master-slave clock unified architecture, which includes a master control device and an image acquisition camera. The master control device embeds a high-precision timer, which generates a series of precise TTL (Transistor-Transistor Logic) trigger pulses at a predetermined imaging frame rate (e.g., 100 to 1000 frames per second, with frame rate jitter less than 1 ppm). The timing of the trigger pulse generation is strictly synchronized with the electrophysiological signal sampling clock within the system, ensuring a high degree of consistency between the reference time base for electrophysiological data acquisition and the imaging trigger time base. This is generally achieved through mutual frequency locking between the master control device and the high-precision crystal oscillator within the system, or between the master control device and the PLL (phase-locked loop) within the system. The image acquisition camera is a professional-grade scientific camera equipped with a high-speed CMOS sensor. This scientific camera has high-speed image data output capability and independent exposure-valid and frame-valid TTL signal output ports. After issuing the image acquisition trigger pulse, the master control device continuously monitors the level changes of these two TTL signal output ports. Specifically, when the CMOS sensor array inside the camera actually begins exposure, the effective exposure signal instantly rises from a low level (e.g., from 0V to 3.3V or 5V). The main control device immediately captures this rising edge and defines it as the actual start time of exposure for that frame. When the camera finishes exposing the frame and prepares to write the data to the image buffer, the effective frame signal also instantly rises from a low level. The main control device also captures this rising edge and defines it as the end time of exposure for that frame. After obtaining the start and end times of the entire exposure window for each frame, the software within the main control device uses a timestamp centroid algorithm to calculate the final high-precision timestamp for that frame. The basic idea of ​​this algorithm is to use the arithmetic midpoint of the image exposure time period as the final timestamp, thereby accurately mapping the time period representing photon integration to a point on the time axis. For example, if the exposure start time is T_start and the exposure end time is T_end, the final high-precision timestamp T_timestamp is calculated as T_timestamp = (T_start + T_end) / 2.This precise timestamp based on the center of the complete exposure window enables unambiguous and accurate alignment of image data and continuous electrophysiological signals, achieving sub-microsecond level absolute accuracy in time synchronization. Each frame of image data and its high-precision timestamp are then forcibly bound and stored in the system's unit for subsequent high-fidelity, high-efficiency spatiotemporal joint analysis. To demonstrate the superiority of this alignment method, assuming a camera capture frequency of 200 frames per second and an exposure time of 2 milliseconds, in traditional methods, if only the trigger pulse is used as the timestamp, and there is a 1-millisecond delay between the trigger and the start of the actual exposure, the timestamp will be 1 millisecond delayed from the centroid of the actual light signal. However, according to the applicant's method, even with the same trigger delay as the traditional method, because it captures the start and end of the actual exposure, the calculated timestamp will always fall within the center of the exposure window, ensuring alignment accuracy within 10 microseconds. In contrast, the error of the traditional method can reach the millisecond level. Therefore, the alignment accuracy of this method is two orders of magnitude higher than that of the traditional method.

[0028] Furthermore, the spatial alignment module addresses the dynamic focal plane drift issue of the entire porous plate in full-panel high-throughput imaging, ensuring that the cell growth layer and imaging objective remain confocal throughout the imaging area. The module's hardware embodies ultra-precise mechanics and advanced sensing technology. It includes a precision stage platform supporting the porous plate. This stage platform is manufactured from materials with low coefficients of thermal expansion (e.g., Invar alloys or special ceramic composites) and precision-machined to minimize thermal deformation and mechanical vibration. At least three high-resolution, high-load-capacity closed-loop piezoelectric nanopositioners are located at the bottom of the stage platform, connected to it in an equilateral triangular or polygonal (e.g., quadrilateral) geometric layout. These piezoelectric nanopositioners drive the porous plate to dynamically track the entire imaging area. These piezoelectric nanopositioners feature sub-nanometer displacement resolution (e.g., 0.5 nm to 2 nm) and stroke ranging from tens to hundreds of micrometers (e.g., 100 μm to 300 μm), along with built-in capacitive displacement sensors for real-time, closed-loop position control with repeatability of ±10 nm and response frequencies of hundreds of hertz. Additionally, a non-contact, high-precision laser displacement sensor is another crucial component. This sensor utilizes triangulation or confocal sensing principles. Mounted at a fixed position above the stage platform, its optical axis is parallel to the objective lens's optical axis and within the measurement field of view for mapping plate height information. Furthermore, a large-stroke coarse focusing Z-axis platform driven by a stepper motor provides a 25 mm stroke with 1 μm steps for initial focusing and switching between perforated plates of different thicknesses. The Z-axis platform is equipped with an encoder to confirm its position. The entire system's control and data processing are handled by a computing unit, typically incorporating a GPU for accelerated graphics processing. The computing unit is typically equipped with a multi-core processor and large memory to process large amounts of image data simultaneously, perform real-time image processing and deep learning inference, parameter correction and analysis. Its processing power can meet the data flow of hundreds of megapixels per second to ensure real-time tracking of the focal plane.

[0029] The calibration and compensation of the spatial alignment module is multi-level and proactive, including three steps: initial static calibration, long-term macroscopic dynamic drift compensation, and high-speed real-time microscopic dynamic focal plane tracking. These three steps are closely connected and interlocking.

[0030] In the initial static calibration phase, coarse adjustment of the partitioned images and preliminary terrain construction are performed first. The software system logically divides the entire field of view of the image acquisition camera into a fixed-size partitioned grid, such as 10×10 or 16×16 partitions, each corresponding to an area of ​​approximately 0.5mm×0.5mm, thus ensuring sufficient cell image features in each partition. A stepper motor-driven Z-axis platform drives the stage platform to perform constant-step scanning within a preset Z-axis range (e.g., scanning 200 micrometers above and below the target focal plane in 5-micrometer steps, for a total of 81 Z-axis positions). At each Z-axis position, the image acquisition camera acquires one frame of image. The computing unit calculates the image sharpness score within each independent partition in parallel. Sharpness scores can be obtained by measuring the image's grayscale gradient (e.g., the Tenengrad operator), the energy of high-frequency components in the Fourier transform (e.g., calculating the proportion of high-frequency energy via a 2D FFT), or some edge feature (e.g., the Sobel or Laplacian operator). These algorithms execute at high speed on GPUs, with an average computation time of less than 1 millisecond per partition image. After scanning, each partition obtains a sharpness-Z-axis position curve. The software unit determines the optimal focal plane height for that partition—the Z-axis position where the sharpness score is highest—by fitting the curve with a Gaussian or parabolic curve. Thus, the system constructs a preliminary set of three-dimensional coordinate points for the porous plate, representing the overall tilt and local warping morphology of the porous plate, on the order of tens of micrometers (e.g., ±20 micrometers).

[0031] Next, fine laser mapping and model correction are performed. On the reference plane selected for coarse adjustment of the partitioned image (e.g., ±50 micrometers above and below), a non-contact high-precision laser displacement sensor is activated. The stage platform moves through a predefined key mapping point matrix via the XY platform (usually combined with a Z-axis platform), such as a dense 50×50 or 100×100 mapping point grid, to ensure coverage of all key areas, with the spacing between mapping points less than 0.5 millimeters. The laser displacement sensor records a Z-axis height value at each measurement point with nanometer-level accuracy. Advanced 3D point cloud processing algorithms are used by the software unit, such as using RANSAC (Random Sample Consensus)-based algorithms to remove outliers and filter noise, followed by ICP (Iterative Closest Point) algorithms for point cloud registration to optimize local fitting, or directly using least squares fitting, radial basis function (RBF) interpolation, thin plate splines, etc., to use this set of high-precision laser data to correct or replace the initial terrain model. The final result is a high-precision 3D topographic map of the plate surface. This map can be a 2D height map or a polynomial surface equation. It contains submicron-level plate surface elevation information and can comprehensively reflect every undulation of the perforated plate, serving as the benchmark for focal plane compensation.

[0032] Finally, initial active tilt and warp compensation are performed. After obtaining the final 3D topographic map data of the plate surface, the software unit uses least squares or higher-order polynomial fitting, such as a quadratic surface model or a higher-order cubic surface model, to capture the most exposed cell growth surface of the warp, fitting a mathematical model. Then, the software unit uses this mathematical model to calculate the specific distances that at least three piezoelectric nanopositioners must extend or shorten to make this surface completely horizontal, i.e., coplanar with the objective lens focal plane. This is a geometric inverse kinematics problem, using matrix operations, for example, calculating a 3×3 or 4×4 homogeneous transformation matrix, and then obtaining the result. The calculation unit gives control commands to the corresponding piezoelectric nanopositioners. The piezoelectric nanopositioners work together to accurately adjust the physical attitude of the stage platform with high bandwidth, high precision, and coplanarity, and then lock it at the optimal focal plane position. This achieves coplanarity between the stage platform and the objective lens focal plane at the micrometer or even submicrometer scale, laying the foundation for the next step of dynamic compensation.

[0033] Furthermore, the dynamic focal plane drift compensation method of the spatial alignment module includes the following two subsystems to cope with focal plane drift at different time scales and amplitude levels.

[0034] Subsystem 1: Real-time Adaptive Compensation for Long-Term Macroscopic Dynamic Deformation. After initial static calibration, for experiments lasting several hours or even days, the system initiates a background dynamic drift monitoring mode. In this mode, the system performs a rapid scan every 1-5 minutes (based on experimental sensitivity) of preset monitoring points or representative areas on the well plate surface (e.g., the four corners and center of each well plate, or the central areas of several key wells). During this scan, the system performs a rapid, low-excitation-dose image sharpness assessment. This assessment uses a sharpness evaluation algorithm similar to that of the initial static calibration, but with a lower sampling rate and excitation time to avoid photobleaching or phototoxicity. Alternatively, the system can use a dedicated edge-sensitive sensor (e.g., a sensor based on the Foucault blade method or a grating sensor) to detect blur and obtain focal plane shift information, with a feedback frequency reaching 100Hz. If the focal plane of any monitoring point deviates from the initial calibration position by more than a micrometer-level preset threshold, the system considers macroscopic dynamic deformation drift to have occurred. Then, based on the trend and degree of drift, the software unit performs a background-based physical attitude fine-tuning. Fine-tuning is achieved through progressive or stepwise displacement commands to the piezoelectric nanopositioners, allowing the stage platform to return to the appropriate focal plane. This maintains consistency in long-term data acquisition. Of course, fine-tuning can also be performed without interrupting main data acquisition. Through intelligent scheduling and resource allocation, such as during pauses in main imaging or through extremely short fine-tuning cycles, the impact on the main imaging task can be minimized.

[0035] Subsystem two, high-speed real-time microscopic dynamic focal plane tracking, is initialized by a pre-trained deep learning model during high-speed image data acquisition. The deep learning model is a convolutional neural network (CNN) architecture designed for this task, such as an encoder-decoder structure like ResNet or U-Net, or the MobileNetV3 architecture. This model is pre-trained on a dataset of tens of thousands of real and synthetic images containing different cell types, cell densities, and degrees of defocus. Data augmentation is performed on the images during training to improve generalization. The model can efficiently infer local metrics or precise sharpness sorting of the focal plane directly from the features of the input raw image. The computational unit processes the acquired image data in real-time at an extremely high frequency. Specifically, it divides the camera's field of view into several sub-regions (e.g., 3×3 or 5×5) and simultaneously evaluates the focal plane of the image data in each sub-region. The deep learning model directly outputs the focal plane offset of each sub-region or a normalized value representing sharpness. The software unit integrates the focal plane shift estimation results from all sub-regions and uses multi-point interpolation algorithms (such as Kriging interpolation or radial basis function interpolation) to calculate the overall minute tilt and Z-axis drift of the current porous plate surface relative to the ideal focal plane. These real-time calculated micron or submicron-level drifts are directly fed back to the controllers of three closed-loop piezoelectric nanopositioners. The piezoelectric actuators perform continuous, high-bandwidth fine-tuning with extremely high response speed and bandwidth, compensating for focal plane drift in real time and with precision. This ensures that the focal plane of the imaging objective remains precisely coplanar with the cell growth layer during high-speed imaging. The process is label-free, non-invasive, and fully integrated into the main imaging workflow, without affecting the biological sample. The input to the deep learning model can be raw grayscale image data or data that has undergone simple normalization, contrast enhancement, and other preprocessing, and the output is a continuous numerical vector representing the focal plane shift.

[0036] Furthermore, the adaptive light source shaping module provides pixel-level uniformity of the excitation light across the entire imaging area of ​​the porous plate. Traditional wide-area illumination suffers from uneven light intensity due to factors such as optical path design, light source characteristics, and uneven sample surfaces, thus affecting quantitative analysis. The light source consists of a matrix array composed of LED units with multiple zones and independently controlled brightness. Each LED unit typically contains multiple high-performance LED chips and is driven by a separate current driver. The current driver features high-precision digital-to-analog conversion (DAC) capabilities, such as a 16-bit DAC, enabling current regulation at the milliampere or even microampere level. This allows for very fine control of the brightness of each LED zone in the excitation light, with a maximum current of up to 1 ampere. The physical layout of the array light source can be a uniform grid (e.g., 8×8 or 16×16) or a non-uniform arrangement with density optimized for specific areas (e.g., the center or edge of the porous plate) to better adapt to the terrain.

[0037] The implementation steps of the self-calibration modeling and correction algorithm include: Step 1: Acquisition of Panel Morphology Information. Before executing this light source calibration procedure, the system first completes the panel morphology compensation of the spatial alignment module and loads the resulting 3D topographic map. The topographic map includes the Z-axis height information of each pixel on the porous panel surface. This topographic map is used for light intensity distance attenuation calculation, because the optical path is actually affected by the surface morphology.

[0038] Step two, intensity contribution map measurement. The calibration procedure sequentially and individually illuminates each LED section of the matrix array light source with a preset reference current (e.g., 100 mA, lower than the maximum LED drive current to avoid premature decay). After illuminating each LED section, the image acquisition camera takes an image to obtain the intensity contribution map of that LED section. The intensity contribution map describes the pixel-level illumination intensity distribution of that LED section within the field of view (i.e., all pixel locations), and includes the luminous efficacy and beam divergence characteristics of the LED itself.

[0039] Step 3: Establishment of the Comprehensive Model. The software unit establishes a comprehensive mathematical model that rigorously describes the correspondence between the illuminance I(x,y) at any point (x,y) on the imaging surface, the driving current C_i of each LED zone, and the actual three-dimensional distance D_i(x,y) from that point to each LED light source. The model strictly follows the inverse square law of light intensity decay with distance, i.e., f(D_i(x,y))=1 / D_i(x,y)^2, and combines the Z-axis distance information obtained from the three-dimensional terrain map to ensure accurate calculation of D_i(x,y). The model can be expressed by the following formula. Where N is the total number of LED partitions, k_i is the intrinsic luminous efficacy coefficient of the i-th LED partition (obtained through step two, which includes the efficiency of the LED itself and the optical path transmission efficiency), and C_i is the driving current of the i-th LED. This model can also be used to account for lens distortion and dispersion effects in the optical path.

[0040] Step four, solving for the optimal current combination. A set of optimal driving current combinations C_1, C_2, ..., C_N is solved in reverse to minimize the variance of the final illuminance I(x,y) across the entire critical region (e.g., the effective cell growth region of the multi-well plate), thereby achieving a light uniformity better than 95% across the entire plate (i.e., coefficient of variation CV less than 5%). The solution method generally employs iterative optimization, such as numerical optimization methods based on gradient descent, Newton's method, or least squares, like the L-BFGS-B algorithm. After multiple iterations, the current is adjusted to inversely calculate the illuminance, gradually approaching the optimal current combination. Its objective function can be defined as... Where I_target is the desired uniform illuminance target value. The optimal current combination obtained is stored in the illumination formula of this orifice plate and can be directly called in subsequent experiments to ensure that the illumination conditions are consistent and highly uniform in each experiment.

[0041] Furthermore, the electrode artifact localization and correction system is used to identify and quantitatively eliminate optical measurement errors introduced by transparent electrodes during photoelectric joint detection. Although transparent electrodes allow electrical signal transmission, they often exhibit localized differences in transmittance, scattering, or reflection, which manifest as spurious peaks in fluorescence images, thus interfering with quantitative fluorescence analysis. Therefore, auxiliary illumination sources are provided to locate and calibrate electrode spurious peaks to obtain accurate fluorescence analysis results. This module includes multiple auxiliary illumination sources distributed at different geometric angles and independently controllable, for example, 4 or 8 channels, illuminating from the sides of the porous plate at oblique angles of 30°, 45°, or 60°, thereby maximizing the contrast between the electrode profile and the background, as refraction and reflection at the electrode edges depend on the incident angle. Additionally, the system includes a specially designed glass calibration plate with a uniformly coated bottom with a photobleachable fluorescent dye (e.g., Rhodamine B or fluorescein embedded in a polymer film, or a uniformly grown quantum dot film directly on glass). The excitation and emission spectra of the fluorescent dye match those of fluorescent probes commonly used by users in actual biological experiments (e.g., calcium probe Fluo-4, voltage probe Di-8-ANEPPS), and the coating is uniform, thus providing a fluorescent signal reference in non-electrode areas.

[0042] The implementation steps of the electrode artifact localization and correction system include: Step 1: Automatic segmentation of the electrode area. The main light source is turned off, and one of the multiple oblique auxiliary lighting sources is turned on. After the image acquisition unit acquires an image, this light source is turned off, and then the next oblique auxiliary lighting source is turned on. The image acquisition unit acquires an image after this light source illuminates the area. This process is repeated until all oblique auxiliary lighting sources have been turned on once, and the image acquisition unit acquires the corresponding image. Since the electrode will produce different shadows, reflections, or scatterings under different illumination angles with different oblique auxiliary lighting sources on, in this step, the software unit performs differential analysis on the images acquired at different illumination angles. For example, one image is subtracted from another, or all images are averaged and then differentially compared with a single image to enhance the electrode's contour contrast. Adaptive thresholding (e.g., Otsu thresholding or local adaptive thresholding) is applied to the differential image to accommodate varying background light intensities. Then, morphological operations (e.g., erosion, dilation, opening, closing operations, using variable-sized structuring elements) are performed to eliminate noise, fill holes, and smooth boundaries, thereby generating an accurate binary electrode mask that corresponds to the pixels of the actual image. This electrode mask can identify the precise location and shape of the transparent conductive electrodes, for example, using edge detection and polygon fitting algorithms, to ensure the accuracy of subsequent corrections.

[0043] Step 2, Quantitative Correction of Artifacts. The cell multi-well plate is replaced with a uniform fluorescence calibration plate. The spatial alignment module is then focused using the uniform fluorescence calibration plate to position it in the focused position. Excitation is performed using the uniform main light source calibrated in Step 1, and the image acquisition camera captures a fluorescence image. Normally, the fluorescent material on the calibration plate is uniformly distributed, so its fluorescence intensity should be uniform. However, due to optical absorption or scattering in the electrode region where the wells are located, the fluorescent material around the wells is severely affected, resulting in a significantly weakened signal and artifacts. The software unit uses the electrode mask generated in Step 1 to calculate the average fluorescence intensity of all regions outside the electrodes. This average fluorescence intensity should be an ideal fluorescence reference intensity without electrode influence. The software generates an artifact correction map. In this map, all pixel values ​​outside the electrode region are defined as 1, while pixel values ​​in the electrode region are greater than 1, representing the factor by which the signal is weakened due to electrode artifacts in this region and needs to be compensated. This factor can be obtained by dividing the average fluorescence intensity of the non-electrode region by the measured fluorescence intensity of the electrode region.

[0044] Step 3: Data Correction Application. In subsequent cell experiments, each frame of the original fluorescence image captured by the image acquisition camera will be multiplied pixel-by-pixel with this corrected spectrum. This calculation is completed in real-time or near real-time in the computing unit, resulting in an image corrected for electrode artifacts. This restores the true fluorescence intensity of areas obscured or interfered with by electrodes, greatly improving the accuracy and reliability of quantitative fluorescence analysis. Example 2

[0045] This embodiment discloses a spatiotemporal calibration method for whole-board high-throughput photoelectric joint detection, referring to... Figure 4 The method covers the fine-grained control and calibration of key spatiotemporal parameters throughout the entire experimental process.

[0046] The method includes the following steps: Step S1: Establish a unified master-slave clock architecture, determine the start and end times of each complete exposure window for each image, and use the arithmetic midpoint of these two times as the high-precision timestamp of the image, and store the image data and timestamp in a strong binding manner.

[0047] In the above steps, during system initialization, the timer configured in the main control device generates a TTL trigger pulse signal synchronized with the electrophysiological signal sampling clock to achieve strict synchronization with the imaging frame rate. For each image, after generating the trigger pulse corresponding to that image, the main control device continues to monitor the level changes of the exposure valid and frame valid signals connected to the image acquisition camera. When the image acquisition camera actually begins exposure, the exposure valid signal changes from low to high. The main control device acquires this rising edge moment and uses it as the start moment of a complete exposure window for that image. After exposure ends, the frame valid signal changes from low to high. The main control device acquires this rising edge moment and uses it as the end moment of a complete exposure window for that image. The software unit acquires the start and end moments of this complete exposure window for each image and uses the arithmetic midpoint of these two moments as the final high-precision timestamp for that image. Each image data and the calculated high-precision timestamp are strongly bound together and stored in the high-speed data storage unit.

[0048] Step S2: Multi-level active whole-plate dynamic focal plane compensation. The compensation includes initial static focal plane calibration, long-term macroscopic dynamic deformation real-time adaptive compensation, and high-speed real-time microscopic dynamic focal plane tracking.

[0049] The initial static focal plane calibration includes the following sub-steps: Step S211, coarse adjustment of partitioned images and preliminary terrain construction: The system software logically divides the complete field of view of the image acquisition camera into equal virtual grids, such as 8×8 or 12×12 sub-regions. A stepper motor drives the stage platform to scan along the Z-axis, for example, within a range of ±200 micrometers with a step size of 10 micrometers. The camera acquires one frame of image at each position. The software calculates the sharpness evaluation value of the image within each independent partition in parallel. The sharpness evaluation value can be the Fourier high-frequency component energy or the local variance of the image, and can be accelerated on a GPU. By Gaussian fitting of the sharpness-Z-axis position curve, the optimal focal plane height of the partition is determined, thereby constructing a preliminary set of three-dimensional coordinate points for the porous plate, revealing its overall tilt and local warping.

[0050] Step S212, Laser Refinement Mapping and Model Correction: On the reference plane initially determined by the partitioning, for example, within any z-axis window from +50 micrometers to -50 micrometers, a non-contact high-resolution laser displacement sensor is activated. The stage XY platform moves to a set of important measurement point arrays, such as a 25×25 grid, and the laser sensor records the Z-axis height of each point. The repeatability of this measurement is better than 0.1 micrometers. This set of high-resolution laser data is used to correct or replace the initial terrain model, using a 3D surface fitting algorithm (such as Kriging interpolation) to generate a final 3D topographic map of the plate surface, which contains sub-micrometer-level elevation information of the plate surface.

[0051] Step S213, Initial Active Tilt and Warp Compensation: The software obtains the final 3D topographic map data and uses the least squares method to fit a spatial mathematical model that best represents the cell growth surface of the well plate, such as a third-order polynomial model, to capture more complex warp information. Using this spatial mathematical model, the software calculates the precise distances by which at least three piezoelectric nanopositioners extend or contract to make the surface horizontal (i.e., coplanar with the objective lens focal plane). The controller sends displacement commands to each piezoelectric nanopositioner, which move synchronously to complete and lock the physical attitude of the stage platform with nanometer-level precision, thereby optimizing the focal plane.

[0052] Long-term macroscopic dynamic deformation real-time adaptive compensation refers to the system activating a background dynamic drift monitoring mode after initial static calibration. This involves periodically checking image sharpness at 3-5 preset key monitoring points or representative areas of the board surface using rapid, low-excitation light exposure. If the focal plane deviates from the initial calibration position, with a drift on the order of micrometers, it is determined to be dynamic deformation drift. Based on the drift trend and magnitude, such as linearly increasing drift, the system can predict future drift and pre-compensate, or directly recalculate and perform a background physical attitude adjustment to bring the board surface back to the optimal focal plane. This process does not interrupt main data acquisition, and appropriate algorithms ensure smooth fine-tuning movements with minimal impact on imaging, such as the low-speed adjustment mode of a piezoelectric positioner.

[0053] High-speed real-time microscopic dynamic focal plane tracking includes: initializing a pre-trained deep learning network simultaneously with high-speed image data acquisition. The computing unit receives and processes image data acquired per (or several) frames in real time, extracts cell structure features through the deep learning network, and performs image processing on each sub-region of the camera's field of view in parallel. The deep learning network directly outputs or infers image sharpness metrics; for example, each sub-region outputs a focal plane offset value from -5 micrometers to +5 micrometers. The focal plane offset estimates for all sub-regions are summarized, and multi-point interpolation methods (e.g., thin-plate spline interpolation, radial basis function interpolation, etc.) are applied to calculate the minute tilt of the current porous plate surface and the corresponding drift along the z-axis. The real-time calculated micrometer- or sub-micrometer-level drift is fed in real-time to the controller of three closed-loop piezoelectric nanopositioners. The piezoelectric actuators perform continuous, high-bandwidth tracking fine-tuning at a high rate to compensate for focal plane drift in real time, ensuring the focal plane remains coplanar with the cell growth layer.

[0054] Step S3, Adaptive Light Source Shaping. Based on the partition contribution model, the LED array light source is self-calibrated. The self-calibration process, combined with the 3D topographic map generated in step S2, establishes an accurate mathematical model that compensates for light intensity distance attenuation, and solves in reverse to obtain a set of optimal driving current combinations to achieve illumination uniformity across the entire board.

[0055] In this step, before performing this calibration, the panel morphology compensation in step S2 is completed first, and the generated 3D topographic map is retrieved. Each LED section of the matrix array light source is sequentially and individually illuminated with a reference current, and images are captured by the camera. These images are then processed by the software unit to obtain the light intensity contribution map of that LED section. The software unit establishes a comprehensive model, for example... This model describes the relationship between the illuminance at the imaging surface location and the driving current of each LED, as well as the actual distance from that point to the light source. The model accurately considers the attenuation of light intensity with distance. The algorithm works backwards to find an optimal combination of driving currents C_1, ..., C_N, minimizing the variance of the final illuminance I(x,y) across the entire critical region, thus achieving an illumination uniformity better than 95% across the entire plate. The optimal current combination is saved as the illumination recipe for that plate, which can be directly called in subsequent experiments to ensure that the illumination conditions are completely consistent and uniform in each experiment.

[0056] Step S4, Electrode Artifact Localization and Correction. Optical artifacts introduced by the transparent electrodes are identified and quantitatively corrected. The correction process involves generating an electrode mask using multi-angle auxiliary illumination, generating an artifact correction spectrum using a specially designed uniform fluorescence calibration plate, and then applying the correction spectrum to subsequently acquired raw fluorescence images.

[0057] In this step, the automatic segmentation of the electrode region (step S41) involves: turning off the main light source and using multiple oblique auxiliary LED light sources for illumination, for example, four LEDs illuminating at 45-degree angles from the N, S, E, and W directions respectively, to acquire images. The software unit performs pairwise difference analysis on the acquired images to enhance the contour contrast of the electrodes. Adaptive thresholding and morphological processing are then applied to the differencing images to generate automatically segmented electrode masks that correspond to the pixels of the actual images, achieving sub-pixel level boundary recognition accuracy.

[0058] Quantitative correction of artifacts (step S42): Replace the cell multi-well plate with a uniform fluorescence calibration plate, and focus according to the focusing procedure of step S2 in the above embodiment to place the calibration plate at the optimal focal plane. Excite with the uniform main light source calibrated in step S3 above and capture a fluorescence image. The software unit uses the electrode mask generated in step S41 to calculate the average fluorescence intensity of all non-electrode regions. Generate an artifact correction map, where the value of non-electrode regions is approximately 1, while the value of electrode regions is greater than 1, indicating the magnitude of correction (compensation) required at that location.

[0059] Application of data correction (step S43): In subsequent cell experiments, the original fluorescence image acquired for each frame is multiplied pixel by pixel with the above-mentioned correction spectrum to correct the image of electrode artifacts. Example 3

[0060] This embodiment specifically illustrates the application and performance advantages of the spatiotemporal calibration system and method for whole-plate high-throughput photoelectric joint detection of the present invention in a specific experimental scenario. In a neuronal network synchronous activity detection experiment, a microelectrode array (MEA) multiwell plate containing 60 wells was used. The bottom of the MEA plate integrated transparent ITO electrodes for simultaneously recording neuronal electrophysiological signals and calcium ion fluorescence signals. The experiment lasted for 4 hours, with an image acquisition frame rate of 100Hz and an exposure time of 5 milliseconds.

[0061] System configuration, refer to Figure 5 : The stage platform, made of Invar alloy, measures 200×200 mm and features a black anodized surface to reduce stray light. Three closed-loop piezoelectric nanopositioners, model PhysikInstrumente P-733.3CD, are mounted on its bottom via high-rigidity connectors. A Micro-Epsilon optoNCDT 1700BL confocal sensor is used for the non-contact laser displacement sensor. The Z-axis platform utilizes a Newport M-ILS250CC linear stepper motor platform. The matrix array light source consists of 8×8 (64 in total) independently controlled custom LED modules, each containing multiple 520nm green LED chips. The auxiliary illumination source comprises four independent white LED arrays, illuminating the four sides of the plate at 45-degree angles; the brightness of each LED array can be independently controlled. The specially designed glass-bottomed calibration plate is the same size as the MEA plate, with a PMMA film uniformly coated on the bottom, into which Rhodamine 6G fluorescent dye (concentration of 100 μmol / L) is uniformly embedded. Its excitation / emission spectrum (excitation peak 529 nm, emission peak 554 nm) is compatible with the calcium probe Fluo-4 used in the experiment (excitation peak 494 nm, emission peak 516 nm).

[0062] Calibration and experimental procedures: 1. Initial static focal plane calibration: Coarse adjustment of partitioned images: The system virtually divides the camera's field of view into a 10*10 grid (each partitioned image is 204×204 pixels in size). The Z-axis platform sweeps across a 200-micrometer range below and above the target focal plane with a step size of 5 micrometers, for a total of 81 Z-axis positions. The computing unit uses an algorithm based on the tenengrad operator (an algorithm that calculates the sum of squared gradients of the image) to perform sharpness estimation in parallel for each partitioned image (average processing time 0.8 milliseconds). By fitting the sharpness-Z curve of each region with a Gaussian, a preliminary topographic map is obtained, showing that the MEA panel has an initial overall tilt of approximately 0.2 degrees and a local warp of 10 micrometers.

[0063] Laser-based fine mapping: A laser displacement sensor scans a 40×40 grid across the MEA (Medium-Advanced Area) surface at 500-micrometer intervals, totaling 1600 points. The acquired laser data undergoes anomaly removal using the RANSAC (Random Sample Consensus) algorithm (threshold 0.5 micrometers), followed by registration and optimization with the preliminary topographic map using the ICP algorithm, outputting a sub-micrometer-level 3D topographic map of the MEA surface.

[0064] Initial active tilt and warp compensation: The software uses a 3D topographic map to fit a third-order polynomial surface model and calculates the displacement of the three piezoelectric nanopositioners. The positioners work together to actively adjust the MEA plate surface to be coplanar with the objective lens focal plane, with the maximum residual deviation between the plate surface and the focal plane being less than ±0.8 micrometers.

[0065] 2. Long-term macroscopic dynamic deformation compensation: During the 4-hour experiment, rapid low-dose image sharpness assessments were performed every 3 minutes at the four corners and center of the MEA plate (exposure time 100 ms, excitation light intensity 10% of normal image acquisition). Once the focal plane deviated by more than ±1.5 micrometers at any position, background fine-tuning was initiated, using a piezoelectric nanopositioner to smoothly pull the focal plane back at a speed of 0.5 micrometers / second. This step was completed within 1 second of the main imaging acquisition being paused.

[0066] 3. High-speed real-time microscopic dynamic focal plane tracking: A pre-trained CNN model is launched, trained on a dataset containing neurons and various off-focus images, including 50,000 labeled images. During 100Hz image acquisition, the computational unit processes the image every 5 frames, dividing the field of view into 3×3 sub-regions. The CNN model outputs the focal plane offset (range ±5 micrometers) for each sub-region. Then, the radial basis function calculates a small overall tilt and Z-axis drift. At a frequency of 200Hz, the focal plane is corrected by feedback before the next measurement.

[0067] 4. Adaptive light source uniformity correction: Obtaining slab morphology information: using the 3D topographic map generated in step 1.

[0068] Light intensity contribution map measurement: Each of the 64 LED modules is lit sequentially, with a reference current of 150 mA, and the contribution map is obtained by taking a picture with a camera.

[0069] Comprehensive model establishment: Based on the inverse square law of light intensity attenuation and three-dimensional topographic maps, a comprehensive model was established. The model, where k_i is obtained through pre-calibration.

[0070] Optimal current combination solution: Using the gradient descent optimization algorithm, the optimal driving current combination for 64 LED modules was solved through 500 iterations, with the goal of achieving an overall illuminance uniformity of better than 98% (i.e., CV less than 2%). The solved current combinations were saved as illumination recipes and automatically loaded before each experiment.

[0071] 5. Electrode artifact correction: Automatic electrode region segmentation: The main light source is turned off, and four auxiliary LED light sources are lit sequentially to acquire images. Image difference operations are performed on the illumination images in adjacent diagonal directions (this method can greatly increase contrast), followed by morphological processing (3×3 circular structuring element, two opening operations, and two closing operations) to obtain an accurate binarized electrode mask with sub-pixel recognition accuracy.

[0072] Quantitative artifact correction: Replace the MEA plate with a uniform fluorescence calibration plate and focus. Excite with the calibrated uniform main light source and capture a fluorescence image. The software calculates the average fluorescence intensity in the non-electrode regions using the electrode mask and then generates an artifact correction spectrum, where the correction factor ranges from 1.2 to 1.6 in the electrode region.

[0073] Data correction application: In neuron experiments, each frame of calcium fluorescence image is multiplied pixel-by-pixel with the artifact correction map in real time.

[0074] Experimental results: Using the above system and method, the synchronous calcium transient activity of neural networks was recorded with high fidelity.

[0075] Time synchronization accuracy: The timestamps of image frames and electrophysiological signal sampling points are aligned with each other by less than 1 microsecond, ensuring the precise correlation of photoelectric signals.

[0076] Focal plane stability: During the 4-hour experiment, the focal plane stability of the entire MEA plate remained within ±0.4 micrometers, ensuring the clarity of cell images and the stability of fluorescence signal acquisition, and the cell movement trajectory was clearly discernible.

[0077] Illumination uniformity: The excitation illumination uniformity of the entire MEA plate surface reaches 98.5% (CV 1.5%), eliminating the fluorescence intensity differences caused by uneven illumination, and enabling direct quantitative comparison of cell fluorescence signals in different regions.

[0078] Electrode artifact correction: After correction, the fluorescence signal attenuation in the ITO electrode area is effectively restored, and the difference in fluorescence signal between the electrode area and the non-electrode area is greatly reduced, significantly improving the accuracy of quantitative analysis and enabling accurate analysis of cells below the electrode.

[0079] This invention, by integrating a multi-level, active calibration mechanism, comprehensively solves the complex technical challenges faced by high-throughput photoelectric joint detection, providing performance and reliability far exceeding traditional methods. The full disclosure of the technical solution ensures that those skilled in the art can readily understand and implement this invention based on the detailed description above, thereby obtaining the expected high-performance experimental results.

[0080] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A space-time calibration system for whole board high-throughput photoelectric combined detection, characterized in that, The system comprises: a time alignment module for accurately determining an image exposure time window and calculating a high-precision timestamp to achieve accurate synchronization of image data and continuous electrophysiological signals; a spatial alignment module for achieving multi-level focal plane compensation through initial physical compensation, macroscopic dynamic drift monitoring, and microscopic real-time dynamic tracking, to ensure that the multi-well plate cell growth layer and the imaging objective focal plane are always coplanar; an adaptive light source shaping system for using a plurality of light-emitting diode array light sources with independently controlled brightness, establishing a lighting model in combination with a three-dimensional topographic map of the plate surface through a self-calibration algorithm, and solving the optimal driving current combination to achieve excitation light uniformity; an electrode artifact positioning and correction system for generating an electrode digital mask using multi-angle auxiliary lighting and generating an artifact correction map using a calibration plate to quantitatively restore optical measurement errors introduced by transparent electrodes.

2. The spatio-temporal calibration system of claim 1, wherein, The time alignment module comprises a master control device for generating a synchronization trigger pulse and listening to a camera exposure state signal, an image acquisition camera for acquiring images and outputting an exposure state signal, and a software unit for calculating a high-precision timestamp according to the exposure state signal and storing the image data and the timestamp.

3. The spatio-temporal calibration system of claim 1, wherein, The spatial alignment module comprises a stage platform for carrying the multi-well plate, a plurality of piezoelectric nanometers for sub-micron level tilt and Z-axis height adjustment of the stage platform, a non-contact high-precision displacement sensor for measuring the Z-axis height of the multi-well plate surface, a stepper motor Z-axis platform for large-range coarse adjustment, and a computing unit for performing image processing and deep learning inference.

4. The spatio-temporal calibration system of claim 3, wherein, The spatial alignment module is configured to perform initial static focal plane calibration, which comprises: partitioned image coarse adjustment and preliminary terrain construction: dividing the camera field of view into a grid, driving the stage to scan, acquiring images, evaluating sharpness, and constructing a preliminary three-dimensional coordinate of the multi-well plate; displacement sensor fine mapping and model correction: enabling the displacement sensor, moving the stage to key mapping points, recording Z-axis height values, correcting the preliminary terrain model, and generating a final three-dimensional topographic map of the plate surface; initial active tilt and warping compensation: obtaining the topographic map and fitting a spatial mathematical model, calculating the extension distance of the piezoelectric nanometer, and driving it to perform physical posture adjustment and locking.

5. The spatio-temporal calibration system of claim 4, wherein, The spatial alignment module is configured to perform dynamic focal plane drift compensation, which comprises: long-time macroscopic dynamic deformation real-time adaptive compensation: after initial static calibration, starting background dynamic drift monitoring; periodically performing rapid sharpness evaluation or blurring detection on the preset monitoring area of the multi-well plate; if the focal plane deviation is detected to be over the threshold, determining that it is a dynamic deformation drift, recalculating and performing background physical posture fine adjustment according to the trend to restore the stage platform to the optimal focal plane; High-speed real-time microscopic dynamic focal plane tracking: inferring focal plane shift from image features during high-speed image data acquisition; real-time processing of image data, real-time processing of image data and extraction of cell features, evaluation of focal plane shift of multiple sub-regions in the camera field of view; comprehensive all sub-region shift estimation, calculation of the overall micro-tilt and Z-axis drift of the multi-well plate; real-time drift feedback to the piezoelectric nanometer positioner controller to drive continuous, high-bandwidth fine-tuning, real-time compensation of focal plane drift, ensuring that the focal plane and the cell growth layer are always coplanar.

6. The spatio-temporal calibration system of claim 1, wherein, The adaptive light source shaping system comprises: a matrix array light source composed of a plurality of independently controlled light-emitting diode units; a current driver for controlling each light-emitting diode unit; a software unit for acquiring a plate topography and a light intensity contribution map, establishing a light illumination mathematical model, and inversely solving an optimal driving current combination to achieve light illumination uniformity in the full plate range.

7. The spatio-temporal calibration system of claim 1, wherein, The electrode artifact positioning and correction system comprises: a plurality of auxiliary illumination light sources distributed at a set geometric angle; a glass bottom calibration plate with a stable fluorescent dye uniformly coated on the bottom; a software unit for generating an accurate electrode mask through image difference operation and morphological operation.

8. A spatio-temporal calibration method for whole board high-throughput photoelectric combined detection, applicable to the spatio-temporal calibration system according to any one of claims 1-7, characterized in that, The method comprises the following steps: Step S1, establish a master-slave clock unified architecture: determine the start and end time of the image exposure window, calculate the arithmetic midpoint as the image timestamp, and bind the image data with the timestamp to achieve accurate synchronization of image data and continuous electrophysiological signals; Step S2, multi-level active whole-plate dynamic focal plane compensation: perform initial static focal plane calibration, long-time macroscopic dynamic deformation real-time adaptive compensation, and high-speed real-time microscopic dynamic focal plane tracking on the multi-well plate to ensure that the multi-well plate cell growth layer and the imaging objective focal plane are always coplanar; Step S3, adaptive light source shaping: based on partition contribution modeling, self-calibration of the light-emitting diode array light source, establishment of a light illumination mathematical model combined with the three-dimensional topography generated in step S2, and solution of the optimal driving current combination to achieve pixel-level uniformity of the excitation light; Step S4, electrode artifact positioning and correction: identify and quantitatively correct the optical artifacts introduced by the transparent electrode, which includes generating an electrode mask using multi-angle auxiliary illumination and generating an artifact correction map using a calibration plate, and then applying the map to the original fluorescence images collected subsequently.

9. The spatio-temporal calibration method of claim 8, wherein, The step S1 specifically comprises: Step S101, establish a master-slave clock unified architecture: generate a trigger pulse sequence synchronized with the electrophysiological signal sampling clock during system initialization; Step S102, accurately capture the exposure window state: listen to the camera exposure state signal and record the start and end time of the exposure; Step S103, timestamp centroid calculation and data binding: obtain the start and end time of the exposure window, calculate the arithmetic midpoint as the timestamp, and store the image data and the timestamp strongly bound.

10. The spatio-temporal calibration method of claim 8, wherein, The step S2 specifically comprises: S201, initial static focal plane calibration: sub-region image coarse adjustment and preliminary topography construction, initial active tilt and warping compensation; S202. Dynamic focal plane drift compensation: long-time macroscopic dynamic deformation real-time adaptive compensation, high-speed real-time microscopic dynamic focal plane tracking.

Images