Microfluidic droplet array visual detection system and method based on fermentation strain screening

Abstract

The present application relates to the technical field of microfluidic visual detection, and specifically discloses a microfluidic droplet array visual detection system and method based on fermentation strain screening, which comprises a pre-calibration and initialization module for jointly calibrating the channel geometry of a microfluidic chip and an optical system to generate a motion-optical joint point spread function database. In the actual operation process of fermentation strain screening, the present application dynamically correlates optical imaging, image restoration and the fluid state of the microfluidic channel to build a closed-loop detection control mechanism that adapts to the fluctuation of the flow field. The system can actively adjust the camera imaging parameters according to the real-time droplet flow rate and perform dynamic focusing, and at the same time, the pre-calibrated joint point spread function database is used to deblur and restore the multi-channel images, thereby overcoming the image distortion problem in the high-speed flow environment from the dual perspectives of physical acquisition and software reconstruction.

Description

Technical Field

[0001] This invention relates to the field of microfluidic visual inspection technology, and in particular to a microfluidic droplet array visual inspection system and method based on fermentation strain screening. Background Technology

[0002] In high-throughput screening of fermentation strains, microfluidic droplet technology is often used to encapsulate and culture single-clonal strains. Existing microfluidic vision inspection systems typically employ fixed-parameter optical imaging devices to acquire fluorescence images of continuously flowing droplet arrays within the channels of a microfluidic chip. Subsequently, image processing algorithms are used to extract the fluorescence characteristics of the droplets to assess the metabolic capacity of the strains and trigger downstream sorting actuators.

[0003] In practical high-throughput screening scenarios, the flow velocity of droplets in microfluidic channels is high and generally exhibits nonlinear fluctuations. Imaging methods with fixed parameters are difficult to match the real-time flow velocity and are prone to motion blur. Combined with the diffraction limitations of the optical system, this results in blurred edges and internal features of the acquired droplet images.

[0004] Meanwhile, traditional image analysis methods typically treat each frame as an independent static target, lacking real-time coordination with the channel's fluid dynamics. This makes it difficult for the system to maintain continuous and accurate droplet target segmentation and trajectory tracking when the flow field fluctuates or the droplets undergo non-rigid deformation. This lack of image restoration capability and trajectory prediction deviation are directly passed on to subsequent stages, causing the calculation of the sorting trigger moment to deviate from the actual physical arrival position of the droplets, resulting in missorting or missed detection of target fermentation strains. Summary of the Invention

[0005] This invention aims to at least partially address one of the technical problems in related technologies. Therefore, the objective of this invention is to propose a microfluidic droplet array visual detection system and method based on fermentation strain screening, in order to improve the accuracy of target fermentation strain identification.

[0006] To achieve the above objectives, a first aspect of the present invention provides a microfluidic droplet array visual inspection system based on fermentation strain screening, comprising:

[0007] The pre-calibration and initialization module is used to jointly calibrate the channel geometry and optical system of the microfluidic chip, generate a database of motion optical joint point diffusion functions, and construct a dynamic sliding region of interest template along the channel flow.

[0008] The active imaging control module is used to dynamically adjust camera parameters based on the real-time flow velocity of the channel, perform three-dimensional morphological perception and dynamic focusing on the local area of ​​the channel centerline, and simultaneously acquire multi-channel fluorescence images with different excitation intensities.

[0009] The image restoration and fusion module is used to perform joint restoration of motion blur and diffraction blur on multi-channel fluorescence images based on the joint point spread function database, and to perform adaptive fusion and directional background correction based on theoretical contrast gain.

[0010] The droplet tracking and segmentation module is used to predict the droplet position for the dynamic sliding region of interest using a Kalman filter, perform non-rigid instance segmentation based on the prediction space constraints, and complete the inter-frame droplet trajectory association.

[0011] The high-dimensional feature extraction and anomaly removal module is used to extract the spatiotemporal incremental features of droplets to generate spatiotemporal high-dimensional feature vectors, and to identify and remove abnormal droplets based on a multi-dimensional anomaly judgment model.

[0012] The closed-loop sorting control module is used to calculate the sorting trigger time based on droplet trajectory prediction to drive the microfluidic sorting valve, and to adaptively correct the system parameters based on the sorting results.

[0013] To achieve the above objectives, a second aspect of the present invention proposes a visual detection method for microfluidic droplet arrays based on fermentation strain screening, the method comprising:

[0014] The channel geometry and optical system of the microfluidic chip are jointly calibrated to generate a database of motion optical joint point diffusion functions and to construct a dynamic sliding region of interest template for flow along the channel.

[0015] Based on the real-time flow velocity of the channel, the camera parameters are dynamically adjusted to perform three-dimensional morphology perception and dynamic focusing on the local area of ​​the channel centerline, and multi-channel fluorescence images with different excitation intensities are acquired simultaneously.

[0016] Based on the joint point spread function database, motion blur and diffraction blur are jointly restored in multi-channel fluorescence images, and adaptive fusion and directional background correction are performed based on theoretical contrast gain.

[0017] The droplet position is predicted for the dynamic sliding region of interest using a Kalman filter, and non-rigid instance segmentation is performed based on the prediction space constraints to complete the inter-frame droplet trajectory association.

[0018] Extract the spatiotemporal incremental features of droplets to generate a high-dimensional spatiotemporal feature vector, and identify and remove abnormal droplets based on a multi-dimensional anomaly detection model;

[0019] The sorting trigger time is calculated based on droplet trajectory prediction to drive the microfluidic sorting valve, and the system parameters are adaptively corrected based on the sorting results.

[0020] To achieve the above objectives, a third aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory. When the computer program is executed by the processor, it implements the above-described microfluidic droplet array visual detection method based on fermentation strain screening.

[0021] In the actual screening process of fermentation strains, the technical solution of this invention establishes a dynamic correlation between optical imaging, image restoration, and the fluid state of the microfluidic channel, constructing a closed-loop detection and control mechanism adapted to flow field fluctuations. The system can actively adjust camera imaging parameters and perform dynamic focusing based on the real-time droplet velocity. Simultaneously, it utilizes a pre-calibrated joint point spread function database to perform targeted deblurring and restoration of multi-channel images, overcoming image distortion problems in high-speed flow environments from both physical acquisition and software reconstruction perspectives. Combining dynamic sliding regions of interest and prediction space constraints, the system achieves continuous trajectory tracking and non-rigid segmentation of deformed droplets in complex flow phases, ensuring the coherence of spatiotemporal feature extraction and the reliability of anomaly removal. Finally, relying on a trajectory prediction model, it accurately pre-calculates the sorting trigger time and uses sorting feedback for adaptive parameter correction, effectively eliminating the timing deviation caused by fluid velocity fluctuations in the sorting execution action, and improving the accuracy of target fermentation strain identification and final physical sorting. Attached Figure Description

[0022] Figure 1 This is a schematic diagram illustrating the implementation of the microfluidic droplet array visual inspection system based on fermentation strain screening provided by the present invention;

[0023] Figure 2 This is a three-dimensional energy distribution map of the joint point spread function (JM-PSF) of motion optics in the microfluidic droplet array visual inspection system based on fermentation strain screening provided by the present invention.

[0024] Figure 3 This is a comparison image of a multi-channel fluorescence adaptive fusion grayscale profile based on theoretical contrast gain in the microfluidic droplet array visual inspection system based on fermentation strain screening provided by this invention.

[0025] Figure 4 This is a scatter plot of the feature space clustering of the microfluidic droplet array visual inspection system based on fermentation strain screening provided by this invention.

[0026] Figure 5 This is a comparison of the covariance of trajectory tracking errors between standard Kalman filtering and controlled unscented Kalman filtering (C-UKF) under transient nonlinear distorted flow fields in the microfluidic droplet array visual inspection system based on fermentation strain screening provided by this invention;

[0027] Figure 6This is a three-dimensional grayscale topology comparison image of the microbubble optical scattering interference region before and after structural interpolation reconstruction in the microfluidic droplet array visual inspection system based on fermentation strain screening provided by this invention.

[0028] Figure 7 This is a time series response diagram of the rate of change of passive optical anchor Laplace variance and Z-axis focal plane compensation amount under continuous operation conditions in the microfluidic droplet array visual inspection system based on fermentation strain screening provided by the present invention.

[0029] Figure 8 This is a schematic flowchart of the microfluidic droplet array visual detection method based on fermentation strain screening provided by the present invention;

[0030] Figure 9 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0031] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0032] The following description, with reference to the accompanying drawings, describes a microfluidic droplet array visual inspection system, method, and electronic device based on fermentation strain screening, according to embodiments of the present invention.

[0033] Example 1:

[0034] like Figure 1 As shown in the figure, this embodiment provides a microfluidic droplet array visual inspection system based on fermentation strain screening. This system is mainly used in industrial and laboratory scenarios that require continuous, real-time, and high-precision detection and sorting of massive amounts of tiny droplets, such as high-throughput microbial screening, enzyme-directed evolution, and single-cell metabolite analysis.

[0035] Specifically, this embodiment of a microfluidic droplet array visual detection system based on fermentation strain screening includes: a pre-calibration and initialization module, used to jointly calibrate the channel geometry and optical system of the microfluidic chip, generate a joint point spread function database of motion optics, and construct a dynamic sliding region of interest template flowing along the channel; an active imaging control module, used to dynamically adjust camera parameters based on the real-time flow velocity of the channel, perform three-dimensional morphology perception and dynamic focusing on the local area of ​​the channel centerline, and simultaneously acquire multi-channel fluorescence images with different excitation intensities; and an image restoration and fusion module, used to perform motion optics on the multi-channel fluorescence images based on the joint point spread function database. The system employs a combined approach to restore dynamic and diffraction blur, performing adaptive fusion and directional background correction based on theoretical contrast gain. A droplet tracking and segmentation module predicts droplet positions for the dynamically sliding region of interest using a Kalman filter, performs non-rigid instance segmentation based on prediction spatial constraints, and establishes inter-frame droplet trajectory association. A high-dimensional feature extraction and anomaly removal module extracts spatiotemporal incremental features of droplets to generate spatiotemporal high-dimensional feature vectors, identifies and removes abnormal droplets based on a multi-dimensional anomaly detection model, and a closed-loop sorting control module calculates the sorting trigger time based on droplet trajectory prediction to drive the microfluidic sorting valve and provides feedback on the sorting results to adaptively correct system parameters.

[0036] Specifically, the system employs a hierarchical four-level asynchronous adaptive control architecture, including: a sorting synchronization control loop, used to match the movement of microfluidic droplets and issue sorting trigger commands; a parameter adaptive control loop, used to detect the fluid velocity within the channel and continuously refresh the camera frame rate, exposure time, and Kalman filter covariance matrix; an imaging optimization control loop, used to adjust the physical focusing position of the three-dimensional focusing unit and the excitation light radiation intensity parameters; and a model reconstruction control loop, used to extract global sorting feedback results and statistical deviations, self-correct the weight distribution of the motion optics joint point spread function database kernel, and the multi-dimensional anomaly detection model. In practical applications, this four-level asynchronous adaptive control architecture relies on the collaborative operation of a field-programmable gate array (FPGA) and a high-performance industrial computer. The FPGA is primarily responsible for the direct triggering and timing alignment of the underlying hardware devices, such as the synchronous control of camera exposure timing and light source flicker pulses, thereby ensuring the time determinism of the system when acquiring physical signals; the industrial computer handles complex image restoration, feature extraction, and matrix operations. The two hardware levels collaboratively construct a complete data link from physical layer acquisition to logical layer operation.

[0037] For example, the pre-calibration and initialization module is specifically used for: injecting standard fluorescent microspheres into the microfluidic chip, controlling the standard fluorescent microspheres to pass through the detection channel at different constant flow rates to obtain motion blur images; combining the reference image of the stationary microspheres, using a blind deconvolution algorithm to extract the motion blur kernel at each flow rate; combining the numerical aperture of the optical system and the pre-calibrated depth response curve, calculating the optical diffraction point spread function at the corresponding spatial position; and performing a convolution operation between the motion blur kernel and the optical diffraction point spread function to generate a database of motion-optical joint point spread functions mapping different flow rates and channel positions.

[0038] In actual calibration operations, the standard fluorescent microspheres injected into the microfluidic chip are typically polystyrene microspheres with stable excitation and emission spectra. Their physical diameter and refractive index are configured to closely resemble the physical properties of the subsequent actual fermentation droplets. The flow control system is configured with a high-precision injection pump to establish a stable laminar flow field. When the microspheres move in the flow field, due to the physical displacement of the microspheres during the camera exposure time, the photon distribution captured by the imaging sensor will produce a trailing effect along the direction of motion, resulting in motion blur. The role of the blind deconvolution algorithm is to separate the kernel function causing the blur from the blurred image and the prior noise distribution of the system, i.e., the motion blur kernel, using mathematical methods such as maximum likelihood estimation, without knowing the exact information of the real clear image. Meanwhile, the optical system itself is limited by the wave nature of light and has an inherent diffraction limit, which is reflected in the optical diffraction point spread function.

[0039] In this embodiment, the diffraction point spread function is mathematically represented using a Gaussian approximation model, the specific formula of which is as follows:

[0040] ;

[0041] in, The coordinates of the point located on the focal plane are The probability value of the diffraction intensity distribution corresponding to the spatial pixel point; This refers to the spatial coordinate reference value in the horizontal direction on the target surface of the imaging sensor. This refers to the spatial coordinate reference value in the vertical direction on the target surface of the imaging sensor; This parameter, which is the system diffraction standard deviation depending on the numerical aperture of the optical microscope objective and the center wavelength of the excitation light, reflects the inherent defocus range of the optical system.

[0042] By acquiring motion blur kernels at different constant flow rates and performing two-dimensional discrete convolution operations with the optical diffraction point spread function obtained from the above formula, the system can pre-build a multi-dimensional joint point spread function database in memory. This database will serve as the kernel benchmark for inverse deblurring operations in subsequent real-time detection, providing theoretical support at the physical optics level for restoring the true fluorescence distribution.

[0043] like Figure 2 This figure illustrates the three-dimensional energy distribution of the motion optics joint point spread function generated by the system's pre-calibration and initialization modules. The units for both the horizontal and vertical spatial coordinates are micrometers, with data ranges from -20 to +20. The vertical axis represents the normalized light intensity energy distribution, with a data range of 0 to 1.

[0044] As can be seen from the joint point diffusion energy surface marked in the figure, the energy distribution surface exhibits a symmetrical single-peak distribution characteristic in the vertical spatial coordinate direction. The highest light intensity energy is concentrated near the 0-micrometer position and gradually decreases smoothly to both sides. This objectively reflects the static defocus range caused by the inherent diffraction limit of the optical system.

[0045] In the horizontal spatial coordinate direction, the energy distribution surface underwent significant broadening and deformation, forming a high-energy flat-top broadened region spanning approximately 15 micrometers. This broadened region directly maps the actual physical displacement trajectory of the droplet within the microfluidic channel during high-speed flow, along the fluid direction within the camera exposure time—that is, motion blur.

[0046] The surface color smoothly transitions from a low-energy dark blue region at the periphery to a high-energy dark red region at the center, following an increasing energy gradient. This color gradient, along with the asymmetrical three-dimensional surface morphology, objectively quantifies the photon spatial dispersion pattern resulting from the coupling of static optical diffraction and dynamic hydrodynamic displacement. Using this three-dimensional energy distribution data as the computational kernel benchmark for the image restoration and fusion module, it can effectively restore stretched image pixels, correct edge feature distortion of droplets in high-speed flowing environments, and provide highly stable underlying visual data support for subsequent non-rigid instance segmentation and high-dimensional feature extraction.

[0047] Optionally, the dynamic sliding region of interest template is constructed as follows: an initial rectangular template is generated based on the pre-acquired coordinates of the microfluidic chip channel centerline and the average physical diameter of the droplet to be tested; the detection field of view is divided into multiple continuous sliding regions of interest along the channel centerline; a unique temporal identifier is assigned to each sliding region of interest, establishing a mapping relationship between the physical position of the field of view and the droplet's movement time, ensuring that the target droplet is anchored and tracked by a single sliding region of interest throughout the entire cycle of passing through the detection field of view. In microfluidic visual inspection, full-field image processing consumes a large amount of computational resources, and irrelevant disturbances in the background area can easily lead to algorithm misjudgments. By constructing a dynamic sliding region of interest template, the system allocates memory and computing power only within a specific spatial range for pixel-level processing.

[0048] The continuously sliding regions of interest (ROIs) divided along the channel centerline logically form a data processing pipeline. Once a timing identifier is assigned, it essentially becomes a pointer for droplet lifecycle management. The system's underlying layer maintains a hash table structure to strongly bind the timing identifier, the current frame index, and the corresponding spatial boundary coordinates of the sliding ROI, thereby maintaining continuous tracking of a single droplet in the continuous video stream and avoiding identity confusion when multiple targets move simultaneously.

[0049] Specifically, the active imaging control module includes: a flow rate monitoring unit, used to calculate the channel flow rate in real time by the physical displacement of droplets in the sliding region of interest in adjacent frame images; an adaptive exposure unit, used to dynamically adjust the sampling frame rate and exposure time of the camera according to the channel flow rate, and simultaneously control the intensity of the excitation light source; and a three-dimensional focusing unit, used to project an coded structured light pattern onto a preset range area based on the channel centerline, calculate the average depth value of the centerline through phase decoding, and drive the displacement stage to dynamically adjust the optimal optical focal plane.

[0050] In the above configuration, the flow velocity monitoring unit calculates the spatial geometric center offset of the sliding region of interest associated with the same temporal identifier in two adjacent image frames, and divides it by the time interval between the two frames to output the instantaneous channel flow velocity. The adaptive exposure unit works closely with the flow velocity monitoring unit. When an increase in flow velocity is detected in the channel, the adaptive exposure unit correspondingly increases the camera's sampling frame rate to ensure that a sufficient number of image frame sequences can be acquired during the passage of a single droplet through the entire detection field of view, preventing data truncation for feature extraction.

[0051] Meanwhile, the system proportionally shortens the exposure time to suppress the exacerbation of motion blur, and simultaneously increases the radiation power of the LED or laser excitation source to compensate for the decrease in total photon collection caused by the shortened exposure time. The three-dimensional focusing unit introduces a topography measurement technique based on fringe projection. The projected coded structured light pattern is typically a multi-frequency phase-shifting sinusoidal fringe. By performing unwrapping operations on the returned deformed fringe image, the system can calculate the minute depth-direction deformation of the fluid within the channel in real time. This depth deformation data is then converted into a driving voltage signal and applied to the piezoelectric ceramic displacement stage to achieve real-time physical focusing and tracking along the optical axis. This mechanism overcomes the optical defocusing problem caused by the expansion or warping of the flexible substrate under continuous high-pressure liquid injection conditions in microfluidic chips.

[0052] It should also be noted that when the image restoration and fusion module performs joint restoration and adaptive fusion, it adopts an iterative algorithm that introduces a local gradient adaptive regularization term, dynamically adjusts the regularization intensity based on the edge and flat region distribution characteristics of the multi-channel image, and performs joint deblurring; it extracts the gray value of each pixel in the multi-channel restored image and the regional background gray value obtained by the background extraction algorithm, and calculates the theoretical contrast gain value by combining it with the pre-calibrated system noise standard deviation; it generates a droplet region mask based on threshold segmentation, and performs a weighted summation operation on the multi-channel pixels in combination with the theoretical contrast gain value.

[0053] In this step, the multi-channel image originates from fluorescence signals of different intensities or bands segmented by an optical beam splitter assembly. The introduction of a local gradient adaptive regularization term in the iterative deblurring algorithm aims to sharpen droplet boundaries while suppressing noise amplification in the fluid background region. The regularization intensity is dynamically allocated based on the local gradient values ​​calculated by the image's Laplacian operator; weaker regularization constraints are applied to edge regions with large gradients to preserve high-frequency details, while stronger regularization constraints are applied to flat regions with small gradients to smooth noise. Background extraction algorithms are typically implemented based on a combination of morphological opening operations and mean filtering to separate the substrate fluorescence intensity that does not contain the target droplet signal.

[0054] To quantify the effective signal quality of each channel pixel, the system defines a theoretical contrast gain value, calculated as follows:

[0055] ;

[0056] in, The row index in the corresponding image matrix is Column index is The theoretical contrast gain value of the spatial pixel at that location; For the imaging sensor in spatial coordinates The original fluorescent grayscale scalar value captured at a specific pixel; This is the local background fluorescence grayscale reference value of the fluid channel corresponding to the region where the specific pixel is located; This represents the standard deviation of the global static dark current noise and shot noise distributions of the imaging system, extracted through pre-calibration. The physical meaning of this formula is to measure the signal-to-noise ratio (SNR) of the target's effective signal strength relative to the system's inherent background noise.

[0057] For example, a weighted summation operation of multi-channel pixels generates the final adaptive fused image. The calculation formula is:

[0058] ;

[0059] in, This is a preset droplet region weighting constant; Channel number index for fluorescence images; For the first Pixels in a channel image The grayscale scalar value; and These are the fluorescence image channel number indices in the cross-fusion operation, and are related to... Belonging to the same channel index set; and Corresponding to the first With the The grayscale scalar values ​​of each channel; In order to be with the first The basic channel weighting coefficients that are positively correlated with the channel theory contrast gain value; In order to be with the first Channel and the The cross-fusion weighting coefficient is positively correlated with the absolute value of the theoretical contrast gain difference between channels.

[0060] The first term in the formula is primarily responsible for aggregating common high-intensity features from multiple channels, enhancing the image contribution of channels with high signal-to-noise ratios through the weighting coefficients of the basic channels. The second term focuses on extracting nonlinear difference features across channels. This adaptive fusion method based on theoretical contrast gain effectively addresses the engineering dilemma of signal overexposure or insufficient signal-to-noise ratio that easily occurs when a single fluorescence channel is exposed to different concentrations of fermentation products, providing fused image data with a wide dynamic range and stable structure for subsequent high-dimensional feature extraction.

[0061] like Figure 3 This figure demonstrates the comparison of multi-channel fluorescence adaptive fusion grayscale profiles based on theoretical contrast gain. The horizontal axis represents the pixel spatial location in pixels, with a data scale ranging from 0 to 100; the vertical axis represents the fluorescence grayscale intensity in dimensionless units, with values ​​set between 0 and 250.

[0062] The figure contains two contrasting curves: the blue dashed line represents the original grayscale curve of a single channel, and the red solid line represents the adaptive fusion enhanced grayscale curve.

[0063] As can be seen from the trend of the waveform transformation, in the fluid background area outside the pixel spatial position 20 and 80, the original blue grayscale curve has significant irregular fluctuations around the value 30. This objectively reflects the unavoidable interference of background scattered light and sensor shot noise in a single detection channel.

[0064] After multi-channel weighted summation and cross-fusion processing, the red enhanced grayscale curve values ​​in the same background area were significantly suppressed and smoothed to below 10, indicating that background correction effectively reduced the base noise.

[0065] In the high-expression region at the center of the droplet near pixel location 50, the peak value of the original blue grayscale curve only reaches about 120 and the edge gradient is relatively gentle, resulting in a low signal-to-noise ratio of the effective signal.

[0066] In contrast, the peak value of the red enhanced grayscale curve in the center region of the droplet is significantly stretched to around 220, and the waveform edges exhibit steeper rising and falling gradients. This expansion of the central amplitude of the red solid line relative to the blue dashed line, as well as its suppression of background noise, objectively confirms that using the contrast gain value of multi-channel theory to assign weights can effectively aggregate the image contribution of high signal-to-noise ratio channels and extract nonlinear difference features across channels.

[0067] The grayscale profile comparison quantifies the practical effect of the fusion algorithm in improving the dynamic range of the effective fluorescence signal of the droplet and enhancing the contour sharpness, providing reliable underlying image data support for the subsequent system to perform high-precision non-rigid instance segmentation and static fluorescence dimensional feature extraction.

[0068] Specifically, the droplet tracking and segmentation module is used to: define the spatial coordinates of the droplet centroid and its orthogonal velocity components as the state vector of the Kalman filter; dynamically adjust the covariance matrix of the prediction error according to the fluctuation amplitude of the channel flow velocity to predict the expected spatial boundary of the droplet in the sliding region of interest in the current frame; calculate the edge curvature tensor of the image in the sliding region of interest; use the expected spatial boundary as an internal forced marker, and combine it with the edge curvature tensor to perform a marker-controlled watershed transformation to output a precise binary mask of the droplet with non-rigid deformation.

[0069] During the movement of a droplet with the fluid, it is frequently affected by local eddies, fluid compression, or other micro-perturbations, and its actual position often does not strictly follow a uniform linear motion model. The Kalman filter is used here as a state estimator; its state vector, containing coordinate and velocity information, can be combined with kinematic equations to deduce the prior distribution for the next moment. Dynamically adjusting the covariance matrix based on the velocity fluctuation amplitude essentially assigns trust weights between the system's prediction model and actual observations. Significant fluctuations increase the system noise covariance, forcing the filtering algorithm to rely more heavily on actual edge observations in the current frame. The edge curvature tensor is obtained by calculating the eigenvalues ​​of the second-order Hessian matrix of the image; it not only identifies the droplet's edge position but also reflects the degree of local edge curvature.

[0070] Watershed transform is a morphological segmentation algorithm based on topological theory. Direct application of this algorithm can easily lead to severe oversegmentation. This embodiment introduces the expected spatial boundary predicted by Kalman filtering as an internally forced marker pole, limiting the initial marker region of the water immersion process. This ensures that the algorithm ultimately generates only a single and continuous closed contour, i.e., a precise binary mask. This mask can realistically reproduce the non-rigid deformation of a droplet under shear forces in the channel, such as ellipsoidal stretching or even local concavity, thus providing high-precision pixel isolation for extracting geometric features such as area and roundness.

[0071] For example, the multi-dimensional anomaly detection model is constructed based on a four-dimensional feature space. Its working logic includes: extracting geometric dimension features to detect anomalies in morphological distribution and area ratio; extracting static fluorescence dimension features to detect anomalies in fluorescence peak distribution; extracting dynamic metabolic dimension features to detect anomalies in fluorescence intensity variation coefficient and time series fit goodness; extracting motion dimension features to detect anomalies in velocity variance and trajectory deviation; the detection conditions of each dimension feature correspond to the set risk assessment quantitative score, and when the cumulative total score of the droplet exceeds the preset safety threshold, an abnormal droplet rejection instruction is triggered.

[0072] In practical scenarios of fermentation strain screening, the sources of abnormal droplets are complex and diverse. Geometric features are mainly used to identify physical generation defects. For example, abnormal area ratios often indicate droplet merging or breakage in the microfluidic generation array, while morphological distribution distortions may indicate an imbalance in the interfacial tension between the dispersed and continuous phases within the droplet. Static fluorescence features examine the spatial distribution of fluorescence signals. Abnormal fluorescence peak distribution can effectively identify local pseudo-highlight regions caused by the aggregation of impurity fluorophores or microparticle contamination, avoiding misidentification of impurities as target strains. Dynamic metabolic features are the core dimension for characterizing biological activity. During cultivation, fermentation strains consume substrates and release fluorescent metabolites, and their fluorescence intensity should exhibit a specific growth curve pattern over time. Abnormal time series goodness of fit indicates that the droplet may contain dead or dormant bacteria, leading to irregular fluctuations in the fluorescence signal. Motion features are used to assess flow field stability. Abnormal trajectory deviations often indicate micro-blockages or protein adsorption on the channel walls, altering local flow resistance.

[0073] By weighting the aforementioned four-dimensional features and assigning them a quantitative risk assessment score, the system no longer relies on a single absolute threshold for classifying and determining target retention and rejection. Instead, it constructs an expert system with a certain degree of fault tolerance and comprehensive judgment. Abnormal droplets exceeding the preset safety threshold will have their subsequent lifecycle tracking immediately terminated in the algorithm logic and will be marked as discarded items when the sorting instruction is issued, thereby saving sorting computing power and ensuring the purity of the final screened products.

[0074] like Figure 4The figure illustrates the clustering distribution of the multi-dimensional anomaly detection model in the feature space. The three axes in the figure represent the three core extracted features: geometric area ratio, fluorescence coefficient of variation, and motion velocity variance, respectively, with units of dimensionless, percentage, and square millimeters per square second.

[0075] As can be seen from the spatial distribution of various colored scatter points in the figure, the blue scatter points representing normal target droplet clusters are closely clustered in a specific low-risk area in the three-dimensional data space, with their geometric area ratio concentrated around 1.0, fluorescence variation coefficient below 30, and motion velocity variance below 0.5.

[0076] In contrast, the other colored scattering points representing anomalous droplet clusters are significantly deviated from this core region. The red geometrically distorted anomalous clusters are scattered in areas with an area ratio greater than 1.6 or less than 0.4, which intuitively reflects the physical defects of droplet breakage or fusion that occur during the generation of the microfluidic array.

[0077] The green abnormal clusters of metabolic inactivation shifted significantly upwards to over 30 on the fluorescence variation coefficient axis, objectively reflecting the intense and irregular fluctuations in fluorescence signal caused by the internal encapsulation of dead or dormant bacteria.

[0078] The purple flow field trapped anomaly clusters significantly exceed the normal boundary of 0.5 on the velocity variance axis, indicating that the trajectory anomaly is caused by a minor blockage in the local channel.

[0079] The high-discriminative spatial boundaries and clustering distribution of these multidimensional scatter points objectively demonstrate that the multidimensional anomaly detection model can overcome the limitations of single physical or biochemical indicators and establish a high-dimensional risk quantification space with comprehensive classification capabilities. By effectively defining the distribution boundaries between normal and various abnormal droplets at the data level, this model provides a highly accurate target identification basis for the closed-loop sorting control module, thereby ensuring the detection efficiency and product purity of the final fermentation strain screening.

[0080] Specifically, the closed-loop sorting control module accurately calculates the sorting trigger time. The logical relationship is as follows:

[0081] ;

[0082] in, The current absolute system time at which the prediction calculation is performed; The spatial offset distance of the target droplet relative to the physical reference point at the center of the optical detection field of view along the longitudinal direction of the microfluidic channel at the current system moment; The instantaneous physical velocity of the target droplet along the longitudinal extension direction of the microfluidic channel is calculated. The fixed geometrical straight-line distance between the physical reference point at the center of the optical detection field of view and the fluid flow direction changing action port of the microfluidic sorting valve; The total hardware response delay time is the sum of the mechanical movement time of the microfluidic sorting valve body and the signal transmission time of the associated electrical control circuit.

[0083] Microfluidic sorting typically employs physical mechanisms such as dielectrophoresis, surface acoustic waves, or microvalve aerodynamic deflection. Regardless of the mechanism used, there is always a hardware delay between the system issuing a command and the actual deflection of the flow field. The above logical relationship establishes a cross-domain connection between the absolute time domain of the system and the spatial domain of the physical channel, combining the fixed geometric physical straight-line distance with the instantaneous physical velocity to deduce the theoretical flight time of the droplet reaching the sorting node.

[0084] Subsequently, by introducing the total comprehensive hardware response delay time for feedforward timing deduction compensation, it is ensured that the high and low level transition signals output by the control system accurately match the physical arrival window of the target fermentation droplets. The adaptive correction of system parameters based on the sorting results involves setting up quality inspection observation points downstream, statistically analyzing the missorting rate, and feeding the error back into the time compensation parameters in the above formula to offset long-term system-level errors caused by piezoelectric ceramic aging or slow drift due to fluid resistance.

[0085] In summary, given the current state of microfluidic droplet screening, which typically employs fixed-parameter imaging equipment and lacks image-fluid dynamic coupling compensation, existing technologies often face severe technical bottlenecks such as motion blur, feature distortion, and sorting time-series drift in high-speed fluctuating flow fields.

[0086] This embodiment provides a microfluidic droplet array visual inspection system based on fermentation strain screening. By pre-constructing a joint point spread function database of motion optics, it mathematically integrates the underlying optical diffraction limit constraints with dynamic fluid dynamics motion characteristics. During the inspection process, it performs adaptive imaging intervention and three-dimensional shape tracking and focusing based on real-time flow velocity, and achieves multi-channel data fusion by relying on a contrast gain-based model. It uses Kalman filtering and a four-dimensional high-dimensional feature determination model to ensure continuous and stable targeted observation of non-rigid microdroplets. Finally, it uses a rigorous motion timing model to pre-calculate the trigger node of the execution component.

[0087] The overall solution, without altering the underlying micro-nano fabrication structure of existing microfluidic chips, significantly improves the identification and detection efficiency of target fermentation strains in complex flow environments and the success rate of back-end physical sorting through end-to-end collaborative adaptive control across opto-mechanical-electronic-computing software dimensions. It has significant application transformation value and industrial-grade deployment significance.

[0088] Example 2:

[0089] In practical high-throughput fermentation strain screening tasks, the long-term operation of microfluidic-driven pumps and the continuous occurrence of biochemical reactions can lead to dynamic disturbances in both physical dynamics and optical biology dimensions of the system.

[0090] Specifically, the microfluidic droplet array visual detection system based on fermentation strain screening in this embodiment is further equipped with a dual-source anti-interference module to address transient distortions in fluid dynamics and optical scattering interference from microbubbles. This module is specifically used to: collect and calculate the second derivative of the flow velocity within the sliding region of interest in real time, using it as a characteristic parameter characterizing the transient nonlinear acceleration jumps of the microfluidic drive system; when the characteristic parameter exceeds a preset steady-state flow field threshold, a tracking dimensionality reduction protection mechanism is triggered, dynamically switching the Kalman filter in the droplet tracking and segmentation module to a controlled unscented Kalman filter, utilizing... The probability density distribution after the flow field jump is fitted using deterministic sampling points obtained from the unscented transformation. Before executing the processing logic of the image restoration and fusion module, high-frequency spatial brightness gradient features inside and at the edges of the droplets are extracted to identify optical distortion regions exhibiting a topological distribution pattern of transmission dark spots and scattering bright rings. The spatial contours of the optical distortion regions are marked as microbubble interference masks. The microbubble interference masks are mapped and aligned to the fluorescence images of the corresponding channels. Using the gray values ​​of the effective droplet matrix pixels adjacent to the outer edge of the mask, the pixel gray values ​​inside the mask are structurally interpolated and reconstructed using a spatial distance inverse weighting algorithm.

[0091] For example, regarding the characteristic phenomenon of transient distortion in fluid dynamics, in practical microfluidic systems, even though high-precision injection pumps or pressure controllers are used to drive the fluid and pulse dampers are configured, the continuous and dispersed phase fluids inside the flow channel cannot maintain an ideal uniform laminar flow state due to the inherent clearance of the mechanical transmission components, the phase switching of the stepper motor, and the elastic energy release of the fluid channel walls. This minute inherent vibration of the mechanical system is transmitted to the microscale flow field, triggering high-frequency, nonlinear velocity abrupt changes, i.e., transient distortion in fluid dynamics.

[0092] If the detection system still relies on the standard Kalman filter based on the linear assumption in Example 1 to predict the target position, when the acceleration undergoes a drastic nonlinear jump, the prediction covariance matrix of the standard Kalman filter will diverge rapidly, causing the algorithm to lose its lock on the target droplet, resulting in serious problems such as target loss or trajectory mismatch.

[0093] Optionally, when processing the aforementioned transient distortions in fluid dynamics, the dual-source anti-interference module first acquires and calculates the second derivative of the flow velocity within the sliding region of interest in real time, using it as a characteristic parameter to characterize the transient nonlinear acceleration jumps in the microfluidic drive system. After acquiring the flow velocity data from consecutive frames, the system's underlying data processing pipeline performs two derivative operations on the instantaneous flow velocity using a discrete-time difference algorithm.

[0094] To accurately quantify this physical process, the specific mathematical formula for calculating the transient nonlinear acceleration jump characteristic parameter in this embodiment is as follows:

[0095] ;

[0096] in, Characteristic parameters for representing transient nonlinear acceleration jumps in microfluidic drive systems; The instantaneous fluid velocity scalar calculated for the sliding region of interest at the current observation sampling time node; The instantaneous fluid velocity scalar obtained for the sliding region of interest at the previous consecutive sampling time node of the current observation sampling time node; The instantaneous fluid velocity scalar obtained for the sliding region of interest at the two consecutive sampling time nodes preceding the current observation sampling time node; This is a fixed physical time interval constant between two adjacent frames when the imaging control module performs continuous image sampling.

[0097] It is important to note that the physical essence of the above formula is to solve for the rate of change of the time-varying velocity of the flow field, i.e., jerk. In a steady-state flow field or a flow field undergoing slow linear acceleration, The value of this parameter approaches zero or remains at an extremely low level. By calculating this characteristic parameter in real time, the system has the ability to directly sense the underlying mechanical pulsation state inside the microfluidic chip, thus providing a reliable numerical basis for the dynamic switching of subsequent algorithm models.

[0098] Specifically, when the characteristic parameter exceeds a preset steady-state flow field threshold, the system immediately triggers a tracking dimension reduction protection mechanism. The steady-state flow field threshold is a scalar benchmark obtained by recording the basic pulsating noise level under no-load operation during the system's initialization calibration phase and multiplying it by a tolerance coefficient. The core action of the tracking dimension reduction protection mechanism is to dynamically switch the Kalman filter in the droplet tracking and segmentation module to a controlled unscented Kalman filter. The controlled unscented Kalman filter is an advanced estimation model for dealing with strongly nonlinear systems. It does not rely on performing a Taylor expansion of the nonlinear function to obtain the Jacobian matrix (this process easily leads to higher-order term truncation errors), but instead uses deterministic sampling points from the unscented transformation to fit the probability density distribution after the flow field jump.

[0099] For example, in the actual operation of a controlled unscented Kalman filter, the system extracts a set of deterministic sampling points (Sigma points) in the state space according to a specific symmetric sampling strategy, based on the droplet's current state vector (including spatial position and velocity) and the system state covariance matrix. After these Sigma points are propagated through the actual nonlinear fluid motion state transition function, the system recalculates the weighted mean and weighted covariance of the propagated Sigma points, thereby reconstructing the true probability density distribution of the droplet position after experiencing a transient change in the flow field.

[0100] This mechanism avoids the predicament of standard linear Kalman filtering where the predicted frame lags behind the actual physical position of the droplet when faced with sudden acceleration jumps, effectively maintaining the spatial constraint coherence of droplet trajectory association and ensuring zero-frame-loss tracking under high-throughput filtering. Once the value falls below the steady-state flow field threshold and continues for a preset observation period, the system automatically switches back to the standard Kalman filter, which has lower computational overhead, to achieve dynamic optimization of computing resources.

[0101] like Figure 5 This paper presents a comparison of the covariance of trajectory tracking errors between standard Kalman filtering and controlled unscented Kalman filtering under transient nonlinear distorted flow fields. The horizontal axis in the figure represents the horizontal spatial position in micrometers, and the vertical axis represents the vertical spatial position in micrometers.

[0102] The figure includes a solid black line representing the actual trajectory of the droplet, and dashed blue lines representing the standard Kalman filter trajectory and error ellipse, and a solid red line representing the controlled unscented Kalman filter trajectory and error ellipse, respectively, for the two filtering algorithms.

[0103] As can be seen from the graph, the flow field is in a uniform and stable state before the horizontal position of 100 micrometers. The predicted trajectories of the two filtering algorithms are highly consistent with the black true trajectory, and the sizes of the blue and red error ellipses along the line are kept in a very small range, indicating that the prediction error covariance of the system is in a convergent state at this time.

[0104] When the horizontal spatial position crosses the 100-micrometer node, the flow field undergoes a significant nonlinear acceleration jump, and the black true trajectory exhibits a sharply rising quadratic curve shape in the vertical direction. At this point, the blue standard Kalman filter trajectory shows obvious tracking lag, deviating from the true motion path. More significantly, the size of the blue error ellipse along its path rapidly expands and diverges as the position advances. This rapid expansion of geometric size objectively reflects the failure of the basic linear assumptions in dealing with high-frequency acceleration jumps, leading to a divergence in the prediction error covariance matrix and increasing the risk of target tracking loss.

[0105] In contrast, the red controlled unscented Kalman filter trajectory still closely matches the real black curve, and the size of the red error ellipse along the line only expands slightly in the initial stage of the jump, and is then quickly suppressed and maintained in a convergent state with clear boundaries.

[0106] The significant suppression effect of the red error ellipse on size and expansion trend compared to the blue error ellipse objectively confirms the effectiveness of the tracking dimension reduction protection mechanism triggered when the acceleration jump characteristic parameter exceeds the limit. It shows that the deterministic sampling points using unscented transformation can accurately fit the probability density distribution after the flow field jump, ensuring the trajectory constraint coherence and long-term tracking stability of microfluidic droplets under complex fluid pulsation interference.

[0107] Optionally, another core function of the dual-source anti-interference module is to address optical scattering interference from microbubbles. In microfluidic-based fermentation strain screening applications, highly active microorganisms, such as yeast or certain aerobic / anaerobic bacteria, inevitably produce gaseous metabolic products (such as carbon dioxide) when metabolizing in a closed droplet microenvironment containing culture medium. These gases aggregate into extremely small bubbles inside the droplets. Due to the significant refractive index difference between the continuous phase (fluorinated oil), the dispersed phase (aqueous culture medium), and the metabolic gases in the microfluidic system, microbubbles exhibit strong optical distortion effects when excited by a light source of a specific wavelength from the system.

[0108] Specifically, to accurately eliminate this interference, the system first extracts the high-frequency spatial brightness gradient features inside and around the droplet before executing the processing logic of the image restoration and fusion module. The optical distortion of microbubbles exhibits highly regular two-dimensional morphological features on the imaging sensor: in the central region of the microbubble, due to the large-angle refraction or total internal reflection of light at the gas-liquid interface, the transmitted light intensity drops sharply, forming a dark spot; while at the edge boundary region of the microbubble, due to the strong scattering and interference superposition effect of light, a significantly bright ring halo, i.e., a scattering bright ring, is formed.

[0109] The system can clearly extract the high-frequency spatial brightness gradient features that abruptly change by calculating the Sobel or Laplacian operator responses of the two-dimensional pixel matrix in the horizontal and vertical directions. Based on this feature, the algorithm can identify optical distortion regions exhibiting a topological distribution pattern of transmitted dark spots and scattered bright rings. Using this topological distribution pattern as a hard-matching template, it can effectively distinguish between pseudo-fluorescence signals induced by microbubbles and effective fluorescence signals produced by high expression of real fermentation strains, avoiding misclassification as high-yield strains. Once identified and confirmed, the system marks the two-dimensional spatial contour of the optical distortion region as a microbubble interference mask, which logically defines the geometric boundaries of the contaminated pixels.

[0110] It is also important to note that marking the mask is not the end point of interference resistance. If the image containing microbubble cavities is directly input into subsequent modules without processing, it will cause a sharp distortion in the feature extraction of the total fluorescence intensity of the droplets, thereby affecting the anomaly detection of the dynamic metabolic dimension. Therefore, the system further maps and aligns the microbubble interference mask to the fluorescence image of the corresponding channel, and uses the gray values ​​of the effective droplet matrix pixels adjacent to the outer edge of the mask to perform structural interpolation reconstruction of the pixel gray values ​​inside the mask through a spatial distance inverse weighted algorithm.

[0111] For example, the spatial distance inverse weighted algorithm is based on the assumption of proximity of geospatial information, that is, the closer a healthy pixel is to the contaminated area, the higher the similarity between its physical fluorescence expression level and the original fluorescence expression level of the contaminated area. In this embodiment, the calculation formula for structural interpolation reconstruction inside the microbubble interference mask is as follows:

[0112] ;

[0113] in, After calculation using the inverse spatial distance weighting algorithm, the relative coordinates of the target points filled into the microbubble interference mask are: Structural interpolation at the location reconstructs the pixel grayscale scalar value; The total number of effective droplet matrix reference pixels obtained by searching outside the geometric boundary of the microbubble interference mask; The first one retrieved for the search The original fluorescent grayscale scalar value of each effective droplet matrix reference pixel; The target coordinates inside the microbubble interference mask With the The two-dimensional Euclidean geometric distance between the actual spatial coordinates of each effective droplet matrix reference pixel; This is a pre-configured distance attenuation weight constant coefficient used to control the pixel grayscale space-related attenuation rate.

[0114] During actual computation, the algorithm engine scans invalid pixels within the microbubble interference mask point by point, then radiates outwards from that point to find a set of healthy base pixels not covered by the mask. Through weighted calculations using the formula above, the system can smoothly repair signal contamination areas caused by microbubble optical scattering by utilizing the true fluorescence gradient trend of the droplet background. This structural interpolation reconstruction process occurs before all image restoration and feature extraction logic, fundamentally optimizing the input data source. This effectively prevents severe ringing effects caused by strong pseudo-highlight edges in subsequent image restoration modules and ensures the accuracy of static fluorescence dimension features, such as average fluorescence intensity.

[0115] like Figure 6This paper presents a comparison of the three-dimensional grayscale topology before and after structural interpolation reconstruction of the microbubble optical scattering interference region. The units for both horizontal and vertical spatial coordinates in the figure are pixels, with a data range from 1 to 40. The vertical axis represents grayscale intensity in dimensionless units, with a data range from 0 to 250.

[0116] The curved surface formed by the blue dashed grid in the figure represents the interference topology before reconstruction. In the central intersection area of ​​spatial coordinates 20 and 20, the grid shows a sharp downward indentation, with the gray value dropping to around 90. This objectively reflects the transmission dark spot caused by total internal reflection at the center of the microbubble. On the outer periphery of the indentation, the grid bulges upward to form a sharp ring-shaped peak, with the gray value soaring to around 230. This directly reflects the bright scattering ring formed by the strong interference superposition at the gas-liquid interface.

[0117] In stark contrast is the solid colored surface representing the smooth topology after reconstruction, its color smoothly transitioning from a low-grayscale dark blue at the edges to a high-grayscale dark red at the center. On this colored surface, the original sharp indentations and peaks are effectively eliminated, and the overall shape is restored to a smooth Gaussian convexity with a peak value of around 150. This difference in topological morphology between the dashed grid and the solid colored surface in three-dimensional space quantitatively confirms that the spatial distance inverse weighted algorithm can accurately utilize the effective droplet matrix pixels immediately adjacent to the optical distortion region to perform point-by-point structural interpolation within the contaminated microbubble interference mask.

[0118] Before the image restoration logic is executed, the reconstruction process corrects the pseudo-highlight pixels and data holes caused by metabolic gases at the spatial gradient level, effectively preventing feature distortion during subsequent high-dimensional feature extraction, thereby ensuring the accuracy and reliability of the static fluorescence dimension feature evaluation of the fermentation strain.

[0119] In summary, most existing microfluidic vision inspection systems only focus on image processing under static or ideal laminar flow conditions, often ignoring the unavoidable nonlinear pulsation of mechanical pumps and the microbubble optical contamination caused by actual biological fermentation gas production in industrial operations. This leads to technical bottlenecks such as the system tracking being prone to failure and the false positive rate remaining high during long-term operation.

[0120] This embodiment, by adding a dual-source anti-interference module, starts with the monitoring of fluid dynamics features and innovatively uses transient nonlinear acceleration jump feature parameters to guide the adaptive switching of the controlled unscented Kalman filter, effectively solving the problem of target trajectory loss caused by fluid pulses. At the same time, from the perspective of optical biology, it accurately locates the specific topological scattering mode of the bubble and performs low-level pixel-level structural interpolation reconstruction, thus immunizing against fluorescence feature contamination caused by metabolic gases.

[0121] The overall technical solution significantly enhances the robustness of the system under real and complex working conditions by embedding an active anti-interference mechanism at the front end of the algorithm logic chain. It ensures the high precision and high reliability of the microfluidic droplet array visual inspection system based on fermentation strain screening when performing high-throughput and long-cycle screening tasks, and has extremely high practical application value and industrialization significance.

[0122] Example 3:

[0123] In practical high-throughput screening pipelines for fermentation strains, microfluidic detection equipment typically needs to operate continuously for tens of hours or even days. During this long-term operation, the high-frequency flickering excitation light source and high-speed rotating sensors inside the equipment generate significant heat accumulation, causing thermal deformation of optical components and mechanical fixtures. Simultaneously, slight fluctuations in ambient temperature and continuous fluid shear work also lead to gradual changes in the dynamic viscosity of the continuous and dispersed phases within the microfluidic chip. These two slowly accumulating physical phenomena couple together, causing the system's optical focal plane and hydrodynamic state to deviate from the initial calibration reference, resulting in severe performance drift.

[0124] Specifically, to address the system performance degradation caused by the coupling of hardware thermal deformation and fluid dynamic viscosity under prolonged continuous operation, the microfluidic droplet array visual inspection system based on fermentation strain screening in this embodiment is further equipped with a long-term self-calibration module for operating status, used to compensate for performance drift caused by the coupling of hardware thermal deformation and fluid dynamic viscosity. This module is specifically used for:

[0125] A passive optical anchor point is constructed by extracting the static high-frequency texture features of the fixed physical boundary of the microfluidic chip channel within the detection field of view. The spatial Laplacian variance change rate of the passive optical anchor point is continuously calculated within a time observation window to quantify the thermally induced focal plane drift of the system's optical components. When the thermally induced focal plane drift exceeds a preset tolerance threshold, a step drive signal is generated based on the drift and directly superimposed on the displacement stage in the active imaging control module to perform physical reset of the optical focal plane during the detection process. The major and minor axis morphological ratios of the target droplets within the sliding region of interest are continuously extracted. After removing transient distortion interference using low-pass filtering, the baseline offset of the morphological ratio within the steady-state time window is calculated as a proxy physical variable for the fluid's dynamic viscosity drift. A point diffusion spatial perturbation matrix is ​​constructed based on the morphological ratio baseline offset. The spatial perturbation matrix is ​​used as a dynamic multiplication factor and applied to the motion optical joint point diffusion function database generated by the pre-calibration and initialization module to generate an evolved joint point diffusion function, which is then input to the image restoration and fusion module.

[0126] For example, the extraction of static high-frequency texture features from the fixed physical boundaries of the microfluidic chip channels within the detection field of view to construct passive optical anchors is a prerequisite for the system to perform in-situ optical hardware compensation. In the manufacturing process of microfluidic chips, the microchannels formed by photolithography using materials such as polydimethylsiloxane or glass do not have perfectly smooth sidewall physical boundaries; rather, they possess inherent physical roughness or processing textures at the microscale. Because the microfluidic chip is rigidly held by a fixture during detection, these texture features on the channel sidewalls remain highly static in the absolute spatial coordinates of the detection field of view.

[0127] The system uses an edge detection algorithm to identify these stationary microstructures in the non-droplet flow region of the field of view, extracts their geometric contours, and registers them as passive optical anchors. The advantage of using passive optical anchors is that the system does not require additional fabrication and implantation of dedicated artificial alignment markers inside the microfluidic chip, thereby reducing the cost and complexity of chip manufacturing.

[0128] Optionally, the spatial Laplacian variance change rate of the passive optical anchor can be continuously calculated within a time observation window to quantify the thermally induced focal plane drift of the system's optical components. Due to the thermal expansion effect caused by continuous system operation, the relative physical distance between the microscope objective and the microfluidic chip will slowly drift at the micrometer level, causing the imaging of the passive optical anchor to gradually deviate from the optimal focal plane. This defocusing phenomenon manifests in image processing as the loss of high-frequency details in the image. The Laplacian operator, as a second-derivative operator, is highly sensitive to high-frequency edges and texture details in images.

[0129] In this embodiment, the specific mathematical formula for calculating the spatial Laplace variance change rate is as follows:

[0130] ;

[0131] in, The scalar value of the spatial Laplace variance rate of change of the degree of defocusing of the optical components in the quantization system is calculated. This is a global statistical variance scalar of all pixel response values ​​obtained after performing a Laplacian convolution operation on the image pixel matrix of the passive optical anchor at the current sampling time; The initial baseline global statistical variance scalar is obtained by performing a Laplacian convolution operation on the image pixel matrix of the passive optical anchor point at the absolute reference moment when the system has just started and completed the initial calibration; The duration observation window length constant is a pre-defined value used to smooth transient noise interference.

[0132] It is also important to note that the system can obtain the quantitative trend of focus drift in real time through the calculation of the above formula. When the system is in a precise focusing state, the image texture of the passive optical anchor is the clearest, and the variance of the Laplacian response is extremely large; while when thermal deformation causes defocusing, the image becomes smooth, the high-frequency response weakens sharply, and the variance decreases significantly. The introduction of the baseline variance and the time observation window length parameter in the formula transforms the absolute blur assessment into a relative rate of change assessment, effectively eliminating the interference of inherent surface roughness differences caused by different chip processing batches, and ensuring the objectivity and universality of the thermal focus plane drift assessment results.

[0133] Specifically, when the thermally induced focal plane drift exceeds a preset tolerance threshold, the system generates a step drive signal based on the drift and directly superimposes it onto the displacement stage in the active imaging control module to perform a physical reset of the optical focal plane during the detection process. The tolerance threshold refers to the maximum acceptable blur tolerance scalar allowed within the depth of field of the microscopic optical system. In the actual control chain, the computing unit in the industrial computer backend continuously monitors the rate of change of the spatial Laplace variance. Once it is determined that the system has substantially deviated from the optimal depth of field region, the control algorithm uses a pre-calibrated variance-displacement response mapping relationship to reverse-calculate the required physical spatial displacement compensation.

[0134] Subsequently, the digital-to-analog converter transforms this spatial displacement into a voltage-formatted step drive signal. The superposition operation is crucial in this process: the system does not interrupt the ongoing high-frequency dynamic tracking task, but instead electrically fuses the slowly generated step drive signal onto the existing control bias voltage of the active imaging control module via an adder, driving the piezoelectric ceramic displacement stage to produce micron-level reverse mechanical creep in the Z-axis direction. This uninterrupted background compensation mechanism achieves dynamic and smooth physical reset of the optical focal plane, ensuring constant sharpness of the original optical acquisition signal during long-term continuous operation.

[0135] like Figure 7 This figure illustrates the time-series response of the Laplacian variance change rate of the passive optical anchor point and the Z-axis focal plane physical compensation amount under continuous operation. The horizontal axis represents the continuous operation time in minutes, with a data range of 0 to 120 minutes. The left vertical axis represents the Laplacian variance change rate as a percentage, ranging from -15 to +5; the right vertical axis represents the Z-axis focal plane physical compensation amount in micrometers, ranging from 0 to 12.5.

[0136] The figure contains two core correlation curves: the solid blue line is the Laplace variance change rate curve, and the dashed red line is the Z-axis focal plane physical compensation curve.

[0137] As can be seen from the waveform trend of the blue solid line, with the passage of continuous operation, the Laplacian variance change rate gradually decreases due to the slight deformation caused by the thermal expansion of the system's optical components. This objectively quantifies that the imaging system is slowly deviating from the optimal focal plane. When the value of the blue solid line drops to the preset tolerance threshold of -10% after about 20 minutes of operation, the system control logic is triggered. At this time, the red dashed line shows a step increase, with the value instantly rising from 0 micrometers to about 2.5 micrometers. This indicates that the system generates a step drive signal based on the drift amount and directly superimposes it onto the displacement stage, performing a physical focal plane reset action.

[0138] At the same moment the red dashed line experiences a step compensation, the blue solid line rapidly recovers from -10% to near the zero baseline, indicating that the high-frequency texture details of the optical anchor point image have been restored, and the system has returned to precise focusing. Over the next 100 minutes, the multiple bottoming outs of the blue solid line and the step-like rise of the red dashed line form a tight closed-loop response correspondence.

[0139] The data curves generated by this dual-axis linkage objectively confirm that the long-term self-calibration module can continuously monitor thermally induced focal plane drift during the detection process and achieve dynamic smooth reset by applying reverse mechanical displacement in the background, thus ensuring that the sharpness of the original optical acquisition signal of the microfluidic vision inspection system remains constant under long-term continuous high-load operation.

[0140] For example, to address the fluid dynamic viscosity change problem caused by long-term operation, the system continuously extracts the morphological ratio of the major and minor axes of the target droplet within the sliding region of interest, replacing complex fluid dynamics physical compensation with software-level data evolution. In the fermentation strain screening system, the physical viscosity of the culture medium and the continuous phase fluorinated oil is extremely sensitive to temperature changes. The increase in fluid temperature caused by long-term operation leads to a decrease in the dynamic viscosity of the continuous phase. Under constant injection pump thrust, the change in fluid viscosity directly alters the wall shear force and fluid drag force experienced by the droplet within the microchannel. According to fluid mechanics principles, microdroplets in the channel are not perfectly regular spheres, but rather exhibit stretched ellipsoidal characteristics. The slow drift of shear force causes a corresponding change in the degree of stretching of this ellipsoid, i.e., a gradual deflection in the ratio of the droplet's geometric major axis to its geometric minor axis.

[0141] Therefore, by measuring the average major-minor axis morphological ratio of a large number of droplets, the true changes in the micro-hydrodynamic viscosity inside the flow channel can be directly or indirectly inferred. This method avoids the need to integrate expensive and easily disruptive microelectromechanical viscosity sensors into the micro-channels.

[0142] Optionally, after removing transient distortion interference using a low-pass filter, the baseline offset of the morphological ratio within the steady-state time window is calculated as a proxy physical variable for the fluid's dynamic viscosity drift. The morphological ratio of a single droplet is often affected by transient high-frequency stretching caused by the nonlinear pulsation of the mechanical pump described in Example 2 when passing through the detection channel. To extract the slowly varying trend component characterizing viscosity change, the system applies an infinite impulse response low-pass filter to digitally filter the continuously acquired major and minor axis morphological ratio sequences, truncating all transient jitter high-frequency components above a preset cutoff frequency.

[0143] The low-frequency data sequence remaining after filtering and smoothing represents the average morphological baseline of the fluid within a steady-state time window. The difference between the current average morphological baseline and the initial baseline established during the system's pre-calibration phase is calculated, and the resulting morphological ratio baseline offset is defined as the proxy physical variable for fluid dynamic viscosity drift. This proxy physical variable is extracted using a purely visual algorithm, achieving an accurate mapping of deep physical flow field environmental parameters.

[0144] Specifically, to eliminate the risk of underlying model failure caused by fluid dynamic viscosity drift in image restoration algorithms, the system constructs a point diffusion spatial perturbation matrix based on the morphological ratio baseline offset. In the motion optics joint point diffusion function database generated by the pre-calibration and initialization modules, each restoration kernel is generated based on the motion blur trailing pattern formed by a specific calibration flow rate and calibration fluid viscosity. When the fluid viscosity changes slowly, causing droplet deformation and changes in the edge velocity field distribution, the actual length of the motion blur trailing image and the energy distribution function generated on the photosensitive element will also undergo spatial stretching or compression. Therefore, directly using the original database at the calibration time for image restoration is highly likely to produce severe edge ghosting and deblurring failure.

[0145] This embodiment reconstructs the original restored kernel by introducing a point diffusion space perturbation matrix. The specific mathematical formula is as follows:

[0146] ;

[0147] in, The relative index coordinates in the constructed two-dimensional point diffusion spatial perturbation matrix are: Local perturbation weighting factor at the location; This refers to the local relative index variable along the horizontal pixel direction in the two-dimensional point diffusion kernel matrix; This refers to the local relative index variable along the vertical pixel direction in the two-dimensional point diffusion kernel matrix; This is a scalar value representing the baseline offset of the morphological ratio, which characterizes the change in fluid viscosity, extracted after low-pass filtering. The dimensionless transformation constant coefficient between the pre-established coupled morphological stretching coefficient and the motion blur trailing broadening effect; The global normalization constant coefficient is used to ensure the conservation of local energy distribution in a two-dimensional matrix.

[0148] It is important to note that the physical core of the above formula is based on the proxy physical variable of fluid dynamic viscosity drift, generating a Gaussian-distributed deformation matrix with adaptive broadening characteristics in two-dimensional space. The magnitude of the morphological ratio baseline offset directly regulates the effective variance size of this spatial perturbation matrix.

[0149] Optionally, the system uses the spatial perturbation matrix as a dynamic multiplication factor, applying it to the motion optical joint point spread function database generated by the pre-calibration and initialization module, to generate an evolved joint point spread function, which is then input to the image restoration and fusion module. In this operation, the data processing unit extracts the original motion optical joint point spread function matrix corresponding to the current flow velocity and performs element-wise Hadamard product operations with the generated two-dimensional point spread spatial perturbation matrix. This matrix-level multiplication operation essentially applies a spatial domain attenuation and deformation correction envelope to the original restoration kernel.

[0150] The revised evolutionary joint point spread function (JDP) closely matches the motion blur characteristics of the real microscopic fluid after drift in terms of morphological features and energy decay gradient. Subsequently, the image restoration and fusion module abandons the outdated calibration kernel and directly loads the JDP to perform joint deconvolution operations on motion blur and diffraction blur. This process completes a digital evolution update at the software level, effectively decoupling the complex binding relationship between fluid dynamic viscosity drift and visual restoration algorithms. The system can autonomously achieve anti-fading compensation in digital space without physical shutdown to adjust fluid pump pressure and formulation.

[0151] In summary, considering the shortcomings of existing microfluidic screening systems, current detection equipment typically assumes that the optical and fluid viscosity parameters of the system remain absolutely constant when operating continuously for extended periods. When accumulated heat causes optical defocusing or changes in fluid resistance characteristics, existing technologies often result in inaccurate droplet feature extraction or complete loss of tracking. This necessitates physical shutdown and cleaning, along with significant time spent recalibrating the system, which severely limits the potential of microfluidic technology for large-scale industrial production lines.

[0152] The technical solution provided in this embodiment, which is a long-term self-calibration module for configuring the operating status, cleverly utilizes the fixed channel boundary of the detection field of view to construct a static passive optical anchor point, and establishes a background hardware optical focal plane sensorless reset closed loop that does not interfere with the main detection task. At the same time, it also proposes to use the macroscopic statistical law of the ratio of the major and minor axes of droplets as a proxy measurement variable for the microscopic dynamic viscosity drift of fluid, thereby driving the digital topology evolution update of the underlying deblurring algorithm kernel.

[0153] This solution deeply integrates in-situ displacement compensation of optical hardware with adaptive digital evolution of the kernel matrix of fluid software, significantly improving the robustness and parameter stability of the microfluidic droplet array visual inspection system based on fermentation strain screening under harsh long-cycle, high-load industrial screening conditions. It greatly extends the effective maintenance-free operation cycle of the equipment and ensures the continuity and sorting reliability of high-throughput parallel screening operations in ultra-large-scale strain libraries.

[0154] Example 4:

[0155] like Figure 8 As shown, this embodiment provides a visual detection method for microfluidic droplet arrays based on fermentation strain screening. In practical high-throughput screening applications of microfluidic droplet technology for fermentation strains, existing technologies typically rely on optical imaging devices with fixed parameters. However, the droplet velocity within the microfluidic channel is high and generally exhibits nonlinear fluctuations. Imaging methods with fixed parameters struggle to match the real-time dynamically changing velocity, resulting in droplet images that are highly susceptible to distortion of edge and internal features due to both motion blur and diffraction limitations of the optical system. Furthermore, traditional static image analysis methods lack real-time coordination with the underlying fluid dynamics. When the flow field fluctuates or the droplet undergoes force deformation, target loss and trajectory prediction deviations are prone to occur, causing the final calculated sorting trigger time to deviate from the droplet's actual physical arrival position, leading to missorting or missed detection of fermentation strains.

[0156] To address the technical problems existing in the prior art, this embodiment, based on the visual inspection system and related hardware architecture described in Embodiments 1 to 3, provides a corresponding control method. Specifically, this microfluidic droplet array visual inspection method based on fermentation strain screening includes the following steps:

[0157] Step 1: Jointly calibrate the channel geometry and optical system of the microfluidic chip to generate a kinematic optical joint point spread function database and construct a dynamic sliding region of interest template flowing along the channel. In actual laboratory or industrial applications, this step is usually performed before the formal injection of the fermentation strain sample. The operator fixes the microfluidic chip, made of polydimethylsiloxane (PDMS) or glass, onto the stage of an inverted fluorescence microscope and connects it to a high-precision syringe pump or other fluid drive device. The system controls the syringe pump to inject fluorescent microspheres with standard known physical properties (such as diameter and fluorescence emission wavelength) into the channel. The calibration algorithm module inside the industrial computer analyzes the motion trail of the microspheres at different constant flow rates, combines it with the numerical aperture parameters of the microscope objective, and generates a kinematic optical joint point spread function database covering multiple operating conditions through convolution operations, and preloads it into high-speed random access memory (RAM).

[0158] Meanwhile, based on the physical centerline coordinates of the microfluidic channel, the system divides the visual field into a dynamic sliding region of interest template, thereby defining the physical boundary of subsequent image processing and significantly reducing the computational resource consumption of irrelevant background areas.

[0159] Step Two: Based on the real-time flow velocity of the channel, the camera parameters are dynamically adjusted to perform three-dimensional topographic perception and dynamic focusing on the local area of ​​the channel centerline, and multi-channel fluorescence images with different excitation intensities are acquired simultaneously. To address the unavoidable minute pulsations in the fluid during actual operation, this step achieves dynamic adaptation through closed-loop linkage of the underlying hardware. The detection device is typically equipped with a high-speed scientific-grade complementary metal-oxide-semiconductor (sCMOS) camera and a multi-wavelength LED or laser excitation source. The computing unit extracts flow velocity information by processing continuous image frames and uses a field-programmable gate array (FPGA) to send down a series of microsecond-level hardware trigger timing signals, dynamically shortening or extending the camera's exposure time and the light source flicker intensity proportionally to physically suppress motion blur.

[0160] During this process, the system drives the structured light projection device to project an coded pattern onto the channel. The phase calculation algorithm obtains the defocusing amount caused by thermal expansion or chip deformation in real time, and outputs an analog control voltage to drive the piezoelectric ceramic Z-axis precision displacement stage rigidly connected to the objective lens, keeping the high-frequency imaging of the droplet always within the optimal optical focal plane. Subsequently, through the beam splitter prism assembly, multiple synchronously triggered cameras acquire multi-channel fluorescence images with different excitation intensities from different optical paths.

[0161] Step 3: Based on the joint point spread function database, joint restoration of motion blur and diffraction blur is performed on the multi-channel fluorescence image, along with adaptive fusion and directional background correction based on theoretical contrast gain. This step primarily relies on workstations equipped with graphics processing units (GPUs) for high-concurrency computation. The system retrieves the most matching joint point spread function kernel from memory based on real-time monitored channel flow rates and uses a deconvolution algorithm to mathematically reconstruct the input fluorescence image. This process, at the software level, offsets the high-speed motion blur and diffraction-limited halo that physical hardware cannot completely eliminate.

[0162] After obtaining the restored image, the algorithm calculates the signal-to-noise ratio (SNR) of each pixel in the multi-channel image, i.e., the theoretical contrast gain value. The system uses this gain value as a weighting parameter to perform a matrix-level weighted fusion operation on the fluorescence signals acquired through different optical paths. This operation, combined with morphological background extraction techniques, removes the autofluorescence and scattered light background at the bottom of the channels, thereby generating a droplet fluorescence distribution image with high dynamic range and high fidelity, laying the data foundation for subsequent feature recognition.

[0163] Step 4: Predict the droplet position for the dynamically sliding region of interest using a Kalman filter, and perform non-rigid instance segmentation based on the predicted spatial constraints to complete the inter-frame droplet trajectory association. In microchannels, droplets typically exhibit irregular ellipsoidal or teardrop-shaped non-rigid deformations due to shear forces from the tube wall and fluid compression. This step uses the Kalman filter algorithm to establish a transition matrix with the droplet's physical coordinates and velocity as state variables, and deduce the spatial position boundary of the droplet at the next imaging moment. This predicted boundary is then used as the spatial constraint pole of an image segmentation algorithm (such as a marker-controlled watershed algorithm). By limiting the computational range of the segmentation algorithm, the system can effectively overcome droplet deformation and background noise interference, and accurately depict the physical contour edge of a single droplet.

[0164] Meanwhile, based on the prediction and update mechanism of Kalman filtering, the system assigns and maintains an independent identity for each droplet in the field of view, realizing continuous trajectory tracking in multi-frame video streams.

[0165] Step 5: Extracting the spatiotemporal incremental features of the droplets to generate a high-dimensional spatiotemporal feature vector. Anomaly droplets are identified and eliminated based on a multi-dimensional anomaly detection model. After completing single-target tracking, the system enters the biological feature evaluation stage. An industrial computer extracts the geometric area, instantaneous velocity, and metabolic fluorescence intensity of the internally encapsulated fermentation strains from the target droplet frame by frame. Since the metabolic process of the fermentation strains has obvious time-series characteristics, the system not only records static data but also focuses on the temporal increment changes of these parameters throughout the entire observation field, thereby combining them to generate a high-dimensional spatiotemporal feature vector characterizing the combined physical and biochemical state of the droplet.

[0166] These vectors are then input into a pre-defined multi-dimensional anomaly detection model. This model can identify states such as no change in fluorescence signal (dead bacteria or empty droplets), area anomalies (droplet breakage or fusion), and velocity variance anomalies (trapped droplets), and label the corresponding targets as anomalies in the data stream to remove them, ensuring that only highly active target strain droplets are included in the final sorting queue.

[0167] Step Six: Calculate the sorting trigger time based on droplet trajectory prediction to drive the microfluidic sorting valve, and adaptively correct system parameters based on the sorting results. This step is the transformation from visual inspection to physical execution. The downstream end of the microfluidic chip typically integrates physical sorting devices such as dielectric electrophoresis sorting electrodes, surface acoustic wave generators, or pneumatic microvalves. Based on the stable trajectory and instantaneous velocity parameters obtained in Step Four, and combined with the known geometric and physical distance from the detection field center to the sorting execution port, the system pre-calculates the absolute theoretical time node for the target droplet to reach the sorting area.

[0168] Subsequently, by subtracting the inherent hardware response delay of the high-voltage amplifier and electrical control circuit of the sorting device, the central processing unit issues control commands at precise trigger moments to drive the sorting valve to deflect the target droplet into the collection channel. Simultaneously, a verification observation area is set up at the end of the sorting channel to statistically analyze the missorting rate and feed the error back to the control center. This allows for dynamic fine-tuning of system parameters such as the focal plane bias voltage and the Kalman filter covariance matrix, maintaining system stability during long-term continuous operation.

[0169] Through the coordinated implementation of the above steps, the method provided in this embodiment effectively eliminates the timing deviation caused by fluid velocity fluctuations in the sorting execution, and establishes a deep dynamic correlation between optical imaging, image restoration, and the fluid state of the microfluidic channel. This method not only achieves continuous and stable tracking of deformed droplets in complex flow phases, ensuring the accuracy of spatiotemporal biochemical feature extraction, but also significantly improves the success rate of target fermentation strain identification and final physical sorting in high-throughput screening scenarios, demonstrating high feasibility for industrial application.

[0170] Example 5:

[0171] Corresponding to the above embodiments, the present invention also proposes an electronic device.

[0172] like Figure 9 The diagram shows a structural schematic of an electronic device according to the present invention. The electronic device 100 includes a processor 101 and a memory 103. The processor 101 and the memory 103 are connected, for example, via a bus 102. Optionally, the electronic device 100 may further include a transceiver 104. It should be noted that in practical applications, the transceiver 104 is not limited to one unit, and the structure of this electronic device 100 does not constitute a limitation on the embodiments of the present invention.

[0173] Processor 101 may be a CPU, a general-purpose processor, a DSP, an ASIC, an FPGA, or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in connection with this disclosure. Processor 101 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0174] Bus 102 may include a pathway for transmitting information between the aforementioned components. Bus 102 may be a PCI bus or an EISA bus, etc. Bus 102 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 9 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0175] The memory 103 stores a computer program corresponding to the microfluidic droplet array visual inspection method based on fermentation strain screening according to the above embodiments of the present invention. This computer program is executed under the control of the processor 101. The processor 101 executes the computer program stored in the memory 103 to implement the content shown in the aforementioned method embodiments.

[0176] Among them, electronic devices 100 include, but are not limited to: mobile terminals such as laptops and PADs (tablet computers) and fixed terminals such as desktop computers. Figure 9 The electronic device 100 shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present invention.

[0177] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A microfluidic droplet array visual inspection system based on fermentation strain screening, characterized in that, include: The pre-calibration and initialization module is used to jointly calibrate the channel geometry and optical system of the microfluidic chip, generate a database of motion optical joint point diffusion functions, and construct a dynamic sliding region of interest template for flow along the channel. The active imaging control module is used to dynamically adjust camera parameters based on the real-time flow velocity of the channel, perform three-dimensional morphological perception and dynamic focusing on the local area of ​​the channel centerline, and simultaneously acquire multi-channel fluorescence images with different excitation intensities. The image restoration and fusion module is used to perform joint restoration of motion blur and diffraction blur on multi-channel fluorescence images based on the joint point spread function database, and to perform adaptive fusion and directional background correction based on theoretical contrast gain. The droplet tracking and segmentation module is used to predict the droplet position for the dynamic sliding region of interest using a Kalman filter, perform non-rigid instance segmentation based on the prediction space constraints, and complete the inter-frame droplet trajectory association. The high-dimensional feature extraction and anomaly removal module is used to extract the spatiotemporal incremental features of droplets to generate spatiotemporal high-dimensional feature vectors, and to identify and remove abnormal droplets based on a multi-dimensional anomaly judgment model. The closed-loop sorting control module is used to calculate the sorting trigger time based on droplet trajectory prediction to drive the microfluidic sorting valve, and to adaptively correct the system parameters based on the sorting results.

2. The system according to claim 1, characterized in that, The precalibration and initialization module is specifically used for: Standard fluorescent microspheres are injected into a microfluidic chip, and the standard fluorescent microspheres are controlled to pass through the detection channel at different constant flow rates to obtain motion-blurred images. By combining a reference image of a stationary microsphere, a blind deconvolution algorithm is used to extract motion blur kernels at various flow velocities; By combining the numerical aperture of the optical system and the pre-calibrated depth response curve, the optical diffraction point diffusion function at the corresponding spatial location is calculated. The motion blur kernel is convolved with the optical diffraction point spread function to generate a database of motion-optical joint point spread functions that map different flow velocities and channel positions.

3. The system according to claim 1, characterized in that, The dynamic sliding interest area template is constructed as follows: An initial rectangular template is generated based on the pre-acquired coordinates of the microfluidic chip channel centerline and the average physical diameter of the droplet to be tested. The detection field of view is divided into multiple continuous sliding regions of interest along the center line of the channel; Each sliding region of interest is assigned a unique temporal identifier, and a mapping relationship between the physical location of the field of view and the droplet's movement time is established to ensure that the target droplet is anchored and tracked by a single sliding region of interest throughout the entire cycle of passing through the detection field of view.

4. The system according to claim 1, characterized in that, The active imaging control module includes: The flow rate monitoring unit is used to calculate the channel flow rate in real time by measuring the physical displacement of droplets within the sliding region of interest in adjacent frame images; An adaptive exposure unit is used to dynamically adjust the camera's sampling frame rate and exposure time proportionally according to the channel flow rate, and simultaneously control the intensity of the excitation light source. The three-dimensional focusing unit is used to project an coded structured light pattern onto a preset range area with the channel centerline as a reference, calculate the average depth value of the centerline through phase decoding, and drive the displacement stage to dynamically adjust the optimal optical focal plane.

5. The system according to claim 1, characterized in that, When the image restoration and fusion module performs joint restoration and adaptive fusion: An iterative algorithm incorporating a local gradient adaptive regularization term is employed to dynamically adjust the regularization intensity based on the distribution characteristics of edges and flat regions in multi-channel images to perform joint deblurring. Extract the grayscale value of each pixel in the multi-channel restored image and the grayscale value of the regional background obtained by the background extraction algorithm, and calculate the theoretical contrast gain value by combining it with the pre-calibrated system noise standard deviation. A droplet region mask is generated based on threshold segmentation, and a weighted summation operation is performed on the multi-channel pixels in combination with the theoretical contrast gain value.

6. The system according to claim 5, characterized in that, The multi-channel pixels are weighted and summed to generate the final adaptive fusion and enhancement image. The calculation formula is: ； in, This is a preset droplet region weighting constant; Channel number index for fluorescence images; For the first Pixels in a channel image The grayscale scalar value; and These are the fluorescence image channel number indices in the cross-fusion operation, and are related to... Belonging to the same channel index set; and Corresponding to the first With the The grayscale scalar values ​​of each channel; In order to be with the first The basic channel weighting coefficients that are positively correlated with the channel theory contrast gain value; In order to be with the first Channel and the The cross-fusion weighting coefficient is positively correlated with the absolute value of the theoretical contrast gain difference between channels.

7. The system according to claim 1, characterized in that, The droplet tracking and segmentation module is specifically used for: The spatial coordinates of the droplet's centroid and its orthogonal velocity components are defined as the state vector of the Kalman filter. The covariance matrix of the prediction error is dynamically adjusted based on the fluctuation amplitude of the channel flow velocity to predict the expected spatial boundary of the droplet in the sliding region of interest in the current frame. Calculate the edge curvature tensor of the image within the sliding region of interest; Using the expected spatial boundary as an internal forced marker, and combining it with the edge curvature tensor, a marker-controlled watershed transform is performed to output a precise binary mask of a droplet with non-rigid deformation.

8. The system according to claim 1, characterized in that, The multi-dimensional anomaly detection model is constructed based on a four-dimensional feature space, and its working logic includes: Extract geometric dimension features to determine anomalies in morphological distribution and area ratio; Extracting static fluorescence dimensional features to identify abnormal fluorescence peak distribution; Extract dynamic metabolic dimension features to identify anomalies in fluorescence intensity variation coefficient and time series goodness of fit; Extract motion dimension features to identify velocity variance and trajectory deviation anomalies; The judgment conditions for each dimension of characteristics correspond to the set risk assessment quantitative score. When the cumulative total score of the droplet exceeds the preset safety threshold, the abnormal droplet rejection instruction is triggered.

9. The system according to claim 1, characterized in that, The closed-loop sorting control module accurately calculates the sorting trigger time. The logical relationship is as follows: ； in, To perform pre-calculated current absolute system time; This represents the real-time coordinates of the droplet along the longitudinal direction of the channel at the current system moment; The instantaneous velocity of the droplet along the longitudinal direction of the channel; The fixed physical distance from the center reference point of the optical detection field of view to the actuation port of the microfluidic sorting valve; The overall hardware response delay time for the microfluidic sorting valve and electrical control circuit.

10. A visual inspection method for microfluidic droplet arrays based on fermentation strain screening, characterized in that, include: The channel geometry and optical system of the microfluidic chip are jointly calibrated to generate a database of motion optical joint point diffusion functions and to construct a dynamic sliding region of interest template for flow along the channel. Based on the real-time flow velocity of the channel, the camera parameters are dynamically adjusted to perform three-dimensional morphology perception and dynamic focusing on the local area of ​​the channel centerline, and multi-channel fluorescence images with different excitation intensities are acquired simultaneously. Based on the joint point spread function database, motion blur and diffraction blur are jointly restored in multi-channel fluorescence images, and adaptive fusion and directional background correction are performed based on theoretical contrast gain. The droplet position is predicted for the dynamic sliding region of interest using a Kalman filter, and non-rigid instance segmentation is performed based on the prediction space constraints to complete the inter-frame droplet trajectory association. Extract the spatiotemporal incremental features of droplets to generate a high-dimensional spatiotemporal feature vector, and identify and remove abnormal droplets based on a multi-dimensional anomaly detection model; The sorting trigger time is calculated based on droplet trajectory prediction to drive the microfluidic sorting valve, and the system parameters are adaptively corrected based on the sorting results.

Images