A method and system for lymphocyte feature analysis based on spatiotemporal mask

Abstract

The application relates to a lymphocyte feature analysis method and system based on a space-time mask, and belongs to the fields of medical image processing and cell detection. The method comprises the following steps: collecting a dynamic video of lymphocytes; extracting multiple key images and dividing feature patch units, performing random mask on part of the units of part of the frames, and reserving a reference frame; inputting the masked images into a pre-trained space-time feature extraction network, reconstructing a complete space-time feature sequence based on the reference frame; constructing a standard feature library of normal lymphocytes; comparing the feature sequence of a sample to be detected with the standard feature library, calculating a feature difference degree, and determining that the sample is abnormal if the feature difference degree exceeds a preset threshold, otherwise, the sample is normal; and outputting a determination result as intermediate information of in-vitro analysis. The application avoids detection errors caused by manual operation, enriches the analysis dimension of cell abnormality detection, can accurately capture the motion behavior abnormality of lymphocytes in the initial stage of change, and provides a brand-new technical scheme for automatic analysis of in-vitro lymphocyte samples.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing and cell detection technology, and relates to a method and system for lymphocyte feature analysis based on spatiotemporal masking. Background Technology

[0002] Lymphocyte abnormality screening is a crucial part of clinical hematological testing and immune assessment. Currently, there are two main types of routine testing methods: peripheral blood smear combined with static microscopic morphological observation, and flow cytometry immunophenotyping. However, both techniques have fundamental limitations.

[0003] Blood smears disrupt the original state of cells, resulting in the loss of dynamic information. The preparation process involves forced interventions such as spreading, drying, fixing, and staining, directly compressing the cells and causing cell shrinkage, nuclear deformation, distorted cytoplasmic staining, and even artificially creating "pseudo-abnormal morphologies." What is observed is not the true state of a living cell, but a distorted artificial specimen. Furthermore, blood smears only provide static planar images, completely losing the microscopic movement behaviors of lymphocytes in a living state, such as membrane tremors, peristaltic shifts, and movement trajectories—these dynamic manifestations are the core reflection of their functional state.

[0004] Normal lymphocytes exhibit morphological heterogeneity, while the morphological changes of slightly abnormal lymphocytes are often extremely subtle, such as slightly enlarged cell bodies, slightly irregular nuclei, and mild changes in cytoplasm. Such "gray areas" cannot be reliably distinguished from physiological fluctuations in static images.

[0005] Flow cytometry has information blind spots, making it unsuitable as a rapid screening method. Although it can perform quantitative analysis of multiple immunophenotypic parameters, the cells are in a high-speed flow state, making it impossible to observe morphological details and motility; the labeling process has a potential impact on cell viability, and the operation is complex, time-consuming, and costly.

[0006] In summary, existing technologies have shortcomings in the following key dimensions: 1) They cannot obtain the true morphology of lymphocytes under non-interventional conditions; 2) They cannot simultaneously capture static structure and dynamic movement behavior; 3) They cannot effectively identify slightly abnormal lymphocytes; 4) They lack quantitative criteria to distinguish between physiological morphological fluctuations and abnormal cell changes. Therefore, there is an urgent need for a novel lymphocyte analysis method that can simultaneously acquire the static and dynamic characteristics of cells under in vivo, label-free, and non-interventional conditions, and achieve efficient and accurate identification of abnormally morphological lymphocytes using quantifiable multi-dimensional standards, providing a reliable auxiliary analytical basis for blood cell morphology testing. Summary of the Invention

[0007] The purpose of this invention is to address the shortcomings of existing technologies by providing a method and system for lymphocyte feature analysis based on spatiotemporal masking. This method constructs a spatiotemporal masking mechanism, extracts static and dynamic features from video frames, reconstructs the masked features, and constructs a standard feature library to determine the discrimination threshold, thereby enabling the judgment of the reconstructed features and thus determining whether the lymphocyte morphology is abnormal.

[0008] The technical solution adopted in this invention is as follows:

[0009] A method for analyzing lymphocyte features based on spatiotemporal masking for non-disease diagnostic purposes, comprising the following steps:

[0010] S1. Place the lymphocyte sample in the observation container, add maintenance solution, and use it for imaging;

[0011] S2. Obtain dynamic video of lymphocytes in the sample loaded in S1;

[0012] S3. Extract at least three frames of images at equal intervals from the dynamic video, divide each frame of image into multiple feature patch units, perform random masking on a preset proportion of patch units in some of the frames, and retain at least one complete frame as the reference frame for cell dynamic changes.

[0013] S4. Static and dynamic features are extracted based on the reference frame and continuous frames to form the reference original feature vector. Based on the image after random masking in S3 and the complete image, combined with the pre-trained spatiotemporal feature extraction network, the spatiotemporal feature sequence is reconstructed.

[0014] S5. Select normal lymphocyte samples from healthy individuals and perform steps S1 to S4 under the same detection conditions. Based on the spatiotemporal feature sequences of each sample, construct a standard feature library.

[0015] S6. Train a support vector machine based on a standard feature library and determine the feature difference threshold. Calculate the feature difference of the spatiotemporal feature sequence obtained by S1-S4 for the lymphocyte sample to be analyzed, and determine whether it is abnormal based on the threshold.

[0016] In the above technical solution, further, S2 includes: placing the lymphocyte sample in an observation container containing maintenance solution, continuously acquiring dynamic video of the lymphocytes under a microscope, with a magnification of 200× to 1000×, a frame rate of 15 to 30 frames / second, and an acquisition time of 3 to 10 minutes.

[0017] Furthermore, the preset ratio mentioned in S3 is 20% to 50%, and the random mask adopts a non-fixed region strategy, with the patch position of the mask being randomly generated in each round of processing.

[0018] Furthermore, in S4, static and dynamic features are extracted to form a baseline original feature vector. The static features are extracted based on the baseline frame and include at least cell size, mean cytoplasmic gray value, cytoplasmic texture parameters, nucleus roundness, and number of nucleoli. The dynamic features are calculated based on the centroid coordinate sequence of continuous frames and include at least cell tremor frequency and tremor amplitude, offset peristalsis speed and direction change rate, motion trajectory tortuosity and space occupancy. The static and dynamic features are normalized and then concatenated to form the baseline original feature vector.

[0019] Furthermore, the spatiotemporal feature extraction network is specifically as follows:

[0020] The image branch takes at least three frames of temporal images, including the randomly masked image in S3 and the complete image, as inputs. The reference original feature vector is used as input to the feature branch. The spatiotemporal feature extraction network includes a three-dimensional convolutional layer, a three-dimensional pooling layer, a feature fusion layer, and a fully connected layer in sequence. The reference original feature vector is introduced into the feature fusion layer as a ground truth constraint. Combined with the intra-frame and inter-frame feature correlation rules, the network assists in completing the missing features in the masked region. The complete feature vectors of each frame are stacked in temporal order to output a complete spatiotemporal feature sequence.

[0021] Furthermore, the spatiotemporal feature network is pre-trained using a self-supervised mask reconstruction task. All training samples are dynamic microscopic videos of labeled healthy lymphocytes. The training method includes the following steps:

[0022] A dataset is constructed using multiple dynamic videos of lymphocytes. Each training sample in the dataset includes at least three time-series images: an image after S3 random masking and the complete image; a baseline original feature vector; and the corresponding real complete spatiotemporal feature sequence without masking. The dataset is divided into training and validation sets. The training loss function is the mean squared error (MSE) between the reconstructed spatiotemporal feature sequence and the corresponding real complete spatiotemporal feature sequence without masking. The validation set is used to monitor the loss. Pre-training is complete when the loss decreases less than a preset threshold for K consecutive iterations. K is typically between 10 and 15, and the preset threshold is 10. -6 .

[0023] Furthermore, in S5, a standard feature library is constructed. Specifically, steps S1-S4 are repeated N times for a normal lymphocyte sample from the same healthy individual under the same detection conditions. The average value of the spatiotemporal feature sequences obtained N times is used as the standardized feature vector of the sample. N can usually be 3-6.

[0024] The number of normal lymphocyte samples from the healthy individuals is 100 to 1000.

[0025] Furthermore, in S6, a support vector machine is trained based on a standard feature library, and the feature difference of each sample in the standard feature library is calculated. The 95th percentile is taken as the threshold from small to large. The trained support vector machine is used to calculate the feature difference of the spatiotemporal feature sequence of the lymphocytes to be analyzed, and whether it is abnormal is determined according to the threshold.

[0026] Furthermore, if the characteristic difference of the spatiotemporal feature sequence of the lymphocyte to be analyzed exceeds the threshold, it is determined to be an abnormal lymphocyte; otherwise, it is normal.

[0027] This invention also provides a lymphocyte feature analysis system based on spatiotemporal masking, comprising:

[0028] The spatiotemporal feature sequence reconstruction module is used to extract at least three frames of images at equal intervals from the dynamic video of lymphocyte samples, divide each frame of image into multiple feature patch units, perform random masking on a preset proportion of patch units in some of the frames, and retain at least one complete image as the reference frame for cell dynamic changes; and extract static and dynamic features based on the reference frame and consecutive frames to form a reference original feature vector, and reconstruct the spatiotemporal feature sequence based on the randomly masked image and the complete image, combined with a pre-trained spatiotemporal feature extraction network.

[0029] The standard feature library module stores the spatiotemporal feature sequences of each sample obtained by processing normal lymphocyte samples from healthy individuals through the spatiotemporal feature sequence reconstruction module.

[0030] The anomaly detection module trains a support vector machine based on the spatiotemporal feature sequences stored in the standard feature library module and determines the feature difference threshold. It calculates the feature difference of the spatiotemporal feature sequence obtained by the spatiotemporal feature sequence reconstruction module for the lymphocyte sample to be analyzed, and determines whether it is abnormal based on the threshold.

[0031] The present invention has at least the following beneficial effects:

[0032] This invention utilizes an inverted microscope to achieve continuous dynamic imaging of live lymphocytes. The entire detection process requires no cell fixation or staining, nor any physical or chemical interventions, thus preserving the morphology and autonomous movement characteristics of lymphocytes in their natural physiological state to the greatest extent possible. Traditional clinical blood smear testing requires a series of manual operations such as spreading, drying, fixing, and staining, which can easily cause cell shrinkage and deformation, as well as cytoplasmic staining deviations, failing to reproduce the true original state of the cells. This invention effectively avoids the detection errors caused by these manual operations, ensuring the authenticity and accuracy of the test results.

[0033] Unlike traditional technologies that rely solely on static images for morphological observation and completely lose information about dynamic cell changes, this invention establishes a spatiotemporal mask feature extraction mechanism that can simultaneously acquire both static morphological and dynamic motion features of lymphocytes. These two types of feature information complement and coordinate with each other, enriching the analytical dimensions of cell abnormality detection and effectively compensating for the limitations of single static detection. This significantly improves the sensitivity and accuracy of lymphocyte abnormality identification. Furthermore, relying solely on static morphological indicators makes it difficult to identify abnormal lymphocytes with subtle morphological changes in the early stages. This invention focuses on subtle dynamic parameters such as cell tremors, motion shifts, and trajectory changes, accurately capturing abnormal motion behavior in the early stages of lymphocyte changes. It possesses excellent identification capabilities for slightly abnormal lymphocytes and can be effectively applied to peripheral blood lymphocyte sample testing and analysis scenarios.

[0034] Meanwhile, the spatiotemporal feature extraction network used in this invention is pre-trained using a self-supervised mask reconstruction method, eliminating the need for extensive manual sample annotation. Stable prior knowledge of the model can be constructed solely through dynamic microscopic videos of massive amounts of normal lymphocytes. In practical applications, only a small number of anomaly-labeled samples are needed to fine-tune the discrimination threshold before deployment, greatly reducing the cost threshold for model training and clinical deployment, and facilitating practical application.

[0035] In addition, all the detection features extracted by this invention correspond to clear cell physiological and behavioral characteristics, with no meaningless latent features, good interpretability, and are well-suited for blood cell testing and analysis, facilitating understanding and verification of test results by medical personnel. Overall, this invention is simple to operate, highly efficient, and provides reliable results, offering a novel technical solution for automated analysis of in vitro lymphocyte samples, possessing high practical value and broad application prospects. Attached Figure Description

[0036] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0037] Figure 1 This is a flowchart illustrating a method for lymphocyte feature analysis based on spatiotemporal masking according to the present invention.

[0038] Figure 2 This is a schematic diagram of a lymphocyte feature analysis method based on spatiotemporal masking according to the present invention;

[0039] Figure 3 This is a comparison diagram of the dynamic trajectories of normal lymphocytes and abnormal lymphocytes. Detailed Implementation

[0040] The specific embodiments of the present invention will now be described with reference to the accompanying drawings. Figure 1 As shown, the lymphocyte feature analysis method based on spatiotemporal masking provided by this invention mainly includes sample preparation, dynamic image acquisition, spatiotemporal masking processing, feature extraction and reconstruction, standard feature library construction, and anomaly determination based on the standard feature library.

[0041] S1. Sample preparation: Place the lymphocyte sample in an observation container, add maintenance solution, and set aside.

[0042] S2, Dynamic Image Acquisition: Continuously acquire dynamic microscopic videos of the samples prepared in S1;

[0043] S3. Spatiotemporal masking: Extract at least three frames of images at equal intervals from the dynamic video, divide each frame of image into multiple feature patch units, perform random masking on a preset proportion of patch units in some of the frames of image, and retain at least one complete frame of image as a reference frame.

[0044] S4. Feature Extraction and Reconstruction: Static features are extracted based on the baseline frame, and dynamic features are extracted based on the continuous frames. The two are combined into a baseline original feature vector. At the same time, the masked image, together with the complete image, is input into the pre-trained spatiotemporal feature extraction network to reconstruct the complete spatiotemporal feature sequence.

[0045] S5. Construction of Standard Feature Library: Select normal lymphocyte samples from healthy individuals, repeat steps S1 to S4 under the same conditions, and construct a standard feature library based on the spatiotemporal feature sequences of each sample.

[0046] S6. Abnormal Cell State Indication: A support vector machine is trained based on a standard feature library and a judgment threshold is determined. The spatiotemporal feature sequence obtained after processing the sample to be tested through S1 to S4 is input into the support vector machine, its feature difference is calculated, and the cell state is indicated according to the threshold.

[0047] The method of the present invention will be described in conjunction with the embodiments, and the steps are as follows:

[0048] I. Sample Preparation

[0049] Two mL of fresh peripheral blood was collected, and a mononuclear cell layer was separated using density gradient centrifugation. After washing twice with phosphate-buffered saline (PFS), the lymphocytes were resuspended in RPMI-1640 medium. The cell density was adjusted to approximately 1 × 10⁻⁶ cells / mL. 6 Cells / mL, take an appropriate amount and seed into a 35 mm glass bottom culture dish or a dedicated cell imaging chamber, add in vivo imaging maintenance solution (containing 10% fetal bovine serum and 1% penicillin antibody Hanks balanced salt solution) preheated to 37°C to ensure that the cells remain active during observation.

[0050] II. Dynamic Image Acquisition

[0051] The observation container holding the sample was placed on the stage of an inverted microscope. A 40× objective lens with a 10× eyepiece was used for observation at a total magnification of 400x. The frame rate was set to 20 frames per second, and continuous acquisition was performed for 5 minutes to record a complete dynamic video of the lymphocytes. The entire image acquisition process preserved the original state of the cells to the greatest extent possible, thereby realistically capturing the autonomous movement characteristics of lymphocytes, including dynamic behaviors such as cell membrane tremors, cell displacement, and peristalsis.

[0052] III. Video Frame Extraction and Spatiotemporal Masking

[0053] After video acquisition, three key frames are extracted at equal intervals from the recorded dynamic video, corresponding to the start of the video, the midpoint at 2.5 minutes of acquisition, and the end of the video. Each frame is uniformly divided into 4×4 patch units, with each patch approximately 1 / 16th of the entire image size. During processing, random masking is performed only on the start and end frames, randomly masking 30% of the patch units by setting pixels to zero. The midpoint frames remain intact throughout, serving as the baseline reference images for subsequent feature extraction. The masked areas are randomly generated and not fixed to specific locations within the image. This approach effectively avoids the problem of the model overfitting to fixed-region features, forcing the model to rely on the remaining effective patch information to fully learn the microscopic spatial morphology of lymphocytes and the dynamic temporal changes throughout the entire process, thus improving the accuracy of feature extraction and detection.

[0054] IV. Feature Extraction

[0055] After completing the video frame extraction and masking preprocessing, the complete intermediate frame without any masking is used as the reference frame. Feature extraction is performed block by block for all unobstructed visible patch areas in the reference frame.

[0056] The cell features extracted by this method are divided into two categories: static features and dynamic features. After extraction, all features are normalized to a value range of 0-1. Static features mainly include cell body size, average cytoplasmic grayscale value, cytoplasmic contrast and texture parameters, nucleus roundness, and nucleolus number, among other microscopic morphological indicators. Dynamic features are calculated based on the cell centroid coordinate sequence of consecutive image frames, specifically including dynamic motion indicators such as cell tremor frequency, tremor amplitude, offset creep speed, rate of change of motion direction, trajectory tortuosity, and spatial occupancy. The normalized static and dynamic features are then concatenated and fused in a preset fixed order to generate the original feature vector x corresponding to the baseline patch. raw This data serves as auxiliary input for subsequent network model training and detection analysis. The static feature extraction method for lymphocytes is as follows:

[0057] 1) The formula for cell size is:

[0058]

[0059] Where N pix S represents the total number of pixels within the cell outline. p The actual area (μm) corresponding to a single pixel 2 / pixel);

[0060] 2) Mean gray value of cytoplasm

[0061]

[0062] Where Ω c Let n be the set of pixels in the cytoplasm region. c I(i,j) represents the number of pixels in the cytoplasmic region; I(i,j) represents the pixel grayscale value.

[0063] 3) Cellular texture (contrast Con, correlation Corr) is calculated based on the gray-level co-occurrence matrix, specifically including:

[0064]

[0065]

[0066] Where p(i,j) is the joint probability of gray levels i and j in the gray-level co-occurrence matrix (GLCM) of the cytoplasmic region; μ i σ i The mean and standard deviation of grayscale values ​​in the row direction of GLCM; μ j σ j The mean and standard deviation of gray values ​​in the column direction;

[0067] 4) Roundness of the cell nucleus

[0068]

[0069] Among them, A nuc Nucleus area (unit: pixels) 2 ); P nuc The perimeter of the cell nucleus (in pixels);

[0070] 5) Number of nucleoli

[0071]

[0072] Where, μ nuc σ nuc The mean and standard deviation of the nucleus and cytoplasm gray values ​​are given, and Roundness is the local circularity of the candidate connected region (based on 4πA / P). 2 (Calculation); indicates searching within the kernel for grayscale values ​​exceeding μ. nuc +2σ nucCount the connected regions with a circularity greater than 0.7.

[0073] The method for extracting dynamic features of lymphocytes is as follows:

[0074] 1) Tremor frequency

[0075]

[0076] Where M represents the total number of times the cell's movement direction changed significantly during the observation period, which can be defined as the number of times the direction angle changed by more than 30°. The time interval (s) between two adjacent frames is N, and N is the total number of consecutive frames involved in the calculation.

[0077] 2) Tremor amplitude ( The value is used to characterize the overall fluctuation of the instantaneous displacement of the cell; the larger the value, the more the cell "shakes".

[0078]

[0079]

[0080] Where t is the temporal index of consecutive frames in the original dynamic video; (x t ,y t (x) represents the pixel coordinates of the cell centroid in frame t; t+1 ,y t+1 ) represents the pixel coordinates of the cell centroid in frame t+1; dt represents the Euclidean distance of the lymphocyte centroid movement between frames t and t+1, i.e., the instantaneous displacement between adjacent frames, in μm.

[0081] 3) Offset creep speed ( It characterizes the average migration rate of cells during the observation period, that is, the average distance the cell moves per unit time, reflecting the speed of the overall displacement of the cell.

[0082]

[0083] Where N is the total number of consecutive frames in the calculation, Δt is the inter-frame time interval, and dt is the instantaneous displacement between adjacent frames, calculated as above;

[0084] 4) Rate of change of direction

[0085]

[0086] Where N is the total number of video frames, t is the frame index, and Δt is the time interval between two adjacent frames. This represents the minimum angular difference between the cell motion direction angles of frame t+1 and frame t (unit: radians or degrees). Let be the angle of cell motion direction in frame t;

[0087] 5) Trajectory curvature ( () represents the actual path length of cell movement divided by linear displacement.

[0088]

[0089]

[0090] Where N is the total number of video frames, and dt represents the instantaneous displacement between frame t and frame t+1, in μm; (x N y N (x1, y1) represents the coordinates of the cell centroid in the Nth frame (end frame), in μm; (x1, y1) represents the coordinates of the cell centroid in the 1st frame (start frame), in μm.

[0091] 6) Space occupancy rate

[0092]

[0093] in, A is the area of ​​the ellipse at the 95% confidence level of the centroid locus. FOV The total field of view is determined by the field number (FN) of the microscope imaging system and the magnification of the objective lens. Where r is the field of view radius (unit: μm) 2 ).

[0094] The above features are concatenated to form the original feature vector x. raw ,Right now:

[0095] x raw =[A,μ c ,Con,Corr,C nuc N nl ,f tr A tr ,v cr ,R dir ,T ort ,S occ ] T .

[0096] V. Feature Reconstruction

[0097] 5.1 Network Input Configuration

[0098] The three time-series images that have undergone masking processing are combined with the previously obtained original feature vector x. raw They are input into the pre-trained spatiotemporal feature extraction network.

[0099] This embodiment uses a three-dimensional convolutional neural network (3D CNN) to build the feature extraction network. For the specific structure, please refer to [reference needed]. Figure 2 The network is configured with a dual-input branch structure. The image branch takes three masked temporal images as input, with dimensions of 3×H×W, corresponding to the start, middle, and end keyframes of the video, respectively. The feature branch takes the original feature vector x extracted from the complete intermediate reference frame as input. raw This provides truth constraints and references for the network feature reconstruction process.

[0100] 5.2 Network Internal Structure

[0101] The main structure of the network consists of three 3D convolutional layers, one 3D max-pooling layer, one feature fusion layer, and two fully connected layers. Each 3D convolutional layer uses a 3×3×3 kernel with a stride and padding of 1. Batch normalization and ReLU activation are applied after each convolutional layer to ensure network training stability. The max-pooling layer uses a 2×2×2 window with a stride of 2 to compress feature dimensions and filter effective feature information.

[0102] The core function of the feature fusion layer is to concatenate and integrate the spatiotemporal features of the image after pooling with the x-values ​​from the previous input. raw The first feature layer incorporates real feature information from the baseline frame, providing a precise reference for subsequent feature reconstruction. The subsequent fully connected layer is responsible for dimensional mapping and information completion of the fused multidimensional features, improving the overall feature representation effect.

[0103] After forward propagation through the network, the final output is a complete spatiotemporal feature vector x with a dimension of 512. full This feature vector integrates two types of effective information: measured cell features directly extracted from unmasked areas of the image, and inferred features derived by the network from autonomous prediction and completion of masked areas, thus achieving a complete and continuous spatiotemporal representation of cell features.

[0104] 5.3 Network Self-Supervised Pre-training Process

[0105] This network is pre-trained using a self-supervised mask reconstruction task, requiring no manual data annotation throughout the process. It uses only dynamic microscopic videos of healthy lymphocytes as training material. During the pre-training dataset construction phase, a large number of cell dynamic videos are collected in batches. Frame extraction and a 30% random masking process are uniformly performed according to the aforementioned method, while simultaneously retaining the original feature vector x corresponding to the complete intermediate frames. raw This forms standardized pre-training samples. A single training sample consists of three parts: three time-series images after random masking, the original feature vector x of the intermediate frame, and so on. rawThe dataset, along with the unmasked, complete spatiotemporal feature sequences, is then divided into training and validation sets according to a standard ratio for network iterative training and performance verification.

[0106] The pre-training process calculates the error using the mean squared error loss function and compares it with the feature x reconstructed from the network output. full The difference from the true complete spatiotemporal feature sequence. Based on x from intermediate complete frames. raw As a benchmark, the network autonomously learns the spatial correlation and temporal variation patterns between visual patches and mask patches in cell images, thus achieving autonomous learning of feature reconstruction capabilities. The Adam optimizer was used for training, with an initial learning rate set to 10. -4 The batch sample size was set to 32, and the total number of iterations was set to a maximum of 200. During the training process, the learning rate was reduced to 0.8 times the original value every 20 iterations to gradually optimize the training accuracy.

[0107] During training, the validation set is used to monitor the fluctuations in the loss value in real time to determine the network's convergence status. When the decrease in the validation set loss is less than 10 for 15 consecutive rounds, the convergence status is determined. -6 When the values ​​show no rebound fluctuations, the network can be considered to have reached the convergence standard, and the overall pre-training work is complete. After the network pre-training converges, the weight parameters of all convolutional and fully connected layers are fixed. When performing cell feature reconstruction work in the future, only the forward inference operation of the network is performed, and the network weights are no longer updated or optimized.

[0108] 5.4 Principles of Sample Feature Reconstruction

[0109] Once in the feature reconstruction stage, the intermediate frame without masking is used as the core reference throughout. All patch units in this frame can be used to extract complete and accurate original feature vectors, precisely corresponding to the actual physiological state of lymphocytes at the current moment, providing reliable ground truth constraints for the network feature reconstruction process. In contrast, 30% of the patch information in both the video's start and end frames is lost due to masking operations.

[0110] Considering the continuous changes in the morphology and motility of lymphocytes over a short period of time, this method relies on x obtained from intermediate frames. raw Features are used as constraints. By combining the feature information of the unmasked effective patch in a single frame and the feature correlation between adjacent time frames, the static morphological features and dynamic motion features missing in the masked area are reasonably predicted and filled in.

[0111] Based on the above mechanism, taking the complete intermediate frame as the benchmark, the effective patch unit features that are not masked in the start frame and end frame are integrated. Through forward inference operation of the network, the missing feature information of each frame is completed, and finally the temporal and spatiotemporal feature sequence containing complete static morphological information and dynamic motion information of cells is reconstructed.

[0112] VI. Construction of a Standard Feature Library for Normal Lymphocytes and One-Class SVM Boundary Training

[0113] (1) Construction of standard feature library:

[0114] This method selects 500 normal lymphocyte samples from different healthy individuals as the baseline dataset to ensure the universality of the baseline features. All samples are processed under uniform detection conditions, and the aforementioned sample processing and feature reconstruction procedures S1 to S4 are repeated. Each sample is detected five times, resulting in five sets of 512-dimensional reconstructed feature vectors x. full To reduce the interference of single detection errors on feature accuracy, the average of the feature vectors obtained from five detections of the same sample is calculated, and the calculated average vector is used as the final standardized feature vector x for that sample. i， The function is: Standardized feature vectors from 500 healthy samples were compiled to establish a standard feature library for normal lymphocytes. The mean μ of all feature dimensions in this library was also calculated. ref With standard deviation σ ref This provides benchmark parameters for the feature normalization processing of subsequent test samples.

[0115] (2) One-Class SVM boundary training:

[0116] The 500 standardized feature vectors in the feature library were randomly divided in a 4:1 ratio to obtain 400 training set samples and 100 validation set samples. The training set data was used to train a single-class support vector machine model, using a radial basis function (RBF) kernel. The model parameters were iteratively updated by solving the corresponding objective function. The training optimization objective was: st .

[0117] After iterative convergence, the core parameters ω and ρ of the model are determined, and the boundary training of the anomaly detection model is completed. The generalization ability of the model is verified by relying on the validation set to avoid overfitting.

[0118] (3) Standardization of features and calculation of variance of the test sample:

[0119] For the lymphocyte sample to be tested, strictly follow the steps S1 to S4 described above to complete sample preparation, image acquisition, and feature reconstruction, and obtain the reconstructed feature vector x of the sample to be tested. full Using the pre-stored mean μ from the health sample feature library. ref and standard deviation σ ref The feature vector to be tested is standardized to eliminate the interference caused by differences in feature dimension units. The standardization formula is: , where μref σ ref The mean and standard deviation of the healthy sample feature library are used. The standardized feature vector x is input into the trained One-Class SVM model, and the feature variance D of the samples is calculated using a decision function. This variance characterizes the degree of deviation between the tested cell features and the healthy baseline features. The formula for calculating the variance is: The kernel function expression is: γ is the kernel width parameter of the radial basis kernel function. Let L2 norm be the square of the two sets of eigenvectors.

[0120] VII. Anomaly Detection

[0121] After the lymphocyte sample to be tested undergoes sample preparation, image acquisition, and feature reconstruction through the aforementioned S1 to S4 processes, a corresponding 512-dimensional spatiotemporal feature vector can be obtained. This feature vector is then imported into a trained One-Class SVM model to calculate the feature dissimilarity D of the sample.

[0122] The threshold T in this scheme is determined based on the statistical distribution of the difference in the validation set of the standard feature library. Specifically, the 95th percentile of the difference data in the validation set is selected as the critical threshold. If the difference D calculated for the sample to be tested is greater than this threshold, the lymphocyte can be determined to be in an abnormal state; otherwise, it is considered to be in a normal cell state.

[0123] Furthermore, the present invention also provides a lymphocyte feature analysis system based on spatiotemporal masking, comprising:

[0124] The spatiotemporal feature sequence reconstruction module is used to extract at least three frames of images at equal intervals from the dynamic video of lymphocyte samples, divide each frame of image into multiple feature patch units, perform random masking on a preset proportion of patch units in some of the frames, and retain at least one complete image as the reference frame for cell dynamic changes; and extract static and dynamic features based on the reference frame and consecutive frames to form a reference original feature vector, and reconstruct the spatiotemporal feature sequence based on the randomly masked image and the complete image, combined with a pre-trained spatiotemporal feature extraction network.

[0125] The standard feature library module stores the spatiotemporal feature sequences of each sample obtained by processing normal lymphocyte samples from healthy individuals through the spatiotemporal feature sequence reconstruction module.

[0126] The anomaly detection module trains a support vector machine based on the spatiotemporal feature sequences stored in the standard feature library module and determines the feature difference threshold. It calculates the feature difference of the spatiotemporal feature sequence obtained by the spatiotemporal feature sequence reconstruction module for the lymphocyte sample to be analyzed, and determines whether it is abnormal based on the threshold.

[0127] Example 1

[0128] The following uses a complete data stream of a specific sample to demonstrate the computational process of this invention from feature extraction to anomaly detection.

[0129] I. Feature Extraction and Standardization of Healthy Control Samples

[0130] After selecting a healthy control sample and processing it according to steps S1 to S4, the original values ​​of static and dynamic features extracted from the baseline frame and continuous frame motion sequences are as follows:

[0131] (a) Measured values ​​of static characteristics

[0132]

[0133] (ii) Measured values ​​of dynamic characteristics

[0134] Dynamic features are calculated based on a sequence of centroid coordinates from consecutive frames. This example uses 100 frames for calculation, with a frame rate of 20 frames per second and a frame interval Δt = 0.05 seconds.

[0135]

[0136] (III) Normalization and Standardization

[0137] The above 12 feature values ​​are concatenated in a fixed-dimensional order to obtain the original feature vector of the healthy sample:

[0138] x raw = [82,130,0.28,0.82,0.94,1,2.0,1.1,3.2,0.7,1.2,0.12] T

[0139] To reduce the random error of a single test, the same healthy sample was tested five times under the same conditions, resulting in five raw feature vectors. The average value was then used as the standardized feature vector x for that sample. i :

[0140]

[0141] After substituting the data, the calculation yielded:

[0142] xi= [83, 128, 0.29, 0.83, 0.93, 1, 2.1, 1.0, 3.3, 0.7, 1.2, 0.13] T .

[0143] II. Feature Extraction of the Sample to be Tested

[0144] Take a sample of lymphocytes to be tested, which is a health screening sample from a routine physical examination that indicates that the lymphocyte status needs to be verified. Process it according to steps S1 to S4. The measured values ​​of the extracted static features are listed in the table below.

[0145] (a) Measured values ​​of static characteristics

[0146]

[0147] (ii) Dynamic features are calculated based on a continuous frame centroid coordinate sequence, and the measured values ​​are as follows:

[0148]

[0149] (III) Original feature vector of the sample to be tested

[0150] Following the same dimensional order as the healthy samples, the 12 feature values ​​of the test sample are concatenated to form the original feature vector:

[0151] x=[95,122,0.38,0.71,0.85,2,4.2,2.8,5.8,2.1,2.0,0.22] T .

[0152] III. Feature Reconstruction

[0153] The three masked images and the original feature vector x of the test sample are input into a pre-trained 3DCNN network. After forward propagation, the network outputs the complete reconstructed spatiotemporal feature vector x of the test sample. full ,

[0154] x full = [94, 124, 0.37, 0.73, 0.86, 2, 4.1, 2.7, 5.6, 2.0, 1.9, 0.21] T As can be seen, the values ​​of the reconstructed vector and the original feature vector are quite close in all dimensions, indicating that the network can preserve the original features well in the unmasked region, while effectively completing the masked region.

[0155] IV. Calculation of Feature Difference D

[0156] The feature difference is calculated using a One-Class SVM model, and the model parameters ω and ρ are obtained by pre-training from a standard feature library.

[0157] (a) Model parameters

[0158] The standard feature library (500 healthy samples) was trained using a One-Class SVM with a Radial Basis Function (RBF) kernel function of γ=0.1. The partial values ​​of the model parameter ω are as follows:

[0159]

[0160] Hyperplane bias parameters: ρ = 1.35; preset threshold T = 0.30 (determined by the 95th percentile of the variance in the validation set samples).

[0161] (II) Kernel Function Calculation

[0162] Using the feature vector x of the test sample and the standardized feature vector x of the healthy sample i Taking the calculation of the square of the L2 norm as an example:

[0163]

[0164] Substitute the radial basis kernel function:

[0165]

[0166] (Note: In the actual SVM decision function, the sample to be tested needs to be weighted and summed with all healthy sample vectors in the standard feature library after kernel function calculation. Here, only the calculation process with a single healthy sample is shown as an example. This weighted summation process has been completed in the training phase and solidified into the model parameters. It does not need to be repeated during actual detection.)

[0167] After comprehensive calculation of multiple healthy samples, the following results were obtained: Then, the feature difference degree is calculated:

[0168] .

[0169] V. Threshold Comparison and Cell Recognition Hints

[0170] The feature variance D of the sample to be tested is 0.37, and the preset threshold T is 0.30. Since the D value exceeds the threshold, the sample is determined to be abnormal lymphocytes. The specific deviations for each dimension are as follows:

[0171]

[0172] The results of the actual measurements show that the core reason for the difference D in the tested samples exceeding the judgment threshold is that dynamic characteristics such as lymphocyte tremor frequency, rate of change in movement direction, and trajectory tortuosity showed significant abnormal deviations. In contrast, static morphological indicators such as cell size and nucleus roundness, although showing some fluctuations compared to healthy samples, still fall within the normal physiological fluctuation range of cells. A single static characteristic is insufficient to determine cellular abnormalities. Figure 3This visually demonstrates the difference in the centroid motion trajectories of cells in healthy control samples and test samples within the same observation period. The motion trajectories of healthy lymphocytes are generally smooth and regular, with stable motion directions and a consistently low rate of directional change. In contrast, the motion trajectories of the abnormal cells in the test are tortuous and chaotic, with obvious cell tremors and extremely large fluctuations in the rate of directional change, which corroborates the aforementioned quantitative characteristics.

[0173] This practical case fully demonstrates that, compared to traditional static morphological observation, this invention focuses on analyzing and judging the dynamic characteristics of cells. It can accurately capture subtle abnormalities at the cellular motility level in the early stages, before significant changes in the appearance of lymphocytes, achieving highly sensitive early auxiliary identification of abnormal cells. Furthermore, it should be noted that this invention only quantitatively analyzes the static morphology and dynamic motility characteristics of isolated lymphocytes, outputting characteristic difference information reflecting the degree of deviation from the cell state. This information is intermediate reference information, only assisting in lymphocyte identification; it cannot directly provide a diagnostic conclusion regarding whether a subject has a specific disease, nor can it provide a diagnostic result containing any disease name. In clinical applications, it can only be used for blood cell testing analysis; the final clinical diagnosis must be made by a physician based on a comprehensive judgment of medical history, physical signs, and other examination results.

[0174] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0175] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0176] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0177] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0178] The embodiments described above are merely some preferred embodiments of the present invention, and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, all technical solutions obtained by equivalent substitution or equivalent transformation fall within the protection scope of the present invention.

Claims

1. A method for analyzing lymphocyte features based on spatiotemporal masking for non-disease diagnostic purposes, characterized in that, Includes the following steps: S1. Place the lymphocyte sample in the observation container, add maintenance solution, and use it for imaging; S2. Obtain dynamic video of lymphocytes in the sample loaded in S1; S3. Extract at least three frames of images at equal intervals from the dynamic video, divide each frame of image into multiple feature patch units, perform random masking on a preset proportion of patch units in some of the frames, and retain at least one complete frame as the reference frame for cell dynamic changes. S4. Static and dynamic features are extracted based on the reference frame and continuous frames to form the reference original feature vector. Based on the image after random masking in S3 and the complete image, combined with the pre-trained spatiotemporal feature extraction network, the spatiotemporal feature sequence is reconstructed. S5. Select normal lymphocyte samples from healthy individuals and perform steps S1 to S4 under the same detection conditions. Based on the spatiotemporal feature sequences of each sample, construct a standard feature library. S6. Train a support vector machine based on a standard feature library and determine the feature difference threshold. Calculate the feature difference of the spatiotemporal feature sequence obtained by S1-S4 for the lymphocyte sample to be analyzed, and determine whether it is abnormal based on the threshold.

2. The method for lymphocyte feature analysis based on spatiotemporal masking according to claim 1, characterized in that, S2 includes: placing the lymphocyte sample in an observation container containing maintenance solution, continuously acquiring dynamic video of the lymphocytes under a microscope, with a magnification of 200× to 1000×, a frame rate of 15 to 30 frames / second, and an acquisition time of 3 to 10 minutes.

3. The method for lymphocyte feature analysis based on spatiotemporal masking according to claim 1, characterized in that, The preset ratio mentioned in S3 is 20% to 50%, and the random mask adopts a non-fixed region strategy. The patch position of the mask is randomly generated in each round of processing.

4. The method for lymphocyte feature analysis based on spatiotemporal masking according to claim 1, characterized in that, In step S4, static and dynamic features are extracted to form a baseline original feature vector. The static features are extracted based on the baseline frame and include at least cell size, average cytoplasmic gray value, cytoplasmic texture parameters, nucleus roundness, and number of nucleoli. The dynamic features are calculated based on the centroid coordinate sequence of continuous frames and include at least cell tremor frequency and tremor amplitude, offset peristalsis speed and direction change rate, motion trajectory tortuosity and space occupancy. The static and dynamic features are normalized and then concatenated to form the baseline original feature vector.

5. The method for lymphocyte feature analysis based on spatiotemporal masking according to claim 1, characterized in that, The spatiotemporal feature extraction network is specifically as follows: The image branch takes at least three frames of temporal images, including the randomly masked image in S3 and the complete image, as inputs. The reference original feature vector is used as input to the feature branch. The spatiotemporal feature extraction network includes a three-dimensional convolutional layer, a three-dimensional pooling layer, a feature fusion layer, and a fully connected layer in sequence. The reference original feature vector is introduced into the feature fusion layer as a ground truth constraint. Combined with the intra-frame and inter-frame feature correlation rules, the network assists in completing the missing features in the masked region. The complete feature vectors of each frame are stacked in temporal order to output a complete spatiotemporal feature sequence.

6. The method for lymphocyte feature analysis based on spatiotemporal masking according to claim 5, characterized in that, The spatiotemporal feature network was pre-trained using a self-supervised mask reconstruction task. All training samples were dynamic microscopic videos of labeled healthy lymphocytes. The training method included the following steps: A dataset is constructed by taking multiple dynamic videos of lymphocytes. Each training sample in the dataset includes at least three time-series images: an image after random S3 masking and a complete image, a baseline original feature vector, and the corresponding real complete spatiotemporal feature sequence without masking. The dataset is divided into a training set and a validation set. The training loss function is the mean square error (MSE) loss between the reconstructed spatiotemporal feature sequence and the corresponding real complete spatiotemporal feature sequence without masking. The validation set is used to monitor the loss. When the loss decreases less than a preset threshold for K consecutive iterations, the pre-training is completed.

7. The method for lymphocyte feature analysis based on spatiotemporal masking according to claim 1, characterized in that, In S5, a standard feature library is constructed. Specifically, steps S1-S4 are repeated N times for a normal lymphocyte sample from the same healthy individual under the same detection conditions. The average value of the spatiotemporal feature sequences obtained N times is used as the standardized feature vector of the sample. The number of normal lymphocyte samples from the healthy individuals is 100 to 1000.

8. The method for lymphocyte feature analysis based on spatiotemporal masking according to claim 1, characterized in that, In S6, a support vector machine is trained based on a standard feature library. The feature difference of each sample in the standard feature library is calculated, and the 95th percentile is taken as the threshold from small to large. The trained support vector machine is used to calculate the feature difference of the spatiotemporal feature sequence of the lymphocytes to be analyzed, and whether it is abnormal is determined according to the threshold.

9. The method for lymphocyte feature analysis based on spatiotemporal masking according to claim 8, characterized in that, If the characteristic difference of the spatiotemporal feature sequence of the lymphocyte to be analyzed exceeds the threshold, it is determined to be an abnormal lymphocyte; otherwise, it is normal.

10. A lymphocyte feature analysis system based on spatiotemporal masking, characterized in that, include: The spatiotemporal feature sequence reconstruction module is used to extract at least three frames of images at equal intervals from the dynamic video of lymphocyte samples, divide each frame of image into multiple feature patch units, perform random masking on a preset proportion of patch units in some of the frames, and retain at least one complete image as the reference frame for cell dynamic changes; and extract static and dynamic features based on the reference frame and consecutive frames to form a reference original feature vector, and reconstruct the spatiotemporal feature sequence based on the randomly masked image and the complete image, combined with a pre-trained spatiotemporal feature extraction network. The standard feature library module stores the spatiotemporal feature sequences of each sample obtained by processing normal lymphocyte samples from healthy individuals through the spatiotemporal feature sequence reconstruction module. The anomaly detection module trains a support vector machine based on the spatiotemporal feature sequences stored in the standard feature library module and determines the feature difference threshold. It calculates the feature difference of the spatiotemporal feature sequence obtained by the spatiotemporal feature sequence reconstruction module for the lymphocyte sample to be analyzed, and determines whether it is abnormal based on the threshold.

Images