An Automated Cell State Analysis Method Based on Image Recognition
By constructing an automatic cell state analysis method based on deep multi-scale feature fusion and dynamic attention guidance, the problems of low cell state discrimination accuracy and insufficient multimodal information fusion in existing technologies are solved. This method achieves high robustness and high accuracy in automatic cell state discrimination, and is suitable for high-throughput drug screening and precision medical diagnosis.
Patent Information
- Application Number
- CN202611583547.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-13
- Publication Date
- 2026-11-17
AI Technical Summary
Existing cell image analysis technologies suffer from low accuracy in cell state identification in complex biological sample environments, weak generalization ability, high dependence on manual intervention, and insufficient fusion of multimodal information, making it difficult to achieve highly robust and accurate automatic cell state identification.
We construct an automatic cell state analysis method based on deep multi-scale feature fusion and dynamic attention guidance. Through cascaded neural networks, we achieve accurate segmentation of cell instances, adaptive extraction of multi-scale contextual features, and dynamic weighted fusion of cross-channel spatiotemporal information. Combined with time gating mechanism and dynamic channel attention decoder, we form an end-to-end analysis closed loop.
It significantly improves the ability to characterize complex states, enhances the objectivity, consistency, and reproducibility of analysis results, reduces reliance on human intervention, and is suitable for high-throughput drug screening and precision medicine diagnosis.
Smart Images

Figure CN123118686A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the interdisciplinary field of artificial intelligence and medical image analysis, specifically involving an automatic cell state analysis method based on image recognition. Background Technology
[0002] With the rapid development of life science research and the biopharmaceutical industry, real-time monitoring and precise analysis of cell state play a crucial role in disease diagnosis, drug screening, and cell therapy. Traditional methods for assessing cell viability mainly rely on manual microscopic observation or biochemical detection methods based on fluorescent labels. The former is highly subjective, inefficient, and difficult to quantify, while the latter requires the introduction of exogenous dyes or probes, which may interfere with normal cellular physiological activities or even induce toxic effects. In recent years, label-free, non-invasive image recognition technology has gradually become an important tool for cell state analysis due to its ability to continuously and dynamically capture cell morphology and behavioral characteristics. Its core lies in extracting visual representations related to cellular physiological activities from time-series image sequences.
[0003] Among them, the temporal image analysis method based on pixel intensity changes focuses on the dynamic evolution of local optical density or grayscale values of cells during culture. This method assumes that the degree of cellular metabolic activity is closely related to its ability to modulate the refractive index of the surrounding medium, thus manifesting as fluctuations in pixel intensity in phase contrast or bright-field imaging. By quantifying the amplitude and frequency of pixel intensity changes in a specific region per unit time, it can theoretically reflect key physiological indicators such as cell membrane fluidity, intracellular substance transport, and overall metabolic level, providing an objective basis for judging the functional state of cells.
[0004] While some studies have attempted to analyze cell movement or deformation using image sequences, current technologies generally suffer from the following drawbacks: First, most methods focus only on morphological features in static images, neglecting the continuous evolution of pixel intensity over time, thus failing to effectively distinguish between transient perturbations and persistent changes in activity. Second, they lack the ability to model pixel intensity changes at multiple scales, making it difficult to simultaneously capture rapid transient responses and slow trend drifts, resulting in blurred boundaries between "insufficiently active" and "overactive" states. Third, existing algorithms typically use fixed thresholds or simple statistics for classification, failing to consider the impact of different cell types, culture conditions, or imaging parameters on the baseline of intensity changes, thus limiting generalization ability. These problems make it difficult for current image recognition methods to achieve highly robust and accurate automatic cell state identification in complex biological scenarios, necessitating an intelligent analysis scheme that can deeply integrate temporal pixel dynamic features and adaptively define activity levels. Summary of the Invention
[0005] This invention relates to an automated cell state analysis method based on image recognition, aiming to address the core problems of existing cell image analysis technologies, such as low accuracy in cell state discrimination under complex biological sample environments, weak generalization ability, high dependence on manual intervention, and insufficient multimodal information fusion. Currently, in the fields of life science research and clinical diagnosis, accurate cell state discrimination is a crucial step in assessing drug response, disease progression, and cell function. Traditional methods mainly rely on manual microscopic observation or automated image processing workflows based on simple threshold segmentation and morphological feature extraction. These methods typically employ image preprocessing strategies with fixed parameters, combined with manually designed geometric or texture features, which are then input into a shallow classifier for state determination.
[0006] However, the aforementioned existing technologies have significant drawbacks. First, biological sample imaging commonly suffers from uneven illumination, strong background noise, cell overlap and adhesion, and large differences in staining intensity, resulting in poor robustness of segmentation algorithms based on fixed thresholds or edge detection, making it difficult to reliably acquire complete single-cell regions. Second, manually designed features have limited dimensions, failing to effectively characterize the subtle and complex morphological, textural, and subcellular structural changes of cells under different physiological or pathological states, especially for transitional cells such as those in early apoptosis, stress responses, or atypical proliferation, where the discrimination accuracy drops significantly. Third, existing systems typically separate image acquisition, preprocessing, feature extraction, and classification decisions into independent steps, lacking an end-to-end collaborative optimization mechanism, leading to the accumulation of errors at each stage of the process. Finally, most methods utilize only single-channel bright-field or fluorescence images, failing to effectively integrate heterogeneous image data from multiple channels and time phases, thus losing crucial spatiotemporal correlation information regarding the dynamic evolution of cell states.
[0007] To overcome the aforementioned technical bottlenecks, this invention proposes an automatic cell state analysis method based on deep multi-scale feature fusion and dynamic attention guidance. This method constructs a complete closed-loop analysis framework from raw image input to cell state semantic output. Its core lies in achieving accurate cell instance segmentation, adaptive extraction of multi-scale contextual features, and dynamic weighted fusion of cross-channel spatiotemporal information through a cascaded neural network architecture.
[0008] As one embodiment of the present invention, the automatic cell state analysis method based on image recognition includes the following steps: acquiring a multi-channel microscopic image sequence of the cell culture sample to be analyzed, wherein the multi-channel microscopic image sequence includes at least one bright-field channel image and at least one specifically fluorescently labeled channel image, the channel images are strictly aligned in space and have a uniform timestamp identifier; performing a standardization preprocessing on the multi-channel microscopic image sequence, the standardization preprocessing including illumination correction, background subtraction, contrast adaptive enhancement, and pixel size normalization, to eliminate systematic biases introduced by equipment and operation; and inputting the preprocessed multi-channel microscopic image sequence into a cascaded cell instance segmentation network. The cascaded cell instance segmentation network consists of a shared low-level feature extractor, a foreground-background coarse segmentation head, and an instance boundary refinement head. The shared low-level feature extractor employs an encoder-decoder structure. The encoder part consists of multiple stacked convolutional blocks, each containing a convolutional layer, a batch normalization layer, and an activation function layer. The decoder part fuses feature maps from different levels of the encoder through skip connections and performs upsampling through transposed convolutions. The foreground-background coarse segmentation head generates a probability heatmap of the cell region based on the top-level feature map of the decoder. The instance boundary refinement head uses the coarse segmentation result as an attention mask to weight the high-resolution feature maps of the intermediate layers of the encoder to generate... The cell instance boundary contours are generated with sub-pixel precision. Based on the cell instance mask output by the cascaded cell instance segmentation network, multi-channel region of interest (ROI) image patches for each independent cell are cropped from the original multi-channel image. The multi-channel ROI image patches for each cell are input into a multi-scale feature pyramid encoder. The multi-scale feature pyramid encoder extracts the feature responses of the cell at three scales: local detail, mesoscopic structure, and global morphology, through parallel dilated convolution branches with different receptive fields. The feature responses at each scale are then concatenated along the channel dimension to form a multi-scale fused feature vector. The multi-scale fused feature vectors of the same cell at different time points are arranged in chronological order to form a... A cell state temporal feature matrix is generated. This matrix is then input into a bidirectional long short-term memory network with a time-gated mechanism. The forward pass of the bidirectional long short-term memory network captures the historical evolution trend of the cell state, and the backward pass captures the potential impact of future states on the current moment. The time-gated mechanism dynamically adjusts the retention and forgetting ratio of the information flow based on the signal-to-noise ratio of the current cell features. The final hidden state vector output by the bidirectional long short-term memory network is then input into a dynamic channel attention decoder. The dynamic channel attention decoder first calculates the importance weight of each feature channel, and then performs a weighted summation of the feature vectors to generate a refined state representation vector that focuses on the most discriminative feature.Based on the refined state representation vector, the final state category label of the cell is output through a fully connected classification layer. The state category label includes normal proliferation, early apoptosis, late apoptosis, necrosis, mitosis, stress response, and resting state.
[0009] As one embodiment of the present invention, the training process of the cascaded cell instance segmentation network adopts a two-stage joint optimization strategy. In the first stage, a large amount of labeled cell image dataset is used to supervise the training of the foreground-background coarse segmentation head with the cross-entropy loss function. In the second stage, after freezing some weights of the shared low-level feature extractor, the instance boundary refinement head is fine-tuned with the weighted sum of the boundary intersection-union loss function and the Hausdorff distance loss function as the optimization objective to ensure the geometric consistency between the segmentation boundary and the ground truth annotation.
[0010] As one embodiment of the present invention, the dilated convolution branch in the multi-scale feature pyramid encoder has a dilation rate of 1, 3, and 5, respectively, and the corresponding receptive field coverage ranges are the cell nucleus region, the entire cytoplasm region, and the local microenvironment region containing neighboring cells, thereby realizing multi-level information capture from micro to macro.
[0011] As one embodiment of the present invention, the specific implementation of the time-gating mechanism is as follows: inputting the cell state temporal feature matrix into a bidirectional long short-term memory network with a time-gating mechanism, including: Let the input at time t be The cell state at the previous moment was The hidden state at the previous moment was Forgotten Gate The calculation formula is , Input gate The calculation formula is , Candidate cell state The calculation formula is , Current cell state The calculation formula is .
[0012] in Here, is the sigmoid function, W and b are the learnable weight matrix and bias term, and ⊙ represents element-wise multiplication.
[0013] As one embodiment of the present invention, the process of calculating channel weights by the dynamic channel attention decoder includes: first, performing global average pooling on the input feature vector to obtain a channel descriptor; then, passing the descriptor through a bottleneck structure containing two fully connected layers, wherein the first fully connected layer compresses the number of channels to one-sixteenth of the original number of channels, and the second fully connected layer restores it to the original number of channels; finally, normalizing the output value to between zero and one using the Sigmoid function, which serves as the attention weight for each channel.
[0014] As one embodiment of the present invention, the method further includes an uncertainty quantification module. This module is based on Monte Carlo Dropout technology. During the inference phase, it performs multiple forward propagations on the same input to calculate the variance of the probability distribution of the output state category. If the variance exceeds a preset threshold, the cell is marked as "state uncertain" and a manual review process is triggered.
[0015] Compared with existing technologies, the advantages and positive effects of this invention are as follows: This invention effectively solves the segmentation inaccuracies caused by cell adhesion and blurred boundaries by constructing a cascaded cell instance segmentation network, providing high-quality single-cell input for subsequent analysis. Through a multi-scale feature pyramid encoder, it systematically captures multi-level morphological and texture information of cells from subcellular organelles to the microenvironment, significantly improving the ability to represent complex states. The introduction of a bidirectional long short-term memory network with a time-gated mechanism fully utilizes the temporal dependence of cell state evolution, making the discrimination of transitional and transient states more accurate. The dynamic channel attention decoder can adaptively focus on the most discriminative feature channels, suppressing interference from irrelevant or noisy features. The entire method forms an end-to-end, multimodal, and time-aware analytical closed loop, significantly reducing reliance on human intervention and improving the objectivity, consistency, and reproducibility of the analysis results, providing reliable technical support for high-throughput drug screening, precision medical diagnosis, and basic cell biology research. Attached Figure Description
[0016] Figure 1 This is a schematic diagram of the overall technical architecture of the automatic cell state analysis method based on image recognition proposed in this invention; Figure 2 This is a schematic diagram of the core principle framework of the cascaded cell instance segmentation network in this invention; Figure 3 This is a logical flowchart of the multi-scale feature pyramid encoding and cell state temporal modeling in this invention; Figure 4 This is a flowchart of the logic flow of the dynamic channel attention decoding and uncertainty quantization module in this invention. Figure 5This is a schematic diagram of the multi-level interaction relationship and data flow between multi-channel microscopic image input and cell state semantic output in this invention. Detailed Implementation
[0017] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0019] Example 1 This invention provides an automatic cell state analysis method based on image recognition. This method constructs an end-to-end, multimodal, and time-series-aware deep learning analysis framework to achieve high-precision and robust discrimination of the state of individual cells in complex biological samples. The method takes a multi-channel microscopic image sequence as input, and through steps such as standardized preprocessing, cascaded cell instance segmentation, multi-scale feature extraction, temporal modeling, dynamic attention decoding, and uncertainty quantification, it finally outputs a state category label for each cell. The specific implementation of this method will be described in detail below, in conjunction with step S in the invention description.
[0020] The method first performs step S1: acquiring a multi-channel microscopic image sequence of the cell culture sample to be analyzed. This multi-channel microscopic image sequence is acquired using an inverted fluorescence microscope system equipped with a multi-channel imaging module. This system includes a high-resolution CMOS image sensor, a set of switchable excitation and emission filters, a controllable intensity LED light source array, and a precision motorized stage. In a complete imaging process, the system sequentially activates the bright-field illumination mode and the excitation light sources of multiple specific fluorescent labeling channels, capturing image data of the corresponding wavelength bands. All channel images are acquired at the same physical location and under the same focal plane, and timestamps are strictly consistent via hardware synchronization signals. The spatial resolution of each image channel is 0.3 micrometers per pixel, and the image size is 2048 x 2048 pixels. The number of fluorescence channels is determined according to the experimental design, but includes at least one DAPI channel for labeling the cell nucleus and one green fluorescence channel for labeling the cell membrane or a specific protein. All raw images are stored in lossless TIFF format, along with metadata files recording information such as acquisition time, channel type, exposure time, gain parameters, and objective magnification.
[0021] The S2 step then proceeds: standardization preprocessing is performed on the multi-channel microscopic image sequence. This preprocessing process aims to eliminate systematic biases introduced by differences in imaging equipment, operating environment, and sample preparation, ensuring that the input data for subsequent analysis has consistent statistical characteristics. The preprocessing operation includes four consecutive sub-steps. The first sub-step is illumination correction, employing a non-uniform illumination compensation algorithm based on polynomial surface fitting. Specifically, the illumination intensity distribution is extracted from the background image without cell regions, fitted with a two-dimensional quadratic polynomial surface, and this surface is used as a divisor to normalize the original image pixel by pixel. The second sub-step is background subtraction, which separates high-frequency detail components in the image through morphological top-hat transformation, effectively removing slowly changing background fluorescence noise. The third sub-step is contrast adaptive enhancement, applying a limited contrast adaptive histogram equalization algorithm with a neighborhood window size set to 128 x 128 pixels and a cropping threshold set to 0.02, to enhance the visibility of fine structures within cells without amplifying noise. The fourth sub-step is pixel size normalization. Based on the objective magnification information in the metadata, all images are resampled to a uniform spatial scale, i.e., 0.5 micrometers per pixel, using a bicubic interpolation algorithm to ensure geometric fidelity. After completing the above four sub-steps, all channel images are stacked into a four-dimensional tensor, with dimensions of height, width, number of channels, and time point.
[0022] Next, step S3 is performed: the preprocessed multi-channel microscopic image sequence is input into a cascaded cell instance segmentation network. This network consists of a shared low-level feature extractor, a foreground-background coarse segmentation head, and an instance boundary refinement head. The shared low-level feature extractor adopts a U-shaped encoder-decoder structure. The encoder part consists of five stacked convolutional blocks, each containing two consecutive 3x3 convolutional layers, followed by batch normalization layers and modified linear unit activation functions. Downsampling is performed between convolutional blocks using 2x2 max-pooling layers, halving the feature map size and doubling the number of channels at each level, ultimately resulting in a deep semantic feature map that is one-sixteenth the size of the original image. The decoder part also contains five upsampling blocks. Each block first restores the feature map size to the previous level through a 2x transposed convolution, then concatenates the channels with the feature map of the corresponding level in the encoder through skip connections, and finally performs feature fusion and refinement through two 3x3 convolutional layers. The foreground-background coarse segmentation head receives the feature map from the top layer of the decoder and maps it to a single-channel probability heatmap through a 1x1 convolutional layer. Each pixel value in this heatmap represents the probability that the location belongs to the cell's foreground region. The instance boundary refinement head receives the probability heatmap output by the coarse segmentation head and binarizes it into a foreground mask. This mask is used as a spatial attention mechanism to weight the high-resolution feature map output from the third layer of the encoder channel by channel. The weighted feature map is then processed by a lightweight convolutional module, ultimately outputting the precise boundary contour of each cell instance with sub-pixel accuracy. The network training employs a two-stage joint optimization strategy. The first stage uses the cross-entropy loss function to supervise the coarse segmentation head, with the optimization objective being to minimize the pixel-level difference between the predicted probability heatmap and the true foreground mask. The second stage, with the weights of the first three convolutional blocks of the encoder frozen, fine-tunes the instance boundary refinement head using the weighted sum of the boundary intersection-union loss function and the Hausdorff distance loss function as the optimization objective. The boundary intersection ratio loss function measures the degree of overlap between the predicted boundary and the true boundary, while the Hausdorf distance loss function constrains the geometric deviation between the two in the worst case. The weight ratio between the two is set to 2:1.
[0023] After completing step S3, step S4 is executed: based on the cell instance mask output by the cascaded cell instance segmentation network, multi-channel region of interest (ROI) image patches for each individual cell are cropped from the original multi-channel image. For each successfully segmented cell instance, its mask defines an irregular two-dimensional region. To facilitate subsequent processing, this region is enclosed by an outer rectangle, forming a rectangular clipping box. The size of the clipping box is dynamically determined according to the actual size of the cell, with its length and width each extended outward by twenty pixels to retain some surrounding microenvironment information. All channel images are simultaneously clipped according to this clipping box, generating a three-dimensional image patch of size H x W x C, where H and W are the height and width of the clipping box, and C is the total number of channels. To eliminate the interference of cell orientation on feature extraction, each image patch is rotated and aligned according to its principal axis, making its major axis parallel to the horizontal direction. The aligned image patches are uniformly scaled to a fixed size of 128 x 128 pixels using a bilinear interpolation algorithm.
[0024] The S5 step then proceeds: Multi-channel region-of-interest (ROI) image patches for each cell are input into a multi-scale feature pyramid encoder. At the heart of this encoder is a parallel dilated convolutional branch structure designed to simultaneously capture cellular contextual information at different spatial scales. The input image patches first pass through a shared initial convolutional layer to generate a base feature map. This feature map is then simultaneously fed into three parallel dilated convolutional branches. The first branch uses a standard 3x3 convolution with a dilation rate of 1, covering a receptive field of approximately 30 micrometers, primarily responding to local details such as chromatin texture and nucleolar structure within the cell nucleus. The second branch uses a 3x3 dilated convolution with a dilation rate of 3, expanding its effective receptive field to approximately 90 micrometers, capable of capturing the morphology of the entire cytoplasm, cytoskeleton distribution, and mesoscopic structures such as intracellular granules. The third branch uses a 3x3 dilated convolution with a dilation rate of 5, further extending its receptive field to approximately 150 micrometers, sufficient to cover the cell body and several adjacent cells, thereby encoding cell density and interaction information of the local microenvironment. Each branch outputs a feature map with the same number of channels, which are then concatenated along the channel dimension to form a multi-scale feature map that integrates micro, meso, and macro information. This feature map is then compressed into a spatially independent multi-scale fused feature vector with a dimension of 1,536 through a global average pooling layer.
[0025] Next, step S6 is executed: the multi-scale fused feature vectors of the same cell at different time points are arranged in chronological order to form a cell state temporal feature matrix. Assuming a specific cell is observed at T consecutive time points, its corresponding T multi-scale fused feature vectors are stacked into a T-row, 1536-column matrix. Each row of this matrix represents the comprehensive state representation of the cell at a certain moment, and the order of the rows strictly follows chronological order.
[0026] Then, step S7 is executed: the cell state temporal feature matrix is input into a bidirectional long short-term memory network with a time-gated mechanism. This network consists of a forward LSTM layer and a backward LSTM layer. The forward LSTM layer processes the input sequence in chronological order, and its hidden states... The hidden states encode the historical state evolution information from the initial time to time t. The backward LSTM layer processes the input sequence in reverse chronological order, and its hidden states... The code encodes the impact of the future state, backtracking from the final time step to time t, on the current state. At each time step t, the hidden states in both directions are concatenated. This serves as a bidirectional context representation at that moment. A time-gating mechanism is embedded within the LSTM unit to dynamically adjust the information flow. Let the input at time t be... (i.e., the multi-scale fusion feature vector in row t), the cell state at the previous time step is The hidden state at the previous moment was Forgotten Gate The calculation formula is , Input gate The calculation formula is , Candidate cell state The calculation formula is , Current cell state The calculation formula is .
[0027] in Here, is the sigmoid function, W and b are the learnable weight matrix and bias term, and ⊙ denotes element-wise multiplication. This gating mechanism can adaptively determine how much historical information to retain and how much new information to introduce based on the signal-to-noise ratio of the current input features, thereby effectively suppressing noise interference and strengthening key state transition signals.
[0028] The S8 step is then executed: the final hidden state vector output by the bidirectional long short-term memory network is input into a dynamic channel attention decoder. The goal of this decoder is to extract the most discriminative information from the high-dimensional feature vector.
[0029] Its workflow consists of three steps: The first step is to perform global average pooling on the input vector to obtain a channel descriptor, where each element of the descriptor represents the global response intensity of the corresponding feature channel. The second step involves feeding the channel descriptor into a bottleneck structure containing two fully connected layers. The first fully connected layer compresses the number of channels to one-sixteenth of the original number (i.e., sixty-four) for dimensionality reduction and information exchange; the second fully connected layer restores the number of channels to the original one thousand and twenty-four. A modified linear unit activation function is used between the two fully connected layers. The third step involves normalizing the output of the second fully connected layer using the Sigmoid function to obtain a weight vector between zero and one. Each element of this vector represents the importance weight of the corresponding feature channel. Finally, the original input vector is multiplied channel-by-channel by this weight vector to obtain a refined state representation vector. This vector focuses on the feature channels that contribute the most to the current cell state discrimination.
[0030] Finally, step S9 is executed: Based on the refined state representation vector, the final state category label of the cell is output through a fully connected classification layer. This fully connected classification layer contains a weight matrix with an input dimension of 1024 and an output dimension of 7, corresponding to seven predefined cell state categories: normal proliferation, early apoptosis, late apoptosis, necrosis, mitosis, stress response, and resting state. The output of the classification layer is processed by a softmax function to generate a seven-dimensional probability distribution vector. The category with the highest probability value is determined as the final state label of the cell.
[0031] Furthermore, the method includes an uncertainty quantification module. This module is activated during the inference phase. Specifically, Dropout is enabled at a rate of 0.5 in all network layers before the dynamic channel attention decoder. Twenty independent forward propagations are performed on the temporal feature matrix of the same cell, each yielding a seven-dimensional probability distribution. The variance of these twenty probability distributions for each category is calculated. If the variance of any category exceeds a preset threshold of 0.05, the cell's state determination is deemed unreliable, it is marked as "state uncertain," and a review task is generated and pushed to the manual review workstation.
[0032] Throughout the methodology, each step is seamlessly integrated via a rigorous data interface. Steps S1 and S2 ensure the quality and consistency of the input data; Step S3 provides high-quality single-cell instance segmentation results, forming the foundation for all subsequent single-cell analyses; Steps S4 and S5 map the pixel space to a high-dimensional semantic feature space; Steps S6 and S7 utilize a temporal model to uncover the dynamic evolution of cell states; and Steps S8 and S9 transform complex high-dimensional features into explicit semantic labels through attention mechanisms and classifiers. An uncertainty quantification module serves as the final line of defense for quality control, effectively guaranteeing the reliability of the analysis results. This method forms a complete and self-consistent analytical loop, enabling fully automated parsing from raw images to cell state semantics without any manual intervention.
[0033] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.
Claims
1. An image recognition-based automatic analysis method of a cell state, characterized by, include: A multi-channel microscopic image sequence of a cell culture sample to be analyzed is obtained. The multi-channel microscopic image sequence includes at least one bright-field channel image and at least one specific fluorescently labeled channel image. The channel images are strictly aligned in space and have a uniform timestamp identifier. The multi-channel microscopic image sequence is subjected to a normalization preprocessing, which includes illumination correction, background subtraction, adaptive contrast enhancement, and pixel size normalization. The preprocessed multi-channel microscopic image sequence is input into a cascaded cell instance segmentation network. The cascaded cell instance segmentation network consists of a shared low-level feature extractor, a foreground-background coarse segmentation head, and an instance boundary refinement head. The shared low-level feature extractor adopts an encoder-decoder structure. The encoder part is composed of multiple stacked convolutional blocks. Each convolutional block contains a convolutional layer, a batch normalization layer, and an activation function layer. The decoder part fuses feature maps from different levels of the encoder through skip connections and performs upsampling through transposed convolution. The foreground-background coarse segmentation head generates a probability heatmap of the cell region based on the top-level feature map of the decoder. The instance boundary refinement head uses the coarse segmentation result as an attention mask to weight the high-resolution feature maps of the intermediate layers of the encoder to generate cell instance boundary contours accurate to the sub-pixel level. Based on the cell instance mask output by the cascaded cell instance segmentation network, multi-channel region of interest image blocks for each individual cell are cropped from the original multi-channel image; Each cell's multi-channel region of interest image patch is input into a multi-scale feature pyramid encoder. The multi-scale feature pyramid encoder extracts the cell's feature responses at three scales: local detail, mesostructure, and global morphology through parallel dilated convolution branches with different receptive fields. The feature responses at each scale are then concatenated along the channel dimension to form a multi-scale fused feature vector. Arrange the multi-scale fused feature vectors of the same cell at different time points in chronological order to form a cell state temporal feature matrix. The cell state temporal feature matrix is input into a bidirectional long short-term memory network with a time-gated mechanism. The forward pass of the bidirectional long short-term memory network captures the historical evolution trend of the cell state, and the backward pass captures the potential impact of the future state on the current moment. The time-gated mechanism dynamically adjusts the retention and forgetting ratio of the information flow according to the signal-to-noise ratio of the current cell features. The final hidden state vector output by the bidirectional long short-term memory network is input into a dynamic channel attention decoder. The dynamic channel attention decoder calculates the importance weight of each feature channel and performs a weighted summation of the feature vectors to generate a refined state representation vector that focuses on the most discriminative feature. Based on the refined state representation vector, the final state category label of the cell is output through a fully connected classification layer. The state category label includes normal proliferation, early apoptosis, late apoptosis, necrosis, mitosis, stress response, and resting state.
2. The image recognition-based automatic analysis method of a cell state according to claim 1, characterized by, The multi-channel microscopic image sequence undergoes normalization preprocessing, including: An illumination correction was performed on the original image using a non-uniform illumination compensation algorithm based on polynomial surface fitting. Background subtraction is performed using morphological top-hat transformation to remove slowly changing background fluorescence noise; An adaptive contrast enhancement algorithm is applied to limit the contrast adaptive histogram equalization algorithm. Based on the objective magnification information in the metadata, all images are resampled to a uniform spatial scale, and pixel size normalization is achieved using a bicubic interpolation algorithm.
3. The image recognition-based automatic analysis method of a cell state according to claim 2, characterized by, The training process of the cascaded cell instance segmentation network adopts a two-stage joint optimization strategy: The first stage uses the cross-entropy loss function to supervise the training of the foreground-background coarse segmentation head; In the second stage, after freezing some weights of the shared underlying feature extractor, the instance boundary refinement head is fine-tuned using the weighted sum of the boundary intersection-union loss function and the Hausdorff distance loss function as the optimization objective.
4. The image recognition-based automatic analysis method of a cell state according to claim 3, characterized by, Each cell's multi-channel region of interest image patch is input into a multi-scale feature pyramid encoder, including: The input image patch is passed through a shared initial convolutional layer to generate a base feature map; The basic feature map is simultaneously fed into three parallel dilated convolution branches; The feature maps output from the three branches are concatenated along the channel dimension and then compressed into a multi-scale fused feature vector through a global average pooling layer.
5. The image recognition-based automatic analysis method of a cell state according to claim 4, characterized by, The cell state temporal feature matrix is input into a bidirectional long short-term memory network with a time-gated mechanism, including: Let the input at time t be The cell state at the previous moment was The hidden state at the previous moment was Forgotten Gate The calculation formula is , Input gate The calculation formula is , Candidate cell states The formula for calculating , Current cell state The calculation formula is , in Here, is the sigmoid function, W and b are the learnable weight matrix and bias term, and ⊙ represents element-wise multiplication.
6. The image recognition-based automatic analysis method of a cell state according to claim 5, characterized by, The final hidden state vector output by the bidirectional long short-term memory network is input into a dynamic channel attention decoder, including: Global average pooling is performed on the input feature vector to obtain a channel descriptor; The channel descriptor is passed through a bottleneck structure containing two fully connected layers, where the first fully connected layer compresses the number of channels to one-sixteenth of the original number of channels, and the second fully connected layer restores it to the original number of channels; The output value is normalized to between zero and one using the Sigmoid function, and used as the attention weight for each channel. The original feature vector is then weighted channel by channel.
7. The image recognition-based automatic analysis method of a cell state according to claim 6, characterized by, Based on the refined state representation vector, the final state category label of the cell is output through a fully connected classification layer, including: The refined state representation vector is input into a fully connected layer with an output dimension of seven, corresponding to seven categories: normal proliferation, early apoptosis, late apoptosis, necrosis, mitosis, stress response, and resting state. The Softmax function is applied to the output of the fully connected layer to generate a seven-dimensional probability distribution, and the one with the highest probability is taken as the final state category label.
8. The image recognition-based automatic analysis method of a cell state according to claim 7, characterized by, The method also includes an uncertainty quantification module, which is used to perform multiple forward propagations on the same input during the inference phase, calculate the variance of the probability distribution of the output state category, and if the variance exceeds a preset threshold, the cell is marked as "state uncertain" and a manual review process is triggered.
9. An image recognition-based automatic analysis system for cell state, characterized by, include: A multi-channel microscopic image sequence acquisition module is used to acquire a multi-channel microscopic image sequence of a cell culture sample to be analyzed. The multi-channel microscopic image sequence includes at least one bright-field channel image and at least one specific fluorescent label channel image. The channel images are strictly aligned in space and have a uniform timestamp identifier. The standardization preprocessing module is used to perform standardization preprocessing on the multi-channel microscopic image sequence. The standardization preprocessing includes illumination correction, background subtraction, contrast adaptive enhancement, and pixel size normalization. A cascaded cell instance segmentation module is used to input preprocessed multi-channel microscopic image sequences into a cascaded cell instance segmentation network. The cascaded cell instance segmentation network consists of a shared low-level feature extractor, a foreground-background coarse segmentation head, and an instance boundary refinement head. The shared low-level feature extractor adopts an encoder-decoder structure. The encoder part is composed of multiple stacked convolutional blocks. Each convolutional block contains a convolutional layer, a batch normalization layer, and an activation function layer. The decoder part fuses feature maps from different levels of the encoder through skip connections and performs upsampling through transposed convolution. The foreground-background coarse segmentation head generates a probability heatmap of cell regions based on the top-level feature map of the decoder. The instance boundary refinement head uses the coarse segmentation result as an attention mask to weight the high-resolution feature maps of the intermediate layers of the encoder to generate cell instance boundary contours accurate to the sub-pixel level. The multi-channel region of interest (ROI) cropping module is used to crop out the multi-channel ROI image block of each individual cell from the original multi-channel image based on the cell instance mask output by the cascaded cell instance segmentation network. The multi-scale feature pyramid encoding module is used to input the multi-channel region of interest image blocks of each cell into a multi-scale feature pyramid encoder. The multi-scale feature pyramid encoder extracts the feature responses of the cell at three scales: local details, mesostructure and global morphology through parallel dilated convolution branches of different receptive fields, and splices the feature responses of each scale along the channel dimension to form a multi-scale fused feature vector. The cell state temporal feature matrix construction module is used to arrange the multi-scale fused feature vectors of the same cell at different time points in chronological order to form a cell state temporal feature matrix. A bidirectional long short-term memory network processing module is used to input the cell state temporal feature matrix into a bidirectional long short-term memory network with a time-gated mechanism. The forward pass of the bidirectional long short-term memory network captures the historical evolution trend of the cell state, and the backward pass captures the potential impact of the future state on the current moment. The time-gated mechanism dynamically adjusts the retention and forgetting ratio of the information flow according to the signal-to-noise ratio of the current cell features. The dynamic channel attention decoding module is used to input the final hidden state vector output by the bidirectional long short-term memory network into a dynamic channel attention decoder. The dynamic channel attention decoder calculates the importance weight of each feature channel and performs a weighted summation of the feature vectors to generate a refined state representation vector that focuses on the most discriminative feature. The cell state classification output module is used to output the final state category label of the cell through a fully connected classification layer based on the refined state representation vector. The state category label includes normal proliferation, early apoptosis, late apoptosis, necrosis, mitosis, stress response, and resting state.
10. The image recognition-based automatic analysis system of a cellular state according to claim 9, characterized by, The multi-scale feature pyramid encoding module is used for: The input image patch is passed through a shared initial convolutional layer to generate a base feature map; The basic feature map is simultaneously fed into three parallel dilated convolution branches, wherein the first branch uses a standard 3x3 convolution with a dilation rate of 1, the second branch uses a 3x3 dilated convolution with a dilation rate of 3, and the third branch uses a 3x3 dilated convolution with a dilation rate of 5. The feature maps output from the three branches are concatenated along the channel dimension and then compressed into a multi-scale fused feature vector through a global average pooling layer.
