Label-free cell activity detection method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of cell activity detection, and discloses an unmarked cell activity detection method and device, computer equipment and a storage medium, and the method comprises the following steps: obtaining a bright field cell image to be detected; and performing cell activity detection based on the bright field cell image to be detected and a trained deep learning model to obtain a classification result of the bright field cell image to be detected, the deep learning model comprises a first convolution module, a second convolution module, a third convolution module, a fourth convolution module, a first up-sampling layer, a second up-sampling layer, a third up-sampling layer, a first splicing layer, a second splicing layer, a third splicing layer, a first coding layer, a second coding layer and a third coding layer. The first Jamba model, the second Jamba model, the third Jamba model, the first detection head, the second detection head and the third detection head can analyze a bright field cell image to be detected through the trained deep learning model, so that the activity of each cell in the image can be detected.

Description

Technical Field

[0001] The present invention relates to the technical field of cell activity detection, and in particular to a label-free cell activity detection method, device, computer equipment and storage medium. Background Art

[0002] Digital microfluidics (DMF) based on EWoD technology is a technology that uses electric fields, pressure or other physical methods to manipulate droplets at the micrometer scale. This technology has great potential in the biomedical field because it can efficiently and accurately control the movement, segmentation, fusion and mixing of trace liquids. Compared with traditional continuous microfluidics, digital microfluidics technology is more flexible in droplet manipulation and can achieve high-throughput analysis. Therefore, it is widely used in biological sample processing, chemical synthesis, drug screening, DNA sequencing and other fields.

[0003] In biological experiments, accurate detection of cell activity is one of the key factors for the success of the experiment, especially in single-cell research, where the activity and state of each cell will directly affect the results and conclusions of the experiment. At present, the commonly used method for detecting cell activity in digital microfluidics systems is fluorescence detection, which has the problems of changing the biological behavior of cells, increasing the complexity and operating costs of the experiment, and requiring a highly sensitive fluorescence microscope. Summary of the invention

[0004] Based on this, it is necessary to address the technical problem that the detection effect of cell activity in the prior art is poor, and propose a label-free cell activity detection method, device, computer equipment and storage medium.

[0005] In a first aspect, a label-free cell activity detection method is provided, the method comprising:

[0006] Acquire bright field images of cells to be detected;

[0007] Cell activity detection is performed based on the bright field cell image to be detected and the trained deep learning model to obtain a classification result of the bright field cell image to be detected, wherein the deep learning model includes a first convolution module, a second convolution module, a third convolution module, a fourth convolution module, a first upsampling layer, a second upsampling layer, a third upsampling layer, a first splicing layer, a second splicing layer, a third splicing layer, a first encoding layer, a second encoding layer, a third encoding layer, a first Jamba model, a second Jamba model, a third Jamba model, a first detection head, a second detection head, and a third detection head.

[0008] In a second aspect, a label-free cell activity detection device is provided, the device comprising:

[0009] An acquisition module, used for acquiring a bright field cell image to be detected;

[0010] A classification module is used to perform cell activity detection based on the bright field cell image to be detected and the trained deep learning model to obtain a classification result of the bright field cell image to be detected, wherein the deep learning model includes a first convolution module, a second convolution module, a third convolution module, a fourth convolution module, a first upsampling layer, a second upsampling layer, a third upsampling layer, a first splicing layer, a second splicing layer, a third splicing layer, a first encoding layer, a second encoding layer, a third encoding layer, a first Jamba model, a second Jamba model, a third Jamba model, a first detection head, a second detection head, and a third detection head.

[0011] In a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned label-free cell activity detection method when executing the computer program.

[0012] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned label-free cell activity detection method are implemented.

[0013] The label-free cell activity detection method proposed in the present invention obtains a bright field cell image to be detected, and then performs cell activity detection based on the bright field cell image to be detected and a trained deep learning model to obtain a classification result of the bright field cell image to be detected, wherein the deep learning model includes a first convolution module, a second convolution module, a third convolution module, a fourth convolution module, a first upsampling layer, a second upsampling layer, a third upsampling layer, a first splicing layer, a second splicing layer, a third splicing layer, a first encoding layer, a second encoding layer, a third encoding layer, a first Jamba model, a second Jamba model, a third Jamba model, a first detection head, a second detection head, and a third detection head. The bright field cell image to be detected can be analyzed by the trained deep learning model, so that the activity of each cell in the image can be detected. The data collection is simple and fast, and the analysis efficiency is high. It can not only reduce the impact of the experimental environment and significantly improve the experimental speed, but also greatly reduce the experimental cost, eliminate the potential impact of fluorescent dyes on cells, and ensure the physiological integrity of cells. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0015] in:

[0016] Figure 1 A diagram showing an application environment of a label-free cell activity detection method in an embodiment;

[0017] Figure 2 A flowchart of a label-free cell activity detection method in one embodiment;

[0018] Figure 3 Schematic diagram of the convolution module structure of a label-free cell activity detection method in one embodiment;

[0019] Figure 4 Schematic diagram of the coding layer structure of a label-free cell activity detection method in one embodiment;

[0020] Figure 5 A schematic diagram of the Jamba model structure of a label-free cell activity detection method in one embodiment;

[0021] Figure 6 A schematic diagram of the structure of a detection head of a label-free cell activity detection method in one embodiment;

[0022] Figure 7 A schematic diagram of a deep learning model structure of a label-free cell activity detection method in one embodiment;

[0023] Figure 8 is a structural block diagram of a label-free cell activity detection device in one embodiment;

[0024] Fig. 9 is a structural block diagram of a computer device in one embodiment;

[0025] Fig.10 It is a structural block diagram of a computer device in another embodiment. DETAILED DESCRIPTION

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0027] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0028] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] The label-free cell activity detection method provided by the embodiment of the present invention can be applied in Figure 1In the application environment, the client 110 communicates with the server 120 through the network. The server 120 can receive the bright field cell image to be detected through the client 110, and then the server 120 performs cell activity detection based on the bright field cell image to be detected and the trained deep learning model to obtain the classification result of the bright field cell image to be detected, wherein the deep learning model includes a first convolution module, a second convolution module, a third convolution module, a fourth convolution module, a first upsampling layer, a second upsampling layer, a third upsampling layer, a first splicing layer, a second splicing layer, a third splicing layer, a first encoding layer, a second encoding layer, a third encoding layer, a first Jamba model, a second Jamba model, a third Jamba model, a first detection head, a second detection head, and a third detection head. The present invention can analyze the bright field cell image to be detected through the trained deep learning model, and the activity of each cell in the image can be detected. The data collection is simple and fast, and the analysis efficiency is high. It can not only reduce the impact of the experimental environment and significantly improve the experimental speed, but also greatly reduce the experimental cost, eliminate the potential impact of fluorescent dyes on cells, and ensure the physiological integrity of cells. The client 110 may be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, and portable wearable devices. The server 120 may be implemented by an independent server or a server cluster consisting of multiple servers. The present invention is described in detail below through specific embodiments.

[0030] See also Figure 2 As shown, Figure 2 A schematic flow chart of a label-free cell activity detection method provided in one embodiment of the present invention comprises the following steps:

[0031] Step S101: Acquire a bright field image of cells to be detected;

[0032] The bright field cell image to be detected may be a bright field image taken under an ordinary microscope.

[0033] Step S102: Perform cell activity detection based on the bright field cell image to be detected and the trained deep learning model to obtain a classification result of the bright field cell image to be detected, wherein the deep learning model includes a first convolution module, a second convolution module, a third convolution module, a fourth convolution module, a first upsampling layer, a second upsampling layer, a third upsampling layer, a first splicing layer, a second splicing layer, a third splicing layer, a first encoding layer, a second encoding layer, a third encoding layer, a first Jamba model, a second Jamba model, a third Jamba model, a first detection head, a second detection head, and a third detection head.

[0034] Among them, reference Figure 3The model structures of the first convolution module, the second convolution module, the third convolution module, and the fourth convolution module can be the same. Specifically, the convolution module includes a depthwise separable convolution layer, a batch normalization layer, and a SILU activation layer connected in sequence, and the input end of the depthwise separable convolution layer and the output end of the SILU activation layer use a skip connection (SkipConnection), wherein the depthwise separable convolution layer (Depthwise Convolution) is composed of a depthwise convolution (DepthwiseConvolution) and a pointwise convolution (Pointwise Convolution), the depthwise convolution is used to extract spatial features, and the pointwise convolution is used to extract channel features. Using depthwise separable convolution can significantly reduce the number of parameters, reduce the amount of calculation, and improve the ability to express features.

[0035] Batch Normalization layer: It mainly normalizes the output of each layer, so that the input of the next layer is more stable, which helps to reduce the risk of overfitting and improve the generalization ability of the model.

[0036] SILU activation layer: Using SILU activation function as a separate layer, it can adaptively scale the input value, which helps to reduce overfitting. Skip connection: Connect the input directly to the output. This connection method can alleviate the gradient vanishing problem and help the network learn better.

[0037] The model structures of the first upsampling layer, the second upsampling layer, and the third upsampling layer can be the same. Upsample layer: Use bilinear interpolation to increase the size of the feature map, restore the spatial position information of the target, and improve the detection accuracy of cells (small targets). The model structures of the first concatenation layer, the second concatenation layer, and the third concatenation layer can be the same. Concatenation layer: Connect two tensors together in the channel dimension to better capture the relationship between different features in subsequent layers.

[0038] refer to Figure 4 , the model structures of the first coding layer, the second coding layer, and the third coding layer can be the same. Specifically, the coding layer can include sequentially connected image blocks and position coding layers. Image block (Patch Embedding): Divide the image into image blocks of fixed size and flatten them into a one-dimensional vector to adapt to the input of the next layer of network. Positional encoding layer (Positional Encoding): Add positional encoding to each image block to retain the position information of the image block, so that the network can understand the relative position relationship between image blocks.

[0039] refer to Figure 5, the model structures of the first Jamba model (Jamba Module), the second Jamba model, and the third Jamba model can be the same. The Jamba Module includes a Mamba layer, a Mamba MoE layer, a Transformer layer, a Mamba MoE layer, a Mamba layer, and a Mamba MoE layer connected in sequence. The input of each Mamba MoE layer also includes the number of target categories. Transformer layer: This is the layer used in the traditional Transformer model. It is mainly responsible for processing the self-attention mechanism of the input data, allowing the model to consider all positions in the sequence when generating the output. Mamba layer: The Mamba layer is a state-space model. Compared with the Transformer layer, it can be trained in parallel and is more efficient in processing long sequence data. At the same time, it shows superior performance in capturing long-distance dependencies. Mamba MoE layer: A mixed expert layer, which processes specific parts or tasks of the input data through multiple "expert" networks, and then determines the degree or probability of each expert being activated through a gating selection mechanism, thereby improving the accuracy of the model in recognizing cells. Number of target categories: Encode the number of cell categories that need to be predicted, put it into the network as a sequence for training, and finally predict the category of each target and the position of the detection box.

[0040] refer to Figure 6 The model structures of the first detection head, the second detection head, and the third detection head can be the same. The detection head consists of two feedforward neural networks, one of which uses the multi-scale output for target category prediction, and the other feedforward neural network uses the multi-scale output for target bounding box prediction.

[0041] Specifically, refer to Figure 7 , input the bright field cell image to be detected into the first convolution module to obtain a first feature; input the first feature into the second convolution module to obtain a second feature; input the second feature into the third convolution module to obtain a third feature; input the third feature into the fourth convolution module to obtain a fourth feature;

[0042] The fourth feature is input into the third upsampling layer to obtain a fifth feature, and the third feature and the fifth feature are input into the third concatenation layer to obtain a sixth feature;

[0043] The sixth feature is input into the second upsampling layer to obtain a seventh feature, and the second feature and the seventh feature are input into the second concatenation layer to obtain an eighth feature;

[0044] The eighth feature is input into the first upsampling layer to obtain a ninth feature, and the first feature and the ninth feature are input into the first concatenation layer to obtain a tenth feature;

[0045] Inputting the tenth feature into the first encoding layer, the first Jamba model, and the first output head connected in sequence to obtain a first target category prediction and a first target bounding box prediction;

[0046] Inputting the eighth feature into the second encoding layer, the second Jamba model and the second output head connected in sequence to obtain a second target category prediction and a second target bounding box prediction;

[0047] Inputting the sixth feature into the third encoding layer, the third Jamba model and the third output head connected in sequence to obtain a third target category prediction and a third target bounding box prediction;

[0048] The first target category prediction, the first target bounding box prediction, the second target category prediction, the second target bounding box prediction, the third target category prediction, and the third target bounding box prediction are processed by using a bilinear interpolation method to obtain a target category prediction and a target bounding box prediction.

[0049] In one embodiment, the step of performing cell activity detection based on the bright field cell image to be detected and the trained deep learning model to obtain the classification result of the bright field cell image to be detected includes:

[0050] Step S201: Acquire a bright field cell image set and a fluorescent field cell image set;

[0051] In one embodiment, the bright field cell image and the fluorescent field cell image are obtained by photographing a cell sample using a fluorescent microscope, and the cell sample is washed with PBS buffer and centrifuged, and then a cell activity detection reagent is added and incubated at room temperature in the dark.

[0052] As an example, first, prepare the cell sample: prepare the cell suspension, wash with PBS buffer and centrifuge, then add Calcein / PI cell activity detection reagent at room temperature and incubate in the dark to fully stain it; then, collect cell images: use a fluorescence microscope to shoot the cell sample to obtain a large number of high-resolution bright field cell images and fluorescent field cell images. These images contain physical information such as cell morphology, size, transparency, etc., which can reflect the health and activity of the cells. Living cells usually have clear boundaries and uniform morphology, while apoptotic or dead cells may appear deformed or have blurred outlines.

[0053] As another example, 1. Prepare a CHO-S cell suspension, wash with phosphate buffer and centrifuge, add the Calcein / PI cell activity detection kit of Bio-Technology Co., Ltd. and resuspend, and incubate at room temperature in the dark for 30 minutes. 2. Inject the incubated CHO-S cell suspension into the digital microfluidic chip. 3. Use a Nikon Ni-U fluorescence microscope to photograph the cell sample, and record the bright field cell image and the fluorescence field cell image respectively.

[0054] Step S202: preprocessing the bright field cell image set;

[0055] In one embodiment, the preprocessing includes image scaling, noise reduction, normalization and image enhancement. Specifically, image preprocessing: in order to improve the quality of the data, the collected bright field cell images are preprocessed to make them more suitable for the training of the deep learning model. Commonly used image preprocessing methods include the following: ① Image scaling: change the size of the image by nearest neighbor interpolation, bilinear interpolation, bicubic interpolation and other methods, and adjust the image size to adapt to the input of the deep learning model. ② Noise reduction: remove noise from the image by Gaussian Blur, mean filtering, median filtering and other methods to improve the clarity of the image. ③ Normalization: adjust the pixel value of the image to a certain range (such as [0, 1] or [-1, 1]) through normalization, Z-Score Normalization and other methods to reduce the impact of brightness differences between different images on the model. ④ Image enhancement: Increase the quantity and diversity of training data through methods such as image rotation, image cropping, and image flipping to improve the performance and robustness of deep learning models.

[0056] As another example, for the preprocessed bright field cell image, Gaussian filtering (GaussianBlur) is first performed to improve the clarity of the image, and then the pixel value of the image is adjusted to [0, 1] through normalization (Normalization) to reduce the impact of brightness differences between different images on the model, and finally the amount and diversity of data are increased through image rotation, image cropping and image flipping.

[0057] Step S203: Based on the fluorescent field cell image as a reference, the background, living cells, and dead cells in each image in the pre-processed bright field cell image set are marked to obtain a target bright field cell image set;

[0058] In this embodiment, the collected fluorescent field cell image is used as a reference standard, and the LabelImg annotation tool can be used to manually or semi-automatically annotate the pre-processed bright field cell image, and each pixel in the pre-processed bright field cell image can be divided into three categories: background, living cells, and dead cells.

[0059] Step S204: Perform model training based on the target bright field cell image set and the initial deep learning model to obtain a trained deep learning model, wherein the trained deep learning model is used to classify each pixel in the bright field cell image to be detected, and output the classification result of the bright field cell image to be detected, wherein the deep learning model includes a first convolution module, a second convolution module, a third convolution module, a fourth convolution module, a first upsampling layer, a second upsampling layer, a third upsampling layer, a first splicing layer, a second splicing layer, a third splicing layer, a first encoding layer, a second encoding layer, a third encoding layer, a first Jamba model, a second Jamba model, a third Jamba model, a first detection head, a second detection head, and a third detection head.

[0060] Among them, the deep learning model can adopt a convolutional neural network, and the bright field cell image to be detected can be a bright field image taken under an ordinary microscope.

[0061] In this embodiment, in order to quickly distinguish the activity of each cell from the bright field cell image, a single-stage initial deep learning model is used for iterative training. To ensure the accuracy and robustness of the model, the following methods are usually used to improve the performance of the model: ① Use a loss function that measures the difference between the model prediction result and the actual result, usually the following: Mean Squared Error Loss Function (MSE), Cross-Entropy Loss Function (CE), Logarithmic Loss Function (Logarithmic Loss), Categorical Cross-Entropy Loss Function (CCES), KL Divergence Loss Function (Kullback–Leibler Divergence, KLD), etc. ② Use optimization algorithms that continuously adjust their internal parameters to minimize the loss function, usually the following: Gradient Descent (GD), Stochastic Gradient Descent (SGD), Mini-batch Gradient Descent, Momentum, Adaptive Learning Rate Algorithm (Adam), etc. ③ Use regularization techniques to prevent the model from overfitting on the training data, usually the following: L1 regularization, L2 regularization, Dropout regularization, Elastic Net regularization, etc.

[0062] Specifically, the overall error function can be used to train the initial deep learning model, wherein the output of the initial deep learning model includes the target category and the target bounding box. The error function of the target category adopts the multi-classification cross entropy function (Categorical Cross-Entropy, CCES), and the error function of the target bounding box adopts the weighted function of the mean absolute error (L1 Loss) and the intersection over union error (GIoU Loss). The overall error function is as follows: the error function of the target bounding box = 0.4 × mean absolute error + 0.6 × intersection over union error; the error function of the detection head = 0.7 × the error function of the target category + 0.3 × the error function of the target bounding box; the overall error function = 0.6 × the error function of the first detection head + 0.3 × the error function of the second detection head + 0.1 ×

[0063] Error function of the third detection head.

[0064] In one embodiment, the images in the target bright field cell image set are divided into a training set, a validation set, and a test set in a ratio of 7:1:1, wherein the number of cell samples in the training set is 21060, the number of cell samples in the validation set is 3122, and the number of cell samples in the test set is 3234.

[0065] In one embodiment, model training is performed based on a target bright field cell image set and an initial deep learning model, and the parameters of the initial deep learning model are optimized based on a cross-entropy loss function (CE), a stochastic gradient descent (SGD), and Dropout regularization; when the number of model training times of the initial deep learning model meets a preset number of times, the initial deep learning model is used as a trained deep learning model.

[0066] As an example, after obtaining a trained deep learning model, the bright field cell image to be detected can be detected. Specifically, the bright field cell image collected under an ordinary optical microscope is used as the bright field cell image to be detected, and the bright field cell image to be detected is input into the deployed deep learning model for feature extraction. The extracted features include morphological features (such as cell shape and size), optical features (such as brightness and transparency), and texture features (such as the surface structure of the cell). These features are automatically inferred and detected in the deep learning model, and each pixel of the input bright field cell image is classified, and the detection result is output.

[0067] As an example, the trained deep learning model is exported to ONNX format and deployed to the digital microfluidics system as a C++ library. Bright field cell images in actual working scenes are collected as input to the model. The activity of each cell can be obtained without adding fluorescent staining reagents.

[0068] See also Figure 8 As shown, in one embodiment, a label-free cell activity detection device is provided, the device comprising:

[0069] An acquisition module 10 is used to acquire a bright field cell image to be detected;

[0070] The prediction module 20 is used to perform cell activity detection based on the bright field cell image to be detected and the trained deep learning model to obtain the classification result of the bright field cell image to be detected, wherein the deep learning model includes a first convolution module, a second convolution module, a third convolution module, a fourth convolution module, a first upsampling layer, a second upsampling layer, a third upsampling layer, a first splicing layer, a second splicing layer, a third splicing layer, a first encoding layer, a second encoding layer, a third encoding layer, a first Jamba model, a second Jamba model, a third Jamba model, a first detection head, a second detection head, and a third detection head.

[0071] In one embodiment, the prediction module 20 is used to:

[0072] Input the bright field cell image to be detected into the first convolution module to obtain a first feature; input the first feature into the second convolution module to obtain a second feature; input the second feature into the third convolution module to obtain a third feature; input the third feature into the fourth convolution module to obtain a fourth feature;

[0073] The fourth feature is input into the third upsampling layer to obtain a fifth feature, and the third feature and the fifth feature are input into the third concatenation layer to obtain a sixth feature;

[0074] The sixth feature is input into the second upsampling layer to obtain a seventh feature, and the second feature and the seventh feature are input into the second concatenation layer to obtain an eighth feature;

[0075] The eighth feature is input into the first upsampling layer to obtain a ninth feature, and the first feature and the ninth feature are input into the first concatenation layer to obtain a tenth feature;

[0076] Inputting the tenth feature into the first encoding layer, the first Jamba model, and the first output head connected in sequence to obtain a first target category prediction and a first target bounding box prediction;

[0077] Inputting the eighth feature into the second encoding layer, the second Jamba model and the second output head connected in sequence to obtain a second target category prediction and a second target bounding box prediction;

[0078] Inputting the sixth feature into the third encoding layer, the third Jamba model and the third output head connected in sequence to obtain a third target category prediction and a third target bounding box prediction;

[0079] The first target category prediction, the first target bounding box prediction, the second target category prediction, the second target bounding box prediction, the third target category prediction, and the third target bounding box prediction are processed by using a bilinear interpolation method to obtain a target category prediction and a target bounding box prediction.

[0080] In one embodiment, the label-free cell activity detection device is used for:

[0081] Acquire bright field cell image sets and fluorescent field cell images;

[0082] Preprocessing the bright field cell image set;

[0083] Based on the fluorescent field cell image as a reference, the background, living cells, and dead cells in each image in the preprocessed bright field cell image set are marked to obtain a target bright field cell image set;

[0084] Model training is performed based on the target bright field cell image set and the initial deep learning model to obtain a trained deep learning model, wherein the trained deep learning model is used to classify each pixel in the bright field cell image to be detected, and output the target category prediction and the target bounding box prediction of the bright field cell image to be detected.

[0085] In one embodiment, the label-free cell activity detection device is used for:

[0086] The preprocessing includes image scaling, noise reduction, normalization and image enhancement.

[0087] In one embodiment, the label-free cell activity detection device is used for:

[0088] The bright field cell image and the fluorescent field cell image are obtained by photographing the cell sample using a fluorescent microscope. The cell sample is washed with PBS buffer and centrifuged, and then a cell activity detection reagent is added and incubated in the dark at room temperature.

[0089] In one embodiment, the label-free cell activity detection device is used for:

[0090] Model training is performed based on the target bright field cell image set and the initial deep learning model, and the parameters of the initial deep learning model are optimized based on the cross entropy loss function, stochastic gradient descent method and regularization;

[0091] When the model training times of the initial deep learning model meet the preset times, the initial deep learning model is used as the trained deep learning model.

[0092] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig. 9 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client via a network connection. When the computer program is executed by the processor, the functions or steps on the service side of a label-free cell activity detection method are implemented.

[0093] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Fig.10 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, the functions or steps on the client side of a label-free cell activity detection method are implemented.

[0094] In one embodiment, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are implemented:

[0095] Acquire bright field images of cells to be detected;

[0096] Cell activity detection is performed based on the bright field cell image to be detected and the trained deep learning model to obtain a classification result of the bright field cell image to be detected, wherein the deep learning model includes a first convolution module, a second convolution module, a third convolution module, a fourth convolution module, a first upsampling layer, a second upsampling layer, a third upsampling layer, a first splicing layer, a second splicing layer, a third splicing layer, a first encoding layer, a second encoding layer, a third encoding layer, a first Jamba model, a second Jamba model, a third Jamba model, a first detection head, a second detection head, and a third detection head.

[0097] The present invention can analyze the bright field cell images to be detected through the trained deep learning model, and can detect the activity of each cell in the image. The data collection is simple and fast, and the analysis efficiency is high. It can not only reduce the impact of the experimental environment and significantly improve the experimental speed, but also greatly reduce the experimental cost, eliminate the potential impact of fluorescent dyes on cells, and ensure the physiological integrity of cells.

[0098] In one embodiment, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0099] Acquire bright field images of cells to be detected;

[0100] Cell activity detection is performed based on the bright field cell image to be detected and the trained deep learning model to obtain a classification result of the bright field cell image to be detected, wherein the deep learning model includes a first convolution module, a second convolution module, a third convolution module, a fourth convolution module, a first upsampling layer, a second upsampling layer, a third upsampling layer, a first splicing layer, a second splicing layer, a third splicing layer, a first encoding layer, a second encoding layer, a third encoding layer, a first Jamba model, a second Jamba model, a third Jamba model, a first detection head, a second detection head, and a third detection head.

[0101] The present invention can analyze the bright field cell images to be detected through the trained deep learning model, and can detect the activity of each cell in the image. The data collection is simple and fast, and the analysis efficiency is high. It can not only reduce the impact of the experimental environment and significantly improve the experimental speed, but also greatly reduce the experimental cost, eliminate the potential impact of fluorescent dyes on cells, and ensure the physiological integrity of cells.

[0102] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions on the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0103] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0104] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0105] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A label-free cell activity detection method, characterized in that: The label-free cell activity detection method comprises: Acquire bright field images of cells to be detected; Cell activity detection is performed based on the bright field cell image to be detected and the trained deep learning model to obtain a classification result of the bright field cell image to be detected, wherein the deep learning model includes a first convolution module, a second convolution module, a third convolution module, a fourth convolution module, a first upsampling layer, a second upsampling layer, a third upsampling layer, a first splicing layer, a second splicing layer, a third splicing layer, a first encoding layer, a second encoding layer, a third encoding layer, a first Jamba model, a second Jamba model, a third Jamba model, a first detection head, a second detection head, and a third detection head.

2. The label-free cell activity detection method according to claim 1, characterized in that: The step of performing cell activity detection based on the bright field cell image to be detected and the trained deep learning model to obtain the classification result of the bright field cell image to be detected includes: Input the bright field cell image to be detected into the first convolution module to obtain a first feature; input the first feature into the second convolution module to obtain a second feature; input the second feature into the third convolution module to obtain a third feature; input the third feature into the fourth convolution module to obtain a fourth feature; The fourth feature is input into the third upsampling layer to obtain a fifth feature, and the third feature and the fifth feature are input into the third concatenation layer to obtain a sixth feature; The sixth feature is input into the second upsampling layer to obtain a seventh feature, and the second feature and the seventh feature are input into the second concatenation layer to obtain an eighth feature; The eighth feature is input into the first upsampling layer to obtain a ninth feature, and the first feature and the ninth feature are input into the first concatenation layer to obtain a tenth feature; Inputting the tenth feature into the first encoding layer, the first Jamba model, and the first output head connected in sequence to obtain a first target category prediction and a first target bounding box prediction; Inputting the eighth feature into the second encoding layer, the second Jamba model and the second output head connected in sequence to obtain a second target category prediction and a second target bounding box prediction; Inputting the sixth feature into the third encoding layer, the third Jamba model and the third output head connected in sequence to obtain a third target category prediction and a third target bounding box prediction; The first target category prediction, the first target bounding box prediction, the second target category prediction, the second target bounding box prediction, the third target category prediction, and the third target bounding box prediction are processed by using a bilinear interpolation method to obtain a target category prediction and a target bounding box prediction.

3. The label-free cell activity detection method according to claim 2, characterized in that: The step of performing cell activity detection based on the bright field cell image to be detected and the trained deep learning model to obtain the classification result of the bright field cell image to be detected includes: Acquire bright field cell image sets and fluorescent field cell images; Preprocessing the bright field cell image set; Based on the fluorescent field cell image as a reference, the background, living cells, and dead cells in each image in the preprocessed bright field cell image set are marked to obtain a target bright field cell image set; Model training is performed based on the target bright field cell image set and the initial deep learning model to obtain a trained deep learning model, wherein the trained deep learning model is used to classify each pixel in the bright field cell image to be detected, and output the target category prediction and the target bounding box prediction of the bright field cell image to be detected.

4. The label-free cell activity detection method according to claim 3, characterized in that: The preprocessing includes image scaling, noise reduction, normalization and image enhancement.

5. The label-free cell activity detection method according to claim 4, characterized in that: The bright field cell image and the fluorescent field cell image are obtained by photographing the cell sample using a fluorescent microscope. The cell sample is washed with PBS buffer and centrifuged, and then a cell activity detection reagent is added and incubated in the dark at room temperature.

6. The label-free cell activity detection method according to claim 5, characterized in that: The steps of performing model training based on the target bright field cell image set and the initial deep learning model to obtain a trained deep learning model include: Model training is performed based on the target bright field cell image set and the initial deep learning model, and the parameters of the initial deep learning model are optimized based on the cross entropy loss function, stochastic gradient descent method and regularization; When the model training times of the initial deep learning model meet the preset times, the initial deep learning model is used as the trained deep learning model.

7. A label-free cell activity detection device, characterized in that: The label-free cell activity detection device comprises: An acquisition module, used for acquiring a bright field cell image to be detected; A prediction module is used to perform cell activity detection based on the bright field cell image to be detected and the trained deep learning model to obtain a classification result of the bright field cell image to be detected, wherein the deep learning model includes a first convolution module, a second convolution module, a third convolution module, a fourth convolution module, a first upsampling layer, a second upsampling layer, a third upsampling layer, a first splicing layer, a second splicing layer, a third splicing layer, a first encoding layer, a second encoding layer, a third encoding layer, a first Jamba model, a second Jamba model, a third Jamba model, a first detection head, a second detection head, and a third detection head.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the label-free cell activity detection method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the label-free cell activity detection method according to any one of claims 1 to 6 are implemented.

Images