Method and device for identifying the activity of lung cancer organoids based on a sequence image model

Abstract

The present invention relates to a method and device for identifying the activity of lung cancer organoids based on a sequence image model, including: acquiring a sequence image of lung cancer organoids; forming a data set by labeling the sequence image of the lung cancer organoids with feature tags indicating the activity state; constructing a lung cancer organoid activity classification model, segmenting the sequence image of the lung cancer organoids in the data set sample into a plurality of image patches, linearly embedding the image patches and adding position encoding, projecting each image patch into a vector sequence of a fixed length as an input sequence, and at the same time adding an identifier to the input sequence and feeding it into an encoder based on a transformer, and taking the output of the identifier in the encoding layer of the model as the feature output of the sequence image to complete the construction of the lung cancer organoid activity recognition model. The present invention realizes the recognition of the activity of organoids under various drug categories, concentrations and culture cycles, and the recognition result is accurate, which helps to speed up the process of organoid cell experiments.

Description

Technical Field

[0001] The present invention relates to the technical field of computer-aided biological sample recognition, and in particular to a method and device for identifying the activity of lung cancer organoids based on a sequence image model. Background Art

[0002] An organoid is a three-dimensional cell complex formed by inducing the differentiation of stem cells or organ progenitor cells in vitro using 3D culture technology, which is similar to the target organ or tissue in both structure and function. Organoids can summarize the physiological processes of the whole organism, can better simulate the in vivo environment, are suitable for molecular and cell biology analysis, and provide a better solution between the animal and cell levels for fields such as tumor research, drug screening, and regenerative medicine. They have been widely used in various research aspects such as functional tissue induction, disease model establishment, drug screening, anti-inflammatory tests, and clinical research. In the field of lung cancer, lung cancer organoids have been widely used in personalized drug screening for lung cancer. After conducting experiments with different drugs on lung cancer organoids, it is necessary to measure the activity of the organoids through biological measurement methods to obtain the sensitivity of the lung cancer organoids to the drugs.

[0003] Although artificial intelligence deep learning technology has been increasingly widely used in the field of biomedicine, there is still a lack of research on applying artificial intelligence to the biological activity of organoids. In the existing personalized drug screening based on lung cancer organoids, after administering drugs to the organoids, they are continuously cultured for a certain period of time. Multiple attempts are required for the drug category and drug concentration, and finally, the biological activity of the organoids needs to be measured biologically, with a complex process. Therefore, there is an urgent need to develop a method that can identify the activity of organoid cells and can identify the activity of organoids without biological measurement, so as to accelerate the progress of organoid cell experiments and reduce the cost of the entire cell experiment. Summary of the Invention

[0004] Aiming at the deficiencies of the prior art, the present invention provides a method and device for identifying the activity of lung cancer organoids based on a sequence image model, which solves the problems of complex biological measurement processes and slow progress in traditional organoid personalized drug screening methods.

[0005] The technical solution adopted by the present invention is as follows:

[0006] A method for identifying the activity of lung cancer organoids based on a sequence image model, comprising:

[0007] Obtaining sequence images of lung cancer organoids of different samples, where the sequence images of lung cancer organoids of each sample contain the growth information of the organoids under specific drug categories and concentration conditions within a certain growth period; forming a data set by labeling the sequence images of the lung cancer organoids with feature tags indicating the activity state;

[0008] To construct a lung cancer organoid activity classification model, first, an initial model is pre-trained based on a large number of cell images using a neural network, and then a lung cancer organoid activity classification model is constructed based on the initial model. The lung cancer organoid activity classification model is trained and tested using the dataset.

[0009] The trained lung cancer organoid activity classification model is used to identify the activity of organoid images.

[0010] The lung cancer organoid activity classification model uses an encoder based on an improved Transformer. When training the lung cancer organoid activity classification model, the initial model weights are loaded as the parameters of the encoding layer. The lung cancer organoid sequence images are evenly segmented into several image patches. The image patches are linearly embedded and position encoding is added. Each image patch is projected into a vector sequence of a fixed length as the input sequence. At the same time, identifiers are added to the input sequence. Then, the input sequence is fed into the Transformer-based encoder for encoding. Then, the output of the identifier in the encoding layer of the model is used as the feature output for sequence image classification, and then connected to the classification layer to complete the activity classification. The vector values in the identifier are randomly initialized. Through continuous training of the network, the identifier encodes the statistical characteristics of the entire dataset and aggregates the information of other image patches as global features.

[0011] A further technical solution is as follows:

[0012] The lung cancer organoid sequence images of each sample are obtained by the following method:

[0013] Fixed size: The sizes of Z×T images of organoid growth information under specific drug categories and concentration conditions within a certain growth period are unified into a set value. Z is the coordinate position of collecting images along a certain coordinate axis direction, and T is the number of days of drug administration.

[0014] Image enhancement: The Z×T images are enhanced, including one or several of rotation, scaling, translation, flipping, and denoising.

[0015] Sequence splicing: The Z×T images are spliced in the channel dimension to obtain the lung cancer organoid sequence image X∈R H ×W×C , where H, W, and C are the height, width, and number of channels of the image respectively.

[0016] During training, the lung cancer organoid sequence image X∈R H×W×C is reshaped into a two-dimensional slice sequence X P ∈R N×D , where N = HW / P 2is the number of image patches divided, that is, the effective length of the input sequence of the improved Transformer encoder. (P, P) is the resolution of each image patch; D is the constant latent vector used in all layers of the improved Transformer encoder, which is used to linearly project the image patch onto the D dimension.

[0017] The concatenation order of the Z×T images in the channel dimension is sorted in ascending order according to the number of days of drug administration of lung cancer organoids and the coordinates of the image acquisition position.

[0018] The initial model formed by pre-training based on a neural network using large-scale cell images includes:

[0019] The initial model is constructed based on Transformer and learns the features of cell images through unsupervised pre-training on a large-scale cell image dataset.

[0020] The lung cancer organoid activity classification model uses the AdamW optimizer, and the loss function uses the cross-entropy loss function with label smoothing.

[0021] A lung cancer organoid activity recognition device based on a sequence image model includes:

[0022] A high-content imager for acquiring sequence images of lung cancer organoids of different samples. The sequence images of lung cancer organoids of each sample contain the growth information of organoids under specific drug categories and concentration conditions within a certain growth period;

[0023] Lung cancer organoid activity classification and recognition module, which is communicatively connected to a high-content imager, forms a data set by labeling the sequence images of lung cancer organoids with characteristic tags indicating the activity state, and divides the data set into a training set and a test set; uses the training set and the test set to train, learn and test the lung cancer organoid activity classification model; the lung cancer organoid activity classification model uses an encoder based on an improved Transformer. When training the lung cancer organoid activity classification model, the initial model weights are loaded as the parameters of the encoding layer. The sequence images of lung cancer organoids are evenly segmented into several image patches, the image patches are linearly embedded and position encoding is added, each image patch is projected into a vector sequence of a fixed length as the input sequence, and at the same time, identifiers are added to the input sequence. Then the input sequence is fed into the Transformer-based encoder for encoding, and then the output of the identifier in the encoding layer is used as the feature output for sequence image classification, and then connected to the classification layer to complete the activity classification; the vector values in the identifier are randomly initialized. Through continuous training of the network, the identifier encodes the statistical characteristics of the entire data set and aggregates the global features of the information of other image patches. Use the trained lung cancer organoid activity classification model to identify the activity category of the sequence images of lung cancer organoids regularly transmitted by the high-content imager, and obtain the activity category of the corresponding lung cancer organoids under a given drug.

[0024] It also includes a database module for storing the recognition results of the activity categories of lung cancer organoids under a given drug.

[0025] The initial model is pre-trained based on a neural network using a large number of cell images.

[0026] The beneficial effects of the present invention are as follows:

[0027] In the process of the organoid drug administration culture experiment of the present invention, sequence images are collected and made, and the deep learning network model is trained, learned and tested to obtain a classification model that can automatically identify the activity of organoids after drug administration through sequence images. Compared with the traditional biological measurement method, it can realize the recognition of organoid activity under various drug categories, concentrations and culture cycles, contains a lot of characteristic information, and the recognition result is accurate. It helps to speed up the process of organoid cell experiments and reduces the cost of cell experiments.

[0028] The input of the traditional model constructed based on Transformer is one-dimensional sequence data and cannot input multi-dimensional image data such as sequence images. The lung cancer organoid activity classification model of the present invention is constructed based on an improved Transformer deep learning algorithm, which improves the traditional Transformer and processes the sequence images at the input layer to convert the sequence image data into one-dimensional sequence data, so that Transformer can accept sequence images for input.

[0029] In terms of the input of the model, the traditional method is basically a single image, and the method of using sequential images is less adopted. However, the sequential images of the present invention not only include organoids at different heights on the z-axis, but also include images of organoids growing for multiple days. By using such sequential images, not only the information during the growth process of the organoids is included, but also the morphological information of the organoids at different z-axis heights is included. From a data perspective, the sequential images contain richer information and features than single images, which is more conducive to the learning and training of the model.

[0030] For the lung cancer organoid activity classification model of the present invention, first, a large-scale cell image is used to pre-train an initial model based on a neural network, and then sequential images are used as the model input to construct a classification model. The encoding part of the classification model loads the parameter weights of the initial model after pre-training for training, learning, and evaluation testing. Compared with the classification model without loading pre-trained weights, the classification model loaded with pre-trained weights can achieve good training effects with only a small number of training rounds, shortening the training time and improving the overall training efficiency. Description of the Drawings

[0031] Figure 1 It is a structural diagram of the lung cancer organoid activity classification model according to the embodiment of the present invention. Detailed Embodiments

[0032] The following describes the detailed embodiments of the present invention with reference to the drawings.

[0033] The present application provides a method for identifying the activity of lung cancer organoids based on a sequential image model, including:

[0034] S1. Obtain sequential images of lung cancer organoids of different samples, where the sequential images of lung cancer organoids of each sample include the growth information of organoids under specific drug categories and concentration conditions within a certain growth period; form a data set by labeling the sequential images of lung cancer organoids with feature tags indicating the activity state, and divide the data set into a training set and a test set;

[0035] Specifically, the sequential images of lung cancer organoids of each sample are obtained by the following method:

[0036] Fixed size: Unify the sizes of Z×T images of the growth information of organoids under specific drug categories and concentration conditions within a certain growth period, where Z is the coordinate position of collecting images along a certain coordinate axis direction, and T is the number of days of drug administration;

[0037] Image enhancement: Perform enhancement processing on the Z×T images, including one or several of rotation, scaling, translation, flipping, and denoising;

[0038] Sequence splicing: Concatenate Z×T images in the channel dimension to obtain a sequence image of lung cancer organoids X∈R H ×W×C , where H, W, and C are the height, width, and number of channels of the image, respectively.

[0039] Specifically, the concatenation order of the Z×T images in the channel dimension is sorted in ascending order according to the number of days of drug administration of lung cancer organoids and the coordinates of the image acquisition position.

[0040] S2. Construct a lung cancer organoid activity classification model, including: First, use a large number of cell images to pre-train an initial model based on a neural network, and then use the training set and test set to train and test the lung cancer organoid activity classification model;

[0041] The lung cancer organoid activity classification model uses an encoder based on an improved Transformer. The encoder of the improved Transformer processes the sequence image data into a one-dimensional sequence as input, and uses a large number of cell images to pre-train the initial model weights based on the Transformer neural network. In the training of the lung cancer organoid activity classification model, first load the initial model weights as the parameters of the encoding layer of the lung cancer organoid activity classification model; during training, the lung cancer organoid sequence image is evenly divided into several image patches, linearly embed the image patches and add position encoding, project each image patch into a vector sequence of a fixed length as the input sequence, and at the same time add a token to the input sequence and then feed the input sequence into the Transformer-based encoder for encoding. The output of the token in the encoding layer of the model will be used as the feature output of the sequence image, and then connect to the classification layer to complete the activity classification;

[0042] Specifically, the token can encode the statistical characteristics of the entire dataset, and during the entire training process, the token will perform global feature aggregation on the information on other image patches. And since the token does not have the information of the image, the token will not be biased towards a specific image patch during the training process. Therefore, the token can learn the global information of the entire dataset during the training process.

[0043] Specifically, during training, the lung cancer organoid sequence image X∈R H×W×C is reshaped into a two-dimensional patch sequence X P ∈R N×D , where N = HW / P 2 is the number of image patches divided, that is, the effective length of the input sequence of the encoder of the improved Transformer, (P, P) is the resolution of each image patch; D is the constant latent vector used in all layers of the encoder of the improved Transformer, which is used to linearly project the image patch into D dimensions.

[0044] S3. Use the lung cancer organoid activity classification model to identify the activity of the organoid images.

[0045] The technical solution of the present application will be further described below with specific embodiments.

[0046] (1) Obtain a lung cancer case tissue sample, perform organoid cell culture, inoculate the organoid cells into a cell culture well plate for culture, and culture them in different well positions in the well plate according to different cases and administrations.

[0047] (2) Obtain lung cancer organoid sequence images, including image data of lung cancer organoids at different times and different heights (in the Z-axis direction).

[0048] Specifically, place the lung cancer organoid well plate in a fixed position in a high-content imager (which can be purchased), and automatically take pictures of the lung cancer organoid images in the well plate according to the preset photographing process of the high-content imager. The photographing process of the high-content imager is as follows: On the 0th, 1st, 2nd, 3rd, and 4th days of the planned administration, take pictures of the lung cancer organoids at fixed positions in the fixed holes in the well plate at 10 different Z-axis heights. Stack and preprocess a total of 5×10 = 50 images at 5 days of culture and 10 shooting heights in the same well position to form a high-throughput sequence image, that is, the sequence image of one sample. A total of 103,180 samples corresponding to 67 lung cancer patients, 154 drugs, and 10 drug concentrations were collected. After removing the problematic data, a total of 103,040 effective samples were finally collected.

[0049] Processing the 50 high-resolution images includes: fixing each image to a unified size (256×256); performing image enhancement, including but not limited to operations such as rotation, scaling, translation, flipping, and denoising. The same enhancement work needs to be performed on the 50 images in the same sample sequence image to ensure the consistency of multiple sub-images in the same sequence image; splicing the 50 images in the channel dimension to obtain a lung cancer organoid sequence image X∈R H×W×C , where H, W, and C are the height, width, and number of channels of the image respectively. The splicing order of the images in the sequence image in the channel dimension can be specifically sorted according to the administration days T of the lung cancer organoids and the increasing order of the shooting Z-axis height.

[0050] Label the sequence images of lung cancer organoids: By measuring the cell biological activity indicators, obtain the activity indicators of lung cancer organoids, and judge the corresponding activity labels of lung cancer organoids through the biological activity indicators (in this embodiment, a binary classification label is set, that is, an active label and an inactive label). Then randomly group the labeled sequence image dataset, divide it into a training set and a test set, where there are 100,011 in the training set and 3,029 in the test set, and construct a deep learning model for model training. The ultimate goal is to classify the images into two categories: active and inactive.

[0051] (3) Construct a lung cancer organoid activity classification model. First, use a large number of cell images to pre-train an initial model based on a neural network. The initial model backbone network uses a total of about 1 billion self-owned and publicly available cell-related images for pre-training.

[0052] Then use the training set and the test set to perform supervised learning training and testing on the initial model to obtain a lung cancer organoid activity classification model. The lung cancer organoid activity classification model uses an encoder based on an improved Transformer, and the encoder of the improved Transformer takes a one-dimensional sequence as input;

[0053] See Figure 1 , during training, the lung cancer organoid sequence images are first processed by patching. In this embodiment, the lung cancer organoid sequence images are segmented into a total of 256 image patches of 16×16 (that is, the model input sequence length is 256), and the resolution of each image patch is (P, P)=(16, 16). The number of image patches can be adjusted according to different model parameters. For the sake of illustration, Figure 1 the images are segmented into 9 patches in

[0054] Each image patch is flattened into a vector. The size of the flattened vector is C×P×P, that is, each patch becomes a vector with a length of C×P×P, where C is the number of channels of the sequence image and can take the value of 3. Then the dimension of each image patch is C×P×P×T×Z = 3×16×16×5×10 = 38,400. In this embodiment, the encoder based on the improved Transformer uses a constant latent vector size D in all layers, that is, the flattened patch vector is mapped into a D-dimensional vector through a linear layer mapping. The dimension of the linear projection layer is 38,400×D (D = 768). Therefore, the dimension after passing through the linear projection layer becomes 256×768. To perform splitting, a special token is added to the model input sequence, and its output in the encoding layer of the model will be used as the feature output of the sequence image. After adding the token, the data dimension is updated to 257×768.

[0055] Specifically, since the sequential images are divided into multiple patches and input into the model, the position information of the patches themselves in the original sequential images is lost. Therefore, we need to add position encoding to let the model know the position information of each patch in the original sequential images. Add position encoding (as shown by Patch+Position Embedding in the figure). The position encoding is equivalent to a table. The table has a total of N rows, and the size of N is the same as the length of the input sequence. Each row represents a vector, and the dimension of the vector is the same as the dimension of the input sequence (768). Using standard learnable one-dimensional position embeddings, after adding the position encoding information, the dimension remains 257×768;

[0056] The input data with added position encoding is input into the encoding layer (Encoder) of the model for encoding. Through the encoder layer, the encoded output is obtained. The output corresponding to the identifier in the encoded output is used as the final output of the encoder. Then, through the classification layer (MLP Head) of the model, the final lung organoid activity category is obtained, that is, two categories: active lung organoids and inactive lung organoids. That is, the dimension of the last output layer of the MLP Head layer is 2.

[0057] Specifically, the lung organoid activity classification model uses the adamw optimizer, and the loss function uses the cross-entropy loss function with label smoothing. The value obtained by changing the output class probability through label smoothing is then fed into the cross-entropy loss function to obtain the final loss value. Through adamw, the error is backpropagated in the lung organoid activity classification network for network learning to train the lung organoid activity classification model.

[0058] Specifically, after training, the trained model is evaluated using the test set, and metrics such as accuracy, precision, recall, and F1 score are calculated, and hyperparameters are adjusted to improve performance. If the model does not meet expectations, the model can be adjusted or further data processing can be performed. In this embodiment, the trained model is used to evaluate the test set. Finally, through multiple hyperparameter tuning trainings, the accuracy of the model on the test set reaches 97%.

[0059] Specifically, adjust the parameters and structure of the model according to the evaluation results to improve the performance of the model. Use the trained model to predict new cell images and obtain the classification results of their activity states. Finally, the model can be embedded into practical applications for automatic identification of cell activity.

[0060] Specifically, the initial model pre-trained based on neural networks using large-scale cell images includes:

[0061] The initial model is constructed based on the Transformer. After unsupervised pre-training with a large number of cell image datasets, the initial model can well learn the features of cell images. In the subsequent training of the lung cancer organoid activity classification model, the parameter of the encoding layer of the lung cancer organoid activity classification model directly loads the weights of the initial model for training. The lung cancer organoid activity classification model loaded with the weights of the initial model can achieve good results with only a few training epochs. Loading the pre-trained model can improve the training effect and efficiency of the subsequent lung cancer organoid activity classification model.

[0062] This embodiment also provides a lung cancer organoid activity recognition device based on a sequence image model, including:

[0063] A high-content imager for acquiring sequence images of lung cancer organoids of different samples. The sequence images of lung cancer organoids of each sample contain the growth information of organoids under specific drug categories and concentration conditions within a certain growth period;

[0064] A lung cancer organoid activity classification and recognition module, which is communicatively connected to the high-content imager, forms a dataset by marking feature tags indicating the activity state for the sequence images of lung cancer organoids, and divides the dataset into a training set and a test set; uses the training set and the test set to train and test the lung cancer organoid activity classification model, where the lung cancer organoid activity classification model loads the weights of the initial model as the parameters of the encoding layer during training, and the lung cancer organoid activity classification model uses an encoder based on an improved Transformer. The encoder of the improved Transformer processes the sequence image into a one-dimensional sequence as the input: the sequence image of the lung cancer organoid is evenly divided into several image patches, the image patches are linearly embedded and position encoding is added, each image patch is projected into a vector sequence of a fixed length as the input sequence, and then fed into the Transformer-based encoder; at the same time, an identifier is added to the input sequence, and the output of the identifier at the encoding layer of the model will be used as the feature output of the sequence image;

[0065] The lung cancer organoid activity classification model is used to identify the activity category of the sequence images of lung cancer organoids regularly transmitted by the high-content imager, and obtain the activity category of the corresponding lung cancer organoid under a given drug.

[0066] It also includes a database module for storing the recognition results of the activity categories of lung cancer organoids under a given drug. After storing the recognition results of the model in the database, further data analysis can be performed on the model results later, and data support can be provided for the subsequent improvement and optimization of the model, which is beneficial to the iterative update of the subsequent model.

[0067] Those of ordinary skill in the art can understand that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for identifying the activity of lung cancer organoids based on a sequence image model, characterized in that, Comprising: Obtaining lung cancer organoid sequence images of different samples, where each sample's lung cancer organoid sequence image contains organoid growth information under specific drug categories and concentration conditions within a certain growth period; Marking the lung cancer organoid sequence images with feature tags indicating the active state to form a data set; Constructing a lung cancer organoid activity classification model. First, use a large-scale cell image to pre-train based on a neural network to form an initial model, and then construct a lung cancer organoid activity classification model based on the initial model, and use the data set to train, learn, and test the lung cancer organoid activity classification model; Using the trained lung cancer organoid activity classification model to identify the activity of organoid images; The lung cancer organoid activity classification model adopts an encoder based on an improved Transformer. When training the lung cancer organoid activity classification model, the initial model weights are loaded as the parameters of the encoding layer. The lung cancer organoid sequence image is evenly divided into several image patches, the image patches are linearly embedded and position encoding is added, each image patch is projected into a vector sequence of a fixed length as the input sequence, and at the same time, a token is added to the input sequence. Then, the input sequence is fed into the Transformer-based encoder for encoding, and then the output of the token in the encoding layer of the model is used as the feature output for sequence image classification, and then connected to the classification layer to complete the activity classification; the vector values in the token are randomly initialized. Through continuous training of the network, the token encodes the statistical characteristics of the entire data set and aggregates the information of other image patches globally.

2. The method for identifying the activity of lung cancer organoids based on a sequence image model according to claim 1, wherein The lung cancer organoid sequence images of each sample are obtained by the following method: Fixed size: Unify the sizes of Z×T images of organoid growth information under specific drug categories and concentration conditions within a certain growth period. Z is the coordinate position of the image collected along a certain coordinate axis direction, and T is the number of days of drug administration; Image enhancement: Perform enhancement processing on the Z×T images, including one or several of rotation, scaling, translation, flipping, and denoising; Sequence splicing: Splice Z×T images in the channel dimension to obtain the lung cancer organoid sequence image X∈R H×W×C , where H, W, and C are the height, width, and number of channels of the image, respectively.

3. The method for identifying the activity of lung cancer organoids based on a sequence image model according to claim 2, wherein During training, the lung cancer organoid sequence image \(X\in\mathbb{R}\) H×W×C is reshaped into a two-dimensional slice sequence \(X\) P \(\in\mathbb{R}\) N×D , where \(N = HW / P\) 2 is the number of image patches segmented, that is, the effective length of the input sequence of the improved Transformer encoder, \((P, P)\) is the resolution of each image patch; \(D\) is the constant latent vector used in all layers of the improved Transformer encoder for linearly projecting the image patches onto the \(D\)-dimensional space.

4. The method for identifying the activity of lung cancer organoids based on a sequence image model according to claim 2, wherein The splicing order of the Z×T images in the channel dimension is sorted in ascending order according to the number of days of drug administration of the lung cancer organoid and the coordinate of the image collection position.

5. The method for identifying the activity of lung cancer organoids based on a sequence image model according to claim 1, wherein The pre-training of the initial model based on a neural network using a large-scale cell image includes: The initial model is constructed based on the Transformer and pre-trained without supervision through a large-scale cell image data set to learn the features of cell images.

6. The method for identifying the activity of lung cancer organoids based on a sequence image model according to claim 1, wherein The lung cancer organoid activity classification model uses the AdamW optimizer, and the loss function uses the cross-entropy loss function with label smoothing.

7. A lung cancer organoid activity recognition device based on a sequence image model, characterized in that, Comprising: A high-content imager for obtaining lung cancer organoid sequence images of different samples, where each sample's lung cancer organoid sequence image contains organoid growth information under specific drug categories and concentration conditions within a certain growth period; The lung cancer organoid activity classification and recognition module is communicatively connected to the high-content imager, forms a data set by labeling the characteristic tags indicating the activity state for the sequence images of the lung cancer organoids, and divides the data set into a training set and a test set; uses the training set and the test set to train, learn and test the lung cancer organoid activity classification model; The lung cancer organoid activity classification model uses an improved Transformer-based encoder. When training the lung cancer organoid activity classification model, the initial model weights are loaded as the parameters of the encoding layer. The sequence images of the lung cancer organoids are evenly segmented into several image patches, the image patches are linearly embedded and position encoding is added, each image patch is projected into a vector sequence of a fixed length as the input sequence, and at the same time, identifiers are added to the input sequence. Then, the input sequence is fed into the Transformer-based encoder for encoding. Then, the output of the identifier in the encoding layer is used as the feature output for sequence image classification, and then connected to the classification layer to complete the activity classification; the vector values in the identifier are randomly initialized. Through continuous training of the network, the identifier encodes the statistical characteristics of the entire data set and aggregates the global features of the information of other image patches; Uses the trained lung cancer organoid activity classification model to identify the activity categories of the sequence images of the lung cancer organoids regularly transmitted by the high-content imager, and obtains the activity categories of the corresponding lung cancer organoids under a given drug.

8. The lung cancer organoid activity recognition device based on a sequence image model according to claim 7, characterized in that, It also includes a database module for storing the recognition results of the activity categories of the lung cancer organoids under a given drug.

9. The lung cancer organoid activity recognition device based on a sequence image model according to claim 7, wherein The initial model is pre-trained based on a neural network using a large number of cell images.

Images