A method and device for predicting the activation state of an immune cell

Abstract

This application provides a method and apparatus for predicting the activation state of immune cells, comprising: acquiring first immune cell images of first immune cells in a control group and second immune cell images of second immune cells in an experimental group; pairing immune cell images according to the cell image features of immune cells in each first and second immune cell image to obtain multiple immune cell image groups; extracting the difference visual features between the first and second immune cell images in each immune cell image group to predict the activation state of the second immune cell corresponding to the second immune cell image in the immune cell image group. In this way, predicting the activation state through an algorithm avoids the complexity of manual measurement, increases prediction throughput, and reduces experimental costs; compared to a single cell image, using difference visual features better reflects the changes in cell image features during the activation process, reduces the influence of experimental noise, and improves prediction accuracy.

Description

Technical Field

[0001] This application relates to the field of biomedical technology, and in particular to a method and device for predicting the activation state of immune cells. Background Technology

[0002] In the process of developing new drugs, in order to assess the differences between different perturbations for target discovery or drug design, it is necessary to determine the activation status of immune cells, such as Jurkat cells and primary T cells.

[0003] Currently, existing technologies typically involve invasive detection of immune cell activation values ​​during biological experiments using specific activation value measuring instruments to determine the activation state. However, this method has drawbacks such as high instrument costs, complex and time-consuming operation, and unstable measurement values ​​due to experimental noise, thus affecting the efficiency and accuracy of immune cell activation status analysis. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a method and device for predicting the activation state of immune cells. This method pairs immune cell images from experimental and control groups based on the cell image characteristics of the immune cells, and extracts the visual difference features between the paired images to predict the activation state of the immune cells in the experimental group. This algorithmic prediction of activation state avoids the complexity of manual instrument measurements, increases prediction throughput, saves manpower and resources, and reduces experimental costs. Compared to single cell images, using visual difference features better reflects the changes in cell image features during the activation process, reduces the impact of experimental noise on the prediction results, and thus improves prediction accuracy and sensitivity.

[0005] This application provides a method for predicting the activation state of immune cells, the method comprising:

[0006] Images of each first immune cell in the control group and images of each second immune cell in the experimental group were obtained; wherein the control group and the experimental group were set with different experimental culture conditions.

[0007] Based on the cell image features of immune cells in each first immune cell image and each second immune cell image, multiple first immune cell images and multiple second immune cell images are paired to obtain multiple immune cell image groups;

[0008] For each immune cell image group, the difference visual features between the first immune cell image and the second immune cell image in the immune cell image group are extracted; wherein, the difference visual features are used to characterize the visual differences between the first immune cell image and the second immune cell image in cell image features;

[0009] Based on the difference visual features of the immune cell image group, the activation state of the second immune cell corresponding to the second immune cell image in the immune cell image group is predicted.

[0010] Furthermore, according to the cell image features of immune cells in each first immune cell image and each second immune cell image, multiple first immune cell images and multiple second immune cell images are paired to obtain multiple immune cell image groups, including:

[0011] Based on the cell image characteristics of immune cells, a first cell feature value for each first immune cell image and a second cell feature value for each second immune cell image are determined; wherein, the cell image characteristics of immune cells are selected from at least one of color, morphology, and texture;

[0012] The positions of multiple first immune cell images are arranged according to the first cell feature value and the positions of multiple second immune cell images are arranged according to the second cell feature value, using the same sorting method.

[0013] Pairing first and second immune cell images with the same positional order results in an immune cell image group.

[0014] Furthermore, the prediction method also includes:

[0015] Compare the number of images in the first immune cell image and the number of images in the second immune cell image;

[0016] When the number of images is not equal, the cell images corresponding to the smaller number of images are resampled according to the larger number of images, so that the number of images of multiple first immune cell images and multiple second immune cell images after resampling is equal.

[0017] Furthermore, for each group of immune cell images, extracting the visual difference features between the first immune cell image and the second immune cell image in that group includes:

[0018] The first immune cell image in the immune cell image group is input into a pre-trained visual representation extraction model to obtain the first immune cell representation corresponding to the immune cell image group.

[0019] The second immune cell image in the immune cell image group is input into the visual representation extraction model to obtain the second immune cell representation corresponding to the immune cell image group.

[0020] Batch regularization was performed on the first and second immune cell representations corresponding to the immune cell image group, respectively.

[0021] The difference between the second immune cell representation and the first immune cell representation after batch regularization is determined as the difference visual feature corresponding to the immune cell image group.

[0022] Furthermore, the visual representation extraction model includes: a backbone convolution module, a continuous convolution module with residuals, a downsampling convolution module, and a global mean pooling module; the continuous convolution module with residuals includes: depthwise separable convolution units with large kernels, channel boosting convolution units, and channel deflating convolution units; the step of inputting the first and second immune cell images from the immune cell image set into the pre-trained visual representation extraction model to obtain the first and second immune cell representations corresponding to the immune cell image set includes:

[0023] The first and second immune cell images from the immune cell image group are input into the visual representation extraction model. The backbone convolution module performs convolution processing on the first and second immune cell images respectively to obtain a first feature map and a second feature map. The convolution processing includes convolution operation, layer regularization operation and Gaussian error linear unit activation operation.

[0024] The continuous convolution module with residuals performs continuous convolution processing on the first feature map and the second feature map respectively, and then the downsampling convolution module performs downsampling convolution processing on the first feature map and the second feature map after the continuous convolution processing respectively.

[0025] Repeat the previous step until the number of executions reaches a preset threshold. Then, the global mean pooling module performs global mean pooling operations on the first feature map and the second feature map respectively to obtain the first immune cell characterization and the second immune cell characterization corresponding to the immune cell image group.

[0026] Furthermore, acquiring the first immune cell image of each of the multiple first immune cells included in the control group, and the second immune cell image of each of the multiple second immune cells included in the experimental group, includes:

[0027] High-content images of cells in the control group and the experimental group were acquired respectively, and the high-content images of cells in the control group and the experimental group were converted into visualized RGB images through image processing operations;

[0028] The RGB images of the experimental group and the control group are segmented using an instance segmentation model to extract cell images of individual immune cells, resulting in first immune cell images of each of the multiple first immune cells in the control group and second immune cell images of each of the multiple second immune cells in the experimental group.

[0029] Furthermore, the prediction method also includes:

[0030] Set up at least one experimental group;

[0031] Drug targets were screened based on the predicted activation status of the second immune cells in each of the at least one experimental group.

[0032] This application also provides a device for predicting the activation state of immune cells, the device comprising:

[0033] The acquisition module is used to acquire images of the first immune cells of each of the multiple first immune cells in the control group and images of the second immune cells of each of the multiple second immune cells in the experimental group; wherein the control group and the experimental group are set with different experimental culture conditions.

[0034] The pairing module is used to pair multiple first immune cell images and multiple second immune cell images according to the cell image features of immune cells in each first immune cell image and each second immune cell image to obtain multiple immune cell image groups;

[0035] The extraction module is used to extract the difference visual features between the first immune cell image and the second immune cell image in each immune cell image group; wherein, the difference visual features are used to characterize the visual differences between the first immune cell image and the second immune cell image in cell image features;

[0036] The prediction module is used to predict the activation state of the second immune cell corresponding to the second immune cell image in the immune cell image group based on the difference visual features of the immune cell image group.

[0037] This application also provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the method for predicting the activation state of immune cells as described above are performed.

[0038] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the method for predicting the activation state of immune cells as described above.

[0039] This application provides a method and apparatus for predicting the activation state of immune cells. Based on the cell image features of immune cells, it pairs immune cell images from experimental and control groups and extracts the visual difference features between the paired images to predict the activation state of immune cells in the experimental group. This algorithmic prediction of activation state avoids the complexity of manual instrument measurements, increases prediction throughput, saves manpower and resources, and reduces experimental costs. Compared to single cell images, using visual difference features better reflects changes in cell image features during the activation process, reduces the impact of experimental noise on the prediction results, and thus improves prediction accuracy and sensitivity.

[0040] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0041] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 A flowchart of a method for predicting the activation state of immune cells provided in an embodiment of this application is shown;

[0043] Figure 2 This illustration shows a schematic diagram of a paired immune cell image set provided in an embodiment of this application;

[0044] Figure 3 This illustration shows one of the process diagrams for extracting difference visual features provided in an embodiment of this application;

[0045] Figure 4 This is a second schematic diagram illustrating a process for extracting visual features by means of difference provided in an embodiment of this application;

[0046] Figure 5 This paper illustrates the system framework of a method for predicting the activation state of immune cells provided in an embodiment of this application;

[0047] Figure 6A schematic diagram of the structure of an immune cell activation state prediction device provided in an embodiment of this application is shown.

[0048] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. Based on the embodiments of this application, every other embodiment obtained by those skilled in the art without inventive effort falls within the scope of protection of this application.

[0050] Research has found that in the process of developing new drugs, in order to assess the differences between different perturbations for target discovery or drug design, it is necessary to determine the activation status of immune cells, such as Jurkat cells and primary T cells.

[0051] Currently, existing technologies typically involve invasive detection of immune cell activation values ​​during biological experiments using specific activation value measuring instruments to determine the activation state. However, this method has drawbacks such as high instrument costs, complex and time-consuming operation, and unstable measurement values ​​due to fluctuations in experimental conditions, thus affecting the efficiency and accuracy of immune cell activation status analysis.

[0052] Based on this, embodiments of this application provide a method and apparatus for predicting the activation state of immune cells, thereby avoiding the complexity of manual instrument measurement, increasing prediction throughput, saving manpower and resources, reducing experimental costs, and improving prediction accuracy and sensitivity.

[0053] Please see Figure 1 , Figure 1 This is a flowchart illustrating a method for predicting the activation state of immune cells provided in an embodiment of this application. Figure 1 As shown in the embodiments of this application, the prediction method includes:

[0054] S101. Obtain the first immune cell image of each first immune cell in the multiple first immune cells included in the control group, and the second immune cell image of each second immune cell in the multiple second immune cells included in the experimental group.

[0055] In one possible implementation, the immune activation state of immune cells in the prediction method provided in this application can be correlated with fluorescence intensity; that is, the immune activation state can be characterized by fluorescence intensity, for example, the higher the activation level, the stronger the fluorescence intensity. Cells that meet this condition include, for example, Jurkat-NFAT-Luci cells.

[0056] It should be noted that the control group and the experimental group were subjected to different experimental culture conditions. Compared to the control group, the experimental group was subjected to perturbation conditions. Apart from the perturbation conditions, all other experimental culture conditions for the experimental and control groups must be kept consistent to control variables in the biological experiment. Depending on the designed perturbation conditions, at least one experimental group can be set up in the biological experiment to obtain the activation state of immune cells in the experimental group under different perturbation conditions. The perturbation conditions can be designed according to specific experimental needs, and this application embodiment does not impose any limitations. For example, perturbation conditions could include adding different drugs or performing different gene editing on the cells.

[0057] The following example illustrates different methods of gene editing in cells under perturbation conditions. In practice, the steps of a biological experiment may include:

[0058] 1. Applying the disturbance:

[0059] aAPC cells (artificial antigen-presenting cells) were gene-edited using different gene-editing methods. The gene-edited aAPC cells were then co-cultured with Jurkat-NFAT-Luci cells to obtain control and experimental groups. The control group could use a gene-editing method that had been determined to have no effect on activation status, or no gene editing at all. The experimental group used a gene-editing method different from the control group. Gene-editing methods could include gene overexpression, gene knockdown, gene knockout, and gene knock-in, among others.

[0060] 2. Stain Jurkat-NFAT-Luci cells.

[0061] In one possible implementation, step S101 may include:

[0062] S1011. Collect high-content images of cells in the control group and the experimental group respectively, and convert the high-content images of cells in the control group and the experimental group into visualized RGB images through image processing operations.

[0063] In this step, firstly, high-content images of cells in the control group and experimental group are acquired using high-content screening and imaging instruments, respectively; then, the high-content images of cells are converted into a reasonable pixel range through image processing operations to obtain visualized RGB images.

[0064] S1012. The visualized RGB images of the experimental group and the control group are segmented using an instance segmentation model to segment cell images of individual immune cells, thereby obtaining first immune cell images of each first immune cell in the multiple first immune cells included in the control group, and second immune cell images of each second immune cell in the multiple second immune cells included in the experimental group.

[0065] In this step, a pre-trained instance segmentation model can be used to segment the visualized RGB images of the experimental group and the control group, respectively, to obtain the first immune cell image of each of the multiple first immune cells in the control group and the second immune cell image of each of the multiple second immune cells in the experimental group. Corresponding to the above example, if the perturbation condition is applied by co-culturing aAPC cells and Jurkat-NFAT-Luci cells, then when segmenting instances using the instance segmentation model, aAPC cells in the image should be ignored, and only single-cell instances of immune cells should be extracted from the image. The instance segmentation model and the pre-training process are described in detail in the prior art, and this embodiment does not impose any limitations here.

[0066] S102. Based on the cell image features of immune cells in each first immune cell image and each second immune cell image, pair multiple first immune cell images and multiple second immune cell images to obtain multiple immune cell image groups.

[0067] In practice, based on the cell image characteristics of immune cells in immune cell images, a pair of first and second immune cell images with similar cell image characteristics can be selected and paired to form an immune cell image group.

[0068] In one possible implementation, step S102 may include:

[0069] S1021. Based on the cell image characteristics of immune cells, determine the first cell feature value of each first immune cell image and the second cell feature value of each second immune cell image.

[0070] The cell image features of the immune cells are selected from at least one of color, morphology, and texture. In specific implementations, the color (e.g., brightness of each channel), morphology (e.g., area, shape), and texture of the selected cell image features can be quantified and scored according to a preset image feature function. For example, the image feature function can map image features to quantized scores within a fixed range according to a certain mapping relationship. For instance, if the RGB value of cell image x is between 200 and 255, then the quantized score of cell image x regarding morphology in the cell image features after mapping is 90. Then, the quantized scores corresponding to each of the selected cell image features are obtained by weighted summation, averaging, etc., to obtain the cell feature value. Taking the weighted summation to obtain the cell feature value as an example, the formula can be expressed as:

[0071] Score(x)=w1×Area(x)+w2×Intensity(x)+w3×Texture(x)

[0072] In the formula, Score(x) represents the cell feature value of cell image x; Area(x) represents the quantized score of cell image x with respect to morphology in cell image features, taking cell area as an example here; Intensity(x) represents the quantized score of cell image x with respect to color in cell image features, taking brightness as an example here; Texture(x) represents the quantized score of cell image x with respect to texture in cell image features; w1, w2 and w3 represent the weights corresponding to morphology, color and texture in cell image features, respectively. For cell image features that are not selected, their corresponding weights are set to 0 in the formula.

[0073] S1022. Arrange the positions of multiple first immune cell images according to the first cell feature value and the positions of multiple second immune cell images according to the second cell feature value in the same sorting method.

[0074] S1023. Pair the first immune cell image and the second immune cell image with the same position and order to obtain an immune cell image group.

[0075] Please refer to the following: Figure 2 , Figure 2 This is a schematic diagram of a paired immune cell image group provided in an embodiment of this application. Figure 2As shown, in this step, multiple first immune cell images of the control group are sorted according to the first cell feature values ​​determined in S1021, and multiple second immune cell images of the experimental group are sorted according to the second cell feature values ​​determined in S1021. The sorting method is the same for both the control and experimental groups; both can be sorted in ascending or descending order. Then, the sorted cell image lists are paired according to their positions, meaning that first and second immune cell images with the same position are paired to form an immune cell image group. This ensures that the first and second immune cell images in each immune cell image group are similar in cell image features. It is important to note that this similarity is not an absolute numerical similarity, but rather a similarity in relative cell image features within the control and experimental groups. This reduces the influence of batch effects, differences in initial cell states, and different staining concentrations, thereby better assessing the overall impact of different perturbation conditions on the control and experimental groups.

[0076] Experimental results demonstrate that pairing immune cell image groups using the method provided in this application significantly improves overall performance compared to random pairing. This is because the goal of the prediction method is to learn the differences in the activation state of immune cells before and after perturbation, simulating the temporal evolution process. Therefore, selecting immune cells with similar image features makes it easier to analyze subtle changes between them, which is also consistent with biological significance.

[0077] Furthermore, in the process of pairing multiple first immune cell images and multiple second immune cell images to obtain multiple immune cell image groups, the prediction method further includes: comparing the number of images in the first immune cell images and the number of images in the second immune cell images; when the number of images is not equal, resampling the cell images corresponding to the smaller number of images according to the larger number of images, so that the number of images in the multiple first immune cell images and the multiple second immune cell images after resampling is equal.

[0078] Here, when the number of images in the first immune cell image and the number of images in the second immune cell image are not equal, the cell images corresponding to the smaller number of images can be repeatedly sampled according to the larger number of images, expanding them to the same number, so as to facilitate pairing to obtain immune cell image groups.

[0079] S103. For each immune cell image group, extract the visual difference feature between the first immune cell image and the second immune cell image in the immune cell image group.

[0080] Among them, the difference visual feature is used to characterize the visual differences in cell image features between the first immune cell image and the second immune cell image.

[0081] In practice, traditional image processing algorithms or machine learning models can be used to extract the difference visual features corresponding to each immune cell image group.

[0082] Please refer to the following: Figure 3 and Figure 4 , Figure 3 This is one of the schematic diagrams illustrating a process for extracting visual features by difference, provided in an embodiment of this application. Figure 4 This is a second schematic diagram illustrating a process for extracting visual features by difference, provided in an embodiment of this application. In one possible implementation, step S103 may include:

[0083] S1031. Input the first immune cell image in the immune cell image group into the pre-trained visual representation extraction model to obtain the first immune cell representation corresponding to the immune cell image group.

[0084] S1032. Input the second immune cell image in the immune cell image group into the visual representation extraction model to obtain the second immune cell representation corresponding to the immune cell image group.

[0085] For steps S1031 and S1032 above, two immune cell images from an immune cell image group can be input into the visual representation extraction model respectively, and the same network parameters can be used for prediction. The network parameters are shared, and the visual representations of the two immune cell images are output.

[0086] S1033. Perform batch regularization processing on the first and second immune cell representations corresponding to the immune cell image group respectively.

[0087] In this step, batch regularization is performed on the first and second immune cell representations corresponding to each immune cell image group through two different BN layers. This can standardize the data distribution of different batches, alleviate overfitting to some extent, and reduce the impact of batch effects caused by imaging differences (such as staining bias). The overall prediction performance is improved by about 1%.

[0088] S1034. The difference between the second immune cell characterization and the first immune cell characterization after batch regularization is determined as the difference visual feature corresponding to the immune cell image group.

[0089] In this step, the difference between the second immune cell representation and the first immune cell representation is determined. That is, the vector of the first immune cell representation of the control group is subtracted from the vector of the second immune cell representation of the experimental group as the difference visual feature, which is used to predict the activation state.

[0090] Here, the difference between two representations is used to predict the activation state of immune cells, rather than using a single representation, for the following reasons: First, the activation process of immune cells is a change from the initial state (represented by the control group, without perturbation) to the target state (represented by the experimental group, with perturbation). Therefore, predicting the immune activation state based on changes in cell state is more biologically accurate than predicting based on the final state of a single cell. Second, the changes in cells before and after perturbation are small, making them susceptible to interference from irrelevant experimental noise factors, such as batch effects, differences in initial state, and different staining concentrations. The significance of these irrelevant experimental noise factors is far greater than the subtle changes in cells. That is, under different experimental conditions, especially when the activation range is small, immune cells in the same activation state show significant differences in image features, making the model more prone to learning noise. This makes it difficult to distinguish small-scale differences using a single immune cell image, thus making it difficult to predict the activation state.

[0091] However, this application's embodiments note that during the activation process, the degree of change in immune cells tends to be consistent, such as gradual wrinkling of cell edges, increasing mitochondrial activity, and gradual nuclear division. Therefore, predicting the activation state by comparing immune cell changes between the experimental and control groups within the same batch is more sensitive. Thus, this application's embodiments use the visual difference between the experimental and control groups to predict the activation state through changes in image features, effectively amplifying the difference signal, making model learning easier, and resulting in model predictions that more closely approximate actual cell changes.

[0092] It is worth noting that, for steps S1031 and S1032, due to technical difficulties such as similar pixel values ​​and small area in single-cell images, the visual representations extracted by conventionally designed neural networks in the prior art have low discriminative power. Therefore, embodiments of this application provide an optimized visual representation extraction model that can extract visual representations of single-cell images with higher discriminative power. Figure 4 As shown, the visual representation extraction model includes: a backbone convolution module, a continuous convolution module with residuals, a downsampling convolution module, and a global mean pooling module; the continuous convolution module with residuals includes: depthwise separable convolution units with large kernels, channel boosting convolution units, and channel deflating convolution units; then step S1031 may include:

[0093] Step 1: Input the first immune cell image and the second immune cell image from the immune cell image group into the visual representation extraction model. The backbone convolution module performs convolution processing on the first immune cell image and the second immune cell image respectively to obtain the first feature map and the second feature map.

[0094] In practical implementation, the network input of the visual representation extraction model is a 224x224x3 image sample. Through a 4x4 convolution operation in the backbone convolution module, with a stride of 4 and an output channel count of 96, the feature map size is reduced to 56, and the number of channels is increased to 96, i.e., [224,224,3]→[56,56,96]. Compared to general neural network designs, the visual representation extraction model provided in this embodiment removes pooling operations and directly uses convolution to reduce the feature map size, achieving better information interaction and reducing information loss.

[0095] Step 2: The continuous convolution module with residuals performs continuous convolution processing on the first feature map and the second feature map respectively, and then the downsampling convolution module performs downsampling convolution processing on the first feature map and the second feature map after the continuous convolution processing respectively.

[0096] In this step, the feature map [56,56,96] is first input into a set of consecutive convolutional operations with residuals. Each set of consecutive convolutional operations includes three convolutional operations: a depthwise separable convolution with a 7x7 kernel, a 1x1 kernel convolution with 4x increased channels, and a 1x1 kernel convolution with 4x decreased channels. The output feature map remains [56,56,96], i.e., [56,56,96] → [56,56,96]. Compared to general neural network designs, the visual representation extraction model provided in this application uses depthwise separable convolutions with larger kernel sizes (from 3x3 to 7x7). This enhances the receptive field of the network without increasing computational cost. In other words, larger kernel sizes increase computational cost, while depthwise separable convolutions reduce computational cost, thus canceling each other out.

[0097] Next, the feature map [56,56,96] is input into the downsampling convolution operation. The convolution kernel is 3x3 and the stride is 2. That is, the feature map is reduced by 2 times and the output channel is increased by 2 times, i.e., [56,56,96]→[28,28,192].

[0098] Step 3: Repeat step 2 until the number of executions reaches a preset threshold. Then, the global mean pooling module performs global mean pooling operations on the first feature map and the second feature map respectively to obtain the first immune cell characterization and the second immune cell characterization corresponding to the immune cell image group.

[0099] Here, the number of executions can be specifically set according to factors such as the size of the input image or the learning difficulty to enhance the model's learning ability. In this embodiment, the number of executions can be set to 3. Each execution reduces the size of the feature map and increases the number of channels, i.e., [56,56,96]→[28,28,192]→[14,14,384]→[7,7,768]. Finally, the feature map [7,7,768] is converted into a 768-dimensional feature vector through global mean pooling.

[0100] Furthermore, it should be noted that each convolutional process in the visual representation extraction model includes a convolution operation, a layer regularization operation, and a Gaussian error linear unit activation operation. That is, after each convolutional operation, batch regularization and rectified linear unit operations are performed. Compared to general neural network designs, the visual representation extraction model provided in this application uses layer regularization (LN) instead of batch regularization (BN). This is because BN normalizes each dimension of the features within a batch of samples, while LN normalizes all features within each sample. When the differences between sample images are small, using LN instead of BN can improve the difference between different distributions. Simultaneously, a Gaussian error linear unit (GELU) is used instead of a rectified linear unit (ReLU). ReLU sets the output to 0 for values ​​less than 0 and is not differentiable at 0, while GELU has a certain probability of not setting the output to 0 for values ​​less than 0 and is differentiable at 0. Therefore, this improves the stability and sensitivity of the network.

[0101] S104. Based on the difference visual features of the immune cell image group, predict the activation state of the second immune cell corresponding to the second immune cell image in the immune cell image group.

[0102] In one possible implementation, the difference visual features of the immune cell image set are input into an activation state prediction model to obtain the activation state of the second immune cell corresponding to the second immune cell image in the immune cell image set. The activation state prediction model may include a multilayer perceptron (MLP) layer. In the MLP layer, the feature vector dimension is first increased by a factor of N and then decreased by a factor of N through a set of linear operations. For example, when N=4, the feature vector dimension is changed from

[768] →

[3072] →

[768] . The activation state prediction model also includes a softmax layer, which converts the dimension to the number of categories (e.g., 10) through linear operations, i.e.,

[768] →

[10] . In this way, the activation state of the second immune cell in the experimental group is predicted to belong to which of the 10 activation state categories.

[0103] The training process of the model provided in this application embodiment is described in detail in the prior art, and this application embodiment does not impose any limitations here. For example, an optimizer can be used to optimize the cross-entropy loss function of the model. Through backpropagation of the gradients of the model parameters, the parameters in the model are continuously optimized until the loss value no longer decreases, reaching a plateau, i.e., convergence, yielding the trained model. A microplate reader can be used to measure the biofluorescence released during fluorescence oxidation using a luciferase assay, which serves as an indicator of NFAT levels and a criterion for judging the activation state of immune cells, and is used as the true value during model training.

[0104] Furthermore, the prediction method also includes: setting up at least one experimental group; and screening drug targets based on the predicted activation status of the second immune cells in each of the at least one experimental group.

[0105] After predicting the activation state of the second immune cells in each experimental group, by comparing the perturbation conditions set for the experimental and control groups, it is possible to screen out which perturbations, such as changes in aAPC genes (e.g., overexpression, knockdown, knockout, or knock-in), lead to differences in membrane proteins, thereby causing activation or inhibition of immune cells. The proteins involved in these perturbations are the targets; for example, the membrane proteins corresponding to the changed genes in aAPC are the drug targets. For instance, perturbations that lead to a certain level of immune cell activation can be screened, or the perturbations corresponding to the top three experimental groups with the greatest increase in immune cell activation can be screened; the proteins involved in these perturbations are the targets.

[0106] In this way, compared with complex biological experiments, predicting the activation state of immune cells based on difference visual features to screen targets can increase the throughput of target screening, reduce human and material costs, and thus reduce the cost of new product development.

[0107] Please refer to the following: Figure 5 , Figure 5 This application provides a system framework for a method to predict the activation state of immune cells. For example... Figure 5 As shown, the preprocessing steps of the prediction method first include: obtaining cells of the control group and experimental group through gene overexpression experiment; automatically capturing high-content images of cells of the control group and experimental group; converting the high-content images of cells into visualized RGB images through the fluorescence image preprocessing module; and segmenting images of individual immune cells in the control group and experimental group through the single-cell instance segmentation module.

[0108] Next, images of individual immune cells from the control group and the experimental group are input into the algorithm model to predict the activation state of immune cells in the experimental group. Specifically, firstly, the sample combination matching module selects cell images from the experimental group and control group with corresponding image feature rankings, and pairs them to form an immune cell image group; secondly, the optimized neural network in the visual representation extraction module outputs the visual representations of two single cells in an immune cell image group, namely the visual representations of single cells in the experimental group and the control group; thirdly, the residual representation calculation module calculates the difference visual features between the visual representations of single cells in the experimental group and the control group; then, the activation category prediction module uses an MLP network to predict the activation state of immune cells; finally, drug targets are screened based on the activation state of immune cells.

[0109] In summary, the prediction method provided in this application has the following technical effects: First, by reflecting changes in cell image features through the visual difference between the experimental and control groups, the immune activation state can be predicted, which is closer to the biological meaning and reduces the influence of experimental noise, thereby improving prediction accuracy and sensitivity. Second, selecting experimental and control group cell images with similar image features allows for better analysis of subtle changes between cells, which is also consistent with biological significance. Third, the optimized neural network design uses convolution operations instead of pooling operations to reduce the feature map size, achieve better information interaction, and reduce information loss; it uses depthwise separable convolutions with larger kernel sizes (from 3x3 to 7x7) to enhance the receptive field of the network without increasing computational cost; it uses Layer Norm instead of Batch Norm to improve the discriminative power between different distributions when sample differences are small; and it uses Gaussian error linear units (GELU) instead of rectified linear units (ReLU) to improve the stability and sensitivity of the network. Fourth, by using two different batch normalization (BN) layers to perform batch regularization on the visual representation vectors of the experimental and control groups, the data distribution across different batches can be standardized, overfitting can be mitigated, the batch effect in biological experiments can be reduced, and prediction performance can be improved. Fifth, by using an innovatively designed deep learning algorithm framework to predict activation states and then screen targets, the complexity of manual instrument measurements and numerous biological experiments can be avoided, prediction throughput and target screening throughput can be improved, saving manpower and resources, reducing experimental costs and new product development costs.

[0110] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of a device for predicting the activation state of immune cells provided in an embodiment of this application. Figure 6 As shown, the prediction device 600 includes:

[0111] The acquisition module 610 is used to acquire the first immune cell image of each first immune cell in the multiple first immune cells included in the control group, and the second immune cell image of each second immune cell in the multiple second immune cells included in the experimental group; wherein the control group and the experimental group are set with different experimental culture conditions.

[0112] The pairing module 620 is used to pair multiple first immune cell images and multiple second immune cell images according to the cell image features of immune cells in each first immune cell image and each second immune cell image to obtain multiple immune cell image groups;

[0113] Extraction module 630 is used to extract the difference visual features between the first immune cell image and the second immune cell image in each immune cell image group; wherein, the difference visual features are used to characterize the visual differences between the first immune cell image and the second immune cell image in cell image features;

[0114] The prediction module 640 is used to predict the activation state of the second immune cell corresponding to the second immune cell image in the immune cell image group based on the difference visual features of the immune cell image group.

[0115] Furthermore, when the pairing module 620 pairs multiple first immune cell images and multiple second immune cell images according to the cell image features of immune cells in each first immune cell image and each second immune cell image to obtain multiple immune cell image groups, the pairing module 620 is used to:

[0116] Based on the cell image characteristics of immune cells, a first cell feature value for each first immune cell image and a second cell feature value for each second immune cell image are determined; wherein, the cell image characteristics of immune cells are selected from at least one of color, morphology, and texture;

[0117] The positions of multiple first immune cell images are arranged according to the first cell feature value and the positions of multiple second immune cell images are arranged according to the second cell feature value, using the same sorting method.

[0118] Pairing first and second immune cell images with the same positional order results in an immune cell image group.

[0119] Furthermore, the pairing module 620 is also used for:

[0120] Compare the number of images in the first immune cell image and the number of images in the second immune cell image;

[0121] When the number of images is not equal, the cell images corresponding to the smaller number of images are resampled according to the larger number of images, so that the number of images of multiple first immune cell images and multiple second immune cell images after resampling is equal.

[0122] Furthermore, when extracting the visual difference features between the first immune cell image and the second immune cell image in each immune cell image group, the extraction module 630 is used to:

[0123] The first immune cell image in the immune cell image group is input into a pre-trained visual representation extraction model to obtain the first immune cell representation corresponding to the immune cell image group.

[0124] The second immune cell image in the immune cell image group is input into the visual representation extraction model to obtain the second immune cell representation corresponding to the immune cell image group.

[0125] Batch regularization was performed on the first and second immune cell representations corresponding to the immune cell image group, respectively.

[0126] The difference between the second immune cell representation and the first immune cell representation after batch regularization is determined as the difference visual feature corresponding to the immune cell image group.

[0127] Furthermore, the visual representation extraction model includes: a backbone convolution module, a continuous convolution module with residuals, a downsampling convolution module, and a global mean pooling module; the continuous convolution module with residuals includes: depthwise separable convolution units with large kernels, channel boosting convolution units, and channel deflating convolution units; the extraction module 630 is used for:

[0128] The first and second immune cell images from the immune cell image group are input into the visual representation extraction model. The backbone convolution module performs convolution processing on the first and second immune cell images respectively to obtain a first feature map and a second feature map. The convolution processing includes convolution operation, layer regularization operation and Gaussian error linear unit activation operation.

[0129] The continuous convolution module with residuals performs continuous convolution processing on the first feature map and the second feature map respectively, and then the downsampling convolution module performs downsampling convolution processing on the first feature map and the second feature map after the continuous convolution processing respectively.

[0130] Repeat the previous step until the number of executions reaches a preset threshold. Then, the global mean pooling module performs global mean pooling operations on the first feature map and the second feature map respectively to obtain the first immune cell characterization and the second immune cell characterization corresponding to the immune cell image group.

[0131] Furthermore, when acquiring the first immune cell image of each first immune cell in the multiple first immune cells included in the control group, and the second immune cell image of each second immune cell in the multiple second immune cells included in the experimental group, the acquisition module 610 is used to:

[0132] High-content images of cells in the control group and the experimental group were acquired respectively, and the high-content images of cells in the control group and the experimental group were converted into visualized RGB images through image processing operations;

[0133] The RGB images of the experimental group and the control group are segmented using an instance segmentation model to extract cell images of individual immune cells, resulting in first immune cell images of each of the multiple first immune cells in the control group and second immune cell images of each of the multiple second immune cells in the experimental group.

[0134] Furthermore, the prediction device 600 also includes a screening module; the screening module is used for:

[0135] Set up at least one experimental group;

[0136] Drug targets were screened based on the predicted activation status of the second immune cells in each of the at least one experimental group.

[0137] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 7 As shown, the electronic device 700 includes a processor 710, a memory 720, and a bus 730.

[0138] The memory 720 stores machine-readable instructions executable by the processor 710. When the electronic device 700 is running, the processor 710 communicates with the memory 720 via the bus 730. When the machine-readable instructions are executed by the processor 710, they can perform the operations described above. Figure 1 The steps of the method embodiment shown in the illustration for predicting the activation state of immune cells can be found in the method embodiment for specific implementation details, which will not be repeated here.

[0139] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described actions. Figure 1 The steps of the method embodiment shown in the illustration for predicting the activation state of immune cells can be found in the method embodiment for specific implementation details, which will not be repeated here.

[0140] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0141] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0142] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0143] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0144] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0145] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for predicting the activation state of immune cells, characterized in that, The prediction method includes: Images of each first immune cell in the control group and images of each second immune cell in the experimental group were obtained; wherein the control group and the experimental group were set with different experimental culture conditions. Based on the cell image features of immune cells in each first immune cell image and each second immune cell image, multiple first immune cell images and multiple second immune cell images are paired to obtain multiple immune cell image groups; For each immune cell image group, the difference visual features between the first immune cell image and the second immune cell image in the immune cell image group are extracted; wherein, the difference visual features are used to characterize the visual differences between the first immune cell image and the second immune cell image in cell image features; Based on the difference visual features of the immune cell image group, the activation state of the second immune cell corresponding to the second immune cell image in the immune cell image group is predicted. For each group of immune cell images, the step of extracting the visual difference features between the first and second immune cell images in that group includes: The first immune cell image in the immune cell image group is input into a pre-trained visual representation extraction model to obtain the first immune cell representation corresponding to the immune cell image group. The second immune cell image in the immune cell image group is input into the visual representation extraction model to obtain the second immune cell representation corresponding to the immune cell image group. Batch regularization was performed on the first and second immune cell representations corresponding to the immune cell image group, respectively. The difference between the second immune cell representation and the first immune cell representation after batch regularization is determined as the difference visual feature corresponding to the immune cell image group.

2. The prediction method according to claim 1, characterized in that, The process involves pairing multiple first immune cell images and multiple second immune cell images according to the cell image features of immune cells in each first immune cell image and each second immune cell image to obtain multiple immune cell image groups, including: Based on the cell image characteristics of immune cells, a first cell feature value for each first immune cell image and a second cell feature value for each second immune cell image are determined; wherein, the cell image characteristics of immune cells are selected from at least one of color, morphology, and texture; The positions of multiple first immune cell images are arranged according to the first cell feature value and the positions of multiple second immune cell images are arranged according to the second cell feature value, using the same sorting method. Pairing first and second immune cell images with the same positional order results in an immune cell image group.

3. The prediction method according to claim 2, characterized in that, The prediction method further includes: Compare the number of images in the first immune cell image and the number of images in the second immune cell image; When the number of images is not equal, the cell images corresponding to the smaller number of images are resampled according to the larger number of images, so that the number of images of multiple first immune cell images and multiple second immune cell images after resampling is equal.

4. The prediction method according to claim 1, characterized in that, The visual representation extraction model includes: a backbone convolution module, a continuous convolution module with residuals, a downsampling convolution module, and a global mean pooling module; the continuous convolution module with residuals includes: depthwise separable convolution units with large kernels, channel boosting convolution units, and channel deflating convolution units; the step of inputting the first and second immune cell images from the immune cell image set into the pre-trained visual representation extraction model to obtain the first and second immune cell representations corresponding to the immune cell image set includes: The first and second immune cell images from the immune cell image group are input into the visual representation extraction model. The backbone convolution module performs convolution processing on the first and second immune cell images respectively to obtain a first feature map and a second feature map. The convolution processing includes convolution operation, layer regularization operation and Gaussian error linear unit activation operation. The continuous convolution module with residuals performs continuous convolution processing on the first feature map and the second feature map respectively, and then the downsampling convolution module performs downsampling convolution processing on the first feature map and the second feature map after the continuous convolution processing respectively. Repeat the previous step until the number of executions reaches a preset threshold. Then, the global mean pooling module performs global mean pooling operations on the first feature map and the second feature map respectively to obtain the first immune cell characterization and the second immune cell characterization corresponding to the immune cell image group.

5. The prediction method according to claim 1, characterized in that, The acquisition of first immune cell images of each of the multiple first immune cells in the control group and second immune cell images of each of the multiple second immune cells in the experimental group includes: High-content images of cells in the control group and the experimental group were acquired respectively, and the high-content images of cells in the control group and the experimental group were converted into visualized RGB images through image processing operations; The RGB images of the experimental group and the control group are segmented using an instance segmentation model to extract cell images of individual immune cells, resulting in first immune cell images of each of the multiple first immune cells in the control group and second immune cell images of each of the multiple second immune cells in the experimental group.

6. The prediction method according to claim 1, characterized in that, The prediction method further includes: Set up at least one experimental group; Drug targets were screened based on the predicted activation status of the second immune cells in each of the at least one experimental group.

7. A device for predicting the activation state of immune cells, characterized in that, The prediction device includes: The acquisition module is used to acquire images of the first immune cells of each of the multiple first immune cells in the control group and images of the second immune cells of each of the multiple second immune cells in the experimental group; wherein the control group and the experimental group are set with different experimental culture conditions. The pairing module is used to pair multiple first immune cell images and multiple second immune cell images according to the cell image features of immune cells in each first immune cell image and each second immune cell image to obtain multiple immune cell image groups; The extraction module is used to extract the difference visual features between the first immune cell image and the second immune cell image in each immune cell image group; wherein, the difference visual features are used to characterize the visual differences between the first immune cell image and the second immune cell image in cell image features; The prediction module is used to predict the activation state of the second immune cell corresponding to the second immune cell image in the immune cell image group based on the difference visual features of the immune cell image group. When extracting the visual difference features between the first immune cell image and the second immune cell image in each immune cell image group, the extraction module is used to: The first immune cell image in the immune cell image group is input into a pre-trained visual representation extraction model to obtain the first immune cell representation corresponding to the immune cell image group. The second immune cell image in the immune cell image group is input into the visual representation extraction model to obtain the second immune cell representation corresponding to the immune cell image group. Batch regularization was performed on the first and second immune cell representations corresponding to the immune cell image group, respectively. The difference between the second immune cell representation and the first immune cell representation after batch regularization is determined as the difference visual feature corresponding to the immune cell image group.

8. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. The machine-readable instructions are executed by the processor to perform the steps of a method for predicting the activation state of immune cells as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of a method for predicting the activation state of immune cells as described in any one of claims 1 to 6.

Images