Cell image segmentation and function prediction method and device and computer equipment

Abstract

The invention discloses a cell image segmentation and function prediction method and device and computer equipment. The method comprises the steps of obtaining a to-be-processed cell image; inputting the to-be-processed cell image into the segmentation model for segmentation to obtain physical morphological features; inputting the physical morphological characteristics into a function prediction model to predict relevant parameters of the cell biological characteristics and relevant indexes of a set type of curative effect so as to obtain the parameters of the cell biological characteristics and the relevant indexes; and outputting physical morphological characteristics, cell biological characteristic parameters and related indexes. By implementing the method provided by the invention, the undyed bright field cell image can be effectively processed, the cell morphology and function are associated, the method is specially used for segmenting the mesenchymal stem cells, and the problem of instance segmentation under the condition of cell overlapping is solved.

Description

Technical Field

[0001] The present invention relates to artificial intelligence, and more specifically to a method, device and computer equipment for cell image segmentation and function prediction. Background Art

[0002] In recent years, deep learning models have made significant progress in the field of biomedical image analysis, especially in cell segmentation. Among them, models such as U-Net, SAM (Segment Anything Model), and Transformer have been widely used. These models are typically designed to process cell images or tissue slice images that have been specifically stained and labeled to more accurately identify and segment different cell types, such as neurons, cancer cells, and stem cells. However, despite the excellent performance of these models in their respective application scenarios, they still have some limitations and challenges.

[0003] Most existing algorithm models are specifically trained for specific cell types, including optimization for cell images processed with specific staining methods. Consequently, these models often perform poorly when applied to unstained brightfield cell images. Predicting nuclear outlines is particularly challenging. Currently, no algorithm can effectively link morphological parameters of cells or organelles to their biological functions. This means that while existing models can identify and segment cells, it is not possible to directly predict the cell's likely biological function from a cell image. This lack of capability limits the potential value of deep learning in cell biology research. Mesenchymal stem cells, due to their unique physiological properties and complex culture environment, present additional challenges in image segmentation. Current algorithm models generally lack specialized training and support for this specialized cell type, making it difficult to achieve ideal segmentation results in practical applications. Specifically, when working with mesenchymal stem cells, due to the common overlap between cells, existing algorithm models face significant challenges in achieving accurate instance segmentation. This poses a significant obstacle to subsequent research efforts, such as single-cell analysis.

[0004] Therefore, it is necessary to design a new method that can effectively process unstained bright field cell images, associate cell morphology and function, specifically segment mesenchymal stem cells, and solve the instance segmentation problem in the case of cell overlap. Summary of the Invention

[0005] The purpose of the present invention is to overcome the defects of the prior art and provide a method, device and computer equipment for cell image segmentation and function prediction.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a cell image segmentation and function prediction method, comprising:

[0007] Acquire the cell image to be processed;

[0008] Inputting the cell image to be processed into a segmentation model for segmentation to obtain physical morphological features;

[0009] Inputting the physical morphological characteristics into a functional prediction model to predict cell biological characteristic related parameters and indicators related to the set type of therapeutic effect, so as to obtain cell biological characteristic parameters and related indicators;

[0010] Output the physical morphological characteristics, cell biological characteristic parameters and related indicators.

[0011] Its further technical solution is: the segmentation model includes an encoder and a decoder, the encoder includes an initial convolutional layer and a residual block group; the residual block group includes several residual blocks, and the residual block includes a batch normalization layer, a ReLU activation function and multiple convolutional layers.

[0012] The decoder includes an upsampling layer, a dense decoder unit, a skip connection layer, and a final convolutional layer; the decoder also includes a kernel pixel branch and horizontal and vertical distance branches.

[0013] A further technical solution is: inputting the cell image to be processed into a segmentation model for segmentation to obtain physical morphological features, including:

[0014] Inputting the cell image to be processed into a segmentation model, and performing a preprocessing operation on the cell image to be processed to obtain a preprocessing result;

[0015] Using an encoder to extract multi-scale spatial details and features from the preprocessing result while retaining fine structure information to obtain a feature map;

[0016] A decoder is used to restore the image resolution of the feature map, and two branches work together to segment the cell area and refine the boundaries to obtain physical morphological features.

[0017] A further technical solution is: the encoder is used to extract multi-scale spatial details and features from the preprocessing result while retaining fine structure information to obtain a feature map, including:

[0018] Preliminarily extracting features from the preprocessing results through an initial convolutional layer to obtain preliminary features;

[0019] The preliminary features are input into the residual block group to capture information of different scales to obtain a feature map.

[0020] A further technical solution is: the decoder is used to restore the image resolution of the feature map, and two branches work together to segment the cell area and refine the boundaries to obtain physical morphological features, including:

[0021] Upsampling the feature map and performing feature enhancement using a dense decoder unit to obtain an intermediate feature map;

[0022] The feature map and the intermediate feature map are transferred and combined using a skip connection layer to obtain an enhanced feature map;

[0023] Using a kernel pixel branch to predict a binary mask for the enhanced feature map to distinguish the cell area from the background, so as to obtain a mask map;

[0024] Calculating the horizontal and vertical distances of each cell pixel relative to the centroid of the enhanced feature map using horizontal and vertical distance branches to obtain a distance map;

[0025] The mask image and the distance image are processed using a convolutional layer to obtain physical morphological features.

[0026] Its further technical solution is: the function prediction model is obtained by training the LightGBM model with a sample set composed of several different types of physical morphological features.

[0027] The further technical solution is: the function prediction model is obtained by training the LightGBM model with a sample set consisting of several different types of physical morphological features, including:

[0028] The current model is obtained by training the LightGBM model using a sample set composed of several different types of physical morphological features;

[0029] Set the initial prediction value to the average or log-odds mean of the target value in the training set;

[0030] According to the difference between the current model prediction value and the actual value, the negative gradient in each iteration is calculated as the pseudo residual;

[0031] Train a new decision tree using the pseudo residuals as the target variable;

[0032] Determine the optimal output value for each leaf node;

[0033] The new decision tree is weighted by the learning rate and added to the current model;

[0034] Determine whether the current round has reached the set value;

[0035] If the current round reaches the set value, the weighted sum of all decision trees is added to the initial prediction value to obtain a function prediction model;

[0036] If it is determined that the current round has not reached the set value, the difference between the current model predicted value and the actual value is executed, and the negative gradient in each iteration is calculated as a pseudo residual.

[0037] The present invention also provides a cell image segmentation and function prediction device, comprising:

[0038] an acquisition unit, used for acquiring the cell image to be processed;

[0039] a segmentation unit, configured to input the cell image to be processed into a segmentation model for segmentation to obtain physical morphological features;

[0040] A prediction unit, configured to input the physical morphological characteristics into a functional prediction model to predict parameters related to cell biological characteristics and indicators related to the efficacy of a set type, so as to obtain cell biological characteristic parameters and related indicators;

[0041] The output unit is used to output the physical morphological characteristics, cell biological characteristic parameters and related indicators.

[0042] Its further technical solution is: the segmentation unit includes:

[0043] a preprocessing subunit, configured to input the cell image to be processed into a segmentation model and perform a preprocessing operation on the cell image to be processed to obtain a preprocessing result;

[0044] a feature extraction subunit, configured to extract multi-scale spatial details and features from the preprocessing result using an encoder while retaining fine structural information to obtain a feature map;

[0045] The segmentation and refinement subunit is used to restore the image resolution of the feature map using a decoder, and to perform cell region segmentation and boundary refinement through the collaborative work of two branches to obtain physical morphological features.

[0046] The present invention further provides a computer device, comprising a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the above method when executing the computer program.

[0047] The advantages of the present invention over existing technologies are as follows: by inputting unstained brightfield cell images into a deep learning-driven segmentation model for precise instance segmentation, the present invention can effectively identify and extract the physical morphological features of each cell and cell nucleus, even in the presence of cell overlap. These features are then automatically fed into a machine learning-based functional prediction model, which can correlate cell morphology and function, quantitatively predicting parameters reflecting cell biological characteristics and indicators related to specific therapeutic effects. The entire process not only enables automated processing and analysis of brightfield cell images, but also provides data support for studying the physiological state of cells and exploring their potential functions, thereby solving the problem of accurate segmentation and functional prediction in complex cellular environments that traditional methods have difficulty coping with.

[0048] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0050] Figure 1 A schematic diagram of the process of cell image segmentation and function prediction method provided by an embodiment of the present invention;

[0051] Figure 2 A schematic diagram of sub-processes of the cell image segmentation and function prediction method provided by an embodiment of the present invention;

[0052] Figure 3 A schematic diagram of sub-processes of the cell image segmentation and function prediction method provided by an embodiment of the present invention;

[0053] Figure 4 A schematic diagram of sub-processes of the cell image segmentation and function prediction method provided by an embodiment of the present invention;

[0054] Figure 5 A schematic diagram of the structure of a segmentation model provided in an embodiment of the present invention;

[0055] Figure 6 A schematic diagram of the Hover-Net detailed network structure provided by an embodiment of the present invention;

[0056] Figure 7 A schematic block diagram of a cell image segmentation and function prediction device provided by an embodiment of the present invention;

[0057] Figure 8 A schematic block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0059] It will be understood that when used in this specification and the appended claims, the terms “comprises” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0060] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the present invention. As used in the specification and appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0061] It should be further understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0062] See also Figure 1 , Figure 1 A schematic flow chart of a cell image segmentation and function prediction method provided in an embodiment of the present invention. The cell image segmentation and function prediction method is applied to a server, which interacts with a terminal for data. The method is particularly suitable for effective instance segmentation and function prediction of mesenchymal stem cells in unstained bright field cell images, and can accurately distinguish them even in the case of cell overlap. First, a segmentation model comprising an encoder and a decoder is used to process the cell image to be analyzed, wherein the encoder uses a residual block group to extract multi-scale spatial details and fine structure information, and the decoder restores the image resolution and uses the kernel pixel branch and the horizontal and vertical distance branches to work together to refine the cell region boundaries, thereby obtaining physical morphological features. Then, these features are input into the LightGBM function prediction model trained by a variety of different types of physical morphological features to predict cell biological feature parameters and related efficacy indicators. The entire process combines the advantages of deep learning and gradient boosting decision trees to achieve full process automation from image segmentation to function prediction, effectively solves the problem of accurate segmentation in the case of cell overlap, and can associate cell morphology with biological function.

[0063] Figure 1 FIG. 1 is a flow chart of the cell image segmentation and function prediction method provided by an embodiment of the present invention. Figure 1 As shown, the method includes the following steps S110 to S140.

[0064] S110, obtaining an image of cells to be processed.

[0065] In this embodiment, first, it is necessary to determine the source of the cell image. These images are usually obtained by microscopy and are particularly suitable for unstained bright field cell images. For the application scenario in this patent, the focus is on using phase contrast images during the culture process of mesenchymal stem cells (MSCs). This type of image does not rely on specific fluorescent labels or staining techniques and is therefore more suitable for non-invasive observation.

[0066] To ensure the accuracy of subsequent segmentation and function prediction, the acquired cell images must meet certain quality standards:

[0067] Resolution: Sufficient pixel density to ensure that cells and their internal structures (such as the nucleus) can be clearly identified.

[0068] Contrast: Brightfield images may lack significant contrast, which needs to be enhanced through appropriate imaging techniques and post-processing.

[0069] Consistency: All images should maintain consistent lighting conditions, magnification, and other parameters to reduce variability due to external factors.

[0070] S120: Input the cell image to be processed into a segmentation model for segmentation to obtain physical morphological features.

[0071] In this embodiment, the physical morphological features refer to 23 quantitative descriptions of cells and their nuclei output from the segmentation model, such as size, shape, texture and other characteristics.

[0072] Specifically, if Figure 5 and Figure 6 As shown, the segmentation model includes an encoder and a decoder, the encoder includes an initial convolutional layer and a residual block group; the residual block group includes several residual blocks, and the residual block includes a batch normalization layer, a ReLU activation function and multiple convolutional layers.

[0073] The decoder includes an upsampling layer, a dense decoder unit, a skip connection layer, and a final convolutional layer; the decoder also includes a kernel pixel branch and horizontal and vertical distance branches.

[0074] In one embodiment, see Figure 2 , the above-mentioned step S120 may include steps S121 to S123.

[0075] S121 , inputting the cell image to be processed into a segmentation model, and performing a preprocessing operation on the cell image to be processed to obtain a preprocessing result.

[0076] In this embodiment, the preprocessing result refers to optimized image data obtained after normalization, cropping or scaling and noise removal of the input cell image to be processed, so as to facilitate subsequent analysis and processing.

[0077] In order to adapt to the requirements of network training and clarify the boundary conditions of pixel classification tasks, images are usually normalized. This involves adjusting the image brightness and contrast so that images from different sources have similar basic properties.

[0078] Depending on the model input size requirements, the original image may need to be cropped or scaled appropriately. The goal is to ensure that each image can enter the subsequent process at a consistent size while preserving the key structural information of the cell as much as possible.

[0079] Apply filters or other methods to remove noise interference in the image and improve the accuracy of the final segmentation.

[0080] After the above processing, an optimized cell image is obtained, which provides high-quality input data for subsequent feature extraction and segmentation.

[0081] S122. Using an encoder to extract multi-scale spatial details and features from the preprocessing result while retaining fine structure information to obtain a feature map.

[0082] In this embodiment, the feature map refers to the representation of spatial details and structural information at different scales extracted by the encoder, which is used to identify and locate cells and cell nuclei in the image.

[0083] In one embodiment, see Figure 3 , the above-mentioned step S122 may include steps S1221 to S1222.

[0084] S1221. Preliminary feature extraction is performed on the preprocessing result through an initial convolutional layer to obtain preliminary features.

[0085] In this embodiment, preliminary features refer to basic visual elements such as edges and textures extracted from the preprocessed cell images by the initial convolutional layer, which serve as the basis for further feature extraction.

[0086] As the first step in the encoder, the initial convolutional layer primarily extracts basic image features. These features may include low-level visual elements such as edges and textures, which are crucial for understanding the overall layout of cells. The output is a preliminary feature map, which lays the foundation for subsequent deeper feature extraction.

[0087] S1222: Input the preliminary features into the residual block group to capture information of different scales to obtain a feature map.

[0088] Within the residual block group, a series of downsampling and convolution operations gradually capture information at different scales, from local details to global context. This multi-level information extraction approach helps better understand complex cellular structures, especially when cells touch or overlap. The residual connection design allows deep networks to efficiently transfer gradients, making them easier to train and better preserving subtle structural information in the image. The final output feature map contains rich spatial details and feature representations, paving the way for instance segmentation in the decoder stage.

[0089] S123. Use a decoder to restore the image resolution of the feature map, and through the two branches working together, perform cell area segmentation and boundary refinement to obtain physical morphological features.

[0090] In one embodiment, see Figure 4 , the above-mentioned step S123 may include steps S1231 to S1235.

[0091] S1231. Upsample the feature map and perform feature enhancement using a dense decoder unit to obtain an intermediate feature map.

[0092] In this embodiment, the intermediate feature map is generated by performing upsampling operations (such as deconvolution or bilinear interpolation) on the original low-resolution feature map and using densely connected decoder layers to enhance the feature representation, resulting in a higher-resolution feature map containing more detailed information. This process is intended to improve the accuracy of subsequent processing stages.

[0093] S1232. Transfer and combine the feature map and the intermediate feature map using a skip connection layer to obtain an enhanced feature map.

[0094] In this embodiment, the enhanced feature map is obtained by combining the original input feature map with the intermediate feature map after upsampling and feature enhancement. The use of skip connections can effectively preserve key information in the original input, avoid information loss, and enhance the model's ability to learn features of different scales.

[0095] S1233. Use the kernel pixel branch to predict a binary mask for the enhanced feature map to distinguish the cell area from the background, so as to obtain a mask map.

[0096] In this embodiment, the mask image is a binary image generated by processing the enhanced feature map through the kernel pixel branch (i.e., a neural network branch specifically designed to identify cell regions), where each pixel value represents whether the corresponding position belongs to the cell region (usually, the cell region is marked as 1 and the background is marked as 0). This step is mainly used to accurately locate the position of cells.

[0097] S1234. Calculate the horizontal and vertical distances of each cell pixel relative to the centroid for the enhanced feature map using the horizontal and vertical distance branches to obtain a distance map.

[0098] In this example, the distance map is generated by a dedicated branch called the horizontal and vertical distance branch, which calculates the horizontal and vertical distances from each cell pixel to its cell centroid and encodes them into an image. This distance information is crucial for subsequent cell boundary refinement and morphological analysis.

[0099] S1235 . Process the mask image and the distance image using a convolutional layer to obtain physical morphological features.

[0100] The final step involves taking the mask and distance maps generated in the previous two steps as input, further extracting and fusing information through a series of convolution operations, ultimately yielding a feature representation describing the physical morphological properties of the cells. These features, including but not limited to important parameters such as area, perimeter, and shape factor, are crucial for cell classification, recognition, and disease diagnosis.

[0101] In this embodiment, if Figure 5 As shown in the figure, the above segmentation model adopts the encoding-decoding architecture of deep neural networks, which is specifically used for instance segmentation of bright field images of cells. The encoder is built on an improved version of Preact-ResNet50, and uses a convolutional neural network combined with residual connections to perform downsampling operations to extract powerful and representative features. The decoding part consists of a unified upsampling structure, including a kernel pixel (NP) branch and a horizontal-vertical distance (HoVer) branch. The NP branch generates a binary mask to identify whether the pixel belongs to the cell area or the background, providing preliminary area information; while the HoVer branch calculates the horizontal and vertical distance of each cell pixel relative to the center of mass to further separate touching or overlapping cells. These two branches work together to first separate the cell area and then refine the boundary, thereby completing the task of accurate cell instance segmentation.

[0102] To adapt to the characteristics of the input data and optimize the network design, the bright field images were preprocessed, such as normalization operations, to make them more suitable for network training and clarify the boundary conditions of the pixel classification task. The model draws on the multi-scale architecture of U-Net, processing information at different scales through two downsampling paths and a unified upsampling path. The downsampling part captures local details and global context information, and the decoding stage restores spatial resolution through gradual upsampling to ensure accurate boundary segmentation. In addition, the model uses transposed convolution operations instead of traditional fixed interpolation methods, enabling the network to adaptively optimize specific spatial relationships, improving the segmentation performance of complex cellular structures.

[0103] See also Figure 6 ,The Hover-Net core model adopts an encoder-decoder structure, which includes a shared encoder and two parallel decoder branches;

[0104] The encoder is inspired by Preact-ResNet50, specifically, the 32x downsampling of the standard ResNet50 is adjusted to a factor of 8. This is achieved by using a stride of 1 in the first convolutional layer and removing the subsequent max pooling layers, in order to preserve the fine spatial details that are critical for the segmentation task.

[0105] Its structure includes:

[0106] Initial convolutional layer: a 7x7 convolutional layer.

[0107] Residual Blocks: Contains four main stages, applied to 1x, 2x, 4x, 8x downsampling levels relative to the input size after the initial convolution.

[0108] The number of residual units in each stage is 3, 4, 6, and 3 respectively.

[0109] The number of feature map channels increases with the stage: 256, 512, 1024, 2048.

[0110] The residual unit adopts a pre-activation design, including batch normalization (Batch Norm), ReLU activation function, 1x1 and 3x3 convolutional layers, and contains skip connections.

[0111] The decoder branch starts from the final feature map of the encoder and branches into three parallel branches, as follows:

[0112] The general decoder structure includes:

[0113] Upsampling layer: Use nearest neighbor upsampling to increase spatial resolution.

[0114] Dense Decoder Units: Inspired by DenseNet, but may simplify the representation.

[0115] Skip connection layer: Features from the corresponding stage of the encoder are passed through the skip connection and added element-wise with the upsampled features in the decoder.

[0116] Final convolutional layer: A 1x1 convolutional layer is used at the end of each branch to generate the final prediction map.

[0117] Effective convolution is used throughout the decoder, resulting in an output spatial dimension that is smaller than the input dimension.

[0118] Specific branches include:

[0119] The cell / nucleus pixel branch performs binary segmentation (cell / nucleus vs. background), and outputs an 80x80 image (after Softmax, representing the probabilities of the two categories).

[0120] The HoVer branch regresses the horizontal and vertical distances to the centroid of the nucleus, outputting two 80x80 images (one for horizontal distance and one for vertical distance, values ​​are usually normalized to between -1 and 1).

[0121] Through the collaborative work of the NP branch and the HoVer branch, the model can effectively distinguish cells that are touching or overlapping, which is difficult to achieve with traditional methods. This ability is crucial for improving the accuracy of subsequent analysis steps. The use of transposed convolution operations instead of traditional fixed interpolation methods allows the network to adaptively optimize specific spatial relationships, thereby improving the segmentation performance of complex cellular structures.

[0122] By using a shared encoder to process two parallel decoder branches (NP and HoVer), not only does it reduce total training time, but it also fully utilizes the information shared between tasks, significantly improving the accuracy and robustness of cell instance segmentation. Drawing on the multi-scale architecture of U-Net, it can effectively capture local details and global contextual information, ensuring more accurate boundary segmentation while restoring spatial resolution.

[0123] By improving the standard ResNet50, including reducing the downsampling rate to retain fine spatial details that are critical to the segmentation task, and adopting residual units with a pre-activation design, the feature extraction capability of the model is greatly enhanced.

[0124] Using the physical morphological features output by the segmentation model as input, the prediction model can quantitatively predict various biological properties of cells, providing new tools and perspectives for understanding cellular function and its changes. The high degree of automation means that the entire process from image segmentation to feature prediction requires almost no human intervention, which greatly accelerates the experimental process and helps quickly obtain research results.

[0125] In summary, this combined artificial intelligence model has significant advantages in improving the accuracy of cell image analysis, simplifying workflows, saving time and resources, and opening up new avenues for biomedical research.

[0126] S130, inputting the physical morphological characteristics into a functional prediction model to predict cell biological characteristic related parameters and indicators related to the set type of therapeutic effect, so as to obtain cell biological characteristic parameters and related indicators.

[0127] In this embodiment, cell biological characteristic parameters refer to the quantitative physical characteristics obtained from cell image analysis, and related indicators refer to quantitative measurements closely related to specific biological functions or therapeutic effects. By inputting these physical morphological characteristics into the functional prediction model trained based on LightGBM, the biological characteristic parameters of cells and related indicators for evaluating specific types of therapeutic effects can be predicted and obtained.

[0128] Cell biological characteristic parameters refer to parameters that can be used to describe the physiological state of cells, quantified by analyzing the physical morphological characteristics of cells (such as size, shape, and texture). Related indicators refer to quantitative standards or thresholds that are further calculated or predicted based on these cell biological characteristic parameters and are used to evaluate the effectiveness of specific treatments on cells. For example, these indicators may include measurements directly related to therapeutic efficacy, such as cell survival rate and drug sensitivity scores.

[0129] That is, cell biological characteristic parameters refer to the quantitative features obtained from cell image analysis, such as cell area, circumference, diameter, roundness, texture, etc. These characteristics can reflect the physical morphology of cells.

[0130] Relevant indicators refer to quantitative measurements related to specific biological functions or disease treatment effects, such as "Th1 inhibition rate", "TNF-α inhibition rate", "IDO1 positive cell ratio", etc., which reflect the role of cells in immune regulation or other biological processes.

[0131] The functional prediction model is a LightGBM model trained on a sample set consisting of different types of physical morphological features. This model is used to predict cell biological characteristic parameters and indicators related to the efficacy of the set type.

[0132] The function prediction model is obtained by training the LightGBM model with a sample set consisting of several different types of physical morphological features.

[0133] In one embodiment, the function prediction model is obtained by training a LightGBM model using a sample set composed of several different types of physical morphological features, including the following steps S131 to S138.

[0134] S131. A LightGBM model is trained using a sample set consisting of several different types of physical morphological features to obtain the current model.

[0135] In this embodiment, a LightGBM model is trained using a dataset containing multiple physical morphological features to establish a preliminary function prediction model.

[0136] S132. Set the initial prediction value to the average or log-probability mean of the target value of the training set.

[0137] In this embodiment, typically, for regression tasks, the initial prediction value is set to the average of the target values ​​in the training set; for classification tasks, it is set to the mean of the log-odds.

[0138] S133. Based on the difference between the current model prediction value and the actual value, the negative gradient in each iteration is calculated as a pseudo residual.

[0139] In this embodiment, the negative gradient in each iteration is calculated as a pseudo residual based on the difference between the current model's prediction and the actual value. This is to determine the direction for further improvement.

[0140] S134. Use the pseudo residual as the target variable to train a new decision tree.

[0141] In this embodiment, the calculated pseudo residual is treated as a new target variable and used to train a new decision tree. This tree aims to correct the error left by the previous model.

[0142] S135. Determine the optimal output value of each leaf node.

[0143] In this embodiment, for each leaf node, the optimal output value that minimizes the total loss of all samples falling into the node is found.

[0144] S136: Add the new decision tree to the current model after weighting it according to the learning rate.

[0145] In this embodiment, the newly generated decision tree is weighted according to the learning rate and then added to the existing model to gradually optimize the entire prediction model.

[0146] S137, determining whether the current round has reached the set value;

[0147] S138. If the current round reaches the set value, the weighted sum of all decision trees is added to the initial prediction value to obtain a function prediction model;

[0148] If it is determined that the current round has not reached the set value, step S133 is executed.

[0149] If the preset maximum number of iterations is reached (ie the current round reaches the set value), the weighted sum of all decision trees plus the initial prediction value is added to obtain the final function prediction model; otherwise, step S133 is continued to perform the next round of iteration.

[0150] This approach effectively builds an accurate and efficient predictive model that can not only identify complex nonlinear relationships but also process large datasets, thereby achieving precise predictions of cell biological properties. This technology has important applications in medical research, such as helping to understand disease mechanisms and evaluate the effectiveness of treatment options.

[0151] In this embodiment, the function prediction model is based on the concept of Gradient Boosting Decision Tree (GBDT), and the LightGBM framework is used to analyze the relationship between cell morphological characteristics and various immune regulation or biological function indicators, such as "Th1 inhibition rate", "TNF-α inhibition rate" and "IDO1 positive cell ratio". LightGBM discretizes the feature values ​​through a bucketing strategy and uses a tree structure that grows by leaves to effectively reduce the computational cost while ensuring high prediction accuracy. In addition, an early stopping mechanism is applied during training to avoid overfitting. This method can gradually approach the optimal solution, thereby exploring the nonlinear correlation between the cell and its nuclear morphological characteristics and the immune regulation function, and provides an efficient and stable prediction method for parsing the potential connection between cell morphological characteristics and biological effects.

[0152] Specifically, LightGBM is based on the basic framework of GBDT and is an additive model constructed through iterative learning:

[0153] Initialization: Start with a simple initial prediction, usually the mean of the training set target value (regression task) or the mean of the logarithmic probability (classification task). The formula is: F0(x) = argmin γ ∑L(y i ,γ); where x is the input feature, y is the predicted value (corresponding to the biological feature), and γ is a hyperparameter used to control the minimum gain of splitting.

[0154] Iteratively build decision trees (Boosting Rounds): In each round m (from 1 to M, where M is the number of total trees), perform the following steps:

[0155] Calculate negative gradient (pseudo residual): According to the loss function L(y, F) of the current ensemble model, the negative gradient of the model prediction value is calculated, which represents the difference or improvement direction between the current model prediction value and the true value.

[0156] Fitting a decision tree: Using these negative gradients as target values, a new decision tree is trained to fit these pseudo residuals. This is the core of LightGBM optimization.

[0157] Calculate the leaf node output value: For each leaf node of the newly generated tree, calculate an optimal output value that minimizes the total loss of all samples falling into the leaf node.

[0158] Update the ensemble model: Add the newly generated decision trees (multiplied by a learning rate η, also called shrinkage) to the existing ensemble model.

[0159] Final Model: After M rounds of iteration, the final prediction model is the weighted sum of all decision tree predictions (plus the initial prediction). This process is essentially a gradual refinement process, with each round attempting to correct the errors left by the previous model by building a new decision tree. This approach not only improves model accuracy but also ensures computational efficiency.

[0160] Furthermore, in the Gradient Boosted Decision Tree (GBDT) framework, efficiently and accurately constructing a decision tree in each round is its core. LightGBM introduces key innovations in this regard, making it faster and using less memory than traditional GBDT (such as some modes of XGBoost).

[0161] The first is a leaf-wise (leaf-first) decision tree growth strategy. Traditional decision tree building methods, such as CART or XGBoost's level-wise strategy, split the tree layer by layer. This means all leaf nodes on the same layer are attempted to be split, and the split with the highest gain is selected. This approach makes it easy to control the depth of the tree, thus avoiding overfitting, but it is inefficient because many splits do not contribute much to the model.

[0162] In contrast, LightGBM adopts a leaf node priority strategy, which finds the node with the largest split gain (Delta Loss) from all current leaf nodes each time and repeats this process. This enables LightGBM to reduce errors and improve accuracy faster with the same number of iterations because it focuses on optimizing the most promising nodes. However, this may also lead to the generation of deeper decision trees, so parameters (such as max_depth, min_data_in_leaf, etc.) are needed to prevent overfitting. The resulting single tree may be unbalanced, with some branches being very deep and others relatively shallow.

[0163] The feature splitting point search based on histogram is as follows: the traditional method or XGBoost's exact greedy algorithm requires sorting all values ​​of each feature and then traversing all possible splitting points to calculate the information gain, which is computationally expensive.

[0164] LightGBM discretizes continuous floating-point feature values ​​into a series of buckets (Bins) and constructs a histogram of the features based on this. When looking for the best split point, only these limited number of buckets need to be traversed instead of all the original values. This method not only greatly reduces the required memory (because storing histograms is much more space-efficient than storing sorted original values), but also significantly speeds up the amount of computation (the amount of computation required to traverse a bin is much less than traversing all sorted values). In addition, histogram subtraction can further speed up the gain calculation. It is worth noting that this does not change the logical structure of the tree, but optimizes the process of finding the best split point.

[0165] The second is GOSS (Gradient-based One-Side Sampling). Specifically, in GBDT, the gradients of different samples vary in size. Samples with larger absolute gradient values ​​usually mean larger prediction errors and contribute more to model training; conversely, samples with small gradients may indicate that the model has already fitted these samples well, or are noise themselves. GOSS retains all samples with large absolute gradient values ​​and randomly samples from samples with small absolute gradient values. In order to keep the data distribution unchanged, when calculating the information gain, an amplified weight is given to the sampled small gradient samples. This can significantly reduce the amount of data used to calculate the information gain with little loss of accuracy, thereby accelerating the training process. This is a sampling optimization of the data used to calculate the split point gain when building the decision tree in each round, reducing the amount of calculation.

[0166] The third is EFB (Exclusive Feature Bundling). The features in high-dimensional data are usually sparse, and there may be a large number of mutually exclusive features (Exclusive Features), that is, they rarely (or never) take non-zero values ​​at the same time (such as features after One-Hot encoding). The EFB algorithm identifies these mutually exclusive or nearly mutually exclusive features and bundles them into a single "super feature". When constructing a histogram, you only need to build a histogram for this super feature. By cleverly designing the offset, the original different features can be distinguished within a histogram. Doing so greatly reduces the number of effective features and reduces the computational complexity of constructing histograms and finding splitting points, which is especially suitable for high-dimensional sparse data. This is a method of preprocessing features before or during training, reducing the effective feature dimensions involved in decision tree construction.

[0167] In short, LightGBM achieves higher efficiency and lower resource consumption than the traditional GBDT model through innovations in decision tree growth strategy, feature splitting point search, sample sampling, and feature processing.

[0168] S140: Output the physical morphological characteristics, cell biological characteristic parameters and related indicators.

[0169] In this embodiment, the above-mentioned physical morphological characteristics, cell biological characteristic parameters and related indicators are output to the terminal for display.

[0170] For the method of this embodiment, by inputting a bright field phase contrast image of mesenchymal stem cell (MSC) culture, the physical morphological characteristics of cells and their nuclei are automatically extracted and analyzed, and their biological properties are further predicted. The segmentation model receives the bright field phase contrast image as input, uses a trained deep neural network to parse the image, and identifies and segments the outlines of cells and cell nuclei. A segmented image of the cells and cell nuclei is generated, and 23 physical morphological features (such as size, shape, texture, etc.) are calculated. These features are crucial for subsequent functional prediction.

[0171] The prediction model uses 23 physical morphological features obtained from the segmentation model as input data and analyzes the relationship between these features and specific biological properties through a pre-trained machine learning model. It predicts 15 key cell biological characteristic parameters that reflect the physiological state and potential function of the cell.

[0172] The application process is as follows: The user provides a brightfield phase-contrast image of an MSC culture. The segmentation model processes the input image, automatically identifying and segmenting the outlines of cells and nuclei. It then extracts and calculates 23 physical morphological features describing the cells and their nuclei. These physical morphological features are automatically transferred to the prediction model. Based on these features, the prediction model quantitatively predicts 15 parameters reflecting the biological properties of the cells, including but not limited to immunomodulatory capacity and differentiation potential.

[0173] The application of this combined AI model significantly simplifies the tedious manual steps in traditional cell analysis, improving analysis efficiency and accuracy. It not only provides researchers with a new way to rapidly obtain detailed cellular information, but also lays the foundation for further exploration of cell biology mechanisms, disease diagnosis, and treatment optimization. In particular, in the field of personalized medicine, this model can help doctors develop more precise and effective treatment strategies tailored to the patient's cellular characteristics, thereby improving treatment success rates and patient satisfaction.

[0174] In summary, this combined AI model brings innovative solutions to modern biomedical research by integrating advanced image processing technology and machine learning methods, demonstrating great potential in promoting scientific research and technological progress.

[0175] For example:

[0176] By analyzing bright-field phase-contrast images of mesenchymal stem cells (MSCs) in culture, a non-invasive method is provided to evaluate the morphological characteristics of cells and their nuclei, and further predict their biological properties and clinical treatment efficacy indicators.

[0177] The user first provides a bright field phase contrast image of the MSC culture process; after receiving the input image, accurate segmentation of cells and cell nuclei is automatically performed.

[0178] Outputs include: segmented cells and predicted nuclei images;

[0179] 23 morphological features were extracted from these segmented images; these features describe the geometry, size, shape, and other properties of cells and nuclei segmented from brightfield and phase contrast images. They serve as the basis for subsequent functional prediction.

[0180] Using the 23 morphological features extracted above as input, in-depth analysis was performed to predict 17 key cell biological characteristic parameters and 15 clinical treatment efficacy indicators for systemic lupus erythematosus (SLE), as shown in Table 2.

[0181] Table 2. Cell biological characteristic parameters and clinical treatment efficacy indicators

[0182]

[0183]

[0184] Cell biological characteristic parameters include but are not limited to CD29+, CD90+, Th1 cell inhibition rate, etc., which reflect the expression of cell surface markers and immune regulation ability.

[0185] Clinical treatment efficacy indicators include renal coefficient, ds-DNA (U / mL), ANA (U / mL), etc., which are mainly used to evaluate the treatment response of patients with systemic lupus erythematosus, such as the degree of organ damage and the level of inflammatory markers.

[0186] This model is unique in that it provides an efficient, non-invasive method to obtain important information about MSCs without the need for traditional invasive testing methods. This not only accelerates research progress but also improves understanding of disease states and the selection of treatment options. Especially for complex diseases like systemic lupus erythematosus, the ability to accurately predict treatment responses can help develop more personalized treatment strategies, thereby improving patients' quality of life.

[0187] In summary, this method has opened up new avenues for biomedical research by integrating advanced image processing technology and machine learning algorithms, especially showing great potential in the fields of personalized medicine and precision treatment.

[0188] The aforementioned cell image segmentation and function prediction method, by inputting unstained brightfield cell images into a deep learning-driven segmentation model for precise instance segmentation, can effectively identify and extract the physical morphological features of each cell and cell nucleus, even in the presence of cell overlap. These features are then automatically fed into a machine learning-based function prediction model, which correlates cell morphology and function, quantitatively predicting parameters reflecting cell biological properties and indicators associated with specific therapeutic effects. This entire process not only enables automated processing and analysis of brightfield cell images but also provides data support for studying cell physiological states and exploring their potential functions, thereby resolving the challenge of accurate segmentation and function prediction in complex cellular environments that traditional methods struggle to address.

[0189] Figure 7 FIG is a schematic block diagram of a cell image segmentation and function prediction device 300 provided by an embodiment of the present invention. Figure 7 As shown, corresponding to the above cell image segmentation and function prediction method, the present invention also provides a cell image segmentation and function prediction device 300. The cell image segmentation and function prediction device 300 includes a unit for executing the above cell image segmentation and function prediction method, and the device can be configured in a server. Specifically, please refer to Figure 7The cell image segmentation and function prediction device 300 includes an acquisition unit 301 , a segmentation unit 302 , a prediction unit 303 and an output unit 304 .

[0190] The acquisition unit 301 is used to acquire the cell image to be processed; the segmentation unit 302 is used to input the cell image to be processed into a segmentation model for segmentation to obtain physical morphological features; the prediction unit 303 is used to input the physical morphological features into a functional prediction model to predict cell biological feature-related parameters and indicators related to the set type of therapeutic effect to obtain cell biological feature parameters and related indicators; the output unit 304 is used to output the physical morphological features, cell biological feature parameters and related indicators.

[0191] In one embodiment, the segmentation unit 302 includes:

[0192] The preprocessing subunit is used to input the cell image to be processed into the segmentation model and perform preprocessing operations on the cell image to be processed to obtain a preprocessing result; the feature extraction subunit is used to use an encoder to extract multi-scale spatial details and features from the preprocessing result while retaining fine structural information to obtain a feature map; the segmentation and refinement subunit is used to use a decoder to restore the image resolution of the feature map, and through the collaborative work of the two branches, perform cell area segmentation and boundary refinement to obtain physical morphological features.

[0193] In one embodiment, the feature extraction subunit includes:

[0194] A preliminary extraction module is used to preliminarily extract features from the preprocessing results through an initial convolution layer to obtain preliminary features; a residual processing module is used to input the preliminary features into a residual block group to capture information of different scales to obtain a feature map.

[0195] In one embodiment, the segmentation and refinement subunit includes:

[0196] An upsampling module is used to upsample the feature map and use a dense decoder unit to perform feature enhancement to obtain an intermediate feature map; a skip connection module is used to transfer and combine the feature map and the intermediate feature map using a skip connection layer to obtain an enhanced feature map; a binary mask prediction module is used to use a kernel pixel branch to predict a binary mask for the enhanced feature map to distinguish the cell area from the background to obtain a mask map; a distance calculation module is used to use horizontal and vertical distance branches to calculate the horizontal and vertical distances of each cell pixel relative to the center of mass of the enhanced feature map to obtain a distance map; a convolution processing module is used to process the mask map and the distance map using a convolution layer to obtain physical morphological features.

[0197] In one embodiment, the above-mentioned device further includes:

[0198] The prediction model training unit is used to train the LightGBM model using a sample set consisting of several different types of physical morphological features to obtain the current model;

[0199] Set the initial prediction value to the average or log-odds mean of the target value in the training set;

[0200] According to the difference between the current model prediction value and the actual value, the negative gradient in each iteration is calculated as the pseudo residual;

[0201] Train a new decision tree using the pseudo residuals as the target variable;

[0202] Determine the optimal output value for each leaf node;

[0203] The new decision tree is weighted by the learning rate and added to the current model;

[0204] Determine whether the current round has reached the set value;

[0205] If the current round reaches the set value, the weighted sum of all decision trees is added to the initial prediction value to obtain a function prediction model;

[0206] If it is determined that the current round has not reached the set value, the difference between the current model predicted value and the actual value is executed, and the negative gradient in each iteration is calculated as a pseudo residual.

[0207] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned cell image segmentation and function prediction device 300 and each unit can refer to the corresponding description in the aforementioned method embodiment. For the convenience and brevity of description, it will not be repeated here.

[0208] The cell image segmentation and function prediction device 300 can be implemented in the form of a computer program. Figure 8 Runs on the computer equipment shown.

[0209] See also Figure 8 , Figure 8 1 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 500 may be a server, wherein the server may be an independent server or a server cluster composed of multiple servers.

[0210] See Figure 8The computer device 500 includes a processor 502 , a memory, and a network interface 505 connected via a system bus 501 , wherein the memory may include a non-volatile storage medium 503 and an internal memory 504 .

[0211] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions, which, when executed, can enable the processor 502 to perform a cell image segmentation and function prediction method.

[0212] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500.

[0213] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a cell image segmentation and function prediction method.

[0214] The network interface 505 is used to communicate with other devices through the network. Figure 8 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 500 to which the solution of the present application is applied. The specific computer device 500 may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0215] The processor 502 is configured to execute a computer program 5032 stored in the memory to implement the following steps:

[0216] Acquire a cell image to be processed; input the cell image to be processed into a segmentation model for segmentation to obtain physical morphological features; input the physical morphological features into a functional prediction model to predict cell biological feature-related parameters and indicators related to the set type of therapeutic effect to obtain cell biological feature parameters and related indicators; output the physical morphological features, cell biological feature parameters and related indicators.

[0217] The segmentation model includes an encoder and a decoder, the encoder includes an initial convolutional layer and a residual block group; the residual block group includes several residual blocks, and the residual block includes a batch normalization layer, a ReLU activation function and multiple convolutional layers.

[0218] The decoder includes an upsampling layer, a dense decoder unit, a skip connection layer, and a final convolutional layer; the decoder also includes a kernel pixel branch and horizontal and vertical distance branches.

[0219] The function prediction model is obtained by training the LightGBM model with a sample set consisting of several different types of physical morphological features.

[0220] In one embodiment, when the processor 502 implements the step of inputting the cell image to be processed into the segmentation model for segmentation to obtain physical morphological features, the processor 502 specifically implements the following steps:

[0221] The cell image to be processed is input into a segmentation model, and a preprocessing operation is performed on the cell image to be processed to obtain a preprocessing result; an encoder is used to extract multi-scale spatial details and features from the preprocessing result while retaining fine structural information to obtain a feature map; a decoder is used to restore the image resolution of the feature map, and the two branches work together to segment the cell area and refine the boundaries to obtain physical morphological features.

[0222] In one embodiment, when the processor 502 implements the step of extracting multi-scale spatial details and features from the preprocessing result using an encoder while retaining fine structure information to obtain a feature map, the processor 502 specifically implements the following steps:

[0223] The preprocessing result is initially extracted with an initial convolutional layer to obtain preliminary features; the preliminary features are input into a residual block group to capture information of different scales to obtain a feature map.

[0224] In one embodiment, when the processor 502 implements the step of restoring the image resolution of the feature map using a decoder and performing cell region segmentation and boundary refinement through the collaborative work of two branches to obtain physical morphological features, the processor 502 specifically implements the following steps:

[0225] The feature map is upsampled and feature enhanced using a dense decoder unit to obtain an intermediate feature map; the feature map and the intermediate feature map are transferred and combined using a skip connection layer to obtain an enhanced feature map; a kernel pixel branch is used to predict a binary mask for the enhanced feature map to distinguish the cell area from the background to obtain a mask map; the horizontal and vertical distance branches are used to calculate the horizontal and vertical distances of each cell pixel relative to the center of mass of the enhanced feature map to obtain a distance map; the mask map and the distance map are processed using a convolutional layer to obtain physical morphological features.

[0226] In one embodiment, when the processor 502 implements the step of obtaining the function prediction model by training the LightGBM model using a sample set consisting of several different types of physical morphological features, the processor 502 specifically implements the following steps:

[0227] The LightGBM model is trained using a sample set consisting of several different types of physical morphological features to obtain the current model; the initial prediction value is set to the average or logarithmic probability mean of the target value of the training set; according to the difference between the current model prediction value and the actual value, the negative gradient in each iteration is calculated as a pseudo residual; the pseudo residual is used as the target variable to train a new decision tree; the optimal output value of each leaf node is determined; the new decision tree is weighted according to the learning rate and added to the current model; it is judged whether the current round reaches the set value; if the current round reaches the set value, the weighted sum of all decision trees is added and the initial prediction value is added to obtain a functional prediction model; if it is judged that the current round does not reach the set value, the difference between the current model prediction value and the actual value is executed, and the negative gradient in each iteration is calculated as a pseudo residual.

[0228] It should be understood that in the embodiment of the present application, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0229] Those skilled in the art will appreciate that all or part of the steps in the method of the above-described embodiment can be implemented by instructing the relevant hardware through a computer program. The computer program includes program instructions, which can be stored in a storage medium that is computer-readable. The program instructions are executed by at least one processor in the computer system to implement the steps in the method of the above-described embodiment.

[0230] Therefore, the present invention also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein when the computer program is executed by a processor, the processor performs the following steps:

[0231] Acquire a cell image to be processed; input the cell image to be processed into a segmentation model for segmentation to obtain physical morphological features; input the physical morphological features into a functional prediction model to predict cell biological feature-related parameters and indicators related to the set type of therapeutic effect to obtain cell biological feature parameters and related indicators; output the physical morphological features, cell biological feature parameters and related indicators.

[0232] The segmentation model includes an encoder and a decoder, the encoder includes an initial convolutional layer and a residual block group; the residual block group includes several residual blocks, and the residual block includes a batch normalization layer, a ReLU activation function and multiple convolutional layers.

[0233] The decoder includes an upsampling layer, a dense decoder unit, a skip connection layer, and a final convolutional layer; the decoder also includes a kernel pixel branch and horizontal and vertical distance branches.

[0234] The function prediction model is obtained by training the LightGBM model with a sample set consisting of several different types of physical morphological features.

[0235] In one embodiment, when the processor executes the computer program to implement the step of inputting the cell image to be processed into a segmentation model for segmentation to obtain physical morphological features, the processor specifically implements the following steps:

[0236] The cell image to be processed is input into a segmentation model, and a preprocessing operation is performed on the cell image to be processed to obtain a preprocessing result; an encoder is used to extract multi-scale spatial details and features from the preprocessing result while retaining fine structural information to obtain a feature map; a decoder is used to restore the image resolution of the feature map, and the two branches work together to segment the cell area and refine the boundaries to obtain physical morphological features.

[0237] In one embodiment, when the processor executes the computer program to implement the step of extracting multi-scale spatial details and features from the preprocessing result using an encoder while retaining fine structure information to obtain a feature map, the processor specifically implements the following steps:

[0238] The preprocessing result is initially extracted with an initial convolutional layer to obtain preliminary features; the preliminary features are input into a residual block group to capture information of different scales to obtain a feature map.

[0239] In one embodiment, when the processor executes the computer program to implement the steps of restoring the image resolution of the feature map using a decoder and performing cell region segmentation and boundary refinement through the collaborative work of two branches to obtain physical morphological features, the processor specifically implements the following steps:

[0240] The feature map is upsampled and feature enhanced using a dense decoder unit to obtain an intermediate feature map; the feature map and the intermediate feature map are transferred and combined using a skip connection layer to obtain an enhanced feature map; a kernel pixel branch is used to predict a binary mask for the enhanced feature map to distinguish the cell area from the background to obtain a mask map; the horizontal and vertical distance branches are used to calculate the horizontal and vertical distances of each cell pixel relative to the center of mass of the enhanced feature map to obtain a distance map; the mask map and the distance map are processed using a convolutional layer to obtain physical morphological features.

[0241] In one embodiment, when the processor executes the computer program to implement the step of training the LightGBM model using a sample set consisting of several different types of physical morphological features as the function prediction model, the processor specifically implements the following steps:

[0242] The LightGBM model is trained using a sample set consisting of several different types of physical morphological features to obtain the current model; the initial prediction value is set to the average or logarithmic probability mean of the target value of the training set; according to the difference between the current model prediction value and the actual value, the negative gradient in each iteration is calculated as a pseudo residual; the pseudo residual is used as the target variable to train a new decision tree; the optimal output value of each leaf node is determined; the new decision tree is weighted according to the learning rate and added to the current model; it is judged whether the current round reaches the set value; if the current round reaches the set value, the weighted sum of all decision trees is added and the initial prediction value is added to obtain a functional prediction model; if it is judged that the current round does not reach the set value, the difference between the current model prediction value and the actual value is executed, and the negative gradient in each iteration is calculated as a pseudo residual.

[0243] The storage medium may be any computer-readable storage medium that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk.

[0244] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the composition and steps of each example according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0245] In the several embodiments provided herein, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the various units is merely a logical functional division, and actual implementation may employ other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be omitted or not implemented.

[0246] The steps in the methods of the embodiments of the present invention may be adjusted in order, combined, or deleted as needed. The units in the devices of the embodiments of the present invention may be combined, divided, or deleted as needed. Furthermore, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit.

[0247] If this integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the existing technology, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, terminal, or network device, etc.) to execute all or part of the steps of the method described in various embodiments of the present invention.

[0248] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A cell image segmentation and function prediction method, characterized in that: include: Acquire the cell image to be processed; Inputting the cell image to be processed into a segmentation model for segmentation to obtain physical morphological features; Inputting the physical morphological characteristics into a functional prediction model to predict cell biological characteristic related parameters and indicators related to the set type of therapeutic effect, so as to obtain cell biological characteristic parameters and related indicators; Output the physical morphological characteristics, cell biological characteristic parameters and related indicators.

2. The cell image segmentation and function prediction method according to claim 1, characterized in that: The segmentation model includes an encoder and a decoder, the encoder includes an initial convolutional layer and a residual block group; the residual block group includes several residual blocks, and the residual block includes a batch normalization layer, a ReLU activation function and multiple convolutional layers. The decoder includes an upsampling layer, a dense decoder unit, a skip connection layer, and a final convolutional layer; the decoder also includes a kernel pixel branch and horizontal and vertical distance branches.

3. The cell image segmentation and function prediction method according to claim 2, characterized in that: The step of inputting the cell image to be processed into a segmentation model for segmentation to obtain physical morphological features includes: Inputting the cell image to be processed into a segmentation model, and performing a preprocessing operation on the cell image to be processed to obtain a preprocessing result; An encoder is used to extract multi-scale spatial details and features from the preprocessing result while retaining fine structure information to obtain a feature map; A decoder is used to restore the image resolution of the feature map, and two branches work together to segment the cell area and refine the boundaries to obtain physical morphological features.

4. The cell image segmentation and function prediction method according to claim 3, characterized in that: The encoder is used to extract multi-scale spatial details and features from the preprocessing result while retaining fine structure information to obtain a feature map, including: Preliminarily extracting features from the preprocessing results through an initial convolutional layer to obtain preliminary features; The preliminary features are input into the residual block group to capture information of different scales to obtain a feature map.

5. The cell image segmentation and function prediction method according to claim 3, characterized in that: The decoder is used to restore the image resolution of the feature map, and two branches work together to segment the cell area and refine the boundaries to obtain physical morphological features, including: Upsampling the feature map and performing feature enhancement using a dense decoder unit to obtain an intermediate feature map; The feature map and the intermediate feature map are transferred and combined using a skip connection layer to obtain an enhanced feature map; Using a kernel pixel branch to predict a binary mask for the enhanced feature map to distinguish the cell area from the background, so as to obtain a mask map; Calculating the horizontal and vertical distances of each cell pixel relative to the centroid of the enhanced feature map using horizontal and vertical distance branches to obtain a distance map; The mask image and the distance image are processed using a convolutional layer to obtain physical morphological features.

6. The cell image segmentation and function prediction method according to claim 1, characterized in that: The function prediction model is obtained by training the LightGBM model with a sample set consisting of several different types of physical morphological features.

7. The cell image segmentation and function prediction method according to claim 6, characterized in that: The function prediction model is obtained by training the LightGBM model with a sample set consisting of several different types of physical morphological features, including: The current model is obtained by training the LightGBM model using a sample set composed of several different types of physical morphological features; Set the initial prediction value to the average or log-odds mean of the target value in the training set; According to the difference between the current model prediction value and the actual value, the negative gradient in each iteration is calculated as the pseudo residual; Train a new decision tree using the pseudo residuals as the target variable; Determine the optimal output value for each leaf node; The new decision tree is weighted by the learning rate and added to the current model; Determine whether the current round has reached the set value; If the current round reaches the set value, the weighted sum of all decision trees is added to the initial prediction value to obtain a function prediction model; If it is determined that the current round has not reached the set value, the difference between the current model predicted value and the actual value is executed, and the negative gradient in each iteration is calculated as a pseudo residual.

8. A cell image segmentation and function prediction device, characterized in that: include: an acquisition unit, used for acquiring the cell image to be processed; a segmentation unit, configured to input the cell image to be processed into a segmentation model for segmentation to obtain physical morphological features; A prediction unit, configured to input the physical morphological characteristics into a functional prediction model to predict parameters related to cell biological characteristics and indicators related to the efficacy of a set type, so as to obtain cell biological characteristic parameters and related indicators; The output unit is used to output the physical morphological characteristics, cell biological characteristic parameters and related indicators.

9. The cell image segmentation and function prediction device according to claim 8, characterized in that: The segmentation unit includes: a preprocessing subunit, configured to input the cell image to be processed into a segmentation model, and perform a preprocessing operation on the cell image to be processed to obtain a preprocessing result; a feature extraction subunit, configured to extract multi-scale spatial details and features from the preprocessing result using an encoder while retaining fine structural information to obtain a feature map; The segmentation and refinement subunit is used to restore the image resolution of the feature map using a decoder, and to perform cell region segmentation and boundary refinement through the collaborative work of two branches to obtain physical morphological features.

10. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

Images