Cell image analysis method combining morphology and color analysis and application and instrument thereof

Abstract

The invention provides a form and color analysis combined cell image analysis method and application and instrument thereof, and belongs to the technical field of artificial intelligence. The method comprises the following steps: S1, acquiring an image sample; s2, image data preprocessing: carrying out key area cutting and size standardization operation on all collected image samples; step S3, image data annotation: carrying out centroid point selection and classification annotation on the image samples of which the sizes are adjusted; s4, establishing a cellular morphology segmentation and classification model based on the U-Net network model, and performing network training; and S5, verifying the identification effect and precision of the network on the cell information by using the verification set and the test set. The technical scheme of the invention can realize multi-dimensional analysis, is convenient, efficient and non-toxic, has the characteristics of high precision, high automation degree, strong expansibility and the like, and has a very good practical application value.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and in particular relates to a cell image analysis method combining morphology and color analysis, and an application and instrument thereof. Background Art

[0002] MTT colorimetry is a cell activity detection method widely used in biology and medicine to detect cell survival and growth. The detection principle is to use the succinate dehydrogenase in the mitochondria of living cells to reduce exogenous tetramethylthiazolium salt (MTT) to water-insoluble blue-purple crystalline formazan and deposit it in the cells, while dead cells do not have this function. By measuring the content of formazan generated in cells, the growth status and survival rate of cells can be reflected. Dimethyl sulfoxide (DMSO) can dissolve the formazan in cells, and its light absorption value is measured at a wavelength of 490nm using an enzyme-linked immunosorbent assay, which can indirectly reflect the number of living cells. Within a certain range of cell numbers, the amount of MTT crystals formed is proportional to the number of cells. This method has been widely used in the activity detection of some biologically active factors, large-scale anti-tumor drug screening, cytotoxicity tests, and tumor radiosensitivity determination. It is characterized by high sensitivity and economy.

[0003] The disadvantage of the MTT colorimetric method is that the formazan product produced by reduction is insoluble in water and must be dissolved before detection. This not only increases the workload, but also affects the accuracy of the experimental results. In addition, the organic solvent that dissolves formazan is also harmful to the experimenter.

[0004] Therefore, it is more convenient, efficient and non-toxic to use artificial intelligence methods instead of traditional MTT colorimetry. In addition, artificial intelligence methods can more accurately analyze the content of formazan crystals generated by each cell from the perspective of image recognition and classify morphological changes.

[0005] The US patent application with publication number US20230160879A1 discloses: A method for training AI for label-free cell viability determination includes the steps of providing a cell sample, obtaining a fluorescent image and a DHM image of the cell sample, determining a first cell viability of the cell sample according to the fluorescent image of the cell sample, marking the DHM image of the cell sample as a model specifying the first cell viability, and performing AI training by using a model containing the DHM image of the cell sample. However, its application scope is narrow and the accuracy is not high. Summary of the invention

[0006] In order to solve the problems existing in the prior art, the present invention uses the encoder-decoder architecture of U-Net to realize pixel-level segmentation in biomedical images. The innovation of the present invention is that it does not use traditional segmentation to identify cells, but encodes cells by marking the cell centroid point, thereby realizing the classification of cell morphology and staining intensity. By constructing and training the U-Net network model, it is possible to efficiently and accurately complete the task of identifying cell morphology and staining intensity without manual intervention.

[0007] The present invention relates to a cell image analysis method combining morphology and color analysis, which is used to replace conventional cytotoxicity detection methods and can be used for cell viability and morphology classification and identification. Compared with the existing MTT colorimetric method, the present invention mainly improves the means and methods for quantitative analysis of cell information (such as cell viability). The traditional MTT colorimetric method relies on an enzyme-linked immunosorbent assay to measure its light absorption value at a wavelength of 490nm to evaluate cell survival rate, while the present invention adopts advanced artificial intelligence technology, especially the U-Net network model, to achieve high-precision automatic recognition of cell information (such as cell viability and morphology).

[0008] By using the U-Net network model, the present invention can extract complex features from cell images and use these features to perform quantitative analysis of cell information. This method not only improves the accuracy and repeatability of detection, but also greatly shortens the experimental cycle and reduces manual intervention. In addition, the U-Net network can perform detailed classification and identification of cell information, providing richer cell information than traditional colorimetry, which helps to more deeply understand and evaluate cytotoxic reactions.

[0009] In the present invention, "cv2.imshow() function" is a function known in the art for displaying images, and its running syntax format is: cv2.imshow(winname,mat); winname refers to the window name used to display the image, which can be set by the user according to needs, but there should be no Chinese name; mat is the image to be displayed.

[0010] The technical solution of the present invention to solve the above technical problems is as follows:

[0011] A cell image analysis method combining morphology and color analysis is established based on a U-Net network model and includes the following steps:

[0012] Step S1, image sample collection;

[0013] Step S2, image data preprocessing: performing key area cropping and size standardization operations on all collected image samples;

[0014] Step S3, image data labeling: for the resized image samples, centroid points are selected and classified and labeled;

[0015] Step S4, establishing a cell morphology segmentation and classification model based on the U-Net network model, and performing network training;

[0016] Step S5: Use the validation set and the test set to verify the recognition effect and accuracy of the network on cell information.

[0017] Preferably, the method further comprises: S6, developing a software interface for quantitatively counting cell information.

[0018] Specifically, in step S1: the image sample acquisition source is multiple parallel cell culture platforms or tools, each cell culture platform or tool includes multiple independent units, each independent unit is provided with a number of parallel pipes or cell culture areas of arbitrary shapes, these pipes or areas can be connected to a total inlet and a total outlet, or parallel and independent inlets and outlets are set. The area and volume of all cell culture areas are equal to ensure that the fluid can be evenly distributed during the test.

[0019] Preferably, in step S1, the image sample collection source is a lane chip, each chip contains multiple independent units, each independent unit is provided with a plurality of parallel pipes, and these pipes are connected to an inlet and an outlet. The total length and volume of all parallel pipes are equal to ensure that the fluid can be evenly distributed during the test.

[0020] In some specific embodiments, each chip contains 9 independent units, each unit contains 6 parallel straight pipes for cell culture; the size of each straight pipe is 0.4*2.5mm, and the distance between adjacent straight pipes is 0.4mm; the design height of the straight pipe is 60μm to meet the needs of cell monolayer culture and fluid perfusion; each unit contains 1 inlet and 1 outlet, and the fluid is evenly distributed in the 6 parallel pipes; the size of each chip is 22*50mm, and 3 chip repetitive designs can be completed on each 4-inch silicon wafer by photolithography.

[0021] Specifically, the step S2 includes:

[0022] S201, cropping the chip lane structure in the key area of ​​the complete chip image;

[0023] S202, performing standardization processing on lane images of different widths to make the widths uniform; dividing the image into a plurality of equal-length segments in the length direction, ensuring that the size of each local image is maintained within n downsampling layers that are adapted to the U-Net network, where n is an integer;

[0024] S203: Perform data enhancement on the cropped and size-standardized image.

[0025] More specifically, the step S201 is completed using the OpenCV semi-automatic interactive image cropping tool, including:

[0026] (1) The program initializes a series of global variables, including the points list for storing the coordinates of the user's click position, image_index for tracking the index of the currently processed image, the image_paths list for storing the paths of the images to be processed, and scale_factor for controlling the image display ratio; scale_factor is set to 1.0 by default, indicating that the image will be displayed at its original size;

[0027] (2) The program defines a resize_image function, which receives an image object as input and adjusts the image size according to the set scale_factor to ensure that the image has a suitable size ratio when displayed, which is convenient for users to observe and operate;

[0028] (3) The program defines the get_points callback function, which is used to respond to the user's mouse click event. When the user clicks the left mouse button on the image, the function will calculate the coordinates of the click position in the original image size and record these coordinates in the points list to provide necessary data support for subsequent cropping operations.

[0029] (4) Entering the main processing logic, the program traverses all image file paths in the specified folder and stores them in the image_paths list; for each image in the list, the program first tries to read the image and checks whether it is successfully loaded; if the reading fails, an error message is output and the image is skipped; if the reading is successful, the points list is cleared to prepare for processing a new image;

[0030] (5) Before displaying the image, the program uses the cv2.imshow() function to render the image and sets the mouse callback function to get_points, so that the user can mark the area to be cropped by clicking on the screen; in addition, the program provides a series of keyboard shortcut commands, and the user can adjust the image display magnification or reduction ratio by pressing the '+' or '-' key to more accurately locate the cropping boundary;

[0031] (6) When the user has completed the selection of four vertices, he can press the 'q' key to determine the cropping area; at this time, the program will calculate the minimum rectangle surrounding these points based on the selected four vertices and crop the image accordingly. The cropped image will be saved to the specified location; if the user wants to continue processing the next image, he can switch to the next image by pressing the 'n' key.

[0032] Preferably, the size of the local image in step S202 is 768 pixels×512 pixels.

[0033] Further specifically, step S203 includes: defining a set of data enhancement operations, including at least rotation, flipping, brightness adjustment, and contrast change, to simulate various variations of images in a real environment, thereby enriching the training data set; for each image that has been cropped and size-normalized, the program sequentially executes the above-defined data enhancement operations; the rotation includes rotating the image by a random angle to simulate different viewing angles; the flipping includes flipping the image horizontally or vertically to increase the diversity of samples; the brightness adjustment includes adjusting the brightness of the image; the contrast change includes adjusting the contrast of the image to cope with the effects under different lighting conditions; in the process of performing data enhancement, ensure that each transformation keeps the key features of the original image unchanged to avoid the image losing its original biological meaning or information due to excessive enhancement; at the same time, to ensure the effectiveness of data enhancement, multiple variants will be generated for each original image, thereby increasing the size of the training data set.

[0034] Specifically, the step S3 includes:

[0035] S301, annotate the prepared image data set, and organize training samples to establish a training data set;

[0036] S302. Create a data set. The annotated images are organized in a file structure. The training data includes source images in JPG format and corresponding label images in PNG format, where each source image is associated with 4 label images. The size of the label image is the same as the input image and is used for key point detection of the cell centroid. Each cell centroid point position in the image has a corresponding Gaussian distribution peak, and the surrounding pixel values ​​gradually decrease. The model learns the specific position of the key points and has the ability to extract and learn the surrounding pixel features. In the data loading stage, the source image and label image are loaded according to the index and converted into PyTorch tensor format.

[0037] More specifically, the step S301 includes: implementing image data annotation by writing Python code, annotating the cell centroid and morphological category of each cell image;

[0038] First, define a set of category names TypeName to describe the morphological characteristics of different cells, including "normal morphology and color - true positive", "abnormal morphology and no color - true negative", "normal morphology but no color - false negative" and "abnormal morphology but color - false positive", to help users clearly identify and select various cell types when annotating images;

[0039] After opening the image selection folder to start image cell annotation, double-click the cell center to achieve cell annotation; at this time, the software will place a circular annotation point of different colors at the selected cell position to indicate the cell position and its category; the color of the annotation point changes according to the category of the selected cell; if you need to remove a marked cell point, you can double-click the annotation point with the right mouse button, so that the software will remove the point and delete the corresponding information from the annotation list; the tool also includes a "show center" option. When the user checks this option, all detected cell center points will be presented on the image in a visual way, which is convenient for quickly viewing the core position of each cell; there is also a text box on the tool interface to display the total number of cells detected in the current image; the tool also provides image navigation functions, such as forward and backward page buttons and image progress bar sliders, which are convenient for browsing other images in the same folder; the "show3D" button displays image coordinates and pixels in three dimensions; at the same time, a save function is provided, allowing users to save marked images or export the annotated data as Excel tables for further data analysis and processing.

[0040] Preferably, the colors of the marked points are: green represents "normal morphology and color - true positive", blue represents "abnormal morphology and no color - true negative", yellow represents "abnormal morphology but color - false positive" and red represents "normal morphology but no color - false negative".

[0041] Specifically, the step S4 includes:

[0042] S401, select the U-Net network model and make appropriate adjustments, and customize a network structure according to the complexity of the task;

[0043] The network structure includes at least a convolution layer, a pooling layer, an activation function layer, and an output layer;

[0044] During training, within the specified training cycle, traverse the training data set, perform forward propagation to calculate the predicted value, calculate the loss, and perform back propagation to update the network parameters; regularly evaluate the performance of the model on the validation set and save the best model; select the mean square error function as the loss function, and set a learning rate scheduler in the training parameters to reduce the learning rate after a certain period of time.

[0045] More specifically, the network structure includes:

[0046] First, there are convolutional layers and pooling layers. The first convolutional layer layer_00 has an input channel of 3, corresponding to the three color channels of the RGB image, an output channel of L / 4, a convolution kernel size of 3×3, and a padding of 1, keeping the size of the output feature map the same as the input.

[0047] The activation function layer uses the ReLU activation function, which only retains positive values ​​and sets negative values ​​to 0.

[0048] The pooling layer uses the maximum pooling (Max Pooling), the pooling window size is 2×2, and the step size is also 2, which reduces the size of the feature map;

[0049] Next is a combination of consecutive convolutional layers and pooling layers. Each group includes two layers of convolution operations, and the output channels are L / 2, L, 2L, 4L, 8L, and 16L respectively. Each convolution layer is followed by a ReLU activation function, followed by a maximum pooling layer with a pooling window size of 2×2 and a stride of 2. These layers gradually increase the number of output channels while reducing the size of the feature map to extract more advanced features.

[0050] After the encoder part, the network enters the decoder part, and gradually restores the size of the feature map through upsampling operations, and splices it with the feature map of the corresponding layer of the encoder to fuse multi-level information; the upsampling operation uses a bilinear interpolation method to double the size of the feature map; after each layer of upsampling, the feature map size is restored through a convolutional layer (the output channels are gradually reduced to 8L, 4L, 2L, L), and followed by a ReLU activation function to activate the features;

[0051] The final output layer layer_41 has an input channel of L, an output channel of the number of classification categories, a convolution kernel size of 1×1, a step size of 1, and a padding of 0; the Sigmoid activation layer uses Sigmoid as the activation function and outputs the probability that each pixel belongs to each category.

[0052] Specifically, the step S5 includes: recording the average loss of each Epoch during the training process, performing verification after each Epoch, and drawing a training loss and verification accuracy curve. Preferably, the cell information includes but is not limited to cell viability and morphology classification.

[0053] Specifically, the step S6 includes: using the PyQt5 framework to develop a software interface for quantitative statistics of cell viability and morphological classification; providing users with an intuitive operating platform to effectively collect and analyze cell viability data and classify cell morphology; the interface is used to display cell images and their corresponding statistical data, and to help users perform complex analysis operations, the analysis operations including at least any one or more of cell counting, viability assessment, and classification of morphological features.

[0054] In some specific embodiments, the method comprises the following steps:

[0055] Step S1: Image sample collection, reference Figure 2 , Figure 3 ;

[0056] Step S2: Image data preprocessing: perform key area cropping and size standardization operations on all collected image samples; the image preprocessing process is as follows Figure 4 shown.

[0057] Step S3: Image data annotation: Use the customized cell annotation tool to select the centroid points and perform four classification annotations on the resized chip experimental result images. Figure 5 As shown, the annotation tools are Figure 6 shown.

[0058] Step S4: Establish a cell morphology segmentation and classification model based on the U-Net network model and perform network training.

[0059] Step S5: Use the validation set and the test set to verify the recognition effect and accuracy of the network for cell viability and morphology classification.

[0060] Step S6: Develop a software interface for quantifying cell viability and morphological classification, such as Figure 7 shown.

[0061] On the other hand, the present invention also provides application of the aforementioned cell image analysis method in cell viability analysis.

[0062] The cells are cells that can grow on the wall and are convenient for adding drugs and observation, including but not limited to cells derived from animals, plants or microorganisms.

[0063] The cell viability analysis is used for any one or more of activity detection of bioactive factors, anti-tumor drug screening, cytotoxicity testing, tumor radiosensitivity determination, gene expression regulation research and cell metabolism analysis.

[0064] In another aspect, based on the method of the present invention, a cell image analysis instrument is also provided, which includes a program for running all the steps and parameters in the method. In some specific embodiments, the instrument is used to analyze cell viability.

[0065] Beneficial effects of the present invention:

[0066] U-Net is generally used for cell image segmentation, but this technical solution applies it to cell classification and is a multi-dimensional comprehensive classification of cell morphology and staining results, and performs cell counting to achieve multi-dimensional analysis, which is convenient, efficient and non-toxic. Specifically:

[0067] 1. The present invention achieves high-precision cell classification by adopting the U-Net deep learning framework and combining customized image preprocessing steps (such as size standardization, key area cropping, and cell centroid annotation tools).

[0068] 2. The method of the present invention has a high degree of automation and realizes a semi-automatic closed-loop workflow from image acquisition to processing and then to classification, which reduces the necessity of human intervention and improves work efficiency.

[0069] 3. The method of the present invention has good scalability and adaptability. It is not only suitable for the research of specific types of cells, but also easy to expand and apply to the research of other types of cells. By designing a flexible image annotation mechanism and configurable model training parameters, the system can adapt to the morphological characteristics of different types of cells, thereby enhancing the scope of application of the system.

[0070] 4. The present invention designs an intuitive and easy-to-use operating interface, provides a user-friendly graphical interface, simplifies the data analysis process, and enables non-professionals to easily complete complex image processing tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 The figure is a flow chart of the cell image analysis method of the present invention.

[0072] Figure 2 The figure is a schematic diagram of the structure of a chip used in an example embodiment.

[0073] Figure 3 This is a diagram showing the chip experiment results.

[0074] Figure 4 Determine the four-point cropping diagram for image preprocessing.

[0075] Figure 5 Schematic diagram of four cell classification labels.

[0076] Figure 6 Show diagram for the cell annotation tool.

[0077] Figure 7 This is a diagram showing the interface of the MTT cell intelligent analysis system.

[0078] All the attached drawings are screenshots of the operation interface. Some unclear details will not affect the effect determination. DETAILED DESCRIPTION

[0079] The present invention will be further described in detail below in conjunction with specific embodiments. The following embodiments are not intended to limit the present invention, but are only intended to illustrate the present invention. The described embodiments are not all embodiments of the present invention, and all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present invention. The experimental methods used in the following examples are generally conventional under normal conditions, and the materials, reagents, etc. used in the following examples are commercially available unless otherwise specified.

[0080] In a specific embodiment of the present invention, as an example, a microfluidic-based The cytotoxicity test chip is used to verify the technical effect of the present invention. In fact, those skilled in the art can also select other suitable chip types in combination with the cell image analysis method of the present invention according to actual needs. Those skilled in the art can anticipate the corresponding technical effects and have the ability to apply the cell image analysis method of the present invention to other scenarios.

[0081] Example 1

[0082] The present invention provides a microfluidic-based Qualitative and quantitative detection and analysis methods of cell viability of cytotoxicity test chips, such as Figure 1 As shown (flow chart), including:

[0083] Step S1, image sample collection: This technical solution adopts a lane-type chip structure, such as Figure 2 shown.

[0084] The specific operation process of the chip experiment is as follows:

[0085] Prepare the materials and reagents required for the MTT experiment, such as FB cell suspension, lane chip, cell culture medium, MTT solution and related consumables. Use a digital syringe pump to perfuse the cells into the chip and incubate in a 37°C, 5% carbon dioxide incubator for 4 hours. Change to drug culture medium and incubate overnight. After the incubation, wash once with PBS, perfuse MTT culture medium, incubate in a 37°C, 5% carbon dioxide incubator for 4 hours, and use a microscope to adjust the magnification and exposure to take pictures of the cells in the chip to obtain image data.

[0086] Step S2, image data preprocessing: perform key area cropping and size standardization operations on all collected image samples.

[0087] Step S3, image data annotation: using a customized cell annotation tool, the centroid point is selected and four classification annotations are performed on the resized chip experiment result image.

[0088] Step S4: Establish a cell morphology segmentation and classification model based on the U-Net network model and perform network training.

[0089] Step S5: Use the validation set and the test set to verify the recognition effect and accuracy of the network for cell viability and morphology classification.

[0090] Step S6: Develop a software interface for quantitative statistics of cell viability and morphological classification.

[0091] Step S2 includes:

[0092] S201. Crop the key area of ​​the chip lane structure from the complete chip image. Design a semi-automatic interactive image cropping tool based on OpenCV. Users can select four vertices to define the cropping area by clicking on the image, and can zoom in and out of the image to select the cropping area more accurately. The specific steps are as follows:

[0093] (1) The program initializes a series of global variables, including the points list for storing the coordinates of the user's click position, image_index for tracking the index of the currently processed image, the image_paths list for storing the paths of the images to be processed, and scale_factor for controlling the image display ratio. The scale_factor defaults to 1.0, indicating that the image will be displayed at its original size.

[0094] (2) The program defines a resize_image function that accepts an image object as input and adjusts the image size according to the set scale_factor to ensure that the image has an appropriate size ratio when displayed, which is convenient for users to observe and operate.

[0095] (3) The program defines a get_points callback function to respond to the user's mouse click event. When the user clicks the left mouse button on the image, the function will calculate the coordinates of the click position in the original image size and record these coordinates in the points list to provide necessary data support for subsequent cropping operations.

[0096] (4) Entering the main processing logic, the program traverses all image file paths in the specified folder and stores them in the image_paths list. For each image in the list, the program first tries to read the image and checks whether it is successfully loaded. If the read fails, an error message is output and the image is skipped; if the read is successful, the points list is cleared to prepare for processing a new image.

[0097] (5) Before displaying the image, the program uses the cv2.imshow() function to render the image and sets the mouse callback function to get_points so that the user can mark the area to be cropped by clicking on the screen. In addition, the program provides a series of keyboard shortcut commands, and the user can adjust the image display magnification or reduction ratio by pressing the '+' or '-' key to more accurately locate the cropping boundary.

[0098] (6) When the user has finished selecting the four vertices, he can press the 'q' key to determine the cropping area. At this time, the program will calculate the minimum rectangle surrounding these points based on the selected four vertices and crop the image accordingly. The cropped image will be saved to the specified location. If the user wants to continue processing the next image, he can switch to the next image by pressing the 'n' key.

[0099] S202. In order to ensure the consistency and standardization of the image data input into the U-Net neural network for training, the cell experiment images from a complete single lane were accurately preprocessed. Given that such images usually have narrow and long features, the lane images of different widths were first standardized during the processing to make them uniform in width; then, the images were divided into multiple equal-length segments in the length direction to ensure that the size of each local image was strictly maintained at 768 pixels × 512 pixels. This series of preprocessing steps not only helps to improve the efficiency and accuracy of model training, but also lays a solid foundation for subsequent data analysis and pattern recognition.

[0100] The reason why the image size is kept uniform is that the U-Net network is a convolutional neural network architecture commonly used for image segmentation tasks. One of its design features is that it contains multiple downsampling (encoder) and upsampling (decoder) stages, each of which usually involves halving or doubling the input image size. Therefore, in order for each downsampling and upsampling operation in the network to proceed smoothly without size mismatch or padding, the size of the input image should usually be divisible by 2 during each downsampling process until the deepest level in the network is reached. Specifically, if the U-Net network has n downsampling layers, then the width and height of the input image should be 2 n Therefore, 768*512 is a suitable size because it can be divided by 2 multiple times and will not make the network structure too complicated.

[0101] S203, perform data enhancement on the cropped and size-standardized images. Given the limited amount of original image data, a comprehensive data enhancement solution was designed and implemented to improve the robustness and generalization ability of model training. The specific steps are as follows:

[0102] Define a set of data augmentation operations, including but not limited to rotation, flipping, brightness adjustment, contrast change, etc., to simulate various variations that may exist in images in real environments, thereby enriching the training data set. For each image that has been cropped and size-normalized, the program performs the above-defined data augmentation operations in sequence. For example, rotate the image by random angles to simulate different perspectives; flip the image horizontally or vertically to increase the diversity of samples; adjust the brightness and contrast of the image to cope with the effects of different lighting conditions. In the process of performing data augmentation, ensure that each transformation keeps the key features of the original image unchanged to avoid the image losing its original biological meaning or information due to excessive enhancement. At the same time, in order to ensure the effectiveness of data augmentation, multiple variants will be generated for each original image, which significantly increases the size of the training data set.

[0103] After data enhancement is completed, all generated new images will be saved to the specified directory and together with the original images will form the final training data set. This process not only increases the diversity of the data set, but also improves the robustness and generalization ability of model training, so that the model can perform better when facing unknown data. The data enhancement operation adopted by this technical solution increases the amount of image data to 10 times the original amount.

[0104] Step S3 includes:

[0105] S301, annotate the prepared image data set, organize the training samples to establish the training data set. Write Python code to implement an image data annotation tool, such as Figure 3 As shown, each cell image can be annotated with cell centroid and morphological category. In this annotation tool, a set of category names TypeName are first defined to describe the morphological characteristics of different cells, including "normal morphology and color - true positive", "abnormal morphology and no color - true negative", "normal morphology but no color - false negative" and "abnormal morphology but color - false positive". These classifications help users clearly identify and select various cell types when annotating images.

[0106] After starting the image cell annotation by opening the image selection folder, the user can double-click the cell center to annotate the cell. At this point, the software will place a circular annotation point of different colors at the selected cell position to indicate the location of the cell and the category it belongs to. The color of the annotation point changes according to the category of the selected cell, making the annotation results clear at a glance. Green represents "normal morphology and color-true positive", blue represents "abnormal morphology and no color-true negative", yellow represents "abnormal morphology but color-false positive", and red represents "normal morphology but no color-false negative". If you need to remove a marked cell point, the user can double-click the annotation point with the right mouse button, so that the software will remove the point and delete the corresponding information from the annotation list. This operation method simplifies the editing process and enables users to efficiently and accurately mark and manage cells in the image. In addition, the tool also includes a "show center" option. When the user checks this option, all detected cell center points will be presented on the image in a visual way, which is convenient for quickly viewing the core position of each cell. This helps users verify the accuracy of automatic detection and assists manual correction work. There is also a text box on the tool interface to display the total number of cells detected in the current image. This information is very important for statistical analysis and can help users quickly understand the cell density and distribution in the image. The tool also provides image navigation functions, such as page buttons and image progress bar sliders, which are convenient for browsing other images in the same folder; the "show 3D" button displays image coordinates and pixels in three dimensions; at the same time, it provides a save function, allowing users to save marked images or export the annotated data as Excel tables for further data analysis and processing.

[0107] S302, create a data set, the annotated images are organized in a file structure, the training data includes source images (JPG format) and corresponding label images (PNG format), each source image is associated with 4 label images. The size of the label image is the same as the input image. The difference from the segmentation task label file is that the present invention is a key point detection of the cell centroid. The label file is in a heat map format. Each cell centroid point position in the image has a corresponding Gaussian distribution peak, and the surrounding pixel values ​​gradually decrease. Such a design helps the model learn the specific location of the key points, and has a better extraction and learning ability for the surrounding pixel features; in the data loading link, the source image and label image are loaded according to the index and converted into the PyTorch tensor format.

[0108] Step S4 includes:

[0109] S401, establish a cell morphology segmentation and classification model based on the U-Net network model, and perform network training including: selecting the U-Net network model for appropriate adjustment, and re-customizing a network structure according to the complexity of the task. Figure 3 As shown, the following is a detailed introduction to each layer:

[0110] First, there are the convolutional and pooling layers. The first convolutional layer, layer_00, has 3 input channels (corresponding to the three color channels of the RGB image), L / 4 output channels (where L=32), a convolution kernel size of 3×3, and padding of 1, so that the size of the output feature map can be kept the same as the input. The activation function layer uses the ReLU activation function, which only retains positive values ​​and sets negative values ​​to 0. The pooling layer uses the maximum pooling (Max Pooling), with a pooling window size of 2×2 and a step size of 2. This step will reduce the size of the feature map.

[0111] Next are several consecutive convolutional layers combined with pooling layers, each group includes two layers of convolution operations (output channels are L / 2, L, 2L, 4L, 8L, 16L respectively), each convolution layer is followed by a ReLU activation function, and then a maximum pooling layer with a pooling window size of 2×2 and a stride of 2. These layers gradually increase the number of output channels while reducing the size of the feature map to extract more advanced features.

[0112] After the encoder part, the network enters the decoder part, and gradually restores the size of the feature map through upsampling operations, and splices it with the feature map of the corresponding layer of the encoder to fuse multi-level information. The upsampling operation uses a bilinear interpolation method to double the size of the feature map. After each layer of upsampling, the feature map size is restored through a convolutional layer (the output channels are gradually reduced to 8L, 4L, 2L, L), and followed by a ReLU activation function to activate the features.

[0113] The final output layer layer_41 has an input channel of L, an output channel of the number of classification categories (TYPES=4), a convolution kernel size of 1×1, a step size of 1, and a padding of 0. The Sigmoid activation layer uses Sigmoid as the activation function and outputs the probability that each pixel belongs to each category.

[0114] The design of this model follows the basic idea of ​​the U-Net network. By combining the encoder and decoder, it makes full use of multi-scale feature information and fuses the features of the encoder and the decoder through splicing operations to improve the accuracy of segmentation. In addition, the model also uses upsampling operations to restore the size of the feature map to ensure that the final output is consistent with the input image size.

[0115] During training, within the specified training cycle, the training data set is traversed, forward propagation is performed to calculate the predicted value, loss is calculated, and back propagation is performed to update the network parameters. The performance of the model on the validation set is regularly evaluated, and the best model is saved. The loss function is the mean square error function, and a learning rate scheduler is set for the learning rate in the training parameters. The learning rate is reduced after a certain period of time, which can help the model achieve better convergence to a certain extent.

[0116] Step S5 includes:

[0117] The validation set and test set were used to verify the recognition effect and accuracy of the network for cell viability and morphology classification. The average loss of each Epoch was recorded during the training process, and verification was performed after each Epoch, and the training loss and verification accuracy curves were plotted.

[0118] Step S6 includes:

[0119] A software interface for quantitative statistics of cell viability and morphological classification was developed using the PyQt5 framework. This interface is designed to provide users with an intuitive operating platform so that they can effectively collect and analyze cell viability data and classify cell morphology. Through the rich components and tools provided by Qt5, the interface can not only display cell images and their corresponding statistical data, but also help users perform complex analysis operations such as cell counting, viability assessment, and classification of morphological features.

[0120] Example 2

[0121] Referring to the method of Example 1, human primary fibroblasts were used to detect the Pterocarpus ovata samples, and the test concentration gradient was set for the experiment. The specific experimental operation was referred to step 1 in Example 1. The algorithm model trained by the present invention was used to detect the toxicity concentration of the Pterocarpus ovata samples.

[0122] The result obtained by the algorithm is a classification of four types of cell morphology combined with the purple intensity presented on the formazan crystal image: True Positive (TP): normal cell morphology and purple, True Negative (TN): abnormal cell morphology without obvious purple, False Positive (FP): abnormal cell morphology but purple, False Negative (FN): normal cell morphology but not obvious purple. The basis for judging whether the cell morphology is normal or abnormal is that fibroblasts with strong vitality in a normal survival state are usually flat spindle-shaped with slender synapses. Dead fibroblasts may appear atrophic and irregular in shape, losing their original slender shape.

[0123] The formula for calculating cell viability is as follows: Cell survival rate = (TP + FN) / TCS × 100%, where TCS represents the total cell number (Total Cellular Score). This formula was used to analyze the survival rate of fibroblasts after adding different concentrations of palm wood samples to evaluate the toxicity of the samples to fibroblasts. The experimental results obtained by algorithm analysis and the comparison results of the traditional 96-well plate MTT method are shown in Table 1:

[0124] Table 1 Comparison of the results of cytotoxicity tests of a series of compounds with concentration gradients

[0125]

[0126] Note: The data in the table are cell viability data, "96-well plate" refers to the results measured by the 96-well plate method, and "chip" refers to the results measured by the chip method of the present invention.

[0127] According to the data comparison analysis in Table 1, it can be seen that the cell viability data calculated by the algorithm of the present invention is highly consistent with the results obtained by the traditional 96-well plate method, indicating that the algorithm has good reliability. This consistency is not only reflected in the average value, but also in the comparison of the standard deviation (SD), indicating that the algorithm has a strong ability to control experimental errors.

Claims

1. A cell image analysis method combining morphology and color analysis, characterized in that: Based on the U-Net network model, the following steps are included: Step S1, image sample collection; Step S2, image data preprocessing: perform key area cropping and size standardization operations on all collected image samples; Step S3, image data annotation: perform centroid point selection and classification annotation on the resized image samples; Step S4, establish a cell morphology segmentation and classification model based on the U-Net network model, and perform network training; Step S5, use the validation set and test set to verify the network's recognition effect and accuracy of cell information.

2. The cell image analysis method according to claim 1, characterized in that: The method also includes: S6. Develop a software interface for quantitative statistical cell information.

3. The cell image analysis method according to claim 1, characterized in that: In the step S1: the image sample acquisition source is multiple parallel cell culture platforms or tools, each cell culture platform or tool includes multiple independent units, each independent unit is provided with a number of parallel pipes or cell culture areas of arbitrary shapes, these pipes or areas can be connected to a total inlet and a total outlet, or parallel and independent inlets and outlets can be set; the area and volume of all cell culture areas are equal to ensure that the fluid can be evenly distributed during the test.

4. The cell image analysis method according to claim 1, characterized in that: The step S2 comprises: S201, cropping the chip lane structure in the key area of ​​the complete chip image; S202, performing standardization processing on lane images of different widths to make the widths uniform; dividing the image into a plurality of equal-length segments in the length direction, ensuring that the size of each local image is maintained within n downsampling layers that are adapted to the U-Net network, where n is an integer; S203, performing data enhancement on the cropped and size-standardized image; The step S201 is completed using a semi-automatic interactive image cropping tool written in the OpenCV library, including: (1) The program initializes a series of global variables, including the points list for storing the coordinates of the user's click position, image_index for tracking the index of the currently processed image, the image_paths list for storing the paths of the images to be processed, and scale_factor for controlling the image display ratio; scale_factor is set to 1.0 by default, indicating that the image will be displayed at its original size; (2) The program defines a resize_image function, which receives an image object as input and adjusts the image size according to the set scale_factor to ensure that the image has a suitable size ratio when displayed, which is convenient for users to observe and operate; (3) The program defines the get_points callback function, which is used to respond to the user's mouse click event. When the user clicks the left mouse button on the image, the function will calculate the coordinates of the click position in the original image size and record these coordinates in the points list to provide necessary data support for subsequent cropping operations. (4) Entering the main processing logic, the program traverses all image file paths in the specified folder and stores them in the image_paths list; for each image in the list, the program first tries to read the image and checks whether it is successfully loaded; if the reading fails, an error message is output and the image is skipped; if the reading is successful, the points list is cleared to prepare for processing a new image; (5) Before displaying the image, the program uses the cv2.imshow() function in the OpenCV library to render the image and sets the mouse callback function to get_points, so that the user can mark the area to be cropped by clicking on the screen; in addition, the program provides a series of keyboard shortcut commands, and the user can adjust the image display magnification or reduction ratio by pressing the '+' or '-' key to more accurately locate the cropping boundary; (6) When the user has completed the selection of four vertices, the user can press the 'q' key to determine the cropping area; at this time, the program will calculate the minimum rectangle surrounding these points based on the selected four vertices and crop the image accordingly; the cropped image will be saved to the specified location; if the user wants to continue processing the next image, he can switch to the next image by pressing the 'n' key; In step S202, the preferred size of the local image is 768 pixels×512 pixels; Step S203 includes: defining a set of data enhancement operations, including at least rotation, flipping, brightness adjustment, and contrast change, to simulate various variations of images in a real environment, thereby enriching the training data set; for each image that has been cropped and size-normalized, the program sequentially performs the above-defined data enhancement operations; the rotation includes rotating the image by a random angle to simulate different viewing angles; the flipping includes flipping the image horizontally or vertically to increase the diversity of samples; the brightness adjustment includes adjusting the brightness of the image; the contrast change includes adjusting the contrast of the image to cope with the effects under different lighting conditions; in the process of performing data enhancement, ensure that each transformation keeps the key features of the original image unchanged to avoid the image losing its original biological meaning or information due to excessive enhancement; at the same time, to ensure the effectiveness of data enhancement, multiple variants will be generated for each original image, thereby increasing the size of the training data set.

5. The cell image analysis method according to claim 1, characterized in that: The step S3 comprises: S301, annotate the prepared image data set, and organize training samples to establish a training data set; S302, create a data set, the annotated images are organized in a file structure, the training data includes source images in JPG format and corresponding label images in PNG format, each source image is associated with 4 label images; the size of the label image is the same as the input image, which is used for key point detection of cell centroids, and each cell centroid point position in the image has a corresponding Gaussian distribution peak, and the surrounding pixel values ​​gradually decrease; the model learns the specific position of the key points and has the ability to extract and learn the surrounding pixel features; in the data loading stage, the source image and label image are loaded according to the index, and converted into PyTorch tensor format; The step S301 includes: implementing image data annotation by writing Python code, annotating the cell centroid and morphological category of each cell image; First, define a set of category names TypeName to describe the morphological characteristics of different cells, including "normal morphology and color - true positive", "abnormal morphology and no color - true negative", "normal morphology but no color - false negative" and "abnormal morphology but color - false positive", to help users clearly identify and select various cell types when annotating images; After opening the image selection folder to start image cell annotation, double-click the cell center to achieve cell annotation; at this time, the software will place a circular annotation point of different colors at the selected cell position to indicate the cell position and its category; the color of the annotation point changes according to the category of the selected cell; if you need to remove a marked cell point, you can double-click the annotation point with the right mouse button, so that the software will remove the point and delete the corresponding information from the annotation list; the tool also includes a "show center" option. When the user checks this option, all detected cell center points will be presented on the image in a visual way, which is convenient for quickly viewing the core position of each cell; there is also a text box on the tool interface to display the total number of cells detected in the current image; the tool also provides image navigation functions, such as forward and backward page buttons and image progress bar sliders, which are convenient for browsing other images in the same folder; the "show 3D" button displays image coordinates and pixels in three dimensions; at the same time, a save function is provided, allowing users to save the marked image or export the annotated data as an Excel table for further data analysis and processing.

6. The cell image analysis method according to claim 1, characterized in that: The step S4 comprises: S401, select the U-Net network model and make appropriate adjustments, and customize a network structure according to the complexity of the task; The network structure includes at least a convolution layer, a pooling layer, an activation function layer, and an output layer; During training, within the specified training cycle, traverse the training data set, perform forward propagation to calculate the predicted value, calculate the loss, and perform back propagation to update the network parameters; regularly evaluate the performance of the model on the validation set and save the best model; select the mean square error function as the loss function, and set a learning rate scheduler in the training parameters to reduce the learning rate after a certain period of time.

7. The cell image analysis method according to claim 1, characterized in that: The step S5 comprises: recording the average loss of each Epoch during the training process, performing verification after each Epoch, and drawing a curve of training loss and verification accuracy; The step S6 includes: using the PyQt5 framework to develop a software interface for quantitative statistics of cell viability and morphological classification; providing users with an intuitive operating platform to effectively collect and analyze cell viability data and classify cell morphology; the interface is used to display cell images and their corresponding statistical data, and to help users perform complex analysis operations, the analysis operations at least including any one or more of cell counting, viability assessment, and classification of morphological features.

8. Use of the cell image analysis method according to any one of claims 1 to 7 in cell viability analysis.

9. The use according to claim 8, characterized in that: The cells are cells that can grow adherently and are convenient for drug addition and observation, including cells derived from animals, plants or microorganisms; the cell viability analysis is used for any one or more of the activity detection of bioactive factors, anti-tumor drug screening, cytotoxicity testing, tumor radiosensitivity determination, gene expression regulation research and cell metabolism analysis.

10. A cell image analysis instrument, characterized in that: A program comprising all steps and parameters of the cell image analysis method according to any one of claims 1 to 7.

Images