Microscopic cell image tracking method and device based on graph neural network

Abstract

According to the microscopic cell image tracking method and device based on the graph neural network, the electronic equipment and the storage medium provided by the invention, a training set cell image is acquired and a training label is marked, a cell segmentation model is trained according to the training set cell image, and other data sets are predicted according to the cell segmentation model; according to the tracking method provided by the invention, the dependence on manual design can be reduced, the feature expression ability can be improved, the relationship between the cells can be modeled through the graph structure, the tracking precision under the condition of dense and shielded cells can be improved, and the tracking accuracy can be improved. Meanwhile, in combination with time sequence data, the time continuity of cell movement is captured, and the tracking stability is improved.

Description

Technical Field

[0001] This application relates to the technical field of cell image segmentation, and particularly relates to a microscopic cell image tracking method, device, electronic device and storage medium based on a graph neural network. Background Art

[0002] Observing the behavior of cells helps to better understand their biological mechanisms, such as tissue formation and repair, wound healing, and tumor generation. When studying cell behavior, tracking its movement trajectory is very useful, especially for stem cells. Taking the induced pluripotent stem cells (iPSCs) technology as an example, this technology has been applied to the treatment of diseases such as platelet deficiency, spinal cord injury, macular degeneration, Parkinson's and Alzheimer's. However, this technology still has the problem of inefficiency - the reprogramming ratio of cells in most reprogramming protocols is very low, usually less than 5%, and the ratio of precursor cells in some reprogramming protocols at the early stage of reprogramming is even lower than 0.5%, which greatly limits the research and application of induced pluripotent stem cells in the scientific research and clinical fields. Existing research has confirmed that precursor cells can be identified by analyzing the movement trajectories of cells in the early stage. Currently, obtaining cell movement trajectories mainly relies on manual marking, or on training a deep model based on manual marking to complete the tracking of cell movement trajectories.

[0003] Valen et al. used a deep convolutional neural network to solve the cell image segmentation problem and verified its effectiveness in the segmentation of nuclear fluorescence images. Junya Hayashida et al. proposed Motion and Position Map (MPM), which combines cell detection and cell tracking, and is applicable not only to cell movement but also to cell division, enabling multi-object tracking in a dense cell culture environment. Kazuya Nishimura et al. proposed training a cell tracking network only with cell labels, reducing the workload of making tracking labels. T. Dang et al. combined an integration strategy of multiple deep learning models for nuclear segmentation. Magnusson et al. proposed a method for linking the cell contours generated by segmentation into continuous trajectories. Payer et al. proposed a technique for simultaneously segmenting and tracking cells, which combines a pixel-level metric embedding learning strategy and a recurrent hourglass network. He et al. combined a CNN-based observation algorithm with a particle filtering method for tracking non-rigid and weakly detected cells. Nishimura et al. proposed a weakly supervised cell tracking method that uses detection results (i.e., the coordinates of cell positions) to train a CNN model without association information and effectively identifies cell positions through nuclear staining. Kondratiev AY et al. proposed using the generalized nearest neighbor method to complete cell tracking after cell detection using deep learning. Kraus et al. analyzed microscopic images of yeast cells and other pheromone-inhibited cells using a deep convolutional neural network method. Deep learning has been widely applied in the fields of cell detection and cell tracking. Currently, existing cell tracking algorithms mostly rely on the relative distance between cells in adjacent frames and the intersection over union similarity, etc., and often do not perform well on datasets with large morphological differences and irregular shapes among cells.

[0004] For most complex cell images, compared with traditional cell detection and tracking methods, deep learning methods can achieve better performance. Traditional tracking methods require manual feature design, such as based on cell area, etc., which are difficult to capture the complex morphology and dynamic changes of cells, and feature design is time-consuming and not general; at the same time, in the case of cell density or occlusion, traditional methods are difficult to accurately distinguish and track target cells, resulting in error accumulation; most methods only rely on single-frame images, ignoring the temporal continuity of cell movement and are difficult to handle fast movement or morphological changes. Summary of the Invention

[0005] In view of this, the present invention provides a microscopic cell image tracking method, device, electronic device, and storage medium based on a graph neural network, which can ensure capturing the temporal continuity of cell movement while improving tracking stability.

[0006] To solve the above problems, the present application adopts the following technical solutions:

[0007] One of the objectives of this application is to provide a microscopic cell image tracking method based on a graph neural network. The method includes the following steps:

[0008] Obtain training set cell images and label training tags;

[0009] Train a cell segmentation model based on the training set cell images;

[0010] Make predictions on other data sets according to the cell segmentation model;

[0011] Track cells according to the prediction results;

[0012] Analyze the model prediction results based on the tracking results and remove the mislabeled parts of the predictions.

[0013] In some embodiments, in the step of obtaining training set cell images and labeling training tags, it specifically includes the following steps: Label the corresponding training tags by obtaining the fluorescence images corresponding to the training set cell images.

[0014] In some embodiments, before the step of training a cell segmentation model based on the training set cell images, before obtaining the training set cell images and labeling training tags, it further includes the following steps: Perform data augmentation on the training set cell images.

[0015] In some embodiments, in the step of performing data augmentation on the training set cell images, it specifically includes the following steps: Multiply the pixel values of the training set cell images by a coefficient range to change the pixel value distribution of the image in the entire gray scale space, and the coefficient range is selected from 0.8 to 1.6; or perform data augmentation on the training set cell images by using central cropping, random rotation, elastic transformation, grid distortion, brightness contrast, vertical flipping, horizontal flipping, and contrast-limited adaptive histogram equalization.

[0016] In some embodiments, in the step of training a cell segmentation model based on the training set cell images, the cell segmentation model uses the DeepLabv3+ framework.

[0017] In some embodiments, in the step of making predictions on other data sets according to the cell segmentation model, it specifically includes the following steps: Process other cell data to make its format consistent with that of the training set cell images, and input it into the cell segmentation model to automatically generate corresponding segmentation prediction results.

[0018] In some embodiments, in the step of tracking cells according to the prediction results, it specifically includes the following steps: Use a graph neural network algorithm to track cells.

[0019] In some of these embodiments, in the step of tracking cells using a graph neural network algorithm, the following steps are specifically included:

[0020] Extract multiple features from each cell in the segmentation prediction result. The multiple features constitute the initial input information of the graph neural network nodes. The multiple features include geometric attributes, texture and gray-scale distribution information, and motion parameters in consecutive frames. The geometric attributes include area, perimeter, shape, or centroid coordinates, and the motion parameters include displacement and velocity;

[0021] Regard each cell as a node in the graph, and based on the spatial distance and morphological features of the cells between adjacent frames, optionally construct local adjacency edges within the same frame to capture the local aggregation of cells, thereby forming an overall graph structure containing spatio-temporal information;

[0022] Continuously update the node features during the multi-layer message passing process of the graph neural network, realizing the close association of the features of the same cell between different time frames, and ensuring the distinctiveness of the features between different cells to achieve globally optimal cell matching and generate continuous and smooth cell motion trajectories.

[0023] In some of these embodiments, in the step of analyzing the model prediction result according to the tracking result and removing the mispredicted part of the labels, the following steps are specifically included: For each cell in the t-th frame, calculate the overlapping area with the candidate matching cell in the (t + 1)-th frame, and then calculate the ratio with the area of the cell itself. When this ratio exceeds the preset threshold, it is determined that the labels of these two frames are the same cell, thus completing the preliminary tracking between consecutive frames. When there are bubble impurities interfering in the preliminary tracking result, screen the short-term tracking objects. In the subsequent n frames after the preliminary tracking of the object, judge whether there is a cell label matching the current object in each frame. If each frame can establish a match with the object in the previous frame, that is, the overlapping area ratio exceeds the preset threshold, it indicates that the tracking object shows a relatively continuous matching phenomenon in time. If continuous matching cannot be maintained in the subsequent n frames, the predicted label will be cleared. If continuous matching can be achieved in n consecutive frames, it indicates that it is a real cell, and the label of the object is retained.

[0024] In some of these embodiments, the threshold is 0.1.

[0025] A second object of the present application also provides a microscopic cell image tracking device based on a graph neural network. The device includes:

[0026] An image acquisition unit for acquiring training set cell images and marking training labels;

[0027] A training unit for training a cell segmentation model according to the training set cell images;

[0028] A prediction unit for predicting other data sets according to the cell segmentation model;

[0029] A tracking unit for tracking cells according to the prediction results;

[0030] A discrimination unit for analyzing the model prediction results according to the tracking results to remove the mispredicted partial labels.

[0031] A third object of the present application is to provide an electronic device, including at least one processor, at least one memory, and at least one communication bus. Among them, a computer program is stored on the memory, and the processor reads the computer program in the memory through the communication bus;

[0032] When the computer program is executed by the processor, it implements the microscopic cell image tracking method based on the graph neural network.

[0033] A fourth object of the present application is to provide a storage medium, on which a computer program is stored. It is characterized in that when the computer program is executed by a processor, it implements the microscopic cell image tracking method based on the graph neural network.

[0034] The present application adopts the above technical solutions, and the beneficial effects are as follows:

[0035] The microscopic cell image tracking method, device, electronic device and storage medium based on the graph neural network provided by the present application obtain the training set cell images and mark the training labels, train the cell segmentation model according to the training set cell images, and predict other data sets according to the cell segmentation model; track the cells according to the prediction results, and analyze the model prediction results according to the tracking results to remove the mispredicted partial labels. The tracking method provided by the present application can reduce the dependence on manual design, improve the feature expression ability, model the relationship between cells through the graph structure, improve the tracking accuracy in the case of cell density and occlusion, and at the same time combine time series data to capture the time continuity of cell movement and improve the tracking stability.

[0036] In addition, the microscopic cell image tracking method and device based on the graph neural network provided by the present application realize the preliminary labeling of cell images through fluorescence image processing technology, combined with Gaussian filtering, deep learning methods and digital image processing, thereby providing high-quality training labels for subsequent analysis.

[0037] In addition, the microscopic cell image tracking method and device based on the graph neural network provided by this application perform cell image segmentation based on the DeepLabv3+ framework, combine the ResNet50 network, and use the Atrous Spatial Pyramid Pooling (ASPP) module for multi-scale feature extraction to perform cell segmentation, effectively improving the accuracy of cell segmentation; apply the graph neural network to cell tracking, extract features such as the geometry and texture of cells, construct a graph structure containing spatio-temporal information, and generate smooth cell trajectories to achieve accurate cross-frame matching between cells. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] To more clearly illustrate the technical solutions of the embodiments of this application, the following will briefly introduce the drawings required for use in the embodiments of this application or the description of the prior art. Obviously, the following described drawings are only some embodiments of this application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0039] Figure 1 It is a flowchart of the steps of the microscopic cell image tracking method based on the graph neural network provided in Embodiment 1 of this application.

[0040] Figure 2 It is a comparison diagram of the fluorescence image processing effects provided in Embodiment 1 of this application.

[0041] Figure 3 It is a flowchart of the data augmentation processing provided in Embodiment 1 of this application.

[0042] Figure 4 It is a schematic structural diagram of the microscopic cell image tracking device based on the graph neural network provided in Embodiment 2 of this application.

[0043] Figure 5 It is a schematic structural diagram of the electronic device provided in Embodiment 3 of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0044] The following details the embodiments of this application. The examples of the embodiments are shown in the drawings, where the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions throughout. The embodiments described below with reference to the drawings are exemplary and are intended to explain this application and should not be construed as a limitation of this application.

[0045] In the description of the present application, it should be understood that the orientation or positional relationship indicated by the terms "upper", "lower", "horizontal", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation to the present application.

[0046] In addition, the terms "first" and "second" are only used for descriptive purposes and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the present application, the meaning of "a plurality" is two or more, unless otherwise specifically defined.

[0047] In order to make the objectives, technical solutions and advantages of the present application more clear and understandable, the present application will be further described in detail below with reference to the drawings and embodiments.

[0048] Embodiment 1

[0049] Please refer to Figure 1 , which is a flowchart of the steps of the microscopic cell image tracking method based on a graph neural network provided for Embodiment 1 of the present application, including the following steps S110 to S160. The technical solutions implemented are described in detail below.

[0050] Step S110: Obtain training set cell images and label training tags.

[0051] In this embodiment, in the step of obtaining training set cell images and labeling training tags, the following steps are specifically included: Obtain the corresponding fluorescence images of the training set cell images to label the corresponding training tags.

[0052] It can be understood that the initial tags of the training set cell images can be obtained through various channels. For example, the corresponding labels can be obtained by processing the fluorescence images corresponding to the cell images. Since the data set contains two types of images, bright-field images and fluorescence images, for bright-field images, in order to reduce the computational complexity of subsequent operations, they are converted into 8-bit images, and the tif function included in the tifffile module is used to read and parse the files; for fluorescence images, since the fluorescence images are original red fluorescence images in tiff format, first the tif function included in the tifffile module is used to read and parse the files, and then because there are problems with point light source imaging in the red fluorescence images as Figure 2 shown, the processing effects of the Gaussian filtering method, the deep learning method Cellbow, and the digital image processing method are further processed, and finally the initial label image after fluorescence image processing is obtained.

[0053] In this embodiment, through fluorescence image processing technology, combined with Gaussian filtering, deep learning methods (such as Cellbow), and digital image processing, preliminary labeling of cell images is achieved, thus providing high-quality training labels for subsequent analysis.

[0054] Further, before the step of training the cell segmentation model according to the training set cell images before obtaining the training set cell images and labeling the training labels, the following steps are also included: performing data augmentation processing on the training set cell images.

[0055] Specifically, for the data augmentation processing of the training set cell images in this embodiment, it may include: multiplying the pixel values of the training set cell images by a coefficient range to change the pixel value distribution of the image in the entire gray scale space, and the coefficient range is selected from 0.8 to 1.6.

[0056] Please refer to Figure 3 , in this embodiment, the pixel values of the entire image are multiplied by a coefficient range to change the pixel value distribution of the image in the entire gray scale space, and a coefficient range of 0.8 to 1.6 is selected for contrast enhancement, with an interval of 0.1. When the coefficient is taken as 1, the pixel values of the bright field image do not change.

[0057] Further, for the data augmentation processing of the training set cell images in this embodiment, it may also include expanding the training data set for the data augmentation processing of the training set cell images.

[0058] For example: First, based on a random distribution, all cell data sets are divided into a training set, a validation set, and a test set in a ratio of 80:10:10. Then, methods such as central cropping, random rotation, elastic transformation, grid distortion, brightness contrast, vertical flipping, horizontal flipping, and contrast-limited adaptive histogram equalization (CLAHE) are used to expand the training data set, and the validation data is not increased.

[0059] By performing data augmentation processing on the training set cell images to increase the robustness of the model.

[0060] Step S120: Training a cell segmentation model according to the training set cell images.

[0061] In this embodiment, in the step of training the cell segmentation model according to the training set cell images, the cell segmentation model uses the DeepLabv3+ framework.

[0062] It can be understood that in this embodiment, the DeepLabv3+ framework is adopted, combined with ResNet50 as the encoder backbone network, introducing a deep residual connection mechanism to solve the problem of gradient vanishing / explosion through cross-layer connection. In the encoder design part, the Atrous Spatial Pyramid Pooling (ASPP) module is used to perform dilated convolutions with different dilation rates in parallel, so as to achieve multi-scale feature extraction, expand the receptive field while maintaining the feature map resolution; in the decoder design part, bilinear interpolation is used for upsampling, and the entire training and prediction process is implemented using the Python 3.8 and PyTorch deep learning frameworks.

[0063] This embodiment is based on four NVIDIA Tesla A100 GPUs, and the running environment is the CentOS Linux7.4.1708 operating system. The hardware configuration includes an Intel Xeon E5-2650 v4 CPU and 128GB of memory.

[0064] The cell segmentation model provided by this application combines the ResNet50 network and uses the Atrous Spatial Pyramid Pooling (ASPP) module for multi-scale feature extraction to perform cell segmentation, effectively improving the accuracy of cell segmentation.

[0065] Step S130: Predict other data sets according to the cell segmentation model.

[0066] In this embodiment, in the step of predicting other data sets according to the cell segmentation model, the following steps are specifically included: processing other cell data to make its format consistent with the training set cell images, and inputting it into the cell segmentation model to automatically generate corresponding segmentation prediction results. After the model completes multiple rounds of training, select the optimal model and use this model to predict the training set to obtain the predicted results.

[0067] Step S140: Track cells according to the prediction results.

[0068] In this embodiment, in the step of tracking cells according to the prediction results, the following steps are specifically included: using a graph neural network algorithm to track cells.

[0069] Specifically, in the step of tracking cells using the graph neural network algorithm, the following steps are specifically included: extracting multiple features from each cell in the segmentation prediction result, where the multiple features constitute the initial input information of the graph neural network nodes. The multiple features include geometric attributes, texture and gray-scale distribution information, and motion parameters in consecutive frames. The geometric attributes include area, perimeter, shape, or centroid coordinates, and the motion parameters include displacement and velocity; regarding each cell as a node in the graph, and based on the spatial distance and morphological features of cells between adjacent frames, optionally constructing local adjacent edges within the same frame to capture the local aggregation of cells, thereby forming an overall graph structure containing spatio-temporal information; continuously updating the node features during the multi-layer message passing process of the graph neural network, realizing the close association of features of the same cell between different time frames, and ensuring the distinctiveness of features between different cells to achieve globally optimal cell matching and generate continuous and smooth cell motion trajectories.

[0070] It can be understood that before tracking cells based on the prediction result, it is also necessary to strictly align and preprocess the original cell image and the segmentation result, such as normalization, denoising, and image enhancement, and then extract multiple features from each cell obtained by segmentation. These features constitute the initial input information of the subsequent graph neural network nodes.

[0071] In this embodiment, the graph neural network is applied to cell tracking. By extracting features such as the geometry and texture of cells and constructing a graph structure containing spatio-temporal information, accurate cross-frame matching of cells is realized, and smooth cell trajectories are generated. At the same time, by combining deep learning and the graph neural network, high precision can be achieved in cell image segmentation and tracking. Especially in the modeling of the morphological similarity and motion laws between cells, it has significant advantages compared with traditional methods. Compared with traditional pixel-based tracking methods, the graph neural network can handle more complex inter-cell relationships. Especially when the cell morphology changes or the position changes greatly, it can still maintain a high tracking accuracy.

[0072] Step S150: Analyze the model prediction result according to the tracking result and remove the mispredicted partial markers.

[0073] It can be understood that after obtaining the initial cell prediction result of the model and the spatio-temporal graph constructed based on the graph neural network, the cells in each frame have achieved cross-frame matching through the graph neural network.

[0074] For each cell in the t-th frame, by calculating the overlapping area between it and the candidate matching cell in the (t + 1)-th frame, and then calculating the ratio with the area of the cell itself, when this ratio exceeds a preset threshold (e.g., 0.1 for stem cell tracking tasks), it is determined that the labels of these two frames are the same cell, thus completing the preliminary tracking between consecutive frames; there are interferences such as bubble impurities in the preliminary tracking results. For short-term tracking objects, screening is performed. In the subsequent n frames after the preliminary tracking of the object, it is judged whether there is a cell label matching the current object in each frame. If each frame can establish a match with the object in the previous frame (i.e., the overlapping area ratio exceeds the set threshold), it indicates that the tracking object shows a relatively continuous matching phenomenon in time. Given that the movement trajectories of real stem cells usually have long continuity, while misidentified bubbles or impurities may overlap with surrounding targets in individual frames, and the overall tracking sequence has a short number of consecutive frames. Therefore, if continuous matching cannot be maintained in the subsequent n frames, the predicted label will be cleared. If continuous matching can be achieved in n consecutive frames, it indicates that it may be a real cell, and the label of this object is retained.

[0075] It can be understood that by judging the continuity in cell tracking in this embodiment, misidentified impurities or bubbles can be effectively removed, which is difficult to achieve in traditional methods, especially important when dealing with real-time dynamic cell images.

[0076] The microscopic cell image tracking method based on graph neural network provided by this application can reduce the dependence on manual design, improve the feature expression ability, model the relationship between cells through graph structure, improve the tracking accuracy in the case of cell density and occlusion, and at the same time combine time series data to capture the temporal continuity of cell movement and improve the tracking stability.

[0077] Embodiment 2

[0078] Please refer to Figure 4 , for the microscopic cell image tracking device based on graph neural network provided by this embodiment, the device includes:

[0079] An image acquisition unit 110, configured to acquire training set cell images and mark training labels;

[0080] A training unit 120, configured to train a cell segmentation model according to the training set cell images;

[0081] A prediction unit 130, configured to make predictions on other data sets according to the cell segmentation model;

[0082] A tracking unit 140, configured to track cells according to the prediction results;

[0083] An identification unit 150, configured to analyze the model prediction results according to the tracking results and remove the wrongly predicted partial labels.

[0084] The microscopic cell image tracking device based on the graph neural network provided in Embodiment 2 of the present application. For its detailed implementation, reference can be made to Embodiment 1, which will not be elaborated here.

[0085] The microscopic cell image tracking device based on the graph neural network provided by the present application can reduce the dependence on manual design, improve the feature expression ability, model the relationships between cells through a graph structure, improve the tracking accuracy in the case of cell density and occlusion, and at the same time combine time series data to capture the temporal continuity of cell movement and improve the tracking stability.

[0086] Embodiment 3

[0087] Please refer to Figure 5 , Figure 5 , which is a schematic structural diagram of an electronic device provided in an embodiment of the present application. The electronic device includes: one or more processors, one or more memories, one or more communication interfaces, and one or more programs; the one or more programs are stored in the memory and configured to be executed by the one or more processors.

[0088] The above programs include instructions for performing the following steps:

[0089] Obtain training set cell images and label training tags;

[0090] Train a cell segmentation model according to the training set cell images;

[0091] Make predictions on other data sets according to the cell segmentation model;

[0092] Track cells according to the prediction results;

[0093] Analyze the prediction results of the model according to the tracking results and remove the mislabeled parts of the prediction.

[0094] Among them, all relevant contents of each scenario involved in the above method embodiments can be cited in the function descriptions of the corresponding functional modules, which will not be elaborated here.

[0095] It should be understood that the above memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A part of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0096] In an embodiment of the present application, the processor of the above device may be a Central Processing Unit (CPU), and the processor may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0097] It should be understood that the “at least one” involved in the embodiments of the present application refers to one or more, and the “multiple” refers to two or more. “And / or” describes the association relationship of associated objects and indicates that three relationships may exist. For example, A and / or B may indicate: A exists alone, A and B exist simultaneously, and B exists alone, where A and B may be singular or plural. The character “ / ” generally indicates that the associated objects before and after are in an “or” relationship. “At least one (item)” or similar expressions thereof refer to any combination of these items, including any combination of single item (item) or plural items (items). For example, at least one (item) of a, b, or c may indicate: a, b, c, a - b, a - c, b - c, or a - b - c, where a, b, and c may be single or multiple.

[0098] In addition, unless otherwise stated, the ordinal numbers such as “first” and “second” mentioned in the embodiments of the present application are used to distinguish multiple objects and are not used to limit the order, time sequence, priority, or importance of multiple objects. For example, the first information and the second information are only used to distinguish different information, rather than indicating differences in the content, priority, sending order, or importance of these two pieces of information, etc.

[0099] In the implementation process, each step of the above method may be completed by the integrated logic circuit in the hardware of the processor or the instructions in the form of software. The steps of the method disclosed in combination with the embodiments of the present application may be directly embodied as being executed and completed by the hardware processor, or executed and completed by a combination of the hardware and software units in the processor. The software unit may be located in a mature storage medium in the art such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers, etc. This storage medium is located in the memory, and the processor executes the instructions in the memory and combines its hardware to complete the steps of the above method. To avoid repetition, it will not be described in detail here.

[0100] The embodiments of the present application also provide a computer storage medium, where the computer storage medium stores a computer program for electronic data exchange, and the computer program enables the computer to execute some or all of the steps of any method recorded in the above method embodiments.

[0101] An embodiment of the present application also provides a computer program product. The computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute some or all of the steps of any one of the methods described in the foregoing method embodiments. The computer program product may be a software installation package.

[0102] It should be noted that, for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present application is not limited by the described action sequence, because according to the present application, some steps may be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0103] In the above embodiments, the descriptions of the respective embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0104] In several embodiments provided by the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the above division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces. The indirect coupling or communication connection of the device or unit may be in an electrical or other form.

[0105] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiments of the present application.

[0106] In addition, the functional units in each embodiment of the present application may be integrated into a processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0107] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or TRP, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0108] A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing the relevant hardware through a program, and the program may be stored in a computer-readable memory, which may include a flash drive, ROM, RAM, a magnetic disk or an optical disk, etc.

[0109] The above are only preferred embodiments of the present application, and only specifically describe the technical principles of the present application. These descriptions are only for explaining the principles of the present application and cannot be interpreted as limiting the scope of protection of the present application in any way. Based on the explanation here, any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application, and other specific implementation methods of the present application that can be associated with the technicians in this field without creative work, should be included in the scope of protection of the present application.

Claims

1. A microscopic cell image tracking method based on graph neural network, characterized in that, The method includes the following steps: Obtain training set cell images and label training labels; Train a cell segmentation model according to the training set cell images; Make predictions on other data sets according to the cell segmentation model; Track cells according to the prediction results; Analyze the model prediction results according to the tracking results and remove the mislabeled parts of the prediction.

2. The microscopic cell image tracking method based on a graph neural network according to claim 1, wherein In the step of obtaining training set cell images and labeling training labels, it specifically includes the following steps: label the corresponding training labels by obtaining the fluorescence images corresponding to the training set cell images.

3. The microscopic cell image tracking method based on a graph neural network according to claim 1, characterized in that Before the step of training the cell segmentation model according to the training set cell images, before obtaining the training set cell images and labeling the training labels, the following steps are further included: perform data augmentation processing on the training set cell images.

4. The microscopic cell image tracking method based on a graph neural network according to claim 3, wherein In the step of performing data augmentation processing on the training set cell images, it specifically includes the following steps: multiply the pixel values of the training set cell images by a coefficient range to change the pixel value distribution of the images in the entire gray scale space, and the coefficient range is selected from 0.8 to 1.6; or perform data augmentation on the training set cell images by using center cropping, random rotation, elastic transformation, grid distortion, brightness contrast, vertical flipping, horizontal flipping, and contrast-limited adaptive histogram equalization.

5. The microscopic cell image tracking method based on a graph neural network according to claim 1, wherein In the step of training the cell segmentation model according to the training set cell images, the cell segmentation model uses the DeepLabv3+ framework.

6. The microscopic cell image tracking method based on graph neural network according to claim 1, characterized in that In the step of making predictions on other data sets according to the cell segmentation model, it specifically includes the following steps: process other cell data to make its format consistent with the training set cell images, and input it into the cell segmentation model to automatically generate corresponding segmentation prediction results.

7. The microscopic cell image tracking method based on a graph neural network according to claim 1, characterized in that, In the step of tracking cells according to the prediction results, it specifically includes the following steps: use a graph neural network algorithm to track cells.

8. The microscopic cell image tracking method based on a graph neural network according to claim 7, wherein In the step of using a graph neural network algorithm to track cells, it specifically includes the following steps: Extract multiple features from each cell in the segmentation prediction results, and the multiple features constitute the initial input information of the graph neural network nodes. The multiple features include geometric attributes, texture and gray scale distribution information, and motion parameters in consecutive frames. The geometric attributes include area, perimeter, shape, or centroid coordinates, and the motion parameters include displacement and speed; Regard each cell as a node in the graph, and based on the spatial distance and morphological features of the cells between adjacent frames, optionally construct local adjacent edges within the same frame to capture the local aggregation of the cells, thereby forming an overall graph structure containing spatio-temporal information; Continuously update the node features during the multi-layer message passing process of the graph neural network, realize the close association of the features of the same cell between different time frames, and ensure the distinguishability of the features between different cells to achieve the globally optimal cell matching and generate continuous and smooth cell motion trajectories.

9. The microscopic cell image tracking method based on a graph neural network according to claim 8, characterized in that, In the step of analyzing the prediction result according to the tracking result analysis model and removing the mispredicted partial labels, the following steps are specifically included: for each cell in the t-th frame, by calculating the overlapping area between it and the candidate matching cell in the (t + 1)-th frame, and then calculating the ratio with the area of the cell itself. When the ratio exceeds a preset threshold, it is determined that the labels of these two frames are the same cell, thus completing the preliminary tracking between consecutive frames. When there is interference from bubble impurities in the preliminary tracking result, the short-term tracking objects are screened. In the subsequent n frames after the preliminary tracking of the object, it is judged whether there is a cell label matching the current object in each frame. If a match can be established with the object in the previous frame in each frame, that is, the overlapping area ratio exceeds the preset threshold, it indicates that the tracking object shows a relatively continuous matching phenomenon in time. When continuous matching cannot be maintained in the subsequent n frames, the predicted label will be cleared. If continuous matching can be achieved in n consecutive frames, it indicates that it is a real cell, and the label of the object is retained.

10. The microscopic cell image tracking method based on graph neural network according to claim 9, characterized in that, The threshold is 0.

1.

11. A microscopic cell image tracking device based on a graph neural network, characterized in that, The device includes: An image acquisition unit for acquiring training set cell images and marking training labels; A training unit for training a cell segmentation model according to the training set cell images; A prediction unit for predicting other data sets according to the cell segmentation model; A tracking unit for tracking cells according to the prediction result; A discrimination unit for analyzing the prediction result according to the tracking result analysis model and removing the mispredicted partial labels.

12. An electronic device, characterized in that, It includes at least one processor, at least one memory, and at least one communication bus. Among them, a computer program is stored on the memory, and the processor reads the computer program in the memory through the communication bus; When the computer program is executed by the processor, it implements the graph neural network-based microscopic cell image tracking method according to any one of claims 1 to 10.

13. A storage medium, on which a computer program is stored, characterized in that, When the computer program is executed by the processor, it implements the graph neural network-based microscopic cell image tracking method according to any one of claims 1 to 10.

Images