Cell image classification method and image classification system, electronic device, and storage medium

Abstract

The application provides a cell image classification method and an image classification system, an electronic device and a storage medium, and belongs to the technical field of image processing. The method comprises the following steps: acquiring an original cell image, inputting the original cell image into a pre-trained image segmentation model for image segmentation to obtain a cell segmentation graph and a cell segmentation probability; obtaining a cell uncertainty information graph according to the cell segmentation graph and the cell segmentation probability; sending the cell uncertainty information graph to a client and receiving a decision signal returned by the client according to the cell uncertainty information graph; updating the cell uncertainty information graph by responding to the decision signal; and processing the cell segmentation graph according to the updated cell uncertainty information graph to obtain a cell classification graph. The embodiment of the application can improve the accuracy of cell classification in a cell image.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a cell image classification method and image classification system, electronic device and storage medium. Background Technology

[0002] Currently, due to interference from factors such as uneven illumination, defocusing, and artifacts during the imaging process, cell boundaries in cell images are blurred, thus affecting cell classification results. Summary of the Invention

[0003] The main objective of this application is to propose a cell image classification method, image classification system, electronic device, and storage medium that can improve the accuracy of cell classification in cell images.

[0004] To achieve the above objectives, a first aspect of this application proposes a cell image classification method, applied to a server, the method comprising:

[0005] Obtain raw cell images;

[0006] The original cell image is input into a pre-trained image segmentation model for image segmentation to obtain a cell segmentation map and cell segmentation probability.

[0007] A cell uncertainty information map is obtained based on the cell segmentation map and the cell segmentation probability;

[0008] The cell uncertainty information map is sent to the client, and the decision signal returned by the client based on the cell uncertainty information map is received.

[0009] The cell uncertainty information map is updated in response to the decision signal;

[0010] The cell segmentation map is processed based on the updated cell uncertainty information map to obtain a cell classification map.

[0011] In some embodiments, updating the cell uncertainty information graph in response to the decision signal includes:

[0012] Obtain the target uncertainty threshold carried by the decision signal;

[0013] The cell uncertainty information map is updated based on the target uncertainty threshold.

[0014] In some embodiments, before inputting the original cell image into a pre-trained image segmentation model for image segmentation to obtain a cell segmentation map, the method further includes: training the image segmentation model, specifically including:

[0015] Construct a training sample set, which includes multiple training cell samples, each of which includes a sample cell image and a sample segmentation map of the sample cell image;

[0016] An initial segmentation model is constructed, which includes multiple encoding layers and a corresponding decoding layer for each encoding layer, wherein a dropout layer is set after each encoding layer;

[0017] Each sample cell image is input into the initial segmentation model, and the sample cell image is encoded through the encoding layer and the dropout layer to obtain sample image features;

[0018] The sample image features are decoded by the decoding layer to obtain the target segmentation map;

[0019] Based on the sample segmentation map and the target segmentation map, probability calculations are performed to obtain the predicted probability value;

[0020] The loss value is obtained based on the preset loss function and the predicted probability value;

[0021] The model parameters of the initial segmentation model are adjusted based on the sample segmentation map and the target segmentation map of the sample cell image, and the adjusted initial segmentation model is trained again based on the training sample set until the loss value meets the preset training termination condition to obtain the image segmentation model.

[0022] In some embodiments, processing the cell segmentation map based on the updated cell uncertainty information map to obtain a cell classification map includes:

[0023] The cell segmentation image is subjected to image restoration processing to obtain a restored segmentation image;

[0024] Cell contour information is identified on the repaired segmentation map to obtain the region to be identified, and a set of cell contour information is constructed based on the region to be identified.

[0025] Based on the updated cell uncertainty information map, regions are selected from the repair segmentation map to obtain candidate segmentation maps; wherein, the candidate segmentation maps include multiple first target regions;

[0026] Based on the cell contour information set, the shape of each first target region in the candidate segmentation map is determined to obtain the determination result;

[0027] A cell classification map is obtained based on the judgment result and the candidate segmentation map.

[0028] In some embodiments, after performing cell contour information recognition on the repaired segmentation map to obtain the region to be recognized, and constructing a cell contour information set based on the region to be recognized, the method further includes:

[0029] Each region to be identified is screened based on a preset cell area threshold to obtain a region set; wherein, the region set includes multiple second target regions;

[0030] The average area value is obtained by averaging the area values ​​of all the second target regions.

[0031] The cell number parameter is obtained by summing the number of regions in the region set and the area value of the region set;

[0032] The coefficient of variation parameter is obtained based on the number of regions in the region set, the area value of each second target region, and the average area value; wherein, the coefficient of variation parameter is used to characterize the changes in cell size in the cell segmentation map.

[0033] In some embodiments, obtaining a cell classification map based on the judgment result and the candidate segmentation map includes:

[0034] If the judgment result indicates that the first target region is a preset polygon, the target region is filled with color according to the preset color value that matches the preset polygon to obtain a cell classification map.

[0035] To achieve the above objectives, a second aspect of this application proposes a cell image classification method, applied to a client, the method comprising:

[0036] Send the raw cell image to the server;

[0037] The system receives a cell segmentation map, a cell uncertainty information map, and a cell classification map returned by the server based on the original cell image, wherein the cell segmentation map, the cell uncertainty information map, and the cell classification map are obtained according to a cell image classification method described in the first aspect above.

[0038] A decision signal is sent to the server based on the cell uncertainty information graph.

[0039] To achieve the above objectives, a third aspect of this application proposes a cell image classification system, the system comprising:

[0040] The image acquisition module is used to acquire raw cell images;

[0041] The image segmentation module is used to input the original cell image into a pre-trained image segmentation model to perform image segmentation, and obtain cell segmentation maps and cell segmentation probabilities;

[0042] An uncertainty information map acquisition module is used to obtain a cell uncertainty information map based on the cell segmentation map and the cell segmentation probability.

[0043] The signal receiving module is used to send the cell uncertainty information map to the client and receive the decision signal returned by the client based on the cell uncertainty information map;

[0044] A signal response module is used to update the cell uncertainty information map in response to the decision signal;

[0045] The cell classification module is used to process the cell segmentation map based on the updated cell uncertainty information map to obtain a cell classification map.

[0046] To achieve the above objectives, a fourth aspect of the present application provides an electronic device, the electronic device including a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for implementing connection communication between the processor and the memory, wherein the program, when executed by the processor, implements the method described in the first or second aspect above.

[0047] To achieve the above objectives, a fifth aspect of the present application provides a storage medium, which is a computer-readable storage medium for computer-readable storage, wherein the storage medium stores one or more programs that can be executed by one or more processors to implement the method described in the first or second aspect above.

[0048] This application proposes a cell image classification method, image classification system, electronic device, and storage medium. The method involves acquiring a raw cell image, inputting it into a pre-trained image segmentation model for segmentation, obtaining a cell segmentation map and cell segmentation probabilities, and then generating a cell uncertainty information map based on these probabilities. The cell uncertainty information map is sent to a client, and a decision signal returned by the client based on the map is received. The cell uncertainty information map is updated in response to this decision signal. The updated cell uncertainty information map is then used to process the cell segmentation map to obtain a cell classification map. This application improves the accuracy of cell classification in cell images. Attached Figure Description

[0049] Figure 1 This is a schematic diagram of the first process of the cell image classification method provided in the embodiments of this application;

[0050] Figure 2 This is a schematic diagram of the second process of the cell image classification method provided in the embodiments of this application;

[0051] Figure 3 yes Figure 1 The flowchart of step S150 in the middle;

[0052] Figure 4 yes Figure 1 The flowchart of step S160 in the middle;

[0053] Figure 5 This is a schematic diagram illustrating the generation of the repair segmentation map provided in an embodiment of this application;

[0054] Figure 6 This is a schematic diagram of the third process of the cell image classification method provided in the embodiments of this application;

[0055] Figure 7 This is a schematic diagram of the fourth process of the cell image classification method provided in the embodiments of this application;

[0056] Figure 8 This is a schematic diagram of the specific interactive structure of the cell image classification method provided in the embodiments of this application;

[0057] Figure 9 This is a schematic diagram of the system structure of the cell image classification system provided in the embodiments of this application;

[0058] Figure 10 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0060] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0062] First, let's analyze some of the terms used in this application:

[0063] Artificial Intelligence (AI) is a new branch of computer science that studies, develops, and applies theories, methods, technologies, and systems to simulate, extend, and expand human intelligence. It aims to understand the essence of intelligence and produce intelligent machines that can react in a way similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. AI can simulate the information processes of human consciousness and thought. Furthermore, AI utilizes digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceiving the environment, acquiring knowledge, and using that knowledge to achieve optimal results.

[0064] Polygon Approximation Algorithm: Derived from the Douglas-Peucker algorithm, this algorithm uses point-edge distance as the error metric. Starting with the edge connecting the first and last vertices of the original polygon, it calculates the distances from all intermediate vertices to the edge. The vertex furthest from that edge, if its distance is greater than a specified tolerance, is marked as a key and added to the simplified result. This process is recursively applied to each edge in the current simplification until all vertices of the original polygon are within the allowable error range when their distances to the currently examined edge are within the acceptable error range.

[0065] Corneal endothelial cells are tightly packed, non-regenerative epidermal cells located on the posterior surface of the cornea. They are predominantly hexagonal in shape and play a crucial role in maintaining corneal transparency. Assessment of corneal endothelial cell health relies primarily on the quantification of relevant clinical parameters, such as endothelial cell density (ECD), polymorphism (Coefficient of Variance, CV, or the coefficient of variation in cell size), and hexagonal cell morphology (HEX, or the percentage of hexagonal cells).

[0066] Currently, most corneal endothelial cell analyzers on the market are all-in-one devices, with the algorithm encapsulated within the device. This makes it difficult to compare measurement results from different devices. Therefore, there is no unified evaluation standard for assessing the health status of corneal endothelial cells, affecting the accuracy of corneal endothelial cell health assessments by users of different devices.

[0067] Furthermore, due to interference from factors such as uneven illumination, defocusing, and artifacts during the imaging process, and because existing cell analyzers do not consider the uncertainty of quantification results, cell boundaries become blurred, making precise segmentation difficult and affecting cell classification results.

[0068] Based on this, embodiments of this application provide a cell image classification method, an image classification system, and an electronic device, aiming to improve the accuracy of cell classification in cell images.

[0069] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0070] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0071] The cell image classification method provided in this application relates to the field of artificial intelligence technology. This cell image classification method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the cell image classification method, but is not limited to the above forms.

[0072] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0073] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards of the relevant countries and regions. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirects to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data for the proper functioning of the embodiments of this application obtained.

[0074] Please see Figure 1 , Figure 1 This is an optional flowchart illustrating the cell image classification method provided in this application embodiment, applied to the server side. Figure 1 The method may include, but is not limited to, steps S110 to S160.

[0075] Step S110: Obtain the original cell image;

[0076] Step S120: Input the original cell image into a pre-trained image segmentation model to perform image segmentation, and obtain cell segmentation map and cell segmentation probability;

[0077] Step S130: Obtain a cell uncertainty information map based on the cell segmentation map and cell segmentation probability;

[0078] Step S140: Send the cell uncertainty information map to the client and receive the decision signal returned by the client based on the cell uncertainty information map;

[0079] Step S150: Update the cell uncertainty information map in response to the decision signal;

[0080] Step S160: Process the cell segmentation map according to the updated cell uncertainty information map to obtain a cell classification map.

[0081] Steps S110 to S160 of this embodiment involve acquiring an original cell image, inputting the original cell image into a pre-trained image segmentation model for image segmentation, obtaining a cell segmentation map and cell segmentation probabilities, and then obtaining a cell uncertainty information map based on the cell segmentation map and cell segmentation probabilities. The cell uncertainty information map is sent to a client, and a decision signal returned by the client based on the cell uncertainty information map is received. The cell uncertainty information map is updated in response to the decision signal. The cell segmentation map is then processed based on the updated cell uncertainty information map to obtain a cell classification map. This embodiment improves the accuracy of cell classification in cell images by incorporating uncertainty information.

[0082] In step S110 of some embodiments, the original cell image can be a cell image to be classified, which does not require preprocessing, or it can be a regional cell image obtained by dividing the acquired cell image to be classified into regions. The original cell image can be an image of various types of cells, and is not specifically limited here.

[0083] It should be noted that because the individual images of the cells to be classified are relatively large, but due to camera limitations, the edges are often blurry and the image quality is not high, a process of segmenting smaller regions is chosen. Specifically, to improve the accuracy of cell image segmentation, the process of obtaining regional cell images involves dividing the cell image to be classified into regions according to preset image specifications, resulting in multiple regional cell images. For example, when the cell image to be classified is a corneal endothelial cell image, it can be divided into regions according to preset image specifications, resulting in multiple regional cell images with a size of 192*192.

[0084] It should be noted that the front-end and back-end framework of this application embodiment is flexibly deployable, adopting a server-client architecture. A more powerful model is provided on the server side with high computing power to obtain more accurate clinical parameter quantification and analysis. This model greatly reduces the computing power threshold for the client, and the method can be applied to user-end devices with different performance levels. Specifically, after the server obtains the raw cell image transmitted by the client, it starts the control program, thereby calling the corresponding algorithm model and program to execute the cell image classification method of this application embodiment.

[0085] Please see Figure 2 , Figure 2 This is another optional flowchart of the cell image classification method provided in the embodiments of this application. Before step S120, the method provided in the embodiments of this application may further include: training an image segmentation model, specifically including but not limited to steps S210 to S270.

[0086] Step S210: Construct a training sample set, which includes multiple training cell samples. Each training cell sample includes a sample cell image and a sample segmentation map of the sample cell image.

[0087] Step S220: Construct an initial segmentation model. The initial segmentation model includes multiple encoding layers and a corresponding decoding layer for each encoding layer. Each encoding layer is followed by a dropout layer.

[0088] Step S230: Input each sample cell image into the initial segmentation model, and encode the sample cell image through the encoding layer and dropout layer to obtain the sample image features;

[0089] Step S240: Decode the features of the sample image through the decoding layer to obtain the target segmentation map;

[0090] Step S250: Calculate the probability based on the sample segmentation map and the target segmentation map to obtain the predicted probability value;

[0091] Step S260: Obtain the loss value based on the preset loss function and the predicted probability value;

[0092] Step S270: Adjust the model parameters of the initial segmentation model according to the sample segmentation map and target segmentation map of the sample cell image, and continue to train the adjusted initial segmentation model based on the training sample set until the loss value meets the preset training termination condition to obtain the image segmentation model.

[0093] In step S210 of some embodiments, in order to improve the accuracy of the model in image segmentation of cell images, a training sample set is first constructed. The training sample set includes multiple training cell samples, and each training cell sample includes a sample cell image and a sample segmentation map of the sample cell image.

[0094] It should be noted that the sample cell images can come from an existing cell image library. A training sample set is constructed based on multiple sample cell images and their segmented sample maps. The sample segmentation map is equivalent to the sample label of the sample cell image.

[0095] In step S220 of some embodiments, an initial segmentation model is constructed. This initial segmentation model can be based on a UNet convolutional network model. The initial segmentation model includes multiple encoding layers and a corresponding decoding layer for each encoding layer. Specifically, image features are extracted in the encoding layers through stacked convolutions, and the encoded feature maps are compressed through pooling. In the decoding layers, the encoded and compressed image features are restored through upsampling and convolution. To enable the model to incorporate the uncertainty information of the cell images, a dropout layer is set after each encoding layer, introducing uncertainty into the model.

[0096] It's important to note that the dropout layer randomly disables neurons in the initial segmentation model. By randomly removing neurons during each batch of training, only the nodes that weren't masked need to be trained. Therefore, for the same image, the model parameters used in each segmentation attempt differ, resulting in different predictions and introducing uncertainty into the model. Existing networks without dropout, on the other hand, require learning every single node, meaning the model lacks the ability to handle uncertainty and thus reduces segmentation accuracy.

[0097] In steps S230 and S250 of some embodiments, each sample cell image is input into the initial segmentation model. The sample cell images are encoded using a stacked convolutional encoding layer and a dropout layer to obtain multiple sample image features of different sizes. Then, the encoded and compressed image features are decoded using a decoding layer based on upsampling and convolution to obtain the target segmentation map. Probability calculations are then performed based on the sample segmentation map and the target segmentation map to obtain a predicted probability value.

[0098] It should be noted that the initial segmentation model is equivalent to an approximation of the deep Gaussian model. The target segmentation map can be determined according to the prediction distribution formula, which is shown in formula (1).

[0099] q(y * |x * )=∫p(y * |x * ,ω)q(ω)dω (1)

[0100] Where, x * Let y represent the input variable of the initial segmentation model. * q(y) represents the output variable predicted by the initial segmentation model, i.e., the predicted probability value of the model; ω represents the model parameters of the initial segmentation model; q(y) * |x * ) represents y given a model parameter * The predicted distribution is given by q(ω), which represents the probability density function corresponding to the model parameters.

[0101] It should be noted that, as shown in formula (2), the mean of the multiple predicted probability values ​​of the initial segmentation model is used as the segmentation result of the final model, thereby obtaining the target segmentation map. As shown in formula (3), the variance of the multiple predicted probability values ​​of the initial segmentation model is used as the uncertainty value of the model.

[0102]

[0103]

[0104] Among them, E(y) * Var(y) represents the mean of multiple predicted probability values, N1 represents the number of iterations in the model prediction training, and Var(y) represents the mean of the predicted probability values. * ) represents the uncertainty value of the obtained model.

[0105] In steps S260 to S270 of some embodiments, in order to better optimize the model, an image segmentation model capable of accurate segmentation is trained. A loss value is obtained based on a preset loss function and a predicted probability value. The model parameters of the initial segmentation model are adjusted based on the sample segmentation map and the target segmentation map of the sample cell image. The adjusted initial segmentation model is then trained again based on the training sample set until the loss value meets the preset training termination condition to obtain the image segmentation model.

[0106] It should be noted that the target loss function L obtained based on the preset loss function and the predicted probability value is shown in formula (4).

[0107]

[0108] Where N2 represents the number of image pixels in the target segmentation map, y i This represents the predicted value of the i-th pixel. This represents the predicted probability of a pixel, specifically the probability that the similar point is a cell boundary. When the predicted probability of a certain pixel... When it is less than 50%, the predicted value y i A probability of 0 indicates that the pixel is not a cell boundary; when the predicted probability of a certain pixel is... When the predicted value y is greater than or equal to 50%, i A value of 1 indicates that the pixel represents the cell boundary.

[0109] It should be noted that the preset training termination condition can be when the loss value is less than a preset loss value threshold.

[0110] In step S120 of some embodiments, the server calls a pre-trained image segmentation model, inputs the acquired original cell image into the image segmentation model for image segmentation, and obtains a cell segmentation map and cell segmentation probabilities. The cell segmentation map and the original cell image have the same image size.

[0111] In step S130 of some embodiments, to better reflect the confidence level of image segmentation at each pixel location and provide users with additional visual auxiliary diagnostic information, a cell uncertainty information map is obtained based on the cell segmentation map and the cell segmentation probability. The cell segmentation probability represents the segmentation accuracy of each cell image unit in the cell segmentation map; a higher cell segmentation probability indicates greater segmentation accuracy for that cell image unit, and a lower probability indicates lower segmentation accuracy. The cell segmentation map and the cell uncertainty information map are corresponding. The cell uncertainty information map, obtained from the cell segmentation map and the cell segmentation probability map, is used to inform the user of areas in the original cell image where segmentation is less accurate. Specifically, areas with lower segmentation accuracy are highlighted in the cell uncertainty information map, thus assisting in parameter estimation of the cell segmentation map. The cell segmentation map, the cell uncertainty information map, and the original cell image have the same image size.

[0112] It should be noted that this cell uncertainty information map can also be obtained by performing multiple predictions in an image segmentation model based on the cell segmentation map. Specifically, the original cell image input to the image segmentation model is subjected to multiple random forward predictions. During model prediction, a dropout layer is used to randomly disable the function of some neurons, so that different model parameters are used for each prediction, resulting in different prediction results. The cell segmentation map obtained by the model is equivalent to a segmentation probability map. The cell uncertainty information map is obtained based on the multiple prediction results and the segmentation probability map corresponding to the cell segmentation map. This cell uncertainty information map is used to inform the user of the inaccurate segmentation in the image, representing the visualization of the uncertainty corresponding to the cell segmentation map, and providing the user with visual auxiliary judgment information.

[0113] In step S140 of some embodiments, to improve the accuracy of parameter quantization and visualize the image segmentation results, thereby better demonstrating the confidence level of the segmentation results to the client and enhancing the interactivity between the server and the client, after obtaining the cell uncertainty information map, the cell uncertainty information map is sent to the client, and the decision signal returned by the client based on the cell uncertainty information map is received. Specifically, in practical applications, in order to more effectively exclude the uncertainty information corresponding to invalid pixels in the cell uncertainty information map, the decision signal returned by the client based on the cell uncertainty information map is received.

[0114] Please see Figure 3 In some embodiments, step S150 may specifically include, but is not limited to, steps S310 to S320.

[0115] Step S310: Obtain the target uncertainty threshold carried by the decision signal;

[0116] Step S320: Update the cell uncertainty information map according to the target uncertainty threshold.

[0117] In steps S310 and S320 of some embodiments, the client updates the cell uncertainty information map by responding to the decision signal based on the target uncertainty threshold carried in the decision signal returned by the cell uncertainty information map, i.e., based on the target uncertainty threshold.

[0118] Specifically, firstly, based on the uncertainty value in the cell uncertainty information map, the cell uncertainty information map can be divided into multiple continuous subdomains u, u∈U, with uncertainty values ​​distributed between 0 and 2.5. Then, the calculation of the regional uncertainty metric V is shown in formula (5).

[0119]

[0120] Where, N u This represents the number of pixels falling within each subdomain, and N represents the total number of pixels in the cell uncertainty infographic. If the uncertainty value of pixel i falls within the subdomain u, and I represents the set of pixels whose uncertainty value is greater than the target uncertainty threshold, then the uncertainty metric of this region is used to represent the confidence level obtained from the cell uncertainty information map. This confidence level is sent to the client to assist in parameter estimation and improve the accuracy of image classification.

[0121] It should be noted that, by setting a threshold, the embodiments of this application can hide the parts with low uncertainty, so that the uncertainty information map output by the model only shows the areas with high uncertainty, which can better assist the cell segmentation map in parameter estimation.

[0122] It should be noted that, for example, in the assessment of corneal endothelial cell health, a large image of corneal endothelial cells will be taken. However, due to camera shooting issues, the edges of the image will be relatively blurry and the image quality will not be high. Therefore, regions with high segmentation uncertainty will be selected as representative of the entire image, that is, the set of pixels with uncertainty values ​​greater than the target uncertainty threshold will be selected.

[0123] It should be noted that when a decision signal carrying the target uncertainty threshold is received again, the evaluation is carried out again, that is, the cell uncertainty infographic is updated, and the updated cell uncertainty infographic is returned to the client for further analysis.

[0124] Please see Figure 4 In some embodiments, step S160 may specifically include, but is not limited to, steps S410 to S450.

[0125] Step S410: Perform image restoration processing on the cell segmentation map to obtain the restored segmentation map;

[0126] Step S420: Recognize cell contour information on the repair segmentation map to obtain the region to be recognized, and construct a set of cell contour information based on the region to be recognized;

[0127] Step S430: Select regions from the repair segmentation map based on the updated cell uncertainty information map to obtain candidate segmentation maps; wherein, the candidate segmentation map includes multiple first target regions;

[0128] Step S440: Based on the cell contour information set, perform shape judgment on each first target region in the candidate segmentation map to obtain the judgment result;

[0129] Step S450: Obtain the cell classification map based on the judgment result and the candidate segmentation map.

[0130] In step S410 of some embodiments, if there are problems such as holes or noise on the cell segmentation image, which affect the subsequent processing of the cell segmentation image, it is necessary to perform image restoration processing on the cell segmentation image first, that is, to perform morphological operations on the cell segmentation image to obtain a restored segmentation image.

[0131] It should be noted that morphological operations include erosion, dilation, opening, and closing operations. Specifically, erosion eliminates image boundary points, causing the image to shrink inwards along the boundaries; it can also remove portions smaller than a specified structural element, thus shrinking image boundaries. Dilation expands boundaries. Opening is performed after erosion, eliminating noise while preserving the original shape. Closing is performed after dilation, filling holes. Experiments show that... Figure 5 As shown in the embodiment of this application, by first performing a closing operation on the cell segmentation map and then performing a thinning operation, the obtained cell segmentation map can be repaired to the greatest extent, thereby obtaining the required repaired segmentation map.

[0132] In step S420 of some embodiments, since the cell area is too small, it may be noise in the image. In order to solve the subsequent parameters, the cell contour information of the obtained repair segmentation map is identified to obtain the region to be identified, and a set of cell contour information is constructed based on the region to be identified, wherein the contour of each region to be identified is a set of points.

[0133] Please see Figure 6 In some embodiments, after step S420, the method of this application embodiment may also include, but is not limited to, steps S610 to S640.

[0134] Step S610: Filter the area of ​​each region to be identified according to a preset cell area threshold to obtain a region set; wherein, the region set includes multiple second target regions;

[0135] Step S620: Calculate the average area value by averaging the area values ​​of all second target regions;

[0136] Step S630: Obtain the cell number parameter based on the number of regions in the region set and the sum of the area values ​​of the region set;

[0137] Step S640: Based on the number of regions in the region set, the area value of each second target region, and the average area value, obtain the coefficient of variation parameter; wherein, the coefficient of variation parameter is used to characterize the change in cell size in the cell segmentation map.

[0138] In step S610 of some embodiments, in order to calculate the clinical parameters required by the client, firstly, the area value of each region to be identified can be calculated based on the constructed cell contour information set, and the area of ​​each region to be identified is filtered according to a preset cell area threshold, so as to distinguish the cell contour with a large area from the noise contour with a small area, and obtain a region set. The region set includes multiple second target regions, which represent regions to be identified with an area value greater than the preset cell area threshold. The number of second target regions in the region set is the estimated number of cells in the cell segmentation map, and the area formed by the contour of the second target region is the area of ​​the corresponding cell.

[0139] In steps S620 and S640 of some embodiments, the average area value of all the obtained second target regions is calculated to obtain the average area value. As shown in formula (6), based on the number of regions n and the area value of the region set, The cell number parameter is obtained, which is used to represent the endothelial cell density (ECD) in the clinical parameters. As shown in formula (7), based on the number of regions n in the region set and the area value S of each second target region... i and average area value The coefficient of variation (CV) parameter is obtained. The CV parameter is used to characterize the changes in cell size in the obtained cell segmentation map, which can represent the polymorphism CV in the clinical parameters.

[0140]

[0141]

[0142] In steps S430 to S440 of some embodiments, a region selection is performed on the repair segmentation map based on the updated cell uncertainty information map to obtain a candidate segmentation map, which includes multiple first target regions. Specifically, an uncertainty mask map is constructed based on the updated cell uncertainty information map and the obtained region uncertainty metric. This uncertainty mask map is a binary image, where pixels with an uncertainty greater than the target uncertainty threshold are set to 1, and pixels with an uncertainty less than or equal to the target uncertainty threshold are set to 0. The repair segmentation map is then used to select regions based on the uncertainty mask map to obtain a candidate segmentation map including multiple first target regions, which represent cell regions that contribute to parameter estimation. The contour information of each first target region in the candidate segmentation map is obtained based on the cell contour information set, and the shape of each first target region is judged according to a polygon approximation algorithm to obtain a judgment result, which includes the shape judgment result of each first target region.

[0143] It should be noted that the percentage of hexagonal cells (HEX) in the clinical parameters can also be obtained based on this judgment result.

[0144] It should be noted that a connected component algorithm can be used, or the centroid of each cell can be calculated based on the cell contour information set and marked on the cell segmentation map to obtain the cell counting map corresponding to the original cell image.

[0145] In some embodiments, step S450 may specifically include: if the determination result indicates that the first target region is a preset polygon, filling the target region with color according to the preset color value that matches the preset polygon to obtain a cell classification map.

[0146] Specifically, to better visualize cell classification, different cell shapes are marked with different colors in the cell classification map. That is, if the judgment result indicates that the first target area is a preset polygon, the target area is filled with color according to the preset color value that matches the preset polygon to obtain the cell classification map. For example, hexagonal cells are marked with green, octagonal cells with blue, etc., without specific limitations.

[0147] It should be noted that the server sends the obtained visualization analysis images, such as cell segmentation maps, cell uncertainty information maps, cell classification maps, cell count maps, and related clinical parameters, such as cell number parameters, coefficient of variation parameters, and percentage of hexagonal cells, to the client to provide visualized auxiliary judgment information.

[0148] Please see Figure 7 , Figure 7 This is another optional flowchart illustrating a cell image classification method provided in this application embodiment, applied to a client. Figure 7 The method may include, but is not limited to, steps S710 to S730.

[0149] Step S710: Send the original cell image to the server;

[0150] Step S720: Receive the cell segmentation map, cell uncertainty information map, and cell classification map returned by the server based on the original cell image;

[0151] Step S730: Send a decision signal to the server based on the cell uncertainty information graph.

[0152] In step S710 of some embodiments, the client sends the original cell image to the server. After the server performs cell image classification, the client receives the cell segmentation map, cell uncertainty information map, and cell classification map returned by the server based on the original cell image. The client then sends a decision signal to the server based on the cell uncertainty information map. This decision signal includes a target uncertainty threshold.

[0153] It should be noted that the cell segmentation map, cell uncertainty information map, and cell classification map received by the client are obtained according to a cell image classification method of any of the above embodiments.

[0154] It should be noted that the client can also obtain visual related analysis images such as cell count graphs, and related clinical parameters such as cell number parameters, coefficient of variation parameters, and percentage of hexagonal cells, to provide visual auxiliary judgment information.

[0155] In one specific embodiment, such as Figure 8 As shown, the user is guided to upload the original cell image to be classified through the interactive interface 820 at the front end of the user terminal device 810. A communication connection is established between the user terminal device 810 and the server device 830, and the received original cell image is sent to the server device 830. After the server device 830 obtains the original cell image, it starts a control program, which calls the uncertainty segmentation algorithm and the clinical parameter estimation program to execute the cell image classification method provided in this application. By performing the cell image classification method on the original cell image, the algorithm evaluation result is obtained and returned to the user terminal device 810. The algorithm evaluation result is then visualized in the interactive interface 820. The algorithm evaluation result includes visualized correlation analysis images and relevant clinical parameters. The visualized correlation analysis images include cell segmentation maps, cell uncertainty information maps, cell classification maps, and cell count maps. The relevant clinical parameters include cell number parameters, coefficient of variation parameters, percentage of hexagonal cells, etc., to provide visualized auxiliary judgment information.

[0156] Please see Figure 9This application also provides a cell image classification system that can implement the above-described cell image classification method. The system includes:

[0157] Image acquisition module 910 is used to acquire raw cell images;

[0158] The image segmentation module 920 is used to input the original cell image into a pre-trained image segmentation model to perform image segmentation, and obtain cell segmentation map and cell segmentation probability.

[0159] The uncertainty information graph acquisition module 930 is used to obtain a cell uncertainty information graph based on the cell segmentation graph and the cell segmentation probability.

[0160] The signal receiving module 940 is used to send the cell uncertainty information map to the client and receive the decision signal returned by the client based on the cell uncertainty information map;

[0161] The signal response module 950 is used to update the cell uncertainty information map in response to decision signals;

[0162] The cell classification module 960 is used to process the cell segmentation map based on the updated cell uncertainty information map to obtain the cell classification map.

[0163] The specific implementation of this cell image classification system is basically the same as the specific implementation of the cell image classification method described above, and will not be repeated here.

[0164] This application also provides an electronic device, which includes: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for communication between the processor and the memory. When the program is executed by the processor, it implements the aforementioned cell image classification method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0165] Please see Figure 10 , Figure 10 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0166] The processor 1010 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0167] The memory 1020 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010 using the cell image classification method of the embodiments of this application.

[0168] The input / output interface 1030 is used to implement information input and output;

[0169] The communication interface 1040 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0170] Bus 1050 transmits information between various components of the device (e.g., processor 1010, memory 1020, input / output interface 1030, and communication interface 1040);

[0171] The processor 1010, memory 1020, input / output interface 1030 and communication interface 1040 are connected to each other within the device via bus 1050.

[0172] This application embodiment also provides a storage medium, which is a computer-readable storage medium for computer-readable storage. The storage medium stores one or more programs, which can be executed by one or more processors to implement the above-described cell image classification method.

[0173] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0174] This application provides a cell image classification method, image classification system, electronic device, and storage medium. The method guides the user to upload original cell images through a front-end interactive interface (client). Upon receiving the original cell images, the server initiates a control program, which calls the corresponding algorithm model and program. The original cell images are input into a pre-trained image segmentation model for image segmentation, resulting in a cell segmentation map and cell segmentation probabilities. Based on these cell segmentation maps and probabilities, a cell uncertainty information map is obtained. The cell uncertainty information map is sent to the client, and the server receives a decision signal returned by the client based on the cell uncertainty information map. This decision signal carries a target uncertainty threshold. Updating the cell uncertainty information map based on the target uncertainty threshold better assists in parameter estimation of the cell segmentation map. Then, the server performs image restoration processing on the cell segmentation map to obtain a restored segmentation map. Cell contour information recognition is performed on the restored segmentation map to obtain regions to be identified, and a set of cell contour information is constructed based on these regions. Region selection is performed on the restored segmentation map based on the updated cell uncertainty information map to obtain candidate segmentation maps including multiple first target regions. The shape of each first target region in the candidate segmentation map is judged based on the cell contour information set to obtain the judgment result. A cell classification map is then obtained based on the judgment result and the candidate segmentation map. To better visualize cell classification, if the judgment result indicates that the first target region is a preset polygon, the target region is filled with color according to a preset color value matching the preset polygon, thus obtaining the cell classification map. This embodiment uses a deep learning model to segment cell images, which is more robust than traditional methods based on manual features. Based on the obtained cell segmentation map, automatic estimation of clinical parameters such as ECD, CV, and HEX is achieved. This embodiment adds a dropout layer to the image segmentation model, introducing model uncertainty to visualize the results, showing the confidence level of the segmentation results to the client, enhancing human-computer interaction, and improving the accuracy of parameter quantification. The cell image classification system corresponding to this embodiment has a device-independent algorithm, eliminating the influence of inconsistent algorithms on different devices and providing a unified evaluation standard. Compared with traditional all-in-one machines, this system helps to shorten the algorithm development cycle, reduce software development costs, and more conveniently provide rich visualization analysis and interactive interfaces to meet customer needs. This application adopts a server-client architecture, providing a more powerful model on a server with high computing power to obtain more accurate clinical parameter quantification and analysis. This model greatly reduces the computing power threshold for clients, enabling more flexible deployment and accommodating client devices with different performance levels. This application also improves the accuracy of cell classification in cell images by incorporating uncertainty information.

[0175] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0176] It will be understood by those skilled in the art that Figures 1 to 7 The technical solutions shown do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0177] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0178] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0179] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0180] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0181] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0182] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0183] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0184] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0185] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A cell image classification method, applied to a server, characterized in that, The method includes: Obtain raw cell images; The original cell image is input into a pre-trained image segmentation model for image segmentation to obtain a cell segmentation map and cell segmentation probabilities. The image segmentation model includes multiple encoding layers and a corresponding decoding layer for each encoding layer. A dropout layer is set after each encoding layer to enable the image segmentation model to analyze the uncertainty information of the cell image. A cell uncertainty information map is obtained based on the cell segmentation map and the cell segmentation probability; The cell uncertainty information map is sent to the client, and the decision signal returned by the client based on the cell uncertainty information map is received. Obtain the target uncertainty threshold carried by the decision signal, and update the cell uncertainty information map according to the target uncertainty threshold to filter out regions with high uncertainty; The cell segmentation image is subjected to image restoration processing to obtain a restored segmentation image; Cell contour information is identified on the repaired segmentation map to obtain the region to be identified, and a set of cell contour information is constructed based on the region to be identified. Based on the updated cell uncertainty information map, regions are selected from the repair segmentation map to obtain candidate segmentation maps; wherein, the candidate segmentation maps include multiple first target regions; Based on the cell contour information set, the shape of each first target region in the candidate segmentation map is determined to obtain the determination result; The cell classification map is obtained based on the judgment result and the candidate segmentation map, specifically including: if the judgment result indicates that the first target region is a preset polygon, the target region is filled with color according to a preset color value that matches the preset polygon to obtain the cell classification map.

2. The method according to claim 1, characterized in that, Before inputting the original cell image into a pre-trained image segmentation model for image segmentation to obtain a cell segmentation map, the method further includes: training the image segmentation model, specifically including: Construct a training sample set, which includes multiple training cell samples, each of which includes a sample cell image and a sample segmentation map of the sample cell image; An initial segmentation model is constructed, which includes multiple encoding layers and a corresponding decoding layer for each encoding layer, wherein a dropout layer is set after each encoding layer; Each sample cell image is input into the initial segmentation model, and the sample cell image is encoded through the encoding layer and the dropout layer to obtain sample image features; The sample image features are decoded by the decoding layer to obtain the target segmentation map; Based on the sample segmentation map and the target segmentation map, probability calculations are performed to obtain the predicted probability value; The loss value is obtained based on the preset loss function and the predicted probability value; The model parameters of the initial segmentation model are adjusted based on the sample segmentation map and the target segmentation map of the sample cell image, and the adjusted initial segmentation model is trained again based on the training sample set until the loss value meets the preset training termination condition to obtain the image segmentation model.

3. The method according to claim 1, characterized in that, After performing cell contour information recognition on the repaired segmentation map to obtain the region to be recognized, and constructing a cell contour information set based on the region to be recognized, the method further includes: Each region to be identified is screened based on a preset cell area threshold to obtain a region set; wherein, the region set includes multiple second target regions; The average area value is obtained by averaging the area values ​​of all the second target regions. The cell number parameter is obtained by summing the number of regions in the region set and the area value of the region set; The coefficient of variation parameter is obtained based on the number of regions in the region set, the area value of each second target region, and the average area value; wherein, the coefficient of variation parameter is used to characterize the changes in cell size in the cell segmentation map.

4. A cell image classification method, applied to a client, characterized in that, The method includes: Send the raw cell image to the server; The system receives a cell segmentation map, a cell uncertainty information map, and a cell classification map returned by the server based on the original cell image, wherein the cell segmentation map, the cell uncertainty information map, and the cell classification map are obtained by a cell image classification method according to any one of claims 1 to 3; A decision signal is sent to the server based on the cell uncertainty information graph.

5. A cell image classification system, characterized in that, The system includes: The image acquisition module is used to acquire raw cell images; The image segmentation module is used to input the original cell image into a pre-trained image segmentation model for image segmentation to obtain cell segmentation maps and cell segmentation probabilities. The image segmentation model includes multiple encoding layers and a corresponding decoding layer for each encoding layer. A dropout layer is set after each encoding layer to enable the image segmentation model to analyze the uncertainty information of the cell image. An uncertainty information map acquisition module is used to obtain a cell uncertainty information map based on the cell segmentation map and the cell segmentation probability. The signal receiving module is used to send the cell uncertainty information map to the client and receive the decision signal returned by the client based on the cell uncertainty information map; The signal response module is used to obtain the target uncertainty threshold carried by the decision signal and update the cell uncertainty information map according to the target uncertainty threshold in order to filter out regions with high uncertainty. A cell classification module is used to perform image restoration processing on the cell segmentation map to obtain a restored segmentation map; to perform cell contour information recognition on the restored segmentation map to obtain regions to be identified, and to construct a cell contour information set based on the regions to be identified; to select regions in the restored segmentation map based on the updated cell uncertainty information map to obtain candidate segmentation maps; wherein, the candidate segmentation map includes multiple first target regions; to perform shape judgment on each first target region in the candidate segmentation map based on the cell contour information set to obtain a judgment result; and to obtain a cell classification map based on the judgment result and the candidate segmentation map, specifically including: if the judgment result indicates that the first target region is a preset polygon, to fill the target region with color based on a preset color value matching the preset polygon to obtain a cell classification map.

6. An electronic device, characterized in that, The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for establishing communication between the processor and the memory. The program is executed by the processor. The method as described in any one of claims 1 to 3; or The method as described in claim 4.

7. A storage medium, said storage medium being a computer-readable storage medium for computer-readable storage, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to achieve: The method as described in any one of claims 1 to 3; or The method as described in claim 4.

Images