Platelet classification system based on submicroscopic structure, training and classification method thereof

Abstract

The application discloses a platelet classification system based on submicrostructure. The system is based on a deep convolutional neural network and comprises an input layer and an output layer. The input layer is used for inputting fluorescence channel images and bright field channel images of preset specifications of single platelet present fluorescent labeling signals. The output layer comprises output neurons corresponding to submicrostructure categories, output neurons of irregular category platelets and output neurons of invalid image data. The application classifies, detects, counts and counts submicrostructures of platelets after extracting single platelet cell images through a deep learning method, so that the classification accuracy is improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of deep learning, and more particularly, relates to a method and system for classifying erythrocyte submicrostructure. BACKGROUND

[0002] Platelets are a common type of blood cell in the human body, and play an important role in physiological processes such as hemostatic coagulation, repair of damaged blood vessels, and immune regulation. The submicrostructure of platelets includes microtubules, mitochondria, alpha granules, and dense granules. Abnormalities in the number, shape, and size of these submicrostructures are closely related to certain diseases. For example, in ovarian cancer patients, the microtubular system of platelets is significantly different from that of healthy people. Identifying, classifying, and counting the microtubular structure and alpha granule structure of platelets can provide clues for the study of related diseases and help researchers and doctors screen for related diseases.

[0003] Platelet cells are very small, with a diameter of 2.0-10 μm, which results in a low signal-to-noise ratio in images of platelet cells under a super-resolution microscope. Traditional image processing methods are difficult to distinguish between signal and noise background. In addition, platelet cells are easily activated, and the morphology and function of activated platelet cells change, often appearing in a state of aggregation and adhesion. This causes large image processing errors and greatly interferes with image-based platelet classification systems, resulting in low classification or clustering accuracy and ineffective classification.

[0004] Furthermore, existing cell image segmentation techniques cannot effectively extract single platelet cell images, resulting in difficulties in classifying, detecting, counting, and counting various types of platelet submicrostructures in the body of a subject, such as microtubular structures and alpha granule structures, which further reduces the classification accuracy. SUMMARY

[0005] To overcome the above deficiencies or improve the existing technology, the present application provides a platelet classification system based on submicrostructure, which is based on a deep convolutional neural network.

[0006] The input layer is used to input a single platelet microscopic image of a predetermined specification, which includes a fluorescence channel image and a bright field channel image. The fluorescence channel presents a fluorescence labeling signal.

[0007] The output layer includes output neurons corresponding to submicrostructure categories, irregular category platelet output neurons, and invalid image data output neurons.

[0008] the irregular category platelet output neuron, for outputting a single platelet microscopic image of which the bright field channel has a complete cell structure but the fluorescent channel image does not belong to the category of the output neuron corresponding to the submicrostructure category;

[0009] the invalid image data output neuron, for outputting a single platelet fiber image of which the bright field channel has a complete cell structure and the fluorescent channel image does not detect the fluorescent signal within the platelet cell range.

[0010] Preferably, the submicrostructure-based platelet classification system, the deep convolutional neural network is Resnet, VGGnet, or GoogleNet.

[0011] Preferably, the submicrostructure-based platelet classification system, the single platelet microscopic image is obtained by the following method:

[0012] (1) Obtain bright field and fluorescent images: use a microscope to perform in situ bright field and fluorescent microscopic imaging on platelets with submicrostructure fluorescent labeling, obtain a digitized bright field image and a corresponding digitized fluorescent image in the field of view of the microscope, and perform gray scale normalization processing and pixel size interpolation alignment on the digitized fluorescent image.

[0013] (2) Segment platelets: segment the digitized bright field image obtained in step (1) using an image segmentation algorithm, crop to 4.875 μm x 4.875 μm size, and obtain a single platelet cell region;

[0014] (3) Bright field channel image screening: in the single platelet cell region obtained in step (2); detect the image edge, use roundness, connected domain area, minimum external size, and morphological features to exclude adherent platelet cells and platelet cell regions with incomplete edges, and extract the texture features of the image to exclude bubble regions, and obtain a bright field channel image of a single platelet cell;

[0015] (4) Fluorescent channel image acquisition: crop the same region of the single platelet cell bright field channel image obtained in step (3) from the digitized fluorescent image of the platelet submicrostructure, as the fluorescent channel image of a single platelet cell;

[0016] (5) Single platelet microscopic image combination: superimpose the bright field channel image of a single platelet cell and its corresponding fluorescent channel image of a single platelet cell to obtain a single platelet microscopic image.

[0017] Preferably, the submicrostructure-based platelet classification system, the image segmentation algorithm thereof preferably adopts a Res-Unet segmentation network, which mainly emphasizes that the size of the receptive field is adapted to the actual size of the platelet cell, and the network only needs to focus on extracting low-level semantic features rather than global semantic features, so as to well segment each platelet cell.

[0018] The image texture features are extracted by calculating the gray level co-occurrence matrix of the image.

[0019] Preferably, the submicrostructure-based platelet classification system, the submicrostructure categories thereof are classified according to the distribution type of the fluorescent signal in the fluorescent channel image and the position relationship of the fluorescent signal relative to the edge of the platelet.

[0020] Preferably, the submicrostructure-based platelet classification system, for the microtubule structure of the platelet, is divided into circular ring distribution, diffuse distribution, and aggregated distribution; wherein: the average gray value of the center inside the circular ring and the average gray value of the circular ring area are less than a certain threshold value, the fluorescent signal is in circular ring distribution, and the circular ring is classified; the microtubule fluorescent signal is in circular shape, the inside is solid and the ratio of the average gray value of the circular center to the average gray value of the circular edge is close to 1, and the diffuse distribution is classified; the microtubule fluorescent signal is in circular shape and the signal is stronger towards the center of the circle, and the aggregation is classified.

[0021] Preferably, the submicrostructure-based platelet classification system, for the alpha granule structure, is divided into ring distribution, aggregated distribution, and scattered distribution; wherein, the alpha granule fluorescent image is wrapped into a ring shape, the alpha granule fluorescent image is classified as ring distribution, the alpha granule fluorescent image is aggregated in the center of the cell image, and the alpha granule fluorescent image is classified as aggregated distribution; the alpha granule fluorescent image is scattered in the cell image everywhere, and the alpha granule fluorescent image is classified as scattered distribution.

[0022] According to another aspect of the present application, a training method of the submicrostructure-based platelet classification system is provided.

[0023] Training data acquisition: manually mark a single platelet microscopic image, and filter out submicrostructure categories by manually observing the characteristics of different structures;

[0024] Classified data is trained by supervised learning of a deep learning network.

[0025] Preferably, the training method of the submicrostructure-based platelet classification system, the loss function uses the sum of CosFace loss and centerloss loss, and the specific formula is as follows:

[0026]

[0027] where s is a characteristic vector size artificially set to ensure that the final decision is not affected by the length of the characteristic vector. The parameter m controls the decision range, and θ is the angle between the characteristic vector and the decision vector. λ is a hyperparameter, x i is the characteristic vector of the sample, c j is the reference decision vector corresponding to the category.

[0028] According to another aspect of the present application, a submicrostructure-based platelet classification method is provided, which comprises the following steps:

[0029] S1, obtaining a digital bright-field image of a platelet cell of platelet submicrostructure and a corresponding digital fluorescence image, and performing gray scale normalization processing and pixel size interpolation alignment on the digital fluorescence image to obtain a single platelet microscopic image;

[0030] S2, inputting the single platelet microscopic image obtained in step S1 into the platelet classification system according to any one of claims 1 to 7 to obtain a platelet classification result.

[0031] Overall, the above technical solutions conceived by the present application compared with the prior art

[0032] (1) Since the individual platelets can be well separated, while excluding the interference of irregular categories and invalid categories, the platelet category recognition accuracy is greatly improved;

[0033] (2) Due to the presence of step (5), artificial labels are introduced, and the neural network is trained using supervised learning method, so as to better classify the platelet submicrostructure.

[0034] (3) Since the optimized loss function limits the geometric distribution characteristics between the decision vectors, and balances the inter-class distance and intra-class distance of each category, the classification result is more robust. The probability of ambiguous results is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 is a comparison schematic diagram of the single platelet cell microtubule structure and alpha granule structure fluorescence image and the single platelet cell bright-field image provided for the embodiment of the present application;

[0036] Figure 2 is a flowchart of intelligent recognition and category statistics of platelet microtubule structure based on deep learning method provided for the embodiment of the present application;

[0037] Figure 3 is a digital platelet cell bright-field image provided for the embodiment of the present application;

[0038] Figure 4 A segmented binary image of a digitalized bright field image of platelet cells provided for an embodiment of the present application using OTSU threshold segmentation;

[0039] Figure 5 A segmented binary image of a digitalized bright field image of platelet cells provided for an embodiment of the present application using the present method;

[0040] Figure 6 A final segmented and filtered binary image of a digitalized bright field image of platelet cells provided for an embodiment of the present application;

[0041] Figure 7 A schematic diagram of five categories of microtubular structures of platelet cells provided for an embodiment of the present application;

[0042] Figure 8 A schematic diagram of the proportion of microtubular categories of platelet cells provided for an embodiment of the present application;

[0043] Figure 9 A flow chart of intelligent recognition and category statistics of platelet alpha granule structure based on deep learning method provided for an embodiment of the present application;

[0044] Figure 10 A schematic diagram of four categories of alpha granule structures of platelet cells provided for an embodiment of the present application. DETAILED DESCRIPTION

[0045] In order to make the objectives, technical solutions and advantages of the present application clearer, the following further describes the present application with reference to embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.

[0046] The platelet classification system based on submicrostructure provided by the present application is based on a deep convolutional neural network, preferably Resnet, VGGnet, or GoogleNet.

[0047] The input layer is used to input a single platelet microscopic image of a preset specification, wherein the single platelet microscopic image includes a fluorescence channel image and a bright field channel image, and the fluorescence channel presents a fluorescence labeling signal, such as Figure 1 ;

[0048] The single platelet microscopic image is obtained by the following method:

[0049] (1) Obtain bright field image and fluorescence image: adopt microscope to carry out in situ bright field and fluorescence microscopic imaging on platelets with submicrostructure fluorescent label, obtain digital bright field image and corresponding digital fluorescence image in microscope field of view, and carry out gray scale normalization processing and pixel size interpolation alignment on the digital fluorescence image;

[0050] (2) Segment platelets: segment the digital bright field image obtained in step (1) by using image segmentation algorithm, and cut out a single platelet cell region with a size of 4.875 μm x 4.875 μm;

[0051] The image segmentation algorithm preferably adopts a Res-Unet segmentation network, which mainly emphasizes that the size of the receptive field is adapted to the actual size of the platelet cell, and the network only needs to extract low-level semantic features instead of global semantic features, so that each platelet cell can be well segmented;

[0052] (3) Bright field channel image screening: in the single platelet cell region obtained in step (2); detect the image edge, and screen out the adherent platelet cells and the platelet cell regions with incomplete edges by using roundness, connected domain area, minimum external size, and morphological characteristics, and screen out the bubble region by extracting the texture features of the image, to obtain the bright field channel image of a single platelet cell.

[0053] The image texture features are extracted by calculating the gray level co-occurrence matrix (Gray-level co-occurrence matrix) of the image;

[0054] (4) Obtain fluorescence channel image: cut out the same region as the bright field channel image of a single platelet cell obtained in step (3) from the digital fluorescence image of the submicrostructure of the platelet, as the fluorescence channel image of a single platelet cell;

[0055] (5) Combine single platelet microscopic image: superimpose the bright field channel image of a single platelet cell and its corresponding fluorescence channel image of a single platelet cell to obtain a single platelet microscopic image.

[0056] The output layer includes submicrostructure category corresponding output neurons, irregular category platelet output neurons, and invalid image data output neurons;

[0057] The irregular category platelet output neuron is used to output a single platelet microscopic image whose bright field channel has complete cell structure but whose fluorescence channel image does not belong to the category output by the submicrostructure category corresponding output neuron, i.e., a single platelet microscopic image that is not a regular submicrostructure;

[0058] The invalid image data output neuron is used for outputting a single platelet fiber image with complete cell structure in the bright field channel, and no fluorescent signal in the platelet cell range is detected in the fluorescent channel image.

[0059] The irregular category can be theoretically subdivided into multiple subcategories. Generally, in similar fields such as face recognition systems, a decision boundary of each subcategory is established by using a metric learning method to improve the model interpretability and accuracy. However, the current understanding of the submicrostructure of platelets is limited, and the classification based on the submicrostructure is complex and difficult to exhaust, and the classification effect is poor, so the irregular category is set up, and only the distance between the irregular category and the remaining categories is limited, and the decision boundary of the subcategory is not established separately, and the test results show that for the known submicrostructure categories, the classification performance is good.

[0060] The submicrostructure category is classified according to the distribution type of the fluorescent signal in the fluorescent channel image and the position relationship of the fluorescent signal relative to the platelet edge. Specifically:

[0061] For the platelet microtubule structure, it is divided into circular ring distribution, diffuse distribution and aggregation distribution. Among them, if the fluorescent signal is in a circular ring distribution, and the average gray value of the center inside the circular ring and the average gray value of the circular ring area are less than a certain threshold, the circular ring distribution is classified; if the microtubule fluorescent signal is in a circular shape, the inside is solid and the ratio of the average gray value of the circular center to the average gray value of the circular edge is close to 1, the diffuse distribution is classified; if the microtubule fluorescent signal is in a circular shape and the signal is stronger towards the center of the circle, the aggregation is classified.

[0062] For alpha granule structure, it is divided into ring distribution, aggregation distribution and scattered distribution. Among them, if the alpha granule fluorescent image is wrapped into a ring shape, it is classified as ring distribution, if the alpha granule fluorescent image is aggregated in the center of the cell image, it is classified as aggregation distribution, and if the alpha granule fluorescent image is scattered in the cell image, it is classified as scattered distribution.

[0063] The platelet classification system based on submicrostructure provided by the application is trained according to the following method:

[0064] Training data acquisition: manually mark a single platelet microscopic image, and select the submicrostructure category by manually observing the characteristics of different structures.

[0065] The classified data is trained by supervised learning of a deep learning network.

[0066] In order to ensure the interpretability and subsequent adjustability of the classifier, the loss function uses the sum of CosFace loss and centerloss loss, and the specific formula is as follows:

[0067]

[0068] where s is a artificially set feature vector size to ensure that the final decision is not affected by the length of the feature vector. The parameter m controls the decision range, and theta is the angle between the feature vector and the decision vector. Lambda is a hyperparameter, x i is the feature vector of the sample, c j is the reference decision vector of the corresponding category. The first term of the loss function is mainly the classification loss, which ensures that the distance between classes is far enough. The second term is the intra-class loss, which ensures that the intra-class is tight enough.

[0069] During the training process, the above loss function guides the network to establish the reference decision vector of each category, while limiting each reference decision vector to be as far away from each other as possible and as close as possible within the same category.

[0070] During the inference process, the model encodes the sample using the learned feature information and finally obtains the feature vector. Then the cosine distance is calculated with different reference decision vectors to determine the final category attribution. If the distance between the sample feature vector and each reference decision vector except the irregular category does not meet the threshold, or the distance with the reference decision vector of the irregular category is smaller, it is judged that the sample is the irregular category.

[0071] This decision method can not only ensure that images similar to the irregular category in the training set can be accurately classified, but also ensure that the model has good generalization ability for irregular categories not contained in the data set.

[0072] The platelet classification method based on submicrostructure provided by the application comprises the following steps:

[0073] S1, obtaining a digital bright field image of a platelet cell of a platelet submicrostructure and a corresponding digital fluorescence image, and performing gray scale normalization processing and pixel size interpolation alignment on the digital fluorescence image to obtain a single platelet microscopic image;

[0074] S2, inputting the single platelet microscopic image obtained in step S1 into the platelet classification system provided by the application to obtain a platelet classification result.

[0075] The following is an example:

[0076] Example 1

[0077] The platelet classification system based on microtubules provided in this example, Figure 2 The flowchart of intelligent identification and category statistics of platelet microtubule structure based on a deep learning method provided for the implementation example of the application;

[0078] (1) Batch read-in of the digital bright-field image and the corresponding digital fluorescence image of the platelet cell of the platelet microtubule structure of the detected subject, and pre-processing of the digital fluorescence image;

[0079] (2) Batch input of the bright-field image into the trained bright-field platelet cell segmentation network to segment the platelet cell region; the bright-field platelet cell segmentation network is based on the Res-Unet segmentation model, and since the platelet cell is very small, it will cause the platelet cell signal and noise background under the bright field to be difficult to separate, the image signal-to-noise ratio is low, and the traditional threshold segmentation method is not suitable for this task, and the effect is poor, Figure 3 is a platelet bright-field cell image, and Figure 4 is the result after using OTSU threshold segmentation and performing connected domain filling, it can be found that there are still a large amount of background noise remaining after segmentation, and most of the platelet cell outlines are not closed after segmentation, resulting in only a small amount of platelet cells can be correctly positioned and extracted after connected domain filling, in addition, due to the limitation of imaging conditions, the platelet cell bright-field image will also have some bubbles, such as the red box in Figure 3 , these bubbles should also be removed in segmentation, but the traditional threshold segmentation method can only use the gray scale feature information, so it cannot remove them according to their morphological characteristics. However, the present embodiment is a segmentation network based on deep learning to segment the platelet cell bright-field image, the segmentation network mainly emphasizes that the size of the receptive field is adapted to the actual size of the platelet cell, and in addition, the network only needs to extract low-level semantic features of the image, rather than global semantic features. The segmentation result of the present embodiment is shown in Figure 5 , after segmentation by the segmentation network and connected domain filling, the noise, background and irrelevant bubbles are removed, and each platelet cell is well segmented.

[0080] (3) Screening of suitable platelet cell regions, including removal of platelet cell regions with non-compliant circularity, removal of platelet cell regions with non-compliant connected domain area, and removal of platelet cell regions with non-compliant minimum bounding rectangle aspect ratio;

[0081] Cutting 4.875 μm x 4.875 μm size single platelet cell bright field image. Since platelet cells are prone to adhesion, and the adhesion of platelet cells needs to be filtered, the post-processing stage after segmentation is set up with multiple criteria to filter the adhesion of platelet cells and the edge of the incomplete platelet cells, such as roundness, connected domain area, minimum circumscribed rectangle size, morphological characteristics, etc. In addition, the texture features of the image are extracted to further remove the bubbles and adhesion of platelet cells in the image. The texture features here are extracted by calculating the gray level co-occurrence matrix of the image. The final segmented and filtered binary image is shown in Figure 6 .

[0082] (4) Cutting 4.875 μm x 4.875 μm size single platelet cell bright field image at the center position of each screened platelet cell region and cutting the single platelet cell fluorescence image of the same region as the bright field image in the corresponding digital fluorescence image;

[0083] (5) Artificial observation of the extracted platelet fluorescence image, and artificial classification into corresponding categories according to the characteristics of different structures, single platelet microtubule fluorescence image classification into circular ring distribution, diffuse distribution, aggregation distribution, irregular distribution and invalid data: as shown in Figure 7 , the specific classification rules are as follows:

[0084] 1. If the microtubule fluorescence signal is circular ring distribution, and the average gray value of the center inside the circular ring and the average gray value of the circular ring region are less than a certain threshold, then it is classified as circular ring distribution.

[0085] 2. If the microtubule fluorescence signal is circular, solid inside and the ratio of the average gray value of the circular center to the average gray value of the circular edge region is close to 1, then it is classified as diffuse distribution.

[0086] 3. If the microtubule fluorescence signal is circular and the signal is stronger towards the center of the circle, then it is classified as aggregation distribution.

[0087] 4. If the microtubule fluorescence signal does not belong to the above three categories, then it is classified as irregular category. It is worth noting that the irregular category can be further subdivided into multiple subcategories in theory. In similar fields such as face recognition systems, metric learning methods are used to establish decision boundaries for each subcategory to improve model interpretability. However, in this problem, due to the limited subcategory dataset, it is a relatively significant open set recognition problem. We cannot exhaustively establish corresponding reference vectors for all categories, so here we only limit the distance between the irregular category and the rest of the categories, and do not establish a decision boundary for the subcategory.

[0088] 5. In the process of extracting single platelet cell microtubule fluorescence images from bright field segmented images, there are inevitably some errors in the segmentation data, resulting in some single platelet cell fluorescence images extracted without obvious fluorescence signals (the image is completely black). In addition, for platelet cells with small area, the edges of the 4.875 μm x 4.875 μm single platelet cell fluorescence image extracted may have fluorescence signals of other adjacent platelet cell microtubule structures, which are invalid data that need to be removed and do not participate in the subsequent category statistics.

[0089] (6) Using the classification results obtained in step (5) as a training set, a classification network based on Resnet50 is trained.

[0090] The final model reaches a convergent state after about 100 iterations, and the accuracy rate in the test set is 93.22%, which can accurately identify the category of single platelet microtubule fluorescence images, and also proves that the differences between different categories are obvious, which meets the expectation.

[0091] Here we count the category distribution of microtubules in 10 healthy people and 10 patients, and the specific information is as follows Figure 8 It can be found that the irregular category distribution accounts for about 20%, and if this category is randomly mixed into the remaining category distribution, it will cause a very large error in the statistical result. Objectively, the relatively large number of irregular categories requires us to establish a separate category for irregularities.

[0092] Example 2:

[0093] The platelet classification system based on alpha particles provided in this embodiment, Figure 9 The flowchart of intelligent recognition and category statistics of platelet alpha particle structure based on deep learning method provided for the embodiment of the present application;

[0094] (1) Batch read the digital bright field image and the corresponding digital fluorescence image of the platelet cell of the platelet alpha particle structure of the detected person, and pre-process the digital fluorescence image;

[0095] (2) Batch input the bright field image into the trained bright field platelet cell segmentation network to segment the platelet cell region; the bright field platelet cell segmentation network is based on the Res-Unet segmentation model. Since platelet cells are very small, it is difficult to separate platelet cell signals and noise background under bright field, the image signal-to-noise ratio is low, and the traditional threshold segmentation method is not suitable for this task, and its effect is very poor, Figure 3 is a platelet bright field cell image, and Figure 4is the result after using OTSU threshold segmentation and connected domain filling. It can be found that there are still a large number of background noise residues after segmentation, and most of the platelet cell outlines are not closed after segmentation, resulting in only a small amount of platelet cells being correctly positioned and extracted after connected domain filling. In addition, due to the limitation of imaging conditions, the bright field image of the platelet cell will also have some bubbles, as shown in the red box in Figure 3 , which should also be removed in segmentation, but the traditional threshold segmentation method can only use the gray scale feature information, so it cannot remove them according to their morphological features. The present embodiment is a segmentation network based on deep learning to segment the bright field image of the platelet cell. The segmentation network mainly emphasizes that the size of the receptive field is adapted to the actual size of the platelet cell. In addition, the network only needs to extract low-level semantic features of the image, rather than global semantic features. The segmentation result of the present embodiment is shown in Figure 5 . After segmentation by the segmentation network and connected domain filling, the noise, background and irrelevant bubbles are removed, and each platelet cell is well segmented.

[0096] (3) Post-processing the segmentation result of the bright field image to select suitable platelet cell regions. The post-processing steps include removing platelet cell regions with non-conforming circularity, removing platelet cell regions with non-conforming connected domain area, and removing platelet cell regions with non-conforming aspect ratio of minimum bounding rectangle;

[0097] Cutting a single platelet cell bright field image of 4.875 μm x 4.875 μm size. Since platelet cells are prone to adhesion, adherent platelet cells need to be filtered, so the present embodiment sets multiple criteria for post-processing of the segmented platelet cells to filter adherent platelet cells and platelet cells with incomplete edges, such as circularity, connected domain area, minimum bounding rectangle size, morphological features, etc. In addition, the texture features of the image are extracted to further remove bubbles and adherent platelet cells in the image. The texture features are extracted by calculating the gray level co-occurrence matrix of the image. The binary image after segmentation and filtering is shown in Figure 6 , where each individual platelet cell is correctly extracted.

[0098] (4) Cutting a single platelet cell bright field image of 4.875 μm x 4.875 μm size at the center position of each selected platelet cell region and cutting a single platelet cell fluorescence image of the same area as the bright field image in the corresponding digital fluorescence image;

[0099] (5) Artificially observe the extracted platelet alpha granule fluorescence images, and artificially classify them into corresponding categories according to their different structural characteristics. Single platelet alpha granule fluorescence image classification is ring distribution, aggregation distribution, scattered distribution and invalid data, as shown in FIG. 8. Figure 10 The specific classification rules are as follows:

[0100] 1. If the granule signal in the alpha granule fluorescence image is wrapped into a ring, it is classified as ring distribution.

[0101] 2. If the alpha granule fluorescence image is aggregated in the center of the cell image, it is classified as aggregation distribution.

[0102] 3. If the alpha granule fluorescence image is scattered in the cell image, it is classified as scattered distribution.

[0103] 4. The alpha granule and microtubule structure are also set up as invalid data category, including image black image data and image edge with other adjacent platelet cell microtubule structure fluorescence signal image data.

[0104] (6) The classification results obtained in step (5) are used as a training set to train a classification network based on Resnet50.

[0105] The final model reaches a convergence state after about 100 iterations, and the accuracy rate in the test set reaches 93.83%, which can accurately identify the category of single platelet alpha granule fluorescence image, and also proves that the difference between different categories is obvious, which meets the expectation.

[0106] Those skilled in the art will readily understand that the above description is only a preferred embodiment of the present application, and is not intended to limit the present application. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A platelet classification system based on submicroscopic structure, characterized in that, The system is based on a deep convolutional neural network; Its input layer is used to input a single platelet microscopic image of a preset specification. The single platelet microscopic image includes a fluorescence channel image and a bright field channel image. The fluorescence channel presents a fluorescent labeling signal. Its output layer includes output neurons corresponding to submicroscopic structure categories, platelet output neurons for irregular categories, and output neurons for invalid image data; The submicroscopic structure categories are classified according to the distribution type of fluorescence signals in the fluorescence channel image and the positional relationship of the fluorescence signals relative to the platelet edge; for platelet microtubule structures, they are divided into annular distribution, diffuse distribution, and aggregated distribution; for alpha granule structures, they are divided into annular distribution, aggregated distribution, and scattered distribution. The irregular category platelet output neurons are used to output single platelet microscopic images of the category corresponding to the output neurons of the bright field channel having complete cell structure but whose fluorescence channel image does not belong to the submicroscopic structure category. The invalid image data output neuron is used to output an image of a single platelet fiber with an intact cellular structure in the bright field channel, while the fluorescence channel image does not detect fluorescence signals within the platelet cell range.

2. The platelet classification system based on submicroscopic structure as described in claim 1, characterized in that, The deep convolutional neural network is ResNet, VGGNet, or GoogleNet.

3. The platelet classification system based on submicroscopic structure as described in claim 1, characterized in that, The microscopic images of individual platelets were obtained using the following method: (1) Acquiring bright-field and fluorescence images: In situ bright-field and fluorescence microscopy imaging of platelets with submicroscopic fluorescent labels was performed using a microscope to acquire digital bright-field images and corresponding digital fluorescence images within the microscope field of view. The digital fluorescence images were then subjected to grayscale normalization and pixel size interpolation alignment. (2) Platelet segmentation: The digitized bright-field image obtained in step (1) is segmented using an image segmentation algorithm, and a single platelet cell region is obtained by cropping the image to a size of 4.875μm×4.875μm. (3) Bright field channel image screening: The single platelet cell region obtained in step (2); detect the image edge, and use roundness, connected region area, minimum bounding rectangle size, and morphological features to screen out adhered platelet cells and platelet cell regions with incomplete edges, and screen out bubble regions by extracting the texture features of the image to obtain the bright field channel image of a single platelet cell; (4) Acquisition of fluorescence channel images: The same region as the bright field channel image of a single platelet cell obtained in step (3) is cropped from the digital fluorescence image of the submicroscopic structure of platelets and used as the fluorescence channel image of a single platelet cell; (5) Combination of individual platelet microscopic images: The bright field channel image of a single platelet cell is superimposed with the corresponding fluorescence channel image of a single platelet cell to obtain a single platelet microscopic image.

4. The platelet classification system based on submicroscopic structure as described in claim 3, characterized in that, The image segmentation algorithm preferably uses the Res-Unet segmentation network. The Res-Unet segmentation network mainly emphasizes that the size of the receptive field is adapted to the actual size of the platelet cells, and the network only needs to focus on extracting low-level semantic features rather than global semantic features to segment each platelet cell well. The image texture features are extracted by calculating the gray-level co-occurrence matrix of the image.

5. The platelet classification system based on submicroscopic structure as described in claim 1, characterized in that, in: Fluorescent signals exhibiting a ring-like distribution, with the average gray value at the center of the ring being less than a certain threshold compared to the average gray value of the ring-shaped region, are classified as ring-like distributions. Microtubule fluorescence signals exhibiting a circular shape, with a solid interior and the ratio of the average gray value at the center of the circle to the average gray value at the edge of the circle being close to 1, are classified as diffuse distributions. Microtubule fluorescence signals exhibiting a circular shape with the signal becoming stronger towards the center of the circle are classified as aggregated.

6. The platelet classification system based on submicroscopic structure as described in claim 1, characterized in that, in, If the particle signal in an alpha particle fluorescence image is enclosed in a ring, it is classified as a ring distribution; if the particle signal in an alpha particle fluorescence image is clustered in the center of the cell image, it is classified as an aggregated distribution; if the particle signal in an alpha particle fluorescence image is scattered throughout the cell image without a fixed shape, it is classified as a scattered distribution.

7. The training method for a platelet classification system based on submicroscopic structure as described in any one of claims 1 to 6, characterized in that: Training data acquisition: Individual platelet microscopic images are manually labeled, and the characteristics of different structures are manually observed to screen out those that meet the submicroscopic structure category; The pre-classified data is used to train a deep learning network through supervised learning.

8. The training method for the platelet classification system based on submicroscopic structure as described in claim 7, characterized in that: The loss function is the sum of the CosFace loss and the center loss, and the specific formula is as follows: ； in The size of the feature vector is set manually to ensure that the final decision is not affected by the length of the feature vector; parameters Control the scope of decision-making. The angle between the feature vector and the decision vector; For hyperparameters, For the feature vector of the sample, This is the reference decision vector for the corresponding category.

9. A platelet classification method based on submicroscopic structure, characterized in that, Includes the following steps: S1. Obtain digital bright-field images and corresponding digital fluorescence images of platelet cells with submicroscopic platelet structure, and perform grayscale normalization and pixel size interpolation alignment on the digital fluorescence images to obtain a single platelet microscopic image. S2. Input the microscopic image of a single platelet obtained in step S1 into the platelet classification system as described in any one of claims 1 to 6 to obtain the platelet classification result.

Images