A reticulocyte identification and grading system based on blood smears
By using an automated microscope platform and deep learning technology, combined with image segmentation and classification algorithms, we have achieved accurate identification and maturity grading of reticulocytes, solving the problems of inconsistent results and low efficiency in existing methods, and improving the automation and consistency of detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- RUIJIN HOSPITAL AFFILIATED TO SHANGHAI JIAO TONG UNIV SCHOOL OF MEDICINE
- Filing Date
- 2025-10-22
- Publication Date
- 2026-07-28
AI Technical Summary
Existing methods for detecting reticulocytes are susceptible to environmental and human factors, resulting in inconsistent results and low efficiency, and lack a complete automated solution.
Blood smear images were acquired using an automated microscope platform. By combining image segmentation and deep learning technologies, accurate identification, maturity grading, and diagnostic statistics of reticulocytes were achieved. The Reinhard color migration algorithm was used to reduce staining differences, MaskR-CNN was used for cell segmentation, SVM classifier was used for identification, and ResNet-50 and MLP were used for maturity grading.
The system achieves fully automated reticulocyte detection, improving detection efficiency and result consistency, reducing human error and false positive rate, and enhancing the system's adaptability under different conditions.
Smart Images
Figure CN121354096B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of medical cell morphology testing technology, and more specifically, to a reticulocyte identification and grading system based on blood smears. Background Technology
[0002] Reticulocytes are late-stage immature cells in the erythrocyte maturation process. Their cytoplasm contains a small amount of basophilic RNA, which, after staining with a specific dye (such as brilliant creosote blue), reveals a blue-green reticular structure under a microscope. As an important indicator reflecting bone marrow erythropoietic function and erythropoietic activity, reticulocyte analysis is of great significance for the diagnosis, classification, and monitoring of treatment efficacy in anemia.
[0003] Currently, the commonly used methods for detecting reticulocytes in laboratories are flow cytometry and manual microscopy. Flow cytometry typically uses fluorescent dyes (such as thiazole orange) to stain the RNA within reticulocytes. By detecting the forward scattered light (reflecting cell size) and fluorescence signal (reflecting RNA content), the percentage (RET%), absolute value, and maturity genotype (e.g., high fluorescence intensity reticulocytes HFR, medium fluorescence intensity reticulocytes MFR, and low fluorescence intensity reticulocytes LFR) are calculated. Manual microscopy, on the other hand, relies on the laboratory personnel to visually observe stained blood smears under a microscope, typically requiring the counting of reticulocytes in 1000 red blood cells.
[0004] However, both of these traditional methods have significant limitations: flow cytometry is susceptible to interference from contaminating cells or debris in the sample, potentially leading to false positives; its detection performance is also easily affected by factors such as ambient temperature, dye stability, and laser light source characteristics. Manual microscopy is time-consuming and labor-intensive, and its results are easily influenced by factors such as the operator's subjective experience, the quality of blood smear preparation, staining time, and field of view selection, resulting in poor repeatability. For example, the accuracy of manual counting often fluctuates significantly due to uneven blood smear thickness, inconsistent staining conditions, and differences in judgment between observers.
[0005] In recent years, with the development of computer vision and deep learning technologies, numerous studies have focused on developing automated identification methods based on blood smear images. Research indicates that using deep neural networks such as Faster R-CNN can achieve high-precision detection of reticulocytes. Patented solutions also utilize optical microscopy platforms to acquire reticulocyte images and then train a YOLOv5 model with labeled data for identification and counting. However, existing automated methods primarily focus on the detection and counting of reticulocytes, lacking a complete solution that integrates cell detection, classification, and maturity grading from conventional microscopic images.
[0006] In view of this, the present invention proposes a fully automated reticulocyte identification and grading system. This system can automatically acquire blood smear images under a standard microscope and integrate image segmentation and deep learning classification technologies to achieve accurate identification, maturity grading, and diagnostic statistics of reticulocytes, aiming to improve detection efficiency and result consistency. Summary of the Invention
[0007] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a reticulocyte identification and grading system based on blood smears, which solves the problems mentioned in the background art through the following solutions.
[0008] To achieve the above objectives, the present invention provides the following technical solution: a reticulocyte identification and grading system based on blood smears, comprising: The image acquisition module employs an automated microscope platform to continuously scan blood smears at 100x magnification, acquiring high-quality brightfield channel images through autofocus technology. This module monitors image sharpness, contrast, and illumination uniformity to ensure the integrity and usability of each field of view, providing a reliable data foundation for subsequent processing.
[0009] The coloring normalization module addresses color differences caused by different batches and coloring conditions by employing the Reinhard color transfer algorithm. First, a field-view image with ideal coloring and uniform illumination is selected as the global baseline. Then, the color statistical features (mean, standard deviation) of all images to be processed are mapped to this baseline color space, significantly reducing the impact of differences in coloring time, dye batches, and shooting equipment on the performance of subsequent deep learning models.
[0010] The instance segmentation module uses an improved MaskR-CNN deep learning network to accurately detect and segment red blood cells in the field of view image at the pixel level. This network utilizes a pre-trained backbone network to extract multi-scale features, detect the location of each red blood cell, and generate a corresponding segmentation mask. To address challenges such as cell overlap and blurred boundaries, post-processing using morphological operations and boundary correction algorithms ensures that each red blood cell can be accurately separated and cropped into a single-cell image.
[0011] The erythrocyte classification module extracts morphological features from the RGB channels, quantitative blue-purple correlation features from the HSV channels, and color difference features from the LAB channels from the segmented single-cell images. A weighted mechanism is applied during extraction to highlight intracellular blue-purple RNA signals and reticular structure features, reducing the influence of irrelevant noise. The normalized feature vector is then input into an SVM classifier to distinguish between reticulocytes and non-reticulocytes. Through this classification step, the system can accurately screen out reticulocytes with reticular RNA structures.
[0012] The reticulocyte grading module further grades the maturity of identified reticulocytes based on the distribution and morphological characteristics of intracellular RNA residues. This module first extracts deep representations of cell images using a pre-trained ResNet-50, then fuses these deep features with blue-purple / morphological features. The fused features are standardized and dimensionality-reduced before being input into the MLP to reduce redundancy and improve generalization performance. The MLP classifier employs an ordinal regression strategy to maintain the natural sequence relationships between maturity levels, classifying reticulocytes into four levels: 1, 2, 3, and 4.
[0013] The statistical output module summarizes the test results according to two dimensions: field of view and patient, calculates the proportion of reticulocytes and the statistical value of blue-purple intensity at different maturity levels, and generates a visualized diagnostic report.
[0014] Preferably, the staining normalization module specifically includes: performing white balance correction and exposure normalization on all images to be processed and excluding the background using a threshold or Otsu method, calculating statistics only on tissue / cell pixels; converting the reference image and each image to be processed from RGB to Lab color space, calculating the mean and standard deviation of each channel of the reference image, and performing a channel-by-channel linear transformation of the image to be processed on the three Lab channels according to the Reinhard mapping.
[0015] Preferably, the segmentation results of the MaskR-CNN further employ boundary correction and small target enhancement algorithms to ensure accurate segmentation of individual red blood cells when cells overlap or boundaries are blurred.
[0016] Preferably, the boundary correction involves: performing morphological reconstruction on cells with blurred or overlapping edges to ensure the integrity of individual red blood cell outlines; simultaneously, the system applies interpolation and sharpening operations to red blood cells with smaller areas to achieve small target enhancement, avoid segmentation omissions, and finally outputs a single-cell cropped image set, with each cell stored separately and accompanied by corresponding mask information.
[0017] Preferably, the multidimensional features include HSV channel features, LAB channel features, and RGB channel features.
[0018] Preferably, the HSV channel features include: blue-purple staining intensity features, specifically including: average blue-purple intensity, total intensity, and area ratio within the cell; blue-purple texture features, specifically including: contrast, energy, entropy based on the gray-level co-occurrence matrix (GLCM), as well as local binary pattern (LBP), signal variance, and skewness; the LAB channel features include: color features showing the deviation between the blue-purple hue and the baseline hue; the RGB channel features include: morphological features such as cell area, diameter, roundness, and aspect ratio; these multidimensional features are fused and input into a support vector machine (SVM) classifier to achieve automatic identification of reticulocytes and non-reticulocytes.
[0019] Preferably, before the SVM discrimination, the red blood cell classification module employs a weighting mechanism on the extracted features to highlight the intracellular blue-purple RNA signal and network structure features, thereby reducing the impact of noise interference on the classification results.
[0020] Preferably, the ordinal regression strategy employs cumulative probability modeling to reflect the natural sequence relationship of reticulocyte maturity grading, thus avoiding the inconsistency of class boundaries caused by traditional multi-classification methods.
[0021] Preferably, when the reticulocyte grading module concatenates the deep features extracted by ResNet-50 with traditional features, it adopts a feature standardization and dimensionality reduction fusion strategy to reduce dimensionality redundancy and improve the generalization performance of the MLP classifier.
[0022] Preferably, the statistical output module further provides dynamic quality control based on the sample level, including abnormal strong blue-purple detection, image blur warning, and classification confidence histogram, to assist in manual review.
[0023] The technical effects and advantages of this invention are as follows: 1. This invention achieves fully automated processing from sample to report, balancing interpretability (multi-feature fusion) and high classification accuracy (combination of deep learning and traditional image analysis), significantly reducing subjective differences and false negative rates in manual microscopic examination, and improving the efficiency and consistency of clinical testing; 2. Through the transfer learning mechanism, the system can use a small number of labeled samples to adapt across centers and devices, thereby improving the generalization ability and robustness under different imaging conditions and staining protocols. Attached Figure Description
[0024] Figure 1 This is a schematic diagram of the overall structure of the present invention; Figure 2 This is a visual diagram illustrating the grading of reticulocyte maturity in this invention. Detailed Implementation
[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0026] As attached Figure 1-2 The system described is a reticulocyte identification and grading system based on blood smears, comprising an image acquisition module, a staining normalization module, an instance segmentation module, an erythrocyte classification module, a reticulocyte grading module, and a statistical output module. The system achieves end-to-end fully automated processing, reducing manual missed detections and grading discrepancies, and improving clinical testing efficiency and consistency. Specifically, the output of the image acquisition module is electrically connected to the input of the staining normalization module; the output of the staining normalization module is electrically connected to the input of the instance segmentation module; the output of the instance segmentation module is electrically connected to the input of the erythrocyte classification module; the output of the erythrocyte classification module is electrically connected to the input of the reticulocyte grading module; and the output of the erythrocyte classification module is electrically connected to the input of the statistical output module.
[0027] The image acquisition module uses an automated microscope platform to continuously scan blood smears at 100x magnification, acquires high-quality bright-field channel images through autofocus technology, and monitors image sharpness, contrast, and illumination uniformity. The staining normalization module: for color differences caused by different batches and different staining conditions, selects a slice or field-of-view image as a reference image, and performs Reinhard color normalization processing on all images to be processed according to the reference image; In this embodiment, the specific steps are as follows: First, select a uniformly stained field of view or slice with normal lighting and white balance from the same batch as a reference image. Perform white balance correction and exposure normalization on all images to be processed, and exclude the background using thresholding or Otsu's algorithm. Calculate statistics only on tissue / cell pixels. Convert the reference image and each image to be processed from RGB to Lab color space. Calculate the mean and standard deviation of each channel of the reference image and perform a linear transformation on each channel of the image to be processed using Reinhard mapping on the three Lab channels. After transformation, crop the pixel values to the legal range and transform them back to RGB. When calculating the mean and variance, truncated mean or extreme values can be used to improve robustness. When there are significant local lighting differences, perform local exposure correction first and then Reinhard normalization. Record the reference ID and statistics used in all operations for reproducibility. The normalization result serves as the standard input for subsequent segmentation / classification, thereby significantly reducing the influence of batch-to-batch color difference when there are changes in light source, white balance, or staining depth.
[0028] The instance segmentation module uses a MaskR-CNN network to detect and segment red blood cells in the field of view image at the pixel level, and outputs a single-cell cropped image and a corresponding mask. In this embodiment, the specific steps are as follows: After image acquisition, the red blood cell detection and segmentation stage begins. The bright-field image is input into the MaskR-CNN deep learning network for cell detection and segmentation. The pre-trained backbone network ResNet-101 first extracts image features. The Region Proposal Network (RPN) generates candidate cell regions. The classification and masking branches discriminate each candidate region and generate corresponding pixel-level segmentation masks. The system performs post-processing on the output segmentation results. During boundary correction, the system performs morphological reconstruction on cells with blurred or overlapping edges to ensure the integrity of individual red blood cell outlines; simultaneously, the system applies interpolation and sharpening operations to red blood cells with smaller areas to achieve small target enhancement and avoid segmentation omissions. The final output is a set of single-cell cropped images, with each cell stored separately and accompanied by corresponding mask information.
[0029] The red blood cell classification module converts the segmented single-cell image into HSV and LAB channels based on the RGB channels. In the HSV channel, it automatically segments the blue-purple region from the background using k-means clustering. Then, it extracts the following multidimensional features from the RGB, HSV, and LAB channels. In this embodiment, it should be specifically noted that: for each red blood cell image obtained in the red blood cell classification module, the feature extraction module analyzes the RNA staining region and morphology inside the cell, specifically including: 1. Blue-violet intensity characteristics (1) Average blue-violet intensity of cells: The average blue-violet intensity is the average gray value of all pixels within the cell region, and its calculation formula is as follows: ,in, coordinates The pixel grayscale value at that location, This represents the total number of pixels in the cell region.
[0030] (2) Total blue-purple integral intensity: The total blue-purple integral intensity is the sum of the gray values of all pixels within the cell region, and its calculation formula is: .
[0031] (3) Area proportion of blue-purple region: The area proportion of blue-purple region is the ratio of the number of blue-purple pixels to the total number of pixels in the cell region. The calculation formula is as follows: ,in, For grayscale values greater than the threshold The number of pixels.
[0032] 2. Texture features of the blue-purple area (1) Contrast, energy and entropy based on GLCM: constructing gray-level co-occurrence matrix for cell regions After normalization, we get ,but: Contrast: energy: entropy: (2) LBP: The LBP is obtained by comparing the grayscale difference between neighboring pixels and the center pixel. The calculation formula is as follows: ,in, The grayscale value of the center pixel. This represents the grayscale value of the neighboring pixels.
[0033] (3) Signal variance: (4) Signal skewness: 3. Morphological characteristics (1) Cell area: The cell area refers to the number of pixels in the cell region. (2) Cell diameter (equivalent circle diameter): (3) Cell roundness: Roundness refers to the degree to which a cell region approximates an equivalent circle, and its formula is: in, This represents the perimeter of the cell region.
[0034] (4) Cell length-to-width ratio ,in, and These are the long and short sides of the smallest bounding rectangle of the cell, respectively.
[0035] 4. Color Features: The color features described are based on the Lab color space and are obtained by calculating the deviation between the average color of the cell region and a reference blue-violet value. Let the average pixel value within the region be... The mean value of blue-purple was taken from RNA staining areas deemed by doctors to be uniformly stained, with normal light and white balance, and highly representative. The blue-purple tone deviation is: The red blood cell classification module classifies each red blood cell image, extracts morphological features from the bright field image, inputs the extracted features into the SVM model, and outputs whether the red blood cell is a reticulocyte or another type of red blood cell. The system automatically marks all reticulocytes.
[0036] The reticulocyte grading module: Based on the multidimensional features, it increases the maximum pixel intensity in the blue-purple region, and simultaneously integrates the deep representation and prior features extracted by ResNet-50, and inputs them into the multilayer perceptron model. Combined with the ordinal regression strategy, it grades the maturity of reticulocytes. In this embodiment, it is specifically noted that for cell images identified as reticulocytes, the system's grading module extracts more detailed features to assess maturity. First, traditional features in the RNA-stained regions within the reticulocytes are recalculated. These extracted traditional features, based on those selected in S4, are augmented with the maximum pixel intensity in the blue-purple region to better distinguish reticulocytes of different grades. The system grades reticulocytes based on the feature values extracted in S5-1. This module fuses the deep representation extracted by ResNet-50 (a 2048-dimensional vector obtained from the 5th convolutional stage through global average pooling, reduced to 256 dimensions by PCA) with traditional features and inputs it into an MLP model. Combined with an ordinal regression strategy, the reticulocytes are graded into levels 1, 2, 3, and 4.
[0037] The statistical output module calculates the proportion of reticulocytes at each level and the blue-purple intensity statistics, and generates visualized quality control results and diagnostic reports.
[0038] In this embodiment, the following points need to be specifically explained: Statistics and visualization: For each processing batch, the system summarizes the single-cell detection and grading results by field of view level and patient level, and calculates the total number of cells. Reticulocyte count and the percentage of each level (1 / 2 / 3 / 4) For each cell identified as reticulocyte, extract the bright-field to grayscale intensity. The intensity of blue-purple is used as an indicator, and the mean and standard deviation of each level and the overall population are calculated. ( 1, 2, 3, 4 The system generates a visual report based on this data—a field-of-view heatmap (weave red rate), a grade percentage chart, box plots of intensity for each grade, and a blue-purple intensity histogram—facilitating a quick understanding of distribution and anomalies. Quality control, verification, and output: Simultaneously, the statistical output module implements dynamic quality control rules and generates a verification checklist: if the average intensity of a certain field of view... Exceeding the historical average The sum of the values is three times the standard deviation, indicating abnormally high intensity; if the image horizontal blur, indicated by Laplacian variance, is below a threshold, a blurry retake is suggested; if the maximum confidence score for classification / grading is below a threshold set to 0.6, the image is automatically added to the manual review queue. All statistical results, QC tags, samples to be reviewed, and metadata used for calculation (benchmark image ID, normalized parameters, and exposure data) are included. Data such as gain can be saved and exported as CSV files, along with accompanying PDF visualization reports, facilitating rapid assessment and tracing of batch issues by laboratory personnel and feedback of verification results for model optimization.
[0039] In summary, the system automatically acquires bright-field images per field of view under 100× oil immersion. First, the entire batch of images is normalized using the Reinhard method based on a representative field of view selected by the doctor. Then, the normalized field of view is fed into a Mask R-CNN with ResNet-101+FPN as its backbone to complete pixel-level instance segmentation and crop single-cell images. For each single cell, compact traditional features are extracted, and an SVM classification model is used to distinguish between reticulocytes and non-reticulocytes. For samples classified as reticulocytes, ResNet-50 is reused to extract dimensionality-reduced global pooling deep vectors, which are then concatenated with traditional features and input into a lightweight MLP. Ordinal regression is used to classify reticulocytes into levels 1, 2, 3, and 4. The statistics module generates a visual report based on the proportion and intensity distribution of the field of view and the patient, and implements dynamic quality control (abnormal intensity, fuzzy alerts, low confidence verification). The verification results can be fed back for periodic fine-tuning to achieve cross-batch / cross-device adaptation.
[0040] The entire system uses Reinhard staining normalization as the starting point of the process, which significantly reduces the impact of batch staining and illumination differences on downstream models. Then, it adopts a hybrid two-stage strategy of traditional features + SVM → deep feature fusion + MLP + ordinal regression, which balances speed, high recall and interpretability. Finally, it has a built-in dynamic quality control and manual review closed loop, which makes the model output verifiable and traceable.
[0041] All deep learning models used in this invention can be trained on labeled datasets. This system uses 5000 cell images labeled with reticulocytes to train a Mask R-CNN network for accurate red blood cell detection; and thousands of cell images labeled as reticulocytes and ordinary red blood cells to train a ResNet classifier for binary classification. In the final reticulocyte maturity grading module, traditional image processing methods and automatic neural network feature learning are combined for feature extraction, improving the robustness of the research.
[0042] Secondly: The accompanying drawings of the embodiments disclosed in this invention only involve the structures involved in the embodiments disclosed in this invention. Other structures can refer to the general design. In the absence of conflict, the same embodiment and different embodiments of this invention can be combined with each other. In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A reticulocyte identification and grading system based on blood smears, characterized in that, include: Image acquisition module: The automated microscope platform continuously scans blood smears at 100x magnification, acquires high-quality bright-field channel images through autofocus technology, and monitors image sharpness, contrast and illumination uniformity. Staining normalization module: For color differences caused by different batches and different staining conditions, a slice or field-of-view image is selected as a reference image, and all images to be processed are subjected to Reinhard color normalization processing based on the reference image. Instance segmentation module: The MaskR-CNN network is used to detect and segment red blood cells in the field of view image at the pixel level, and outputs single-cell cropped images and corresponding masks; Red blood cell classification module: Converts the segmented single-cell images into HSV and LAB channels based on the RGB channels. In the HSV channel, k-means clustering is used to automatically segment the blue-purple region from the background. Then, the following multidimensional features are extracted from the RGB, HSV and LAB channels. The multidimensional features include HSV channel features, LAB channel features, and RGB channel features; The HSV channel features include: blue-purple staining intensity features, specifically including: average blue-purple intensity, total intensity, and area ratio within the cell; blue-purple texture features, specifically including: contrast, energy, entropy based on the gray-level co-occurrence matrix (GLCM), as well as local binary pattern (LBP), signal variance, and skewness; the LAB channel features include: color features showing the deviation between the blue-purple hue and the baseline hue; the RGB channel features include: morphological features such as cell area, diameter, roundness, and aspect ratio; these multidimensional features are fused and input into a support vector machine (SVM) classifier to achieve automatic identification of reticulocytes and non-reticulocytes. Reticulocyte grading module: Based on the multidimensional features, the maximum pixel intensity in the blue-purple region is increased. At the same time, the deep representation and pre-sequence features extracted by ResNet-50 are fused and input into the multilayer perceptron model. Combined with the ordinal regression strategy, the maturity of reticulocytes is graded. Before the SVM discrimination, the red blood cell classification module uses a weighting mechanism to extract features to highlight intracellular blue-purple RNA signals and network structure features, thereby reducing the impact of noise interference on the classification results. Statistical output module: Calculates the proportion of reticulocytes at each level and the blue-purple intensity statistics, and generates visualized quality control results and diagnostic reports.
2. The reticulocyte identification and grading system based on blood smears according to claim 1, characterized in that: The staining normalization module specifically includes: performing white balance correction and exposure normalization on all images to be processed and excluding the background using a threshold or Otsu method, calculating statistics only on tissue / cell pixels; converting the baseline image and each image to be processed from RGB to Lab color space, calculating the mean and standard deviation of each channel of the baseline image, and performing a linear transformation on each channel of the image to be processed in the Lab three channels according to the Reinhard mapping.
3. The reticulocyte identification and grading system based on blood smears according to claim 1, characterized in that: The MaskR-CNN segmentation results are further enhanced with boundary correction and small target enhancement algorithms to ensure accurate segmentation of individual red blood cells when cells overlap or boundaries are blurred.
4. The reticulocyte identification and grading system based on blood smears according to claim 3, characterized in that: The boundary correction involves morphological reconstruction of cells with blurred or overlapping edges to ensure the integrity of individual red blood cell outlines. Simultaneously, the system applies interpolation and sharpening operations to red blood cells to enhance small targets and avoid segmentation omissions. The final output is a set of single-cell cropped images, with each cell stored separately and accompanied by corresponding mask information.
5. The reticulocyte identification and grading system based on blood smears according to claim 1, characterized in that: The ordinal regression strategy employs cumulative probability modeling to reflect the natural sequence relationships of reticulocyte maturity grading, thus avoiding the inconsistency of class boundaries caused by traditional multi-classification methods.
6. The reticulocyte identification and grading system based on blood smears according to claim 1, characterized in that: When the reticulocyte grading module concatenates the deep features extracted by ResNet-50 with traditional features, it adopts a feature standardization and dimensionality reduction fusion strategy to reduce dimensionality redundancy and improve the generalization performance of the MLP classifier.
7. The reticulocyte identification and grading system based on blood smears according to claim 1, characterized in that: The statistical output module further provides dynamic quality control based on the sample level, including abnormal strong blue-purple detection, image blur warning, and classification confidence histogram, to assist manual review.