An automated method and system for segmenting and counting fundus cells

Abstract

This invention discloses an automatic segmentation and counting method and system for fundus cells. After preprocessing fundus cell images, the data is input into a deep learning model. The deep learning model is trained by introducing a lightweight coupled segmentation head with an embedded attention mechanism module into the YOLOv8-seg cell segmentation model. The attention mechanism module divides the feature map into G sub-features and feeds them into two parallel branches. One branch generates a weighted feature map P. 1×1 Another extracted feature map P 3×3 2D global average pooling is used for P 3×3 and P 1×1 Global spatial encoding generates weights w 3×3 and w 1×1 Perform matrix multiplication on the feature map and weights to generate cross-space interaction weights, and multiply these weights element-wise with x to obtain Y∈R. C×H×W This invention can automatically and accurately select RGCs cells in a fundus model.

Description

Technical Field

[0001] This invention pertains to image processing technology, and particularly relates to an automatic segmentation and counting method and system for fundus cells. Background Technology

[0002] Glaucoma is a group of diseases characterized by damage to optic nerve cells, visual field loss, and developmental abnormalities. It is caused by intraocular pressure (IOP)-related optic neuropathy accompanied by the loss of retinal ganglion cells (RGCs). When glaucoma patients exhibit typical visual field defects, such as blurred or lost vision, more than 50% of retinal ganglion cells may have already been lost. Therefore, significant changes in the number of RGCs provide a basis for the early diagnosis of glaucoma.

[0003] To quantify the impact of glaucoma on retinopathy-cell masses (RGCs), researchers often use mouse models and employ specific markers such as RBPMS, Brn3a, and THY1 for immunohistochemical staining to identify and label RGCs. Despite the availability of various effective markers for RGC identification, accurately counting these cells remains a significant challenge. Manual RGC counting is not only time-consuming and labor-intensive but also susceptible to subjective bias. Therefore, several RGC counting methods based on traditional algorithms have emerged. These methods rely on setting appropriate grayscale thresholds or pre-defined template matching and are sensitive to changes in cell size and shape, making them difficult to count large numbers of RGCs in complex backgrounds. For example, the ImageJ script developed by Geeraerts et al. can semi-automatically quantify RGCs throughout the retina, but requires manual parameter adjustment; while the automated software developed by Guymer et al. can effectively count immunolabeled RGCs, it performs poorly in detecting low-quality RGCs.

[0004] With the maturity of deep learning technology, deep learning-based RGC counting algorithms have emerged. These algorithms can automatically extract abstract features from training data. However, the effectiveness of these algorithms is highly dependent on the quality of the training data, the chosen benchmark model, and the model training strategy. For example, when Masin et al. used U-Net to count RGCs labeled with RBPMS, they were unable to identify larger cells and adjacent cells; while Zhang et al. used YOLOv5 to count Brn3a-labeled RGCs, which performed well on images labeled with cell nuclei but poorly on images labeled with cytoplasm.

[0005] Furthermore, traditional manual selection has problems; the process is time-consuming and easily affected by the experimenter's subjective factors, making it impossible to achieve efficient and accurate cell region selection. Existing automated selection systems are also time-consuming and lack adaptation to images of different pixel sizes, resulting in unsatisfactory selection results when the image size is inconsistent with the reference size.

[0006] Furthermore, existing automatic annotation algorithms perform poorly on low-quality and large images. When researchers capture images of the entire retina using a microscope, images from multiple fields of view need to be stitched together to form a complete retinal image, which can lead to significant edge overlap and degraded image quality. Meanwhile, RGCs are typically very small (approximately 15×15 pixels) and distributed throughout the entire retina (the distance between cells ≤ 20 pixels). Additionally, due to overlap among many RGCs and their lack of alignment on the same horizontal plane, some RGCs exhibit lower image contrast and brightness. When the size of the entire retinal image exceeds 8000×8000 pixels, the image typically contains over 30,000 RGCs. Therefore, both manual detection and recognition algorithm detection require a significant amount of time. Summary of the Invention

[0007] The first objective of this invention is to provide an automatic segmentation and counting method for fundus cells, which can automatically and accurately select RGCs cells in a fundus model and improve the segmentation results under three conditions: low image quality, uncertain image pixel size, and excessively large image size, thereby improving experimental efficiency and the accuracy of segmentation results.

[0008] The first objective of this invention is achieved through the following technical measures: an automatic segmentation and counting method for fundus cells, characterized by comprising the following steps:

[0009] S1. Preprocess the fundus cell images;

[0010] S2. Input the preprocessed fundus cell image into the deep learning model. The deep learning model is trained by introducing a lightweight coupled segmentation head with an embedded attention mechanism module into the YOLOv8-seg cell segmentation model.

[0011] S3. The lightweight coupled segmentation head couples the classification branch and bounding box regression branch of the three branches of the YOLOv8-seg cell segmentation model segmentation head and uses shared parameters to process the three feature maps of different sizes output from the feature integration stage, while retaining an independent segmentation confidence branch for each size feature map; the attention mechanism module processes the input feature map X∈R C×H×W The neural network channels are divided into G sub-features x. i ∈R C / G×H×W These sub-features are then fed into two parallel branches, one of which processes x... i Channel attention is extracted in different directions, and cross-channel interaction is performed using a 1×1 convolutional kernel. Then, x... i Generate weighted feature map P 1×1 ∈R C / G×H×WThe other branch uses a 3×3 convolution kernel to apply to x. i Extract feature map P 3×3 ∈R C / G×H×W Then, 2D global average pooling is applied to P respectively. 3×3 and P 1×1 Perform global spatial encoding and generate weights w in the corresponding spatial dimensions. 3×3 and w 1×1 ;

[0012] S4. Perform matrix multiplication on the feature maps of different spatial dimensions and the corresponding weights in the spatial dimensions to generate cross-spatial interaction weights. The calculation formula is as follows:

[0013] w cross-spatia =sigmoid(P 3×3 ×w 1×1 +P 1×1 ×w 3×3 )

[0014] S5. Integrate cross-spatial interaction weights with x i After performing element-wise multiplication, we get Y∈R C×H×W This yields the segmentation results of individual cells, completing the selection process.

[0015] S6. Process the results of single cell segmentation and count the cells in the fundus.

[0016] This invention introduces a Lightweight Task-aligned Head (LWCH) and an Efficient Multi-Scale Attention (EMA) module into the existing YOLOv8-seg cell segmentation model. Through cross-dimensional interactions between features, it maintains high efficiency and reduces the parameter scale on the baseline, while significantly improving segmentation performance. This invention enables automatic and accurate selection of RGCs in fundus models and improves segmentation results under three conditions: low image quality, uncertain image pixel size, and excessively large images. This improves experimental efficiency and the accuracy of segmentation results.

[0017] When cell shapes are complex and the number of cells is large, storing the cell segmentation results may occupy a very large amount of memory space. Therefore, after obtaining the cell segmentation results, in order to reduce the time and space occupied by dataset processing, this invention performs contour simplification on the cell segmentation results, that is, completes the selection. Contour simplification includes the following steps:

[0018] (1) Obtain the contour point set of each cell from the single cell segmentation result. The contour point set of each cell is a set of points. Convert each set of points into a directed graph set with the beginning and end connected.

[0019] (2) Remove points on the curve recursively while keeping the maximum deviation error of the curve no more than 5%.

[0020] In step S1 of this invention, the preprocessing includes: scaling the imported fundus cell image, and dynamically adjusting the image size according to the current image scaling size and the reference diameter input by the user.

[0021] In step S6 of the present invention, the processing of the segmentation result of a single cell includes: drawing the simplified contour on the image, saving it locally, displaying the drawn segmentation result to the user, and the user saving the cell segmentation result and the corresponding contour point set to Excel for subsequent data processing and calculation.

[0022] This invention uses grouped normalization in a lightweight coupled segmentation head, which divides the feature map into multiple groups along the channel dimension of the neural network, and then calculates the mean and standard deviation of the features within each group to normalize the features.

[0023] The lightweight coupling segmentation head described in this invention provides an output channel number greater than or equal to 128 for the feature map with the largest output size during the feature integration stage.

[0024] In step S2 of this invention, each image used for training the deep learning model has two labeled results, corresponding to the RBPMS channel and the DAPI channel, respectively. This invention demonstrates excellent segmentation performance on datasets stained with DAPI and RBPMS, and can further improve the accuracy of RGCs segmentation by combining these two markers.

[0025] A second objective of this invention is to provide a system using the above-described automatic segmentation and counting method for fundus cells.

[0026] The second objective of this invention is achieved through the following technical measures: a system using the above-described automatic segmentation and counting method for fundus cells, characterized by comprising:

[0027] The image preprocessing module is used to scale the imported fundus cell images and dynamically adjust the image size based on the current image scaling size and the reference diameter input by the user.

[0028] The cell segmentation module has a deep learning model. This deep learning model receives preprocessed fundus cell images. The user selects the cell staining method and confidence level, traverses all images to select an appropriate cell segmentation strategy for segmentation, and outputs the segmentation result of a single cell.

[0029] The results processing module processes the segmentation results of individual cells and counts the cells in the fundus.

[0030] Compared with the prior art, the present invention has the following significant effects:

[0031] (1) This invention delegates the manual marking step to machines and algorithms. Through image processing algorithms, it can segment and count retinal cells, achieving automatic cell region selection. This solves the drawbacks of manual marking being time-consuming, labor-intensive, and mechanically repetitive. At the same time, it can process experimental data and export segmentation results with one click, improving experimental efficiency and accuracy. It can also solve the problem that existing automatic annotation algorithms are not effective on low-quality and large images. In addition, this invention also introduces an attention mechanism module, including a contour simplification algorithm and a dynamic adjustment strategy, to screen and retain the cell regions required by the user for suturing, and enhance the recognition performance of cells in large images and low-quality images.

[0032] (2) This invention integrates complex and redundant steps into a software tool, automates the experimental process, simplifies the processing of experimental data, provides researchers with convenient and reliable experimental tools, and makes experimental operations more efficient and reliable.

[0033] (3) This invention features a diameter calibration function during image preprocessing. It automatically scales the cell images to be processed based on user-input data to adapt to images of different pixel sizes, effectively improving the software's cell segmentation and counting performance for images of varying sizes. Simultaneously, by integrating "Slicing Aided Hyper Inference" counting, the software can transform large images into smaller image fragments. By preserving appropriate overlap between fragments, the original image can be accurately reconstructed from each fragment. This method achieves higher performance when processing large-scale images. This software can efficiently process datasets of different sizes. Attached Figure Description

[0034] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0035] Figure 1 This is a schematic diagram of the process flow and system composition of the method of the present invention;

[0036] Figure 2 This is a diagram showing the system interface of the present invention;

[0037] Figure 3 This is an interface diagram showing the selection of cell staining methods in the system of this invention;

[0038] Figure 4 This is a schematic diagram of the composition structure of the cell segmentation model of the present invention;

[0039] Figure 5 This is a schematic diagram of the attention mechanism module of the present invention;

[0040] Figure 6 This is a flowchart illustrating the contour simplification of cell segmentation results according to the present invention.

[0041] Figure 7 This is an interface display diagram showing the cell segmentation results of the present invention;

[0042] Figure 8 This is an interface display diagram showing the cell segmentation results in a large image in an example of the present invention. Detailed Implementation

[0043] The present invention will now be described in detail with reference to the embodiments and accompanying drawings to help those skilled in the art better understand the inventive concept of the present invention. However, the scope of protection of the claims of the present invention is not limited to the following embodiments. For those skilled in the art, all other embodiments obtained without creative effort without departing from the inventive concept of the present invention are within the scope of protection of the present invention.

[0044] like Figures 1 to 8 The diagram illustrates an automatic segmentation and counting method and system for fundus cells according to the present invention. The automatic segmentation and counting method for fundus cells includes the following steps:

[0045] S1. Preprocess the fundus cell images;

[0046] Preprocessing includes image scaling, reference diameter selection, and image blanking filling to improve the accuracy of the cell segmentation and counting algorithm, ensuring that experimenters can obtain more accurate cell segmentation and counting results.

[0047] After importing images, users can zoom and drag to preview them using the mouse wheel and pointer. They can also adjust the reference cell diameter by inputting a value. The system dynamically adjusts the image size based on the current image zoom level and the user-inputted reference diameter to adapt to segmentation strategies for different cell sizes. Figure 2 The image shown is the system interface for preprocessing images of fundus cells.

[0048] S2. Input the preprocessed fundus cell image into the deep learning model. This deep learning model is trained by introducing a lightweight coupled segmentation head with an embedded attention mechanism module into the YOLOv8-seg cell segmentation model. Users can select a cell staining method and a suitable cell segmentation strategy; see [link to documentation]. Figure 3 .

[0049] The existing YOLOv8-seg cell segmentation model framework uses a finely tuned and optimized CSPDarkNet53 architecture for its backbone network, and PANet as the feature pyramid network for the feature integration stage (neck). The YOLOv8-seg cell segmentation model replaces all the original C3 modules with C2f modules, incorporating more skip connections and channel separation techniques. At the end of the backbone network, YOLOv8-seg employs an SPPF (Spatial Pyramid Pooling Filter) structure, which concatenates three 5×5 max-pooling layers and integrates the output of each layer. This allows YOLOv8-seg to effectively enrich the gradient information flow while maintaining a lightweight design.

[0050] The YOLOv8-seg segment head receives feature maps P3, P4, and P5 of three different sizes from the neck. The YOLOv8-seg segment head consists of the YOLOv8-seg detection head and a YOLACT-based segmentation confidence branch (mask coefficient), employing a decoupled head structure similar to YOLOx. The segment head has three branches: bounding box regression, classification, and segmentation confidence (mask coefficient). In the classification branch, an N... cls The tensor of ×H×W, N cls For the target categories in the dataset, this paper uses 1, and binary cross-entropy is used as the loss during training. A 4*reg_max×H×W tensor is generated in the bounding box regression branch, and CIOU and DFL are used as losses during training, where DFL has a hyperparameter reg_max, which is 16 in this paper. A 32×H×W tensor is generated in the mask coefficient branch, and binary cross-entropy loss is used as the loss during training. Furthermore, the largest feature map P3 is upsampled using Protonet to generate a prototype mask, which is then linearly combined with the Mask Coefficient after NMS to obtain the final mask.

[0051] M=δ(P×C T )

[0052] Where P is the Prototype Mask, δ is the sigmoid, and C T This is the mask coefficient after NMS. The predicted bounding box M is cropped using the NMS result, and then thresholded to obtain the final segmentation result.

[0053] The drawbacks of the YOLOv8-seg architecture are as follows: First, feature maps of different sizes may contain similar spatial information, and using separate segmentation heads may lead to repeated computation of the target. Second, YOLOv8-seg assigns the same parameters to each task branch, which may lead to wasted parameters when processing single-class multi-target segmentation tasks such as RGCs segmentation, especially in the classification branch. In addition, the decoupled head design may also lead to misalignment between different tasks.

[0054] S3. To address the aforementioned problems of YOLOv8-seg, this invention designs a lightweight coupled segmentation head that embeds an attention mechanism module. This lightweight coupled segmentation head... Figure 4 The gray area is represented in the image. The lightweight coupled segmentation head couples the classification branch and the bounding box regression branch of the YOLOv8-seg cell segmentation model's segmentation head and uses shared parameters to process the three feature maps P3, P4, and P5 output from the feature integration stage. That is, a single parameter-shared module processes the three feature maps of different sizes output from the neck, thus achieving effective coupling between the regression and classification branches. Furthermore, to ensure segmentation accuracy, an independent segmentation confidence branch is retained for each size feature map. Secondly, the lightweight coupled segmentation head uses GroupNorm (GN) instead of BatchNorm in the YOLOv8-seg segmentation head. Figure 4 In this context, grouping normalization is represented as Conv_GN. GroupNorm has been proven to effectively improve object detection performance. Its working principle involves dividing the feature map into multiple groups along the channel dimension of the neural network, and then calculating the mean and standard deviation of features within each group to normalize the features. This design reallocates model parameters and enhances the feature interaction between localization and classification tasks, further improving model efficiency and lightweightness. Furthermore, the lightweight coupled segmentation head provides the feature map P3, which has the largest output size, with 128 or more output channels (up from the original 64). Due to the increased number of output channels in feature map P3, Protonet receives richer spatial information to generate a more accurate mask prototype.

[0055] like Figure 5 The diagram shows the structure of the attention mechanism module EMA. The attention mechanism module processes the input feature map X∈R. C ×H×W Divide the channel into G sub-features x i ∈R C / G×H×WThe aim is to learn different semantics across channels; in this paper, G is set to 8. These sub-features are then fed into two parallel branches, one of which processes x... i Channel attention is extracted in different directions, and cross-channel interaction is performed using a 1×1 convolutional kernel. Then, x... i Generate weighted feature map P 1×1 ∈R C / G×H×W (that is) Figure 5 In the concat+conv1x1 method, the other branch passes through a 3×3 convolution kernel (i.e., ...). Figure 5 In the case of conv3x3), x i Extract feature map P 3×3 ∈R C / G×H×W Through parallel subnetworks, that is... Figure 5 (The gray area on the left) EMA can encode information between channels to adjust the weights of different channels and retain fine spatial structure information in the channels.

[0056] Next, EMA proposes a cross-spatial aggregation method across different spatial dimensions to achieve richer feature aggregation. Specifically, it uses 2D global average pooling (i.e., ...) Figure 5 In cross-spatial learning, avgpooling is used to evaluate P. 3×3 and P 1×1 Perform global spatial encoding and generate weights w in the corresponding spatial dimensions using Softmax. 3×3 (The softmax output at the top in Cross-spatial learning) and w 1×1 (The softmax output at the bottom of Cross-spatial learning), Cross-spatial learning is... Figure 5 The gray area on the right side of the middle;

[0057] S3. Perform matrix multiplication on the feature maps of different spatial dimensions and the corresponding weights in the spatial dimensions to generate cross-spatial interaction weights. The calculation formula is as follows:

[0058] w cross-spatial =sigmoid(P 3×3 ×w 1×1 +P 1×1 ×w 3×3 )

[0059] S4. Integrate cross-spatial interaction weights with x i After performing element-wise multiplication, we get Y∈R C×H×WThis yields the results of single-cell segmentation.

[0060] The segmentation result of a single cell is a set of two-dimensional coordinates, where the coordinate values ​​represent the x-coordinate and y-coordinate of the cell contour, in pixels. When the cell shape is complex and the number of cells is large, storing the cell segmentation results can consume a lot of memory. Therefore, after obtaining the segmentation results of a single cell, the contours of the cell segmentation results are simplified to reduce the time and space required for dataset processing.

[0061] like Figure 6 As shown, contour simplification of cell segmentation results includes the following steps:

[0062] (1) Obtain the contour point set of each cell from the single cell segmentation result. The contour point set of each cell is a set of points. Convert each set of points into a directed graph set with the beginning and end connected.

[0063] First, we obtain the contour point set for each cell. Before simplifying the contour, we need to transform this point set into a directed graph with connected ends. Here, we use the Sklansky algorithm, which is efficient in calculating the convex hull of a 2D point set. The basic idea of ​​the Sklansky algorithm is to start from the bottom-left (or top-left, depending on the specific implementation) point, iterate through the graph, and select the next furthest point that can form the convex hull, until we return to the starting point, forming a closed convex hull. In each step, the algorithm maintains a candidate point set and selects the point that forms the largest left turn angle with the current convex hull edge to add to the convex hull. This process is repeated until we return to the starting point. Based on the properties of the algorithm, its time complexity is O(nh), where n is the number of points in the point set and h is the number of vertices on the convex hull. Therefore, this algorithm can achieve good time performance when processing large-scale data point sets.

[0064] (2) Remove points on the curve recursively while keeping the maximum deviation error of the curve no more than 5%.

[0065] Specifically, the Douglas-Puk algorithm is used to simplify the directed contour point set. This is an approximate algorithm used to reduce the number of points in a curve or polyline while preserving its overall shape. The algorithm works by recursively removing points located on the curve while keeping the maximum deviation error of the curve within a given threshold (the epsilon parameter). The value of the epsilon parameter determines the degree to which the contour can be simplified. In this program, epsilon = 5% is used as the eigenvalue to achieve the best balance between simplifying the contour and maintaining accuracy.

[0066] The basic steps of the Douglas-Puk algorithm are as follows:

[0067] 1. Select the starting and ending points on the curve and define them as line segments.

[0068] 2. Calculate the perpendicular distance from all intermediate points to the line segment, and find the point with the largest distance (also known as the outlier).

[0069] 3. Calculate the perpendicular distance from all intermediate points to the line segment and find the point with the largest distance (also known as the outlier). If the largest distance is less than or equal to a given error threshold (epsilon), remove all intermediate points and retain the start and end points as the simplified polyline.

[0070] 4. If the maximum distance is greater than the threshold, treat the outlier as the new endpoint, and recursively repeat steps 1 to 3 for the two curve segments between the starting point and the outlier and between the outlier and the ending point.

[0071] After the recursion is complete, all the retained endpoints form the approximate polygon. Simplifying the cell segmentation results by contouring completes the selection process.

[0072] Finally, the simplified outline is drawn on the image, saved locally, and the segmentation result is displayed to the user, as shown in the image below. Figure 7 .

[0073] S5 is used to process the results of single cell segmentation and to count the cells in the fundus.

[0074] Users can save the cell segmentation results and the corresponding contour point set to Excel for subsequent data processing and calculation.

[0075] A system utilizing the aforementioned automatic segmentation and counting method for retinal cells was developed using the Python language, PYTorch framework, and OpenCV image processing software library. This system employs techniques such as the YOLOv8 deep learning model, the Sklansky algorithm, and the Douglas-Peucker algorithm.

[0076] The system includes:

[0077] The image preprocessing module is used to scale the imported fundus cell images and dynamically adjust the image size based on the current image scaling size and the reference diameter input by the user.

[0078] The cell segmentation module has a deep learning model. This deep learning model receives preprocessed fundus cell images. Users can select an appropriate cell segmentation strategy by choosing a cell staining method, and the module outputs the segmentation results of individual cells.

[0079] The results processing module is used to process the results of single cell segmentation and count the cells in the fundus.

[0080] The system of this invention is an easy-to-use multi-marker RGC segmentation software that can process whole-retinal scan images and large batches of patch images, and supports outputting results in multiple formats to meet different research and clinical needs.

[0081] Example:

[0082] Experimental background: In order to quantify the impact of glaucoma on RGCs, researchers often use mouse models and use specific markers such as RBPMS, Brn3a and THY1 for immunohistochemical staining to identify and label RGCs.

[0083] Experimental objective: To validate the performance of a cell segmentation model by establishing a glaucoma mouse model that complies with the Association for Vision and Ophthalmology Research’s Statement on the Use of Animals in Ophthalmological and Visual Research.

[0084] Experimental preparation:

[0085] Male mice were housed under a 12-hour light / 12-hour dark cycle with free access to standard rodent food and water. The carrier suspension was prepared according to the following formulation: sodium chloride, creatinine, disodium EDTA, sodium carboxymethyl cellulose, polysorbate 80, benzyl alcohol, and sodium bisulfite were added to 100 mL of water for injection. The pH was adjusted to 7, and the mixture was autoclaved at 121°C for 20 minutes, cooled to room temperature, and then stored at 4°C. Subsequently, 0.01 g of dexamethasone was mixed with 1 mL of the pretreated carrier suspension and homogenized in a cryogenic grinder to prepare a 10 mg / mL dexamethasone suspension.

[0086] Experimental instruments: LSM 880 laser confocal microscope (Zeiss, Germany), cryogenic grinder.

[0087] Experimental methods:

[0088] Cell samples were prepared and divided into different datasets, with the test set pre-labeled by the experimenters and the validation set used to verify the model's effectiveness.

[0089] Experimental steps:

[0090] Experimental sample pretreatment: After obtaining intact mouse retinas, they were evenly divided into four segments and blocked at 4°C for 10 hours in working solution containing 0.2% Triton and 5% donkey serum. Following blocking, the retinal tissue underwent phosphate-buffered saline rinsing, followed by 10-hour incubation with primary anti-RBPMS antibody. Afterward, the retina was rinsed three times with PBS for 5 minutes each time, and then immersed in Alexa Fluor. TMIncubate the sample in 647-labeled secondary antibody solution in the dark for 2 hours, followed by three 5-minute PBS rinses. After cleaning away any remaining vitreous cavity tissue and impurities, flatten the retina and remove excess fluid. Prepare a DAPI solution containing anti-fluorescence quenching mounting medium and add it to the sample at a 1:1 ratio for nuclear staining.

[0091] Cellular data acquisition and imaging: Fluorescence signals were acquired using an LSM 880 laser confocal microscope (Zeiss, Germany). For RGC counting, fluorescence images were captured under a 20x objective lens at positions 1, 2, and 3 optic disc head-sized distances from the center of each lobe of the retina. Depending on the actual preparation, 6 to 12 fields of view were captured for each sample, with each field of view measuring 1024 × 1024 pixels.

[0092] Dataset Construction and Model Pre-training: The acquired images underwent detailed annotation. The entire annotation process was handled by three professionals using X-Anylabeling integrated with SAM, with each image being annotated and reviewed by a different person. To accommodate model training needs, the annotated images were further cropped to 512×512 pixels. It should be noted that this cropping method inevitably produces some cell fragments. To reduce the impact on model training, fragmented regions close to the edges were removed. In subsequent statistics, incomplete cells located at image edges were not considered. Our dataset consists of 32 samples, with 16 samples belonging to the Blank group and 16 samples belonging to the model creation group. After removing some excessively out-of-focus or noisy images, our dataset ultimately contains 1000 annotated dual-channel 512×512 pixel images. Notably, each image has two annotation results, corresponding to the RBPMS and DAPI channels respectively, while existing counting algorithms only consider one channel.

[0093] Image preprocessing: After model pre-training, the software imports the images taken for the test set into the system, selects the cell staining method and model confidence level, the user manually inputs the cell diameter, clicks the "Scale Normalization" button to adjust the image pixels, and the user can zoom in and out of the image display using the mouse wheel and drag the mouse to adjust the zoomed image area to observe the specific content of the image.

[0094] Cell segmentation and dataset preservation: See Figure 8Clicking the "Run" button will cause the system to segment and count all cells in the current image based on the staining method and confidence level selected by the user, and generate a new, labeled image. After counting, the outlines of all cells will be displayed, along with the total number of identifiable cells in the image. Clicking the "Save Result" button will then save the cell outline coordinates of each image to an Excel file.

[0095] Experimental conclusion:

[0096] To comprehensively evaluate the performance of the deep learning model of this invention, several key metrics are used, including AP. 50 AP 50-90 These metrics, such as frames per second (FPS), collectively provide a quantitative measure of the model's accuracy, efficiency, and effectiveness.

[0097] Intersection over Union (IoU) is a metric that measures the degree of overlap between the predicted area and the ground truth bounding box. The IoU value represents the degree of overlap between the predicted region and the ground truth bounding box, and is commonly used to evaluate the localization accuracy of a model. The formula for calculating IoU is:

[0098]

[0099] Among them, S predict S represents the area of ​​the predicted region. gt Represents the area of ​​the actual region.

[0100] Precision represents the proportion of samples that are actually classified as positive out of all samples identified as positive by the model. The formula for calculation is:

[0101]

[0102] Recall represents the proportion of samples that are actually positive that are correctly identified as positive by the model. The formula is:

[0103]

[0104] Average precision (AP) is calculated based on precision and recall. It is achieved by plotting precision versus recall curves at different thresholds and then calculating the area under that curve, i.e., the AP value. It's important to note that the RGC counting algorithm includes object detection and instance segmentation algorithms, and their performance is evaluated using the AP of the bounding box and the AP of the mask, respectively. The formula for calculating AP is as follows:

[0105]

[0106] AP provides an evaluation of the model's overall performance across all thresholds. Specifically, AP... 50 and AP50-90 These are two commonly used metrics, representing the average accuracy at an IoU threshold of 0.5 and an IoU threshold from 0.5 to 0.95, respectively. 50-90 The calculation formula is as follows:

[0107]

[0108] To evaluate the inference speed of the model, frames per second (FPS) is used as the evaluation metric, and its calculation formula is as follows:

[0109]

[0110] Where T is the average time required for the model to process one frame, and FPS represents the number of images that the model can complete inference in 1 second.

[0111] First, the deep learning model of this invention was compared with other benchmark models, including different sizes of YOLOv8-seg, such as nano(n), small(s), medium(m), and large(l) configurations. Furthermore, enhanced baselines were considered, in which the Lightweight Coupled Segmentation Head (LWCH) and Attention Mechanism Module (EMA) were integrated, both individually and in combination, into the small YOLOv8-seg model. The table below shows the results of the ablation experiments in this study. The Baseline-3 model demonstrated superior performance. By integrating the Lightweight Coupled Segmentation Head and Attention Mechanism Module, this model achieved an AP50 score of 0.957, 0.020 points higher than YOLOv8-s, and an AP50-95 score of 0.720, 0.013 points higher than YOLOv8-s. In terms of model complexity, Baseline-3 has a total of 10.07 million parameters, approximately 14.5% less than YOLOv8-s's 11.78 million parameters. Meanwhile, Baseline-3 maintained a similar frame rate level to YOLOv8-s, reaching 88.20 FPS.

[0112] In summary, these improvements fully demonstrate the value of the proposed deep learning model (MSTAH-YOLO-seg) in RGCs segmentation tasks. It significantly improves model accuracy while maintaining high-speed processing capabilities and effectively reduces the number of parameters.

[0113] The results of the evaluation benchmark model are shown in the table below:

[0114]

[0115] (Table 1)

Claims

1. An automated method for segmenting and counting fundus cells, characterized in that... Includes the following steps: S1. Preprocess the fundus cell images; S2. Input the preprocessed fundus cell image into the deep learning model. The deep learning model is trained by introducing a lightweight coupled segmentation head with an embedded attention mechanism module into the YOLOv8-seg cell segmentation model. S3. The lightweight coupled segmentation head couples the classification branch and bounding box regression branch of the three branches of the YOLOv8-seg cell segmentation model segmentation head and uses shared parameters to process the three feature maps of different sizes output by the feature integration stage, and retains an independent segmentation confidence branch for each feature map size. The attention mechanism module will input the feature map X∈R C×H×W The neural network channels are divided into G sub-features x. i ∈R C / G×H×W These sub-features are then fed into two parallel branches, one of which processes x... i Channel attention is extracted in different directions, and cross-channel interaction is performed using a 1×1 convolutional kernel. Then, x... i Generate weighted feature map P 1×1 ∈R C / G×H×W The other branch uses a 3×3 convolution kernel to apply to x. i Extract feature map P 3×3 ∈R C / G×H×W Then, 2D global average pooling is applied to P respectively. 3×3 and P 1×1 Perform global spatial encoding and generate weights w in the corresponding spatial dimensions. 3×3 and w 1×1 ; S4. Perform matrix multiplication on the feature maps of different spatial dimensions and the corresponding weights in the spatial dimensions to generate cross-spatial interaction weights. The calculation formula is as follows: w cross-spatial =sigmoid(P 3×3 ×w 1×1 +P 1×1 ×w 3×3 ) S5. Integrate cross-spatial interaction weights with x i After performing element-wise multiplication, we obtain Y∈R C×H×W This yields the segmentation results of individual cells, completing the selection process. S6. Process the results of single cell segmentation and count the cells in the fundus.

2. The automatic segmentation and counting method for fundus cells according to claim 1, characterized in that: The cell segmentation results are then simplified by contouring, which involves the following steps: (1) Obtain the contour point set of each cell from the single cell segmentation result. The contour point set of each cell is a set of points. Convert each set of points into a directed graph set with the beginning and end connected. (2) Remove points on the curve recursively while keeping the maximum deviation error of the curve no more than 5%.

3. The automatic segmentation and counting method for fundus cells according to claim 2, characterized in that: In step S1, the preprocessing includes: scaling the imported fundus cell image and dynamically adjusting the image size according to the current image scaling size and the reference diameter input by the user.

4. The automatic segmentation and counting method for fundus cells according to claim 3, characterized in that: In step S6, the processing of the segmentation results of a single cell includes: drawing the simplified contour on the image, saving it locally, displaying the drawn segmentation results to the user, and the user saving the cell segmentation results and the corresponding contour point set to Excel for subsequent data processing and calculation.

5. The automatic segmentation and counting method for fundus cells according to claim 4, characterized in that: The lightweight coupled segmentation head uses grouped normalization, which divides the feature map into multiple groups along the channel dimension of the neural network, and then calculates the mean and standard deviation of the features within each group to normalize the features.

6. The automatic segmentation and counting method for fundus cells according to claim 5, characterized in that: The lightweight coupling segmentation head provides 128 or more output channels for the feature map with the largest output size during the feature integration stage.

7. The automatic segmentation and counting method for fundus cells according to claim 6, characterized in that: In step S2, each image used for training the deep learning model has two labeled results, corresponding to the RBPMS channel and the DAPI channel, respectively.

8. A system using the automatic segmentation and counting method for fundus cells according to any one of claims 1 to 7, characterized in that... include: The image preprocessing module is used to scale the imported fundus cell images and dynamically adjust the image size based on the current image scaling size and the reference diameter input by the user. The cell segmentation module has a deep learning model. This deep learning model receives preprocessed fundus cell images. The user selects the cell staining method and confidence level, traverses all images to select an appropriate cell segmentation strategy for segmentation, and outputs the segmentation result of a single cell. The results processing module processes the segmentation results of individual cells and counts the cells in the fundus.

Images