Automatic segmentation and counting method and system for fundus cells

Abstract

The invention discloses an automatic segmentation and counting method and system for fundus cells, and the method comprises the steps: preprocessing a fundus cell image, inputting the preprocessed fundus cell image into a deep learning model, and enabling a lightweight coupling segmentation head embedded with an attention mechanism module to be introduced into a YOLOv8-seg cell segmentation model, and carrying out the training, thereby obtaining the deep learning model; the attention mechanism module divides the feature map into G sub-features and sends the G sub-features to two parallel branches, one generates a weighted feature map P1 * 1, the other extracts a feature map P3 * 3, 2D global average pooling is used for global space coding of the P3 * 3 and the P1 * 1, and weights w3 * 3 and w1 * 1 are generated; matrix multiplication is carried out on the feature map and the weight, a cross-space interaction weight is generated, and the weight is multiplied by x element by element to obtain Y which belongs to RC * H * W; according to the method, the RGCs in the fundus model can be automatically and accurately circled.

Description

Technical Field

[0001] The invention belongs to image processing technology, and in particular relates to a method and system for automatically segmenting and counting fundus cells. Background Art

[0002] Glaucoma is a disease characterized by optic nerve cell damage, visual field loss and hypoplasia. It is caused by intraocular pressure (IOP)-related optic neuropathy with loss of retinal ganglion cells (RGCs). When glaucoma patients show typical visual field defects such as blurred or lost vision, the retinal ganglion cells may have suffered more than 50% loss. Therefore, the obvious changes in the number of RGCs provide a basis for the early diagnosis of glaucoma.

[0003] To quantify the effects of glaucoma on RGCs, researchers often use mouse models and perform immunohistochemical staining with specific markers such as RBPMS, Brn3a, and THY1 to identify and mark RGCs. Despite the availability of a variety of effective markers to identify RGCs, accurately counting these cells remains a huge challenge. Manually counting RGCs is not only time-consuming and labor-intensive, but is also accompanied by subjective judgment bias. Therefore, some RGCs counting methods based on traditional algorithms have emerged. These methods rely on setting appropriate grayscale thresholds or preset template matching, and are sensitive to cell size and shape changes, making it difficult to count a large number of RGCs in complex backgrounds. For example, the ImageJ script developed by Geeraerts et al. can semi-automatically quantify RGCs in the entire retina, but manual adjustment of parameters is required; while the automated software developed by Guymer et al. can effectively count immunolabeled RGCs, it performs poorly in detecting low-quality RGCs.

[0004] With the maturity of deep learning technology, deep learning-based RGCs counting algorithms have begun to emerge. These algorithms can automatically extract abstract features from training data. However, the effectiveness of these algorithms is highly dependent on the quality of training data, the selected benchmark model, and the model training strategy. For example, when Masin et al. used U-Net to count RGCs labeled with RBPMS, they were unable to identify larger cells and adjacent cells; while Zhang et al. used YOLOv5 to count RGCs labeled with Brn3a. Although it performed well for images labeled with cell nuclei, it did not work well for images labeled with cytoplasm.

[0005] In addition, traditional manual selection has problems. The process is time-consuming and easily affected by the subjective factors of the experimenter, and it is impossible to achieve efficient and accurate cell area selection. The existing automated selection system takes a long time and lacks adaptation to images of different pixel sizes, resulting in unsatisfactory selection results when the image size is inconsistent with the reference size.

[0006] In addition, existing automatic annotation algorithms do not work well on low-quality images and larger images. When researchers use a microscope to capture images of the entire retina, images from multiple fields of view need to be stitched into a complete retinal image, which can result in significant edge overlap and reduced image quality. At the same time, RGCs are usually very small (about 15×15 pixels) and distributed throughout the retina (the distance between cells is ≤20 pixels). In addition, since many RGCs overlap and are not positioned on the same horizontal plane, the image contrast and brightness of some RGCs are low. When the size of the entire retinal image exceeds 8000×8000 pixels, the image usually contains more than 30,000 RGCs. Therefore, both manual detection and recognition algorithm detection require a lot of time. Summary of the invention

[0007] The first purpose of the present invention is to provide a method for automatic segmentation and counting of fundus cells, which can automatically and accurately select RGCs cells in a fundus model, and improve the segmentation results of the model in three cases: low image quality, uncertain image pixel size, and too large an image, thereby improving experimental efficiency and the accuracy of segmentation results.

[0008] The first object of the present invention is achieved by the following technical measures: a method for automatic segmentation and counting of fundus cells, characterized in that it comprises the following steps:

[0009] S1. Preprocess fundus cell images;

[0010] S2. Input the preprocessed fundus cell image into a deep learning model, wherein the deep learning model is formed by introducing a lightweight coupled segmentation head embedded with an attention mechanism module into a YOLOv8-seg cell segmentation model and training it;

[0011] S3, the lightweight coupled segmentation head couples the classification branch and the bounding box regression branch in the three branches of the YOLOv8-seg cell segmentation model segmentation head and uses shared parameters to process the three feature maps of different sizes output by the feature integration stage, and retains an independent segmentation confidence branch for each feature map of each size; the attention mechanism module takes the input feature map X∈R C×H×W Divide into G sub-features x on the channel of the neural network i ∈R C / G×H×W These sub-features are sent to two parallel branches, one of which extracts channel attention in different directions of x and uses 1×1 convolution kernel for cross-channel interaction, and then generates a weighted feature map P for x 1×1 ∈R C / G×H×W , the other branch extracts the feature map P of x through a 3×3 convolution kernel3×3 ∈R C / G×H×W , and then use 2D global average pooling to P 3×3 and P 1×1 Perform global spatial encoding and generate weights w under the corresponding spatial dimension 3×3 and w 1×1 ;

[0012] S4. Perform matrix multiplication on the feature maps of different spatial dimensions and the weights of the corresponding spatial dimensions to generate cross-spatial interaction weights. The calculation formula is:

[0013] w cross-spatial =sigmoid(P 3×3 × 1×1 +P 1×1 × 3×3 )

[0014] S5. Multiply the cross-space interaction weight by x element by element to obtain Y∈R C×H×W , that is, the single cell segmentation result is obtained and the selection is completed;

[0015] S6. Process the single cell segmentation results and count the fundus cells.

[0016] The present invention introduces a lightweight coupled segmentation head (Light-weighted Task-aligned Head, abbreviated as LWCH) and an attention mechanism module (Efficient Multi-Scale Attention, abbreviated as EMA) into the existing YOLOv8-seg cell segmentation model. Through cross-dimensional interaction between features, high efficiency is maintained and the parameter scale is reduced on the basis of the baseline, while the segmentation performance is significantly improved, so that the present invention can automatically and accurately select RGCs cells in the fundus model, and improve the segmentation results of the model in three cases: low image quality, uncertain image pixel size and too large image, thereby improving the experimental efficiency and the accuracy of the segmentation results.

[0017] When the cell shape is complex and the number of cells is large, saving the cell segmentation results may occupy a very large memory space. Therefore, after obtaining the cell segmentation results, in order to reduce the time and space occupied by data set processing, the present invention simplifies the contours of the cell segmentation results, that is, completes the selection. The contour simplification includes the following steps:

[0018] ⑴ Obtain the contour point set of each cell from the single cell segmentation result. The contour point set of each cell is a set of point sets. Each set of point sets is converted into a directed graph set connected end to end.

[0019] ⑵Recursively remove points on the curve while keeping the maximum deviation error of the curve no more than 5%.

[0020] In the step S1 of the present invention, the preprocessing includes: scaling the imported fundus cell image, and dynamically adjusting the size of the image according to the current image scaling size and the reference diameter input by the user.

[0021] In step S6 of the present invention, the processing of the single cell segmentation result includes: drawing the simplified contour on the image, saving it locally, and displaying the drawn segmentation result to the user, and the user saving the cell segmentation result and the corresponding contour point set to Excel for subsequent data processing and calculation.

[0022] The present invention uses group normalization in the lightweight coupled segmentation head, which divides the feature map into multiple groups in the channel dimension of the neural network, and then calculates the mean and standard deviation of the features in each group to normalize the features.

[0023] The lightweight coupled segmentation head of the present invention provides an output channel number greater than or equal to 128 for outputting a feature map of the maximum size in the feature integration stage.

[0024] In step S2 of the present invention, each image used for training the deep learning model has two annotation results, corresponding to the RBPMS channel and the DAPI channel respectively. The present invention shows excellent segmentation performance in the dataset stained with DAPI and RBPMS, and can further improve the accuracy of RGCs segmentation by combining these two markers.

[0025] The second object of the present invention is to provide a system using the above-mentioned method for automatic segmentation and counting of fundus cells.

[0026] The second object of the present invention is achieved by the following technical measures: a system using the above-mentioned fundus cell automatic segmentation and counting method, characterized in that it includes:

[0027] An image preprocessing module is used to scale the imported fundus cell image and dynamically adjust the image size according to the current image scaling size and the reference diameter input by the user;

[0028] The cell segmentation module has a deep learning model that receives preprocessed fundus cell images. Users select cell staining methods and confidence levels, traverse all images to select appropriate cell segmentation strategies for segmentation, and output single cell segmentation results.

[0029] The result processing module processes the single cell segmentation results and counts the fundus cells.

[0030] Compared with the prior art, the present invention has the following significant effects:

[0031] ⑴ The present invention transfers the manual circle step to a machine and an algorithm to implement. Through the image processing algorithm, the fundus cells can be segmented and counted, and the cell area can be automatically circled, which solves the shortcomings of manual circle, which is time-consuming, labor-intensive and mechanically repetitive. At the same time, the experimental data can be processed and the segmentation results can be exported with one click, thereby improving the experimental efficiency and accuracy. The problem that the existing automatic labeling algorithm does not work well on low-quality images and larger images can be solved. At the same time, the present invention also introduces an attention mechanism module, including a contour simplification algorithm and a dynamic adjustment strategy, to screen and retain the cell areas required by the stitching user, and enhance the recognition performance of cells in larger images and cells in low-quality images.

[0032] ⑵ The present invention integrates complex and redundant steps into a software tool, realizes the automation of the experimental process, simplifies the processing of experimental data, provides researchers with convenient and reliable experimental tools, and makes experimental operations more efficient and reliable.

[0033] ⑶ The present invention has a diameter calibration function when performing image preprocessing. The cell image to be processed is automatically scaled to fit images of different pixel sizes through the data input by the user, which effectively improves the cell segmentation and counting effect of the software for images of different sizes. At the same time, the software can convert large images into smaller image fragments by integrating "Slicing AidedHyper Inference" counting. By retaining appropriate overlapping areas between each fragment, the original image can be accurately reconstructed between each fragment. This method can achieve more efficient performance when processing large-scale images. The software can efficiently process data sets of different sizes. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0035] Figure 1 It is a schematic diagram of the process and system structure of the method of the present invention;

[0036] Figure 2 It is a display diagram of the system interface of the present invention;

[0037] Figure 3 This is an interface display diagram for selecting a cell staining method for the system of the present invention;

[0038] Figure 4 It is a schematic diagram of the composition structure of the cell segmentation model of the present invention;

[0039] Figure 5 It is a structural diagram of the attention mechanism module of the present invention;

[0040] Figure 6 It is a flowchart of the present invention for simplifying the outline of the cell segmentation result;

[0041] Figure 7 It is an interface display diagram of the cell segmentation result of the present invention;

[0042] Figure 8 This is an interface display diagram of the segmentation results of cells in a large image in an example of the present invention. DETAILED DESCRIPTION

[0043] The present invention is described in detail below in conjunction with the embodiments and the accompanying drawings to help those skilled in the art better understand the inventive concept of the present invention. However, the protection scope of the claims of the present invention is not limited to the following embodiments. For those skilled in the art, all other embodiments obtained without creative work without departing from the inventive concept of the present invention belong to the protection scope of the present invention.

[0044] like Figures 1 to 8 As shown, a method and system for automatic segmentation and counting of fundus cells of the present invention are provided, wherein the method for automatic segmentation and counting of fundus cells comprises the following steps:

[0045] S1. Preprocess fundus cell images;

[0046] Preprocessing includes image scaling, reference diameter selection, and image blank area filling to improve the accuracy of the cell segmentation and counting algorithm, ensuring that the experimenter can obtain more accurate cell segmentation and counting results.

[0047] After importing the image, the user can use the mouse wheel and pointer to zoom and drag the preview image, and adjust the reference cell diameter by entering the value of Cell Diameter. The system will dynamically adjust the image size according to the current image zoom size and the reference diameter entered by the user to adapt to the segmentation strategy of cells of different sizes, such as Figure 2 Shown is the system interface for preprocessing fundus cell images.

[0048] S2. Input the preprocessed fundus cell images into the deep learning model. The deep learning model is trained by introducing the lightweight coupled segmentation head embedded with the attention mechanism module into the YOLOv8-seg cell segmentation model. The user selects the appropriate cell segmentation strategy by selecting the cell staining method. Figure 3 .

[0049] The existing YOLOv8-seg cell segmentation model framework uses a fine-tuned and optimized CSPDarkNet53 architecture as its backbone network, and PANet is selected as the feature pyramid network in the feature integration stage (neck). The YOLOv8-seg cell segmentation model replaces all the original C3 modules with C2f modules, which incorporate more skip-layer connections and channel separation technologies. At the end of the backbone network, YOLOv8-seg uses the SPPF (spatial pyramid pooling filter) structure, which connects three layers of 5×5 maximum pooling layers in series and integrates and utilizes the output results of each layer. This allows YOLOv8-seg to effectively enrich the gradient information flow while maintaining its lightweight.

[0050] The segmentation head of YOLOv8-seg receives three feature maps of different sizes P3, P4, and P5 from the neck. The segmentation head of YOLOv8-seg consists of the detection head of YOLOv8-seg and the segmentation confidence branch (mask coefficient) based on YOLACT, and adopts a structure similar to the decoupled head of YOLOx. The segmentation head has three branches: bounding box regression branch, classification branch, and segmentation confidence branch (mask coefficient). In the classification branch, an N cls ×H×W tensor, N cls is the target type in the dataset, which is 1 in this paper, and binary cross entropy is used as the loss during training. A 4*reg_max×H×W tensor is generated in the bounding box regression branch, and CIOU and DFL are used as the loss during training, where DFL has a hyperparameter reg_max, which is 16 in this paper. A 32×H×W tensor is generated in the mask coefficient branch, and binary cross entropy loss is used as the loss during training. In addition, the largest feature map P3 is upsampled by Protonet to generate a Prototype Mask, which is linearly combined with the Mask Coefficient after NMS to obtain the final mask:

[0051] M=δ(P×C T )

[0052] Among them, P is the prototype mask, δ is sigmoid, C T is the mask coefficient after NMS. Use the predicted box after NMS to cut M, and then perform threshold processing to get the final segmentation result.

[0053] The defects of the YOLOv8-seg structural design are: first, feature maps of different sizes may contain similar spatial information, and the use of independent segmentation heads may lead to repeated calculation of targets; second, YOLOv8-seg assigns the same parameters to each task branch, which may lead to parameter waste when processing single-class multi-target segmentation tasks such as RGCs segmentation, especially in the classification branch; in addition, the decoupled head design may also lead to misalignment between different tasks.

[0054] S3. In order to solve the above problems of YOLOv8-seg, the present invention designs a lightweight coupled segmentation head embedded with an attention mechanism module. Figure 4 In the figure, it is represented as a gray area. The lightweight coupled segmentation head couples the classification branch and the bounding box regression branch in the three branches of the YOLOv8-seg cell segmentation model segmentation head and uses shared parameters to process the three feature maps of different sizes P3, P4, and P5 output from the feature integration stage. That is, a parameter-sharing module is used to process the three feature maps of different sizes output by the neck, thereby achieving effective coupling between the regression branch and the classification branch. In order to ensure the accuracy of the segmentation, an independent segmentation confidence branch is retained for each size of the feature map; secondly, group normalization (GroupNorm, abbreviated as GN) is used in the lightweight coupled segmentation head to replace batch normalization (Batchnorm) in the YOLOv8-seg segmentation head. Figure 4 The group normalization is represented as Conv_GN. GroupNorm has been proven to effectively improve the performance of target detection. It works by dividing the feature map into multiple groups in the channel dimension of the neural network, and then calculating the mean and standard deviation of the features within each group to normalize the features. This design redistributes the parameters of the model, and further improves the efficiency and lightweight level of the model by enhancing the feature interaction between the positioning and classification tasks. In addition, the lightweight coupled segmentation head provides an output channel number greater than or equal to 128 for the feature map P3 with the maximum size output in the feature integration stage. The original output channel number is 64. Due to the increase in the output channel number of the feature map P3, Protonet will receive richer spatial information to generate a more accurate mask prototype.

[0055] like Figure 5 As shown in the figure, it is the structure of the attention mechanism module EMA. The attention mechanism module takes the input feature map X∈R C ×H×W Divide into G sub-features x on the channel i ∈R C / G×H×W, which aims to learn different semantics across channels. In this paper, G is set to 8. These sub-features are then sent to two parallel branches, one of which extracts channel attention in different directions of x and uses 1×1 convolution kernels for cross-channel interaction, and then generates a weighted feature map P for x. 1×1 ∈R C / G×H×W (that is Figure 5 The other branch passes through a 3×3 convolution kernel (i.e. Figure 5 conv3x3) extracts feature map P from x 3×3 ∈R C / G×H×W . Through parallel subnetworks (parallel subnetworks, that is, Figure 5 EMA can encode the information between channels to adjust the weights of different channels and retain the fine spatial structure information in the channels.

[0056] Then EMA proposed a cross-space aggregation method in different spatial dimensions to achieve richer feature aggregation. Specifically, 2D global average pooling (i.e. Figure 5 avgpool in Cross-spatial learning) for P 3×3 and P 1×1 Perform global spatial encoding and generate the weight w in the corresponding spatial dimension through Softmax 3×3 (The softmax output above in Cross-spatial learning) and w 1×1 (The softmax output result below in Cross-spatial learning), Cross-spatial learning is Figure 5 The gray area on the middle right;

[0057] S3. Perform matrix multiplication on the feature maps of different spatial dimensions and the weights of the corresponding spatial dimensions to generate cross-spatial interaction weights. The calculation formula is:

[0058] w cross-spatial =sigmoid(P 3×3 × 1×1 +P 1×1 × 3×3 )

[0059] S4. Multiply the cross-space interaction weight by x element by element to obtain Y∈R C×H×W , that is, the single cell segmentation result is obtained;

[0060] The result of a single cell segmentation is a set of two-dimensional coordinates. The values ​​of the coordinates represent the horizontal and vertical coordinates of the cell contour in pixels. When the cell shape is complex and the number of cells is large, saving the cell segmentation results may take up a very large memory space. Therefore, after obtaining the single cell segmentation results, the cell segmentation results are simplified to reduce the time and space occupied by the data set processing.

[0061] like Figure 6 As shown, the outline simplification of the cell segmentation result includes the following steps:

[0062] ⑴ Obtain the contour point set of each cell from the single cell segmentation result. The contour point set of each cell is a set of point sets. Each set of point sets is converted into a directed graph set connected end to end.

[0063] First, we obtain the contour point set of each cell. Before simplifying the contour, we must first convert this set of points into a directed graph set connected end to end. The algorithm we use here is the Sklansky algorithm, which has good efficiency in calculating the convex hull of a two-dimensional point set. The basic idea of ​​the Sklansky algorithm is to start from the bottom left (or top left, depending on the specific convention of the implementation), gradually traverse and select the next farthest point that can form the convex hull until returning to the starting point to form a closed convex hull. In each step, the algorithm maintains a set of candidate points and selects the point that forms the largest left turn angle with the edge of the current convex hull to add to the convex hull. This process is repeated until it returns to the starting point. According to the properties of the algorithm, it can be concluded that the time complexity of the algorithm is O(nh), where n is the number of points in the point set and h is the number of vertices on the convex hull. Therefore, the algorithm can achieve good time performance when processing large-scale data point sets.

[0064] ⑵Recursively remove points on the curve while keeping the maximum deviation error of the curve no more than 5%.

[0065] Specifically, the Douglas-Peucker algorithm is used to simplify the contour of a directed contour point set. This is an approximation algorithm used to reduce the number of points in a curve or polyline while maintaining its overall shape. The algorithm works by recursively removing points on the curve while keeping the maximum deviation error of the curve below a given threshold (epsilon parameter). The value of the epsilon parameter determines the degree to which the contour can be simplified. In this program, epsilon = 5% is used as the eigenvalue to achieve the maximum balance between simplifying the contour and maintaining accuracy.

[0066] The basic steps of the Douglas-Peucker algorithm are as follows:

[0067] 1. Select the starting point and ending point on the curve and define it as a line segment.

[0068] 2. Calculate the perpendicular distance from all the middle points to the line segment and find the point with the largest distance (also called the outlier point).

[0069] 3. Calculate the perpendicular distance from all the middle points to the line segment and find the point with the largest distance (also called an outlier). If the maximum distance is less than or equal to the given error threshold (epsilon), remove all the middle points and keep the starting point and the end point as the simplified polyline.

[0070] 4. If the maximum distance is greater than the threshold, the outlier point is regarded as a new endpoint, and steps 1 to 3 are recursively repeated for the two curve segments from the starting point to the outlier point and from the outlier point to the end point.

[0071] After the recursion is completed, all the retained endpoints form the approximated polygon. The cell segmentation result is simplified to complete the circle selection.

[0072] Finally, the simplified outline is drawn on the image, saved locally, and the segmentation result is displayed to the user. The display effect is as follows: Figure 7 .

[0073] S5 is used to process the single cell segmentation results and count the fundus cells.

[0074] Users can save the cell segmentation results and the corresponding contour point set to Excel for subsequent data processing and calculation.

[0075] A system using the above-mentioned fundus cell automatic segmentation and counting method is developed by using Python language, PYTorch framework and OpenCV image processing software library. This system uses technologies such as YOLOv8 deep learning model, Sklansky algorithm and Douglas-Peucker algorithm.

[0076] The system includes:

[0077] An image preprocessing module is used to scale the imported fundus cell image and dynamically adjust the image size according to the current image scaling size and the reference diameter input by the user;

[0078] The cell segmentation module has a deep learning model that receives the preprocessed fundus cell images. The user selects the appropriate cell segmentation strategy by selecting the cell staining method and outputs the single cell segmentation results.

[0079] The result processing module is used to process the single cell segmentation results and count the fundus cells.

[0080] The system of the present invention is an easy-to-use multi-marker RGC segmentation software that can process full retinal scans and large quantities of patch images, and supports the output of results in multiple formats to meet different research and clinical needs.

[0081] Examples:

[0082] Experimental background: To quantify the effects of glaucoma on RGCs, researchers often use mouse models and perform immunohistochemical staining using specific markers such as RBPMS, Brn3a, and THY1 to identify and label RGCs.

[0083] Experimental purpose: To establish a glaucoma mouse model that complies with the Association for Research in Vision and Ophthalmology's Statement for the Use of Animals in Ophthalmic and Vision Research and to verify the performance of the cell segmentation model.

[0084] Experimental preparation:

[0085] Male mice were housed under a 12-hour light / 12-hour dark cycle with free access to standard rodent chow and water. The vehicle suspension was prepared according to the following recipe: sodium chloride, creatinine, disodium ethylenediaminetetraacetate, sodium carboxymethylcellulose, polysorbate 80, benzyl alcohol, and sodium bisulfite were added to 100 mL of water for injection, the pH value was adjusted to 7, and it was autoclaved at 121°C for 20 minutes, cooled to room temperature, and then stored in a 4°C environment. Subsequently, 0.01 g of dexamethasone was mixed with 1 mL of the pretreated vehicle suspension and homogenized in a cryo-grinder to prepare a 10 mg / mL dexamethasone suspension.

[0086] Experimental instruments: LSM 880 laser confocal microscope (Zeiss, Germany), cryo-grinding apparatus.

[0087] Experimental methods:

[0088] Prepare cell samples and divide them into different data sets, where the test set is pre-labeled by the experimenter and the validation set is used to verify the model effect.

[0089] Experimental steps:

[0090] Pretreatment of experimental samples: After obtaining the complete mouse retina, it was evenly divided into 4 petals and blocked in 5% donkey serum working solution mixed with 0.2% Triton at 4°C for 10 hours. After blocking, the retinal tissue was rinsed with phosphate-buffered saline and then incubated with the primary anti-RBPMS antibody for 10 hours. After that, after three 5-minute PBS rinses, the retina was immersed in Alexa Fluor TMIncubate in the 647-labeled secondary antibody solution in the dark for 2 hours, and rinse with PBS for three times for 5 minutes. After cleaning the remaining vitreous cavity tissue and impurities, flatten the retina and remove excess liquid. Prepare DAPI solution containing anti-fluorescence quenching mounting medium and add it to the sample at a 1:1 ratio for nuclear staining.

[0091] Cell data acquisition and image capture: LSM 880 laser confocal microscope (Zeiss, Germany) was used to acquire fluorescence signals. In order to count RGCs, we took fluorescence images of each retinal leaf at 1, 2, and 3 optic disc sizes from the center under a 20x objective. Depending on the actual preparation, 6 to 12 fields of view were captured for each sample, and the size of each field of view was 1024×1024 pixels.

[0092] Dataset construction and model pre-training: The acquired images were annotated in detail. The entire annotation process was carried out by three professionals, using X-Anylabeling with integrated SAM for annotation, and each image was annotated and reviewed by different people. In order to meet the needs of model training, the annotated images were further cut into 512×512 pixels. It should be pointed out that this cutting method inevitably produces some cell fragments. In order to reduce the impact of its model on training, we deleted the fragment areas close to the edge. In subsequent statistics, we did not consider incomplete cells located at the edge of the image. Our dataset comes from a total of 32 samples, of which 16 samples belong to the Blank group and 16 samples belong to the modeling group. After eliminating some images that are too out of focus or too noisy, our dataset finally contains 1,000 annotated dual-channel 512×512 pixel images. It is worth mentioning that each image has two annotation results, corresponding to the two channels of RBPMS and DAPI, while the existing counting algorithm only considers one channel.

[0093] Image preprocessing: After model pre-training, use the software to import the pictures taken by the test set into the system, select the image cell staining method and model confidence, manually enter the cell diameter, click the "Scale Normalization" button to adjust the image pixels, and the user can use the mouse wheel to scale the image display, adjust the scaled image area by dragging the mouse, and observe the specific content of the image.

[0094] Cell segmentation and dataset storage: see Figure 8, click the "Run" button, the system will segment and count all cells in the current image according to the staining method and confidence level selected by the user, and generate a new, labeled image. After the count is completed, the outlines of all cells will be displayed, and the total number of identifiable cells in the image will be displayed. At this time, the user clicks the "Save Result" button to save the cell outline coordinates of each image to an Excel file.

[0095] Experimental conclusion:

[0096] In order to comprehensively evaluate the performance of the deep learning model of the present invention, several key indicators are used, including AP 50 , A.P. 50-90 , frames per second (FPS), these indicators together provide a quantitative measure of the accuracy, efficiency, and effectiveness of the model.

[0097] The intersection over union (IoU) is a measure of the degree of overlap between the predicted area and the true area. The IoU value represents the overlap between the predicted area and the true box, and is usually used to evaluate the positioning accuracy of the model. The IoU calculation formula is:

[0098]

[0099] Among them, S predict Represents the area of ​​the predicted region, S gt Represents the area of ​​the real region.

[0100] Precision refers to the proportion of samples that are actually positive among all samples identified as positive by the model. The calculation formula is:

[0101]

[0102] The recall rate indicates the proportion of samples that are correctly identified as positive by the model among all samples that are actually positive. The calculation formula is:

[0103]

[0104] The average precision (AP) is calculated based on the precision and recall rate. By plotting the curves of precision and recall rate at different thresholds, the area under the curve is calculated, i.e., the AP value. It should be noted that the RGC counting algorithm includes the object detection and instance segmentation algorithms, and the AP of the bounding box and the AP of the mask are used to evaluate their performance respectively. The calculation formula of AP is as follows

[0105]

[0106] AP can provide an assessment of the overall performance of the model at all thresholds. In particular, AP 50 and AP50-90 There are two commonly used indicators, which represent the average precision at an IoU threshold of 0.5 and an IoU threshold from 0.5 to 0.95. 50-90 The calculation formula is as follows

[0107]

[0108] In order to evaluate the inference speed of the model, frames per second (FPS) is used as the evaluation indicator, and its calculation formula is:

[0109]

[0110] Among them, T is the average time required for the model to process one frame, and FPS represents the number of pictures that the model can complete reasoning in 1 second.

[0111] First, the deep learning model of the present invention is tested against other benchmark models, including different size versions of YOLOv8-seg, such as nano(n), small(s), medium(m), and large(l) configurations. In addition, enhanced baselines are considered in which the lightweight coupled segmentation head (LWCH) and attention mechanism module (EMA) are integrated into the small YOLOv8-seg model individually and in combination. The following table shows the results of the ablation experiments in this study. The Baseline-3 model shows excellent performance. By integrating the lightweight coupled segmentation head and the attention mechanism module, the model achieves a score of 0.957 on the AP50 index, which is 0.020 points higher than YOLOv8-s, and achieves 0.720 on the AP50-95 index, which is 0.013 points higher than YOLOv8-s. In terms of model complexity, Baseline-3 has a total of 10.07 million parameters, which is about 14.5% less than the 11.78 million parameters of YOLOv8-s. At the same time, Baseline-3 maintains a frame rate level similar to YOLOv8-s, reaching 88.20FPS.

[0112] In general, these improvements fully demonstrate the value of the proposed deep learning model (MSTAH-YOLO-seg) in the RGCs segmentation task, which significantly improves the model accuracy while maintaining high-speed processing capabilities and effectively reducing the number of parameters.

[0113] The indicator results for the evaluation benchmark model are shown in the following table:

[0114]

[0115] (Table 1).

Claims

1. A method for automatic segmentation and counting of fundus cells, characterized in that The following steps are involved: S1. Preprocess fundus cell images; S2. Input the preprocessed fundus cell image into a deep learning model, wherein the deep learning model is formed by introducing a lightweight coupled segmentation head embedded with an attention mechanism module into a YOLOv8-seg cell segmentation model and training it; S3, the lightweight coupled segmentation head couples the classification branch and the bounding box regression branch of the three branches of the YOLOv8-seg cell segmentation model segmentation head and uses shared parameters to process the three feature maps of different sizes output by the feature integration stage, and retains an independent segmentation confidence branch for each size of the feature map; The attention mechanism module takes the input feature map X∈R C×H×W Divide into G sub-features x on the channel of the neural network i ∈R C / G×H×W These sub-features are sent to two parallel branches, one of which extracts channel attention in different directions of x and uses 1×1 convolution kernel for cross-channel interaction, and then generates a weighted feature map P for x 1×1 ∈R C / G×H×W , the other branch extracts the feature map P of x through a 3×3 convolution kernel 3×3 ∈R C / G×H×W , and then use 2D global average pooling to P 3×3 and P 1×1 Perform global spatial encoding and generate weights w under the corresponding spatial dimension 3×3 and w 1×1 ; S4. Perform matrix multiplication on the feature maps of different spatial dimensions and the weights of the corresponding spatial dimensions to generate cross-spatial interaction weights. The calculation formula is: w cross-spatial =sigmoid(P 3×3 ×w 1×1 +P 1×1 ×w 3×3 ) S5. Multiply the cross-space interaction weight by x element by element to obtain Y∈R C×H×W , that is, the single cell segmentation result is obtained and the selection is completed; S6. Process the single cell segmentation results and count the fundus cells.

2. The method for automatic segmentation and counting of fundus cells according to claim 1, characterized in that: Simplify the outline of the cell segmentation result, that is, complete the circle selection. The outline simplification includes the following steps: ⑴ Obtain the contour point set of each cell from the single cell segmentation result. The contour point set of each cell is a set of point sets. Each set of point sets is converted into a directed graph set connected end to end. ⑵Recursively remove points on the curve while keeping the maximum deviation error of the curve no more than 5%.

3. The method for automatic segmentation and counting of fundus cells according to claim 2, characterized in that: In the step S1, the preprocessing includes: scaling the imported fundus cell image, and dynamically adjusting the image size according to the current image scaling size and the reference diameter input by the user.

4. The method for automatic segmentation and counting of fundus cells according to claim 3, characterized in that: In step S6, the processing of the single cell segmentation result includes: drawing the simplified contour on the image, saving it locally, and displaying the drawn segmentation result to the user, and the user saving the cell segmentation result and the corresponding contour point set to Excel for subsequent data processing and calculation.

5. The method for automatic segmentation and counting of fundus cells according to claim 4, characterized in that: Group normalization is used in the lightweight coupled segmentation head, which divides the feature map into multiple groups in the channel dimension of the neural network, and then calculates the mean and standard deviation of the features within each group to normalize the features.

6. The method for automatic segmentation and counting of fundus cells according to claim 5, characterized in that: The lightweight coupled segmentation head provides an output channel number greater than or equal to 128 for outputting a feature map of a maximum size in the feature integration stage.

7. The method for automatic segmentation and counting of fundus cells according to claim 6, characterized in that: In step S2, each image used for training the deep learning model has two annotation results, corresponding to the RBPMS channel and the DAPI channel respectively.

8. A system using the method for automatic segmentation and counting of fundus cells according to any one of claims 1 to 7, characterized in that include: An image preprocessing module is used to scale the imported fundus cell image and dynamically adjust the image size according to the current image scaling size and the reference diameter input by the user; The cell segmentation module has a deep learning model that receives preprocessed fundus cell images. Users select cell staining methods and confidence levels, traverse all images to select appropriate cell segmentation strategies for segmentation, and output single cell segmentation results. The result processing module processes the single cell segmentation results and counts the fundus cells.

Images