A dense lesion semi-automatic labeling method for fundus images

Abstract

The application discloses a kind of dense lesion semi-automatic labeling method for fundus image, method includes constructing and training dense lesion segmentation network, macula fovea positioning network and optic cup optic disc segmentation positioning network;Utilize network to process to obtain dense lesion prediction contour area, macular area and optic cup optic disc area respectively;Three labelers are assigned to fundus image, each labeler selects key area and other area from macular area and optic cup optic disc area and carries out labeling operation, to obtain first round labeling result;According to first round labeling result, the lesion labeling consistency index between every two labelers is calculated to determine two labelers to execute second round lesion labeling, and second round labeling result is obtained;Second round labeling result is audited to obtain final labeling result.Therefore, the problem of large difficulty in dense lesion labeling and large difference in labeling by different personnel is solved, and the labeling efficiency is improved, the labeling time is reduced, and the labeling cost is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image processing, in particular to a dense lesion semi-automatic labeling method for fundus images. BACKGROUND

[0002] In recent years, artificial intelligence technology has been widely applied in various fields of our daily life, such as smart city, intelligent retail and smart home, etc., providing various services and conveniences for our daily life. However, the application of artificial intelligence technology in the field of medicine is relatively less, mainly because artificial intelligence technology needs a large amount of labeled images or text information for model training, but medical image resources are generally only owned by hospitals, and the collection of medical image resources is difficult, and the data labeling is more difficult, especially the labeling of dense lesions, which requires professionals to identify and label, and a large amount of manpower and energy is needed.

[0003] There are currently relatively more related works on medical image structure segmentation, structure positioning and lesion segmentation based on artificial intelligence, but how to reasonably use these technologies is relatively less, and the feedback on auxiliary labeling after training on public data sets or a small amount of labeled data sets is relatively less.

[0004] The labeling of dense lesions in medical images is difficult, and in general, the labeling personnel start from zero, which greatly increases the labeling time. At the same time, the labeling difference of different labeling personnel for the same type of lesion in the same medical image is also large, which leads to obvious difference in the final labeling results of different labeling personnel, and the labeling results often cannot be used. Therefore, how to accurately evaluate the labeling consistency of different labeling personnel and how to quickly and accurately select suitable labeling personnel will greatly affect the labeling efficiency and effect of dense lesions. SUMMARY

[0005] The present application aims to at least solve one of the technical problems existing in the prior art. To this end, the present application provides a dense lesion semi-automatic labeling method for fundus images, which solves the problem that the current manual labeling of dense lesions in medical images is difficult and the difference between different manual labeling is large.

[0006] The dense lesion semi-automatic labeling method for fundus images according to the embodiment of the present application comprises the following steps:

[0007] constructing and training a dense lesion segmentation network, a macular fovea positioning network and a cup-disc segmentation and positioning network;

[0008] processing the fundus image to be labeled by using the dense lesion segmentation network to obtain a dense lesion prediction contour region;

[0009] The macular fovea positioning network is used for processing the to-be-labeled fundus image to obtain a macular region;

[0010] The optic cup and disc segmentation positioning network is used for processing the to-be-labeled fundus image to obtain an optic cup and disc region;

[0011] Three labelers are assigned to the to-be-labeled fundus image, each of which selects a key region from the macular region and the optic cup and disc region and determines other regions accordingly, and the key region and the other regions together constitute a full image region of the to-be-labeled fundus image;

[0012] Each labeler performs a first labeling operation on the key region determined by the labeler and performs a second labeling operation on the other regions determined by the labeler to obtain a first round of labeling results of the labeler;

[0013] According to the first round of labeling results, a lesion labeling consistency index between each two labelers is calculated to determine two labelers who perform a second round of lesion labeling;

[0014] The two labelers who perform the second round of lesion labeling perform a third labeling operation on the first round of labeling results of the labelers to obtain a second round of labeling results of the labelers;

[0015] The second round of labeling results of the labelers are merged and submitted to an auditor for adjustment and checking to obtain a final labeling result.

[0016] The dense lesion semi-automatic labeling method for fundus images according to the embodiment of the application has at least the following beneficial effects:

[0017] By using artificial intelligence technology to construct a plurality of network models for fundus image segmentation and positioning, and obtaining a network model with better performance after model training, the to-be-labeled fundus image can be processed to generate a dense lesion prediction contour region to feedback to the dense lesion labeling, and a macular region and an optic cup and disc region are also generated. By using the macular region and the optic cup and disc region to define and divide key regions and other regions, different labelers can perform key labeling operations according to the key regions defined by the labelers, thereby producing first round of labeling results that are not completely consistent, and the labeling consistency between different labelers can be calculated and evaluated, and finally appropriate labelers are assigned to the to-be-labeled fundus image according to the labeling consistency to complete the second round of lesion labeling, thereby ensuring the labeling efficiency and labeling effect of the dense lesion. By using the method of the embodiment of the application, the problem of difficulty in labeling dense lesions in medical images and large differences in labeling between labeling personnel is solved from the root, which can greatly improve the labeling efficiency, reduce the labeling time, and reduce the labeling cost.

[0018] According to some embodiments of the present application, the dense lesion segmentation network is constructed and trained, including the following steps:

[0019] The dense lesion segmentation network is constructed, and the dense lesion segmentation network is composed of a first encoder, a first decoder and a first segmentation head, wherein the first encoder adopts ResNet50, the first decoder adopts UNet++, and the first segmentation head is composed of a convolution layer with a convolution kernel size of 3x3 and an up-sampling operation;

[0020] Prepare dense lesion segmentation training data and dense lesion segmentation test data and perform online histogram scaling;

[0021] The dense lesion segmentation training data is input into the dense lesion segmentation network for model parameter iterative updating, so as to continuously train the dense lesion segmentation network;

[0022] The loss value of the dense lesion segmentation network processing the dense lesion segmentation test data after each round of training is calculated;

[0023] The model with the minimum loss value is determined as the final dense lesion segmentation network.

[0024] According to some embodiments of the present application, the calculation of the loss value of the dense lesion segmentation network processing the dense lesion segmentation test data after each round of training is constrained by the following mathematical model:

[0025]

[0026] Wherein, L dice represents the DiceLoss loss function; N represents the number of pixel points in the lesion segmentation graph; n represents the nth pixel point in the lesion segmentation graph; r n represents the true label of the nth pixel in the lesion segmentation graph; p n represents the predicted probability value of the corresponding pixel; ∈ represents a constant.

[0027] According to some embodiments of the present application, the macular fovea positioning network is constructed and trained, including the following steps:

[0028] The macular fovea positioning network is constructed, and the macular fovea positioning network is composed of a Swin Transformer Tiny network and a feature post-processing module, wherein the Swin Transformer Tiny network is used for feature extraction of the fundus image, and the feature post-processing module is composed of a first full connection layer, a ReLU layer, a Dropout layer, a second full connection layer and a Sigmoid layer;

[0029] Preparation of macular fovea positioning training data and macular fovea positioning test data and online histogram scaling and normalization processing;

[0030] Inputting the macular fovea positioning training data into the macular fovea positioning network for model parameter iterative updating to continuously train the macular fovea positioning network;

[0031] Calculating the loss value of the macular fovea positioning network processing the macular fovea positioning test data after each round of training;

[0032] Determining the model with the minimum loss value as the final macular fovea positioning network.

[0033] According to some embodiments of the present application, the calculation of the loss value of the macular fovea positioning network processing the macular fovea positioning test data after each round of training is constrained by the following mathematical model:

[0034] L Loc = 0.5 x L MSE + 0.5 x L ED ,

[0035]

[0036]

[0037] Wherein, L Loc represents the total loss function of the macular fovea positioning network; L MSE represents the MSELoss loss function; L ED represents the EDLoss loss function; N represents the number of images in each batch during the training stage; n represents the nth image in each batch; x n represents the normalized real horizontal coordinate of the macular fovea in the nth image in each batch; p n represents the predicted horizontal coordinate of the normalized macular fovea in the nth image in each batch; y n represents the normalized real vertical coordinate of the macular fovea in the nth image in each batch; q n represents the predicted vertical coordinate of the normalized macular fovea in the nth image in each batch.

[0038] According to some embodiments of the present application, the optic cup and disc segmentation positioning network is constructed and trained, including the following steps:

[0039] The cup-disc segmentation positioning network is constructed and is composed of a second encoder, a second decoder and a second segmentation head, wherein the second encoder adopts ResNet34, the second decoder adopts UNet, and the second segmentation head is composed of a convolution layer with a convolution kernel size of 3x3 and an up-sampling operation;

[0040] Prepare cup-disc segmentation positioning training data and cup-disc segmentation positioning test data and perform online histogram scaling;

[0041] Input the cup-disc segmentation positioning training data into the cup-disc segmentation positioning network for model parameter iterative updating, so as to continuously train the cup-disc segmentation positioning network;

[0042] Calculate the loss value of the cup-disc segmentation positioning network processing the cup-disc segmentation positioning test data after each round of training;

[0043] Determine the model with the minimum loss value as the final cup-disc segmentation positioning network.

[0044] According to some embodiments of the present application, the calculation of the loss value of the cup-disc segmentation positioning network processing the cup-disc segmentation positioning test data after each round of training is constrained by the following mathematical model:

[0045] L all = 0.5 x L MAE + 0.5 x L dice ,

[0046]

[0047]

[0048] wherein L all represents the total loss function of the cup-disc segmentation positioning network; L MAE represents the MAELoss loss function; L dice represents the DiceLoss loss function; N represents the number of pixel points of the fundus image cup-disc segmentation graph; n represents the nth pixel point in the segmentation graph; r n represents the true label of the nth pixel in the cup-disc segmentation graph; p n represents the predicted probability value of the corresponding pixel; and ε and ∈ both represent constants.

[0049] According to some embodiments of the present application, the cup-disc segmentation positioning network is constructed and is composed of a second encoder, a second decoder and a second segmentation head, wherein the second encoder adopts ResNet34, the second decoder adopts UNet, and the second segmentation head is composed of a convolution layer with a convolution kernel size of 3x3 and an up-sampling operation;

[0050] Determine the macular fovea coordinates from the fundus image to be labeled by using the macular fovea positioning network.

[0051] calculating a macular region radius, the macular region radius being constrained by the following mathematical model:

[0052] R1 = λ1 x min(Width, Height),

[0053] wherein R1 represents the macular region radius; λ1 represents a hyperparameter; Width represents the width of the fundus image; Height represents the height of the fundus image;

[0054] determining the macular region according to the macular fovea coordinates and the macular region radius.

[0055] According to some embodiments of the present application, the processing of the fundus image to be labeled by using the optic cup and disc segmentation positioning network to obtain the optic cup and disc region comprises the following steps:

[0056] obtaining the optic cup and disc position information from the fundus image to be labeled by using the optic cup and disc segmentation positioning network;

[0057] calculating the center point coordinates of the optic cup and disc according to the optic cup and disc position information;

[0058] calculating an optic cup and disc region radius, the optic cup and disc region radius being constrained by the following mathematical model:

[0059] R2 = λ2 x min(Width, Height),

[0060] wherein R2 represents the optic cup and disc region radius; λ2 represents a hyperparameter; Width represents the width of the fundus image; Height represents the height of the fundus image;

[0061] determining the optic cup and disc region according to the center point coordinates of the optic cup and disc and the optic cup and disc region radius.

[0062] According to some embodiments of the present application, the calculating of the lesion labeling consistency index between each two labelers according to the first round labeling results comprises the following steps:

[0063] calculating the intersection and union between the first round labeling results of any two labelers in the full image region to obtain a first intersection result and a first union result;

[0064] dividing the first intersection result by the first union result to calculate the full image region consistency index;

[0065] calculating the intersection and union between the first round labeling results of any two labelers in the key region to obtain a second intersection result and a second union result;

[0066] divide the second intersection result by the second union result to calculate a key area consistency index;

[0067] weight and sum the full image area consistency index and the key area consistency index to calculate the lesion annotation consistency index.

[0068] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent from the description, or can be learned by practice of the application. BRIEF DESCRIPTION OF DRAWINGS

[0069] The above and / or additional aspects and advantages of the present application will become apparent and be readily understood from the following description, taken in conjunction with the accompanying drawings, in which:

[0070] Figure 1 is a flowchart of a dense lesion semi-automatic annotation method for fundus images according to an embodiment of the present application;

[0071] Figure 2 is a structural schematic diagram of a dense lesion segmentation network according to an embodiment of the present application;

[0072] Figure 3 is a segmentation effect diagram of a dense lesion segmentation network on a fundus image according to an embodiment of the present application;

[0073] Figure 4 is a structural schematic diagram of a macular fovea positioning network according to an embodiment of the present application;

[0074] Figure 5 is a structural schematic diagram of a Swin Transformer Tiny network according to an embodiment of the present application;

[0075] Figure 6 is a structural schematic diagram of a feature post-processing module according to an embodiment of the present application;

[0076] Figure 7 is a structural schematic diagram of a cup-disc segmentation positioning network according to an embodiment of the present application;

[0077] Figure 8 is a macular area positioning effect diagram of a macular fovea positioning network on a fundus image according to an embodiment of the present application;

[0078] Figure 9 is a cup-disc area positioning effect diagram of a cup-disc segmentation positioning network on a fundus image according to an embodiment of the present application;

[0079] Figure 10(a) is a mask diagram of a hemorrhage lesion annotation by an annotator for a fundus image according to an embodiment of the present application;

[0080] Fig. 10(b) is a mask image annotated by an annotator for a soft exudation lesion in an eye fundus image according to an embodiment of the present application;

[0081] Fig. 10(c) is a mask image annotated by an annotator for a microaneurysm lesion in an eye fundus image according to an embodiment of the present application;

[0082] Fig. 10(d) is a mask image annotated by an annotator for a hard exudation lesion in an eye fundus image according to an embodiment of the present application;

[0083] Figure 11 Fig. 11 is a schematic diagram of a dense lesion semi-automatic annotation system for eye fundus images according to an embodiment of the present application.

[0084] Reference signs:

[0085] a network construction module 100;

[0086] a dense lesion segmentation module 200;

[0087] a macular fovea positioning module 300;

[0088] a optic cup and disc segmentation positioning module 400;

[0089] a region determination module 500;

[0090] a first round annotation result acquisition module 600;

[0091] an annotation consistency calculation module 700;

[0092] a second round annotation result acquisition module 800;

[0093] a final annotation result acquisition module 900. DETAILED DESCRIPTION

[0094] Embodiments of the present application are described below in detail with reference to the accompanying drawings. The embodiments described below are examples for explaining the present application and should not be construed as limiting the present application.

[0095] In the description of the present application, if there is a description of first, second, etc., it is only for the purpose of distinguishing technical features and should not be understood as indicating or implying relative importance or implying the number of indicated technical features or implying the sequence of the indicated technical features.

[0096] In the description of the present application, it needs to be understood that the orientation description, such as the orientation or position relationship indicated by up, down and the like, is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation of the present application.

[0097] In the description of the present application, it needs to be understood that, unless otherwise explicitly limited, the words such as setting, installing, connecting and the like should be understood in a broad sense, and the person skilled in the art can reasonably determine the specific meaning of the above words in the present application in combination with the specific content of the technical solution.

[0098] The technical solutions of the present application will be described below in conjunction with the drawings. Obviously, the following described embodiments are part of the embodiments of the present application, not all embodiments.

[0099] Referring to Figure 1 As shown in the figure, a flow chart of a dense lesion semi-automatic labeling method for fundus images provided by an embodiment of the present application, the method comprising the following steps:

[0100] Construct and train the dense lesion segmentation network, the macular fovea positioning network and the optic cup and disc segmentation and positioning network;

[0101] The dense lesion segmentation network is used to process the fundus image to be labeled to obtain a dense lesion prediction contour region;

[0102] The macular fovea positioning network is used to process the fundus image to be labeled to obtain a macular region;

[0103] The optic cup and disc segmentation and positioning network is used to process the fundus image to be labeled to obtain an optic cup and disc region;

[0104] Three labelers are assigned to the fundus image to be labeled, each labeler selects a key region from the macular region and the optic cup and disc region and determines other regions accordingly, and the key region and the other regions together constitute the full image region of the fundus image to be labeled;

[0105] Each labeler performs a first labeling operation on the key region determined by himself and a second labeling operation on the other region determined by himself to obtain a first round labeling result of himself;

[0106] According to the first round labeling result, the lesion labeling consistency index between each two labelers is calculated to determine two labelers who perform the second round lesion labeling;

[0107] The two annotators performing the second round of lesion annotation each perform a third annotation operation on their respective first round annotation results to obtain their respective second round annotation results.

[0108] The respective second round annotation results are merged and submitted to an auditor for adjustment and checking to obtain a final annotation result.

[0109] Specifically, as shown in Figure 1 It can be understood that three different deep learning networks are first established and trained. Specifically, the dense lesion segmentation network can predict the location information of the dense lesion on the entire fundus image to outline the dense lesion prediction contour region; the macular fovea positioning network can predict the macular fovea coordinate information to circumscribe the macular region; and the optic cup and disc segmentation positioning network can predict the location information of the optic cup and disc to circumscribe the optic cup and disc region.

[0110] Further, the trained three networks are used to process the fundus images to be annotated, so that the dense lesion prediction contour region, the macular region and the optic cup and disc region segmented or positioned by artificial intelligence technology can be obtained. It can be understood that although artificial intelligence technology can greatly improve the efficiency of medical image processing, in the medical field which is relatively professional, annotators are still needed to annotate the dense lesions. Therefore, multiple annotators are assigned to process the fundus images. It can be understood that according to actual needs, it is relatively reasonable to assign three annotators to each fundus image.

[0111] In the first round of annotation, each annotator performs a first marking operation according to the key region determined by himself, that is, a key marking operation, and performs a second marking operation, that is, a simple annotation operation, on other regions, so as to obtain the first round annotation result. It should be noted that the first marking operation mainly includes adjusting, deleting and adding the lesion prediction contour in the key region; the second marking operation mainly includes adjusting and deleting the lesion prediction contour in the other region. Then the first round annotation results of each annotator are calculated for annotation consistency between each other, and according to the calculation result, two annotators are further determined from the multiple annotators to perform the second round of annotation.

[0112] In the second round of annotation, two annotators each perform a third annotation operation on their respective first round annotation results, that is, adjustment, deletion and addition, so as to obtain the second round annotation results. Finally, the union of the second round annotation results of the two annotators is obtained, and after being audited by a senior annotator, a final annotation result is obtained.

[0113] In the embodiment, a plurality of network models for fundus image segmentation and positioning are constructed by using artificial intelligence technology, and a network model with better performance is obtained after model training, so that the to-be-labeled fundus image can be processed to generate a dense lesion prediction contour area to feed back to dense lesion labeling, and a macular area and a cup-disc area are also generated. By using the macular area and the cup-disc area to customize the division of key areas and other areas, different annotators can perform key annotation operations according to the key areas they divide, thereby generating a first round of annotation results that are not completely consistent, and the annotation consistency between different annotators can be calculated and evaluated, and finally appropriate annotators are assigned to the to-be-labeled fundus image according to the annotation consistency to complete the second round of lesion annotation, so as to finally ensure the labeling efficiency and labeling effect of the dense lesions. By using the method of the embodiment of the application, the problem of difficulty in labeling dense lesions in medical images and large differences in labeling between labeling personnel is solved from the root, which can greatly improve the labeling efficiency, reduce the labeling time, and reduce the labeling cost.

[0114] In some embodiments, the dense lesion segmentation network is constructed and trained, including the following steps:

[0115] The dense lesion segmentation network is constructed, and the dense lesion segmentation network is composed of a first encoder, a first decoder and a first segmentation head, wherein the first encoder adopts ResNet50, the first decoder adopts UNet++, and the first segmentation head is composed of a convolution layer with a convolution kernel size of 3x3 and an up-sampling operation;

[0116] The dense lesion segmentation training data and the dense lesion segmentation test data are prepared and online histogram scaling is performed;

[0117] The dense lesion segmentation training data are input into the dense lesion segmentation network for model parameter iterative updating, so as to continuously train the dense lesion segmentation network;

[0118] The loss value of the dense lesion segmentation network processing the dense lesion segmentation test data after each round of training is calculated;

[0119] The model with the minimum loss value is determined as the final dense lesion segmentation network.

[0120] Specifically, referring to Figure 2 , Figure 2 is a structural schematic diagram of the dense lesion segmentation network of the embodiment of the application. In the embodiment, the dense lesion segmentation network is first constructed, and then the dense lesion segmentation network is trained by using the dense lesion segmentation training data, so as to obtain the final dense lesion segmentation network. Figure 2As can be seen, the encoder of the dense lesion segmentation network is ResNet50, the decoder is UNet++, and the segmentation head is composed of a convolution layer with a kernel size of 3x3 and an up-sampling layer. The segmentation head can restore the width and height of the output feature map to the original input image, and the number of channels of the output feature map is consistent with the number of categories. In the embodiment, the number of categories is 5.

[0121] Further, the dense lesion segmentation training data and the dense lesion segmentation test data are prepared. Specifically, the present embodiment takes diabetic retinal lesion segmentation as an example for illustration, and other dense lesions can be similarly applied to the scene. The diabetic retinal lesion segmentation dataset is selected from the currently published DDR dataset, which contains 757 fundus images and annotations of hard exudation, soft exudation, microaneurysm and hemorrhage lesions in the fundus images. In the data preparation stage, the annotation data is converted, that is, the annotated lesions are replaced by different pixel values, wherein the background is represented by pixel value 0, the hard exudation lesion is represented by pixel value 1, the soft exudation lesion is represented by pixel value 2, the microaneurysm lesion is represented by pixel value 3, and the hemorrhage lesion is represented by pixel value 4. The mask information of the lesion annotation is obtained through the above conversion. Then, the DDR dataset is divided to obtain 680 training data and 77 test data. If there is no public dataset for the dense lesion to be annotated, 50 fundus images can be manually annotated, and then 45 training data and 5 test data are divided.

[0122] Further, the prepared data is processed online. Specifically, the present embodiment performs online histogram scaling on the fundus images and the corresponding dense lesion mask information in the training data and the test data, so that the images are transformed from the original width and height to 512x512. Then, the dense lesion segmentation network is trained using the training data. Specifically, the present embodiment performs optic disc segmentation network training on a workbench equipped with an NVIDIA-A100 GPU, the programming language is Python3.8, and the Pytorch1.13 deep learning framework is used. In the training stage, the Batchsize is set to 8, the optimizer uses AdamW, a total of 30 rounds of training, the initial learning rate is set to 10^(-4), the learning rate is adjusted using cosine annealing, and the final learning rate is 10^(-6). By calculating the loss value after each round of training, the model with the minimum loss value of the dense lesion segmentation network on the test data is obtained as the final dense lesion segmentation network. In some embodiments, the network model also performs online random horizontal flip data augmentation strategy in the training stage.

[0123] Further, the segmentation effect of the dense lesion segmentation network on the fundus image is as shown in Figure 3 The results of the dense lesion segmentation network on the diabetic retinal lesion segmentation test data are shown in Table 1:

[0124] Table 1

[0125] Lesion type Hard exudation Soft exudation Microaneurysm Hemorrhage point Index (Dice) 0.8543 0.8125 0.6993 0.8054

[0126] It should be noted that the Dice index in Table 1 is the most common evaluation index in medical image segmentation tasks, which is a set similarity measure index used to calculate the similarity of two samples, with a value range of [0, 1]. The higher the Dice index, the closer to 1, indicating that the segmentation effect is good. Therefore, the higher the performance of the lesion segmentation network, the less the workload of manual annotation. It can be understood that according to relevant comparisons, the indexes of diabetic retinopathy lesion segmentation shown in Table 1 have been superior to most other related models or algorithms.

[0127] In this embodiment, by using the dense lesion segmentation network, the result of artificial intelligence processing can be provided before the doctor performs manual annotation. The artificial intelligence processing result only needs to be appropriately modified in the manual annotation stage to meet the annotation requirements, thereby greatly improving the annotation efficiency and reducing the annotation cost. And from Table 1, it can be seen that the mean Dice of the lesion segmentation network is about 0.8, with excellent performance and good segmentation effect.

[0128] In some embodiments, the loss value of the dense lesion segmentation network processing the dense lesion segmentation test data after each round of training is calculated, which is constrained by the following mathematical model:

[0129]

[0130] Wherein, L dice represents the DiceLoss loss function; N represents the number of pixel points in the lesion segmentation graph; n represents the nth pixel point in the lesion segmentation graph; r n represents the true label of the nth pixel in the lesion segmentation graph; p n represents the predicted probability value of the corresponding pixel; ∈ represents a constant.

[0131] In some embodiments, the macular fovea positioning network is constructed and trained, including the following steps:

[0132] The macular fovea positioning network is constructed, and the macular fovea positioning network is composed of a Swin Transformer Tiny network and a feature post-processing module, wherein the Swin Transformer Tiny network is used for feature extraction of the fundus image, and the feature post-processing module is composed of a first full connection layer, a ReLU layer, a Dropout layer, a second full connection layer and a Sigmoid layer;

[0133] Prepare macular fovea positioning training data and macular fovea positioning test data and perform online histogram scaling and normalization processing;

[0134] The macular fovea positioning training data is input into the macular fovea positioning network for model parameter iterative updating, so as to continuously train the macular fovea positioning network.

[0135] The loss value of the macular fovea positioning network processing the macular fovea positioning test data after each round of training is calculated.

[0136] The model with the minimum loss value is determined as the final macular fovea positioning network.

[0137] Specifically, referring to Figure 4 , Figure 4 is a structural schematic diagram of the macular fovea positioning network of the embodiment of the present application. The macular fovea positioning network is first constructed in this embodiment, and the macular fovea positioning network is trained by using the macular fovea positioning training data. Figure 4 As can be seen from Figure 5 , the macular fovea positioning network includes a SwinTransformer Tiny feature extractor and a feature post-processing module. In combination with reference to Figure 6 , the Swin Transformer Tiny feature extractor is composed of a patch division, a patch embedding layer, a plurality of Swin Transformer Blocks and a plurality of patch merging layers, and can perform feature extraction on the input fundus image. Specifically, by inputting a fundus image with a size of HxWx3, after processing through the four stages as shown in the figure, a picture with a size of H / 32xW / 32x768 is finally obtained. In combination with reference to Figure 6 , the feature post-processing module is composed of a fully connected layer, a ReLU layer, a Dropout layer, a fully connected layer and a Sigmoid layer, and can further extract and process the features and obtain the final prediction result.

[0138] Further, the macular fovea positioning training data and the macular fovea positioning test data are prepared. Specifically, the macular fovea positioning data set used in the embodiment of the present application is from the combination of three public data sets GAMMA, REFUGE and PALM, wherein the GAMMA data set has 100 training data and 100 test data, the REFUGE data set has 800 training data, and the PALM data set has 800 training data and 400 test data. The above data all contain fundus images and coordinate information of the macular fovea in the fundus images. In this embodiment, 100 training data of GAMMA, 800 training data of REFUGE and 800 training data of PALM are used to form the training data of the final macular fovea positioning data set, a total of 1700; 100 test data of GAMMA and 400 test data of PALM are used to form the test data of the final macular fovea positioning data set, a total of 500.

[0139] Further, the prepared data is processed online. Specifically, the fundus image in the training data and the test data is histogram scaled to an image width and height of 384x384, and the macular fovea coordinates are normalized. For example, the original width and height of the fundus image is 3000x2000, and the original macular fovea coordinates are (750, 400). After normalization, the macular fovea coordinates are (0.25, 0.2).

[0140] Further, the macular fovea positioning network is trained using the training data. Specifically, the macular fovea positioning network model is trained on a workstation equipped with an NVIDIA-A100 GPU, the programming language is Python 3.8, and the Pytorch 1.13 deep learning framework is used. In the training stage, the Batchsize is set to 16, the optimizer is AdamW, a total of 50 rounds of training are performed, the initial learning rate is set to 5x10^(-5), the learning rate is adjusted using cosine annealing, and the final learning rate is 10^(-6). By calculating the loss value after each round of training, the macular fovea positioning network model with the lowest loss value on the test set is obtained as the final macular fovea positioning model.

[0141] Further, the indicators of the macular fovea positioning network model on the test data are shown in Table 2:

[0142] Table 2

[0143] Task type Macular fovea localization Index (normalized Euclidean distance) 0.0082

[0144] It should be noted that the indicators in Table 2 are normalized Euclidean distances, and the macular fovea positioning network outputs the coordinate values of the macular fovea. The normalized Euclidean distance is the distance between the predicted macular fovea coordinates and the reference value. The smaller the normalized Euclidean distance value, the more accurate the prediction of the macular fovea, and the better the performance, i.e., the more accurate the macular fovea region defined by the model. It can be understood that the indicator data in Table 2 can better meet the relevant performance requirements.

[0145] In some embodiments, the loss value of the macular fovea positioning network processing the macular fovea positioning test data after each round of training is constrained by the following mathematical model:

[0146] L Loc = 0.5 x L MSE + 0.5 x L ED ,

[0147]

[0148]

[0149] wherein, L Loc represents the total loss function of the macular fovea positioning network; L MSE represents the MSELoss loss function; L ED represents the EDLoss loss function; N represents the number of images in each batch in the training stage; n represents the nth image in each batch; x n represents the normalized real horizontal coordinate of the macular fovea in the nth image in each batch; p n represents the predicted horizontal coordinate of the normalized macular fovea in the nth image in each batch; y n represents the normalized real vertical coordinate of the macular fovea in the nth image in each batch; q n represents the predicted vertical coordinate of the normalized macular fovea in the nth image in each batch.

[0150] In some embodiments, the optic cup and disc segmentation and positioning network is constructed and trained, including the following steps:

[0151] The optic cup and disc segmentation and positioning network is constructed, and the optic cup and disc segmentation and positioning network is composed of a second encoder, a second decoder and a second segmentation head, wherein the second encoder adopts ResNet34, the second decoder adopts UNet, and the second segmentation head is composed of a convolution layer with a convolution kernel size of 3x3 and an upsampling operation;

[0152] The optic cup and disc segmentation and positioning training data and the optic cup and disc segmentation and positioning test data are prepared and online histogram scaling is performed;

[0153] The optic cup and disc segmentation and positioning training data are input into the optic cup and disc segmentation and positioning network for model parameter iterative updating, so as to continuously train the optic cup and disc segmentation and positioning network;

[0154] The loss value of the optic cup and disc segmentation and positioning network processing the optic cup and disc segmentation and positioning test data after each round of training is calculated;

[0155] The model with the minimum loss value is determined as the final optic cup and disc segmentation and positioning network.

[0156] Specifically, referring to Figure 7 , Figure 7 is a structural schematic diagram of the optic cup and disc segmentation and positioning network of the embodiment of the present application. The optic cup and disc segmentation and positioning network is first constructed in the embodiment, and the structure of the optic cup and disc segmentation and positioning network is shown in Figure 7 As can be seen from the figure, the optic cup and disc segmentation and positioning network is composed of an encoder, a decoder and a segmentation head, and the structure is shown in Figure 7 , wherein the encoder adopts ResNet34, the decoder adopts UNet, and the segmentation head is composed of a convolution layer with a convolution kernel size of 3x3 and an upsampling operation, which can restore the width and height of the output feature map to the width and height of the original input image, and the channel number of the output feature map is consistent with the class number.

[0157] Further, the cup-disc segmentation positioning training data and cup-disc segmentation positioning test data are prepared. Specifically, the cup-disc segmentation positioning data used by the embodiment of the application comes from a combination of multiple public data sets REFUGE, ORIGA650 and GAMMA. The above-mentioned data sets all contain fundus images and fundus image cup-disc segmentation mask images. The data in the above-mentioned data sets are divided into training set and test set according to the ratio of 8:2.

[0158] Further, the prepared data is processed online. Specifically, the embodiment of the application performs online histogram scaling on the fundus images in the training data and test data, so that the images are transformed from the original width and height to 512x512. Then the cup-disc segmentation positioning network is trained using the training data. Specifically, the embodiment of the application trains the cup-disc segmentation positioning network model on a workstation equipped with NVIDIA-A100 GPU, the programming language is Python3.8, and the Pytorch1.13 deep learning framework is used. The Batchsize is set to 8 in the training stage, the optimizer uses AdamW, a total of 30 rounds of training, the initial learning rate is set to 1x10^(-4), the learning rate is adjusted using cosine annealing, and the final learning rate is 10^(-6). By calculating the loss value after each round of training, the cup-disc segmentation positioning network model with the lowest loss value on the test set is obtained as the final cup-disc segmentation positioning model.

[0159] Further, the indicators of the cup-disc segmentation positioning network model on the test data are shown in Table 3:

[0160] Table 3

[0161] Task type Optic cup and disc segmentation Index (Dice) 0.9457

[0162] It should be noted that the indicators in Table 3 are also Dice as in Table 1. It can be understood that according to the relevant comparison, the indicators of the cup-disc segmentation positioning network model shown in Table 3 are better than most other related models or algorithms. It can be understood that the embodiment of the application uses the macular fovea positioning network and the cup-disc segmentation positioning network, so as to provide the annotation key area for the annotator in the artificial annotation stage. After determining the annotation key area, the artificial annotation can focus on whether the annotation of the key area is accurate, so the artificial annotation range is greatly reduced and the annotation efficiency is improved.

[0163] In some embodiments, the loss value of the cup-disc segmentation positioning network processing cup-disc segmentation positioning test data after each round of training is constrained by the following mathematical model:

[0164] L all= 0.5 x L MAE + 0.5 x L dice ,

[0165]

[0166]

[0167] wherein, L all represents the total loss function of the optic cup and disc segmentation positioning network; L MAE represents the MAELoss loss function; L dice represents the DiceLoss loss function; N represents the number of pixel points of the fundus image optic cup and disc segmentation graph; n represents the nth pixel point in the segmentation graph; r n represents the true label of the nth pixel in the optic cup and disc segmentation graph; p n represents the predicted probability value of the corresponding pixel; ε and ∈ both represent constants.

[0168] In some embodiments, the macular fovea positioning network is used to process the fundus image to be labeled to obtain the macular region, including the following steps:

[0169] The macular fovea positioning network is used to determine the macular fovea coordinates from the fundus image to be labeled;

[0170] The macular region radius is calculated, which is constrained by the following mathematical model:

[0171] R1 = λ1 x min(Width, Height),

[0172] wherein, R1 represents the macular region radius; λ1 represents a hyperparameter; Width represents the fundus image width; Height represents the fundus image height;

[0173] The macular region is determined according to the macular fovea coordinates and the macular region radius.

[0174] Specifically, it can be understood that the macular fovea coordinates are first obtained by using the macular fovea positioning network; then the macular region radius is calculated based on the corresponding mathematical formula; finally, the region with the macular fovea coordinates as the center and the radius R1 as the macular region is displayed. Specifically, the schematic diagram of the macular region is shown in Figure 8 The part beyond the image edge will be ignored.

[0175] In some embodiments, the optic cup and disc segmentation positioning network is used to process the fundus image to be labeled to obtain the optic cup and disc region, including the following steps:

[0176] The optic cup and disc segmentation positioning network is used to obtain the optic cup and disc position information from the fundus image to be labeled.

[0177] According to the optic cup and disc position information, the center point coordinates of the optic cup and disc are calculated.

[0178] The optic cup and disc region radius is calculated, and the optic cup and disc region radius is constrained by the following mathematical model:

[0179] R2 = λ2 × min(Width, Height),

[0180] Wherein, R2 represents the optic cup and disc region radius; λ2 represents a hyperparameter; Width represents the fundus image width; Height represents the fundus image height.

[0181] The optic cup and disc region is determined according to the center point coordinates of the optic cup and disc and the optic cup and disc region radius.

[0182] Specifically, it can be understood that first, the optic cup and disc position information is obtained by using the optic cup and disc segmentation positioning network, and the center point coordinates of the optic cup and disc are calculated accordingly; then the optic cup and disc region radius is calculated based on the corresponding mathematical formula; finally, the region with the optic cup and disc center point coordinates as the center and the radius R2 as the region of the optic cup and disc is shown. Specifically, the schematic diagram of the optic cup and disc region is shown in Figure 9 The part beyond the image edge will be ignored.

[0183] In some embodiments, the lesion annotation consistency index between each two annotators is calculated according to the first round of annotation results, including the following steps:

[0184] The intersection and union between the first round of annotation results of any two annotators in the whole image region are calculated to obtain the first intersection result and the first union result;

[0185] The first intersection result is divided by the first union result to calculate the whole image region consistency index;

[0186] The intersection and union between the first round of annotation results of any two annotators in the key region are calculated to obtain the second intersection result and the second union result;

[0187] The second intersection result is divided by the second union result to calculate the key region consistency index;

[0188] The whole image region consistency index and the key region consistency index are weighted and summed to calculate the lesion annotation consistency index.

[0189] Specifically, it can be understood that since three annotators are assigned to the fundus image to be labeled, the fundus image can calculate three lesion annotation consistency indicators. The annotation consistency indicator calculation process for each two annotators is specifically:

[0190] Mask information is generated for each lesion annotation completed by each annotator. It should be noted that, for example, there are four types of diabetic retinal lesions, specifically hemorrhagic lesions, soft exudative lesions, microaneurysm lesions, and hard exudative lesions, so each annotator has four two-dimensional arrays of mask information. In combination with reference to FIG. 10, (a), (b), (c), and (d) of FIG. 10 respectively show the visualization results of the two-dimensional data of the above four types, wherein the white area is the lesion position;

[0191] The annotation consistency indicators of each lesion of two annotators in the whole image area, i.e., the whole image area consistency indicators, are calculated. Specifically, the intersection and union operations of the two mask images of each lesion are performed, and then the intersection area is divided by the union area to obtain the annotation consistency indicator. In particular, if a certain type of lesion is not in the whole image area, the annotation consistency indicator of the lesion is empty and does not participate in the subsequent annotation consistency indicator calculation;

[0192] The annotation consistency indicators of each lesion of two annotators in the macular area, i.e., the key area consistency indicators, are calculated. Specifically, the intersection of the mask image of each lesion and the key area is first taken, and then the intersection is used as the new lesion mask information for annotation consistency indicator calculation. In particular, if a certain type of lesion is not in the key area, the annotation consistency indicator of the lesion is empty and does not participate in the subsequent annotation consistency indicator calculation;

[0193] The lesion annotation consistency indicators are obtained by weighted summation, and the specific calculation formula is as follows:

[0194] IoU all = a x sum of each lesion annotation consistency indicator of the whole image area,

[0195] IoU main = b x sum of each lesion annotation consistency indicator of the key area,

[0196] Score = 0.4 x IoU all + 0.6 x IoU main ,

[0197] wherein IoU all represents the whole image area consistency indicator, and IoU mainThe Score represents the lesion annotation consistency index. In the above formula, a and β will determine the final value through actual conditions, for example, if there are 3 annotation consistency indexes in the full image area of an eye fundus image, then a = 1 / 3; if there are 2 annotation consistency indexes in the key area, then β = 1 / 2.

[0198] In some embodiments, the second round of annotation results of the two annotators performing the second round of annotation are merged, specifically, a mask image is generated for each type of lesion in the two annotation results of the eye fundus image to be annotated, and then the mask images are merged based on the mask images, that is, a logical OR operation is performed, so as to obtain the union.

[0199] It should be noted that the number of annotators in the first round of annotation of the embodiment of the present application can be set to more than 3, that is, the number of annotators is not particularly limited; based on the number of annotators set in the first round, the lesion annotation consistency index between the first round of annotation results of more than 2 annotators can be calculated, so that more than 2 annotators can be determined to perform the second round of annotation; based on the number of annotators set in the first round, more than 2 rounds of annotation can be implemented, so that after multiple rounds of annotation, the annotation results of the two annotators with the best consistency can be finally determined, and the final lesion annotation result can be merged and audited.

[0200] It should also be noted that the method of the embodiment of the present application can be implemented using multiple eye fundus images to be annotated, that is, the number of eye fundus images to be annotated that can be processed is not particularly limited. Therefore, in some embodiments, 200 eye fundus images are selected for annotation, and the time consumption of the original dense lesion annotation method and the dense lesion semi-automatic annotation method for eye fundus images is counted, and the specific statistical results are shown in Table 4, wherein the annotation time of each eye fundus image in each round is an average index.

[0201] Table 4

[0202] Round Original dense lesion annotation method Dense lesion semi-automatic annotation method for fundus images First round of annotation 956 seconds / person*2 persons*200 73 seconds / person*3 persons*200 Second round of annotation 0 212 seconds / person*2 persons*200 Final review 90 seconds / person*1 person*200 66 seconds / person*1 person*200 Total 400400 seconds 141800 seconds

[0203] According to the statistics, it can be seen that the time length of the dense lesion semi-automatic annotation method for eye fundus images is only 35.4% of the time length of the original dense lesion annotation method, and the time required for the final audit link is significantly reduced, that is, there is less content to adjust. Therefore, it can be seen that the dense lesion semi-automatic annotation method for eye fundus images can greatly improve the annotation efficiency of the annotator and reduce the annotation time.

[0204] In addition, as Figure 11 shown, the embodiment of the present application also provides a dense lesion semi-automatic annotation system for eye fundus images, which comprises:

[0205] The network construction module 100 is configured to construct and train the dense lesion segmentation network, the macular fovea positioning network and the optic cup and disc segmentation and positioning network.

[0206] The dense lesion segmentation module 200 is configured to process the image to be labeled by using the dense lesion segmentation network to obtain a dense lesion prediction contour region.

[0207] The macular fovea positioning module 300 is configured to process the image to be labeled by using the macular fovea positioning network to obtain a macular region.

[0208] The optic cup and disc segmentation and positioning module 400 is configured to process the image to be labeled by using the optic cup and disc segmentation and positioning network to obtain an optic cup and disc region.

[0209] The region determination module 500 is configured to assign three labelers to the image to be labeled, and each labeler selects a key region from the macular region and the optic cup and disc region and determines other regions accordingly, and the key region and the other regions jointly constitute a full image region of the image to be labeled.

[0210] The first round of labeling result acquisition module 600 is configured to perform a first labeling operation on the key region determined by each labeler and a second labeling operation on the other region determined by each labeler to obtain a first round of labeling result of each labeler.

[0211] The labeling consistency calculation module 700 is configured to calculate a lesion labeling consistency index between each two labelers according to the first round of labeling result to determine two labelers performing a second round of lesion labeling.

[0212] The second round of labeling result acquisition module 800 is configured to perform a third labeling operation on the first round of labeling result of each labeler to obtain a second round of labeling result of each labeler.

[0213] The final labeling result acquisition module 900 is configured to merge the second round of labeling result of each labeler and submit to an auditor for adjustment and checking to obtain a final labeling result.

[0214] Specifically, referring to Figure 11 It can be understood that the dense lesion semi-automatic labeling system for the fundus image of the embodiment of the application is used to implement the dense lesion semi-automatic labeling method for the fundus image, and the dense lesion semi-automatic labeling system for the fundus image of the embodiment of the application corresponds to the aforementioned dense lesion semi-automatic labeling method for the fundus image. For specific processing process, please refer to the aforementioned dense lesion semi-automatic labeling method for the fundus image, which will not be described here.

[0215] In this embodiment, a plurality of network models for fundus image segmentation and positioning are constructed by using the network construction module 100, and the network models with better performance are obtained after model training, so that the dense lesion segmentation module 200 can be used to process the fundus image to be labeled to generate a dense lesion prediction contour area to feed back to the dense lesion labeling, and the macular fovea positioning module 300 and the optic cup and disc segmentation and positioning module 400 are used to generate a macular region and an optic cup and disc region, respectively. In the region determination module 500, the macular region and the optic cup and disc region are used to define the key region and the other region, and different labelers can perform key labeling operations according to the key region they define in the first round of labeling result acquisition module 600, so as to generate an inconsistent first round of labeling result, and then the labeling consistency between different labelers can be calculated and evaluated in the labeling consistency calculation module 700, and finally appropriate labelers are assigned to the fundus image to be labeled according to the labeling consistency, so as to complete the second round of lesion labeling in the second round of labeling result acquisition module 800. Therefore, the final labeling result obtained in the final labeling result acquisition module 900 ensures the labeling efficiency and labeling effect of the dense lesion. By using the system of the embodiment of the present application, the problems of difficulty in labeling dense lesions in medical images and large differences in labeling between labeling personnel are solved from the root, which can greatly improve the labeling efficiency, reduce the labeling time, and reduce the labeling cost.

[0216] In addition, the embodiment of the present application also provides a kind of dense lesion semi-automatic labeling equipment for fundus image, comprising: at least one control processor and the memory for being connected with at least one control processor communication.

[0217] The memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. In addition, the memory can include a high-speed random access memory and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory can optionally include a memory remotely located relative to the processor, which can be connected to the processor through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0218] The non-transitory software programs and instructions required for the dense lesion semi-automatic labeling method for fundus image according to the above-mentioned embodiments are stored in the memory, and when executed by the processor, the dense lesion semi-automatic labeling method for fundus image according to the above-mentioned embodiments is executed, for example, the method in the above description Figure 1 is executed.

[0219] The system embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purposes of the embodiments.

[0220] In addition, the embodiment of the present application further provides a computer readable storage medium, which stores computer executable instructions, and the computer executable instructions are executed by one or more control processors, so that the one or more control processors execute a dense lesion semi-automatic labeling method for fundus images in the method embodiments. Figure 1

[0221] Those skilled in the art can understand that all or some steps of the above disclosed method and system can be implemented as software, firmware, hardware and appropriate combination thereof. Some or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software can be distributed on a computer readable medium, which can include computer storage media (or non-transitory media) and communication media (or transitory media). As known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. In addition, as known to those skilled in the art, communication media generally includes computer readable instructions, data structures, program modules or other data in modulated data signals such as carrier waves or other transmission mechanisms, and can include any information delivery medium.

[0222] The embodiments of the present application are described in detail above in combination with the drawings, but the present application is not limited to the above embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the purpose of the present application.​

Claims

1. A method for semi-automatic annotation of dense lesions for fundus images, characterized in that, The method comprises the following steps: constructing and training a dense lesion segmentation network, a macular fovea positioning network and a optic cup and disc segmentation positioning network; processing the eye fundus image to be labeled by using the dense lesion segmentation network to obtain a dense lesion prediction contour region; processing the eye fundus image to be labeled by using the macular fovea positioning network to obtain a macular region; processing the eye fundus image to be labeled by using the optic cup and disc segmentation positioning network to obtain an optic cup and disc region; allocating three labelers to the eye fundus image to be labeled, each labeler selecting a key region from the macular region and the optic cup and disc region and determining other regions accordingly, the key region and the other regions together forming a full image region of the eye fundus image to be labeled; each labeler performing a first labeling operation on the key region determined by the labeler and performing a second labeling operation on the other regions determined by the labeler to obtain a first round labeling result of the labeler; calculating a lesion labeling consistency index between each two labelers according to the first round labeling results to determine two labelers performing a second round lesion labeling; the two labelers performing the second round lesion labeling performing a third labeling operation on the first round labeling result of the labeler to obtain a second round labeling result of the labeler; merging the second round labeling results of the labelers and submitting the merged results to an auditor for adjustment and checking to obtain a final labeling result; the method of calculating the lesion labeling consistency index between each two labelers according to the first round labeling results comprises the following steps: calculating an intersection and a union between the first round labeling results of any two labelers in the full image region to obtain a first intersection result and a first union result; dividing the first intersection result by the first union result to calculate a full image region consistency index; calculating an intersection and a union between the first round labeling results of any two labelers in the key region to obtain a second intersection result and a second union result; dividing the second intersection result by the second union result to calculate a key region consistency index; performing weighted summation on the full image region consistency index and the key region consistency index to calculate the lesion labeling consistency index. 2.The method for dense lesion semi-automatic annotation for fundus images according to claim 1, characterized in that, The method of constructing and training the dense lesion segmentation network comprises the following steps: constructing the dense lesion segmentation network, the dense lesion segmentation network comprising a first encoder, a first decoder and a first segmentation head, wherein the first encoder adopts ResNet50, the first decoder adopts UNet++, and the first segmentation head comprises a convolution layer with a convolution kernel size of 3x3 and an up-sampling operation; preparing dense lesion segmentation training data and dense lesion segmentation test data and performing online histogram scaling; inputting the dense lesion segmentation training data into the dense lesion segmentation network to update model parameters iteratively to continuously train the dense lesion segmentation network; calculating a loss value of the dense lesion segmentation network processing the dense lesion segmentation test data after each round of training; The model with the minimum loss value is determined as the final dense lesion segmentation network. 3.The method for dense lesion semi-automatic annotation of fundus images according to claim 2, characterized in that, The loss value of the dense lesion segmentation network processing the dense lesion segmentation test data after each round of training is calculated and is constrained by the following mathematical model: wherein, represents a DiceLoss loss function; represents the number of pixel points in the lesion segmentation map; represents the nth pixel point in the lesion segmentation map; represents the true label of the nth pixel in the lesion segmentation map; represents the predicted probability value of the corresponding pixel; represents a constant. 4.The method for dense lesion semi-automatic annotation of fundus images according to claim 1, characterized in that, The macular fovea positioning network is constructed and trained, including the following steps: The macular fovea positioning network is constructed, and the macular fovea positioning network is composed of a Swin Transformer Tiny network and a feature post-processing module, wherein the Swin Transformer Tiny network is used for feature extraction of the fundus image, and the feature post-processing module is composed of a first full connection layer, a ReLU layer, a Dropout layer, a second full connection layer and a Sigmoid layer; The macular fovea positioning training data and the macular fovea positioning test data are prepared and are subjected to online histogram scaling and normalization processing; The macular fovea positioning training data are input into the macular fovea positioning network for iterative updating of model parameters, so as to continuously train the macular fovea positioning network; The loss value of the macular fovea positioning network processing the macular fovea positioning test data after each round of training is calculated; The model with the minimum loss value is determined as the final macular fovea positioning network.

5. The method for dense lesion semi-automatic annotation of fundus images according to claim 4, characterized in that, The loss value of the macular fovea positioning network processing the macular fovea positioning test data after each round of training is calculated and is constrained by the following mathematical model: wherein, represents the total loss function of the macular fovea positioning network; represents the MSELoss loss function; represents the EDLoss loss function; represents the number of images per batch in the training phase; represents the nth image in each batch; represents the real horizontal coordinate of the normalized macular fovea in the nth image in each batch; represents the predicted horizontal coordinate of the normalized macular fovea in the nth image in each batch; represents the real vertical coordinate of the normalized macular fovea in the nth image in each batch; represents the predicted vertical coordinate of the normalized macular fovea in the prediction of the nth image in each batch. 6.The method for dense lesion semi-automatic annotation of fundus images according to claim 1, characterized in that, The optic cup and disc segmentation positioning network is constructed and trained, including the following steps: The optic cup and disc segmentation positioning network is constructed, and the optic cup and disc segmentation positioning network is composed of a second encoder, a second decoder and a second segmentation head, wherein the second encoder adopts ResNet34, the second decoder adopts UNet, and the second segmentation head is composed of a convolution layer with a convolution kernel size of 3x3 and an up-sampling operation; The optic cup and disc segmentation positioning training data and the optic cup and disc segmentation positioning test data are prepared and are subjected to online histogram scaling; The optic cup and disc segmentation positioning training data are input into the optic cup and disc segmentation positioning network for iterative updating of model parameters, so as to continuously train the optic cup and disc segmentation positioning network; The loss value of the optic cup and disc segmentation positioning network processing the optic cup and disc segmentation positioning test data after each round of training is calculated; The model with the minimum loss value is determined as the final optic cup and disc segmentation positioning network.

7. The method for dense lesion semi-automatic annotation of fundus images according to claim 6, characterized in that, The loss value of the optic cup and disc segmentation positioning network processing the optic cup and disc segmentation positioning test data after each round of training is calculated and is constrained by the following mathematical model: wherein, represents the total loss function of the optic cup and disc segmentation positioning network; represents the MAELoss loss function; represents the DiceLoss loss function; represents the pixel point number of the optic fundus image optic cup and disc segmentation graph; represents the nth pixel point in the segmentation graph; represents the true label of the nth pixel in the optic cup and disc segmentation graph; represents the predicted probability value of the corresponding pixel; and both represent constants. 8.The method for dense lesion semi-automatic annotation of fundus images according to claim 1, characterized in that, The macular fovea positioning network is used to process the fundus image to be labeled to obtain a macular region, including the following steps: The macular fovea positioning network is used to determine the macular fovea coordinates from the fundus image to be labeled; The macular region radius is calculated, and the macular region radius is constrained by the following mathematical model: R1=λ1×min(Width, Height), wherein R1 represents the macular region radius; λ1 represents a hyperparameter; Width represents the fundus image width; and Height represents the fundus image height; The macular region is determined according to the macular fovea coordinates and the macular region radius. 9.The method for dense lesion semi-automatic annotation of fundus images according to claim 1, characterized in that, The processing of the to-be-labeled fundus image by using the optic cup and disc segmentation positioning network to obtain an optic cup and disc region comprises the following steps: The optic cup and disc position information is acquired from the to-be-labeled fundus image by using the optic cup and disc segmentation positioning network. The center point coordinates of the optic cup and disc are calculated according to the optic cup and disc position information. The optic cup and disc region radius is calculated, and the optic cup and disc region radius is constrained by the following mathematical model: R2=λ2×min(Width, Height), wherein R2 represents the optic cup and disc region radius; λ2 represents a hyperparameter; Width represents the fundus image width; Height represents the fundus image height; The optic cup and disc region is determined according to the center point coordinates of the optic cup and disc and the optic cup and disc region radius.

Images