An eye fundus blood vessel drug application region classification method based on a time sequence fusion network

Abstract

The application discloses a fundus blood vessel drug application region classification method based on a time sequence fusion network and relates to the technical field of medical image classification. Step one: after an image sequence and corresponding labels are acquired, an image labeling tool is used to convert the image and label information into a format suitable for neural network learning, and then a boundary box of a drug application region is drawn on the last image and the corresponding category is labeled; the fundus blood vessel drug application region classification method based on the time sequence fusion network, the application considers the characteristics of FFA image drug application region classification and proposes a time sequence fusion target detection network, a group of FFA images are input, the time sequence fusion target detection network can not only locate the drug application region but also extract time sequence information of the drug application region to provide a basis for classification. This not only makes up for the deficiency that a traditional target detection network cannot process sequence images, but also introduces a time sequence information extraction module to simulate an observation process of dynamic changes of a drug application region in an artificial classification process, so that the classification accuracy can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image classification, in particular to a fundus blood vessel drug use area classification method based on a time sequence fusion network. BACKGROUND

[0002] Fluorescein Fundus Angiography (FFA) is an imaging technique used to examine the blood vessel system in the back of the eye, the retina. It is widely used in the diagnosis and monitoring of various fundus diseases. It is a non-invasive examination, mainly through the injection of fluorescein dye, and then taking dynamic images of the fundus blood vessels, helping doctors understand the state of the retinal and choroidal blood vessels.

[0003] FFA works by injecting fluorescein sodium, a yellow fluorescent dye, into the bloodstream, which flows into the blood vessels of the eye. Fluorescein emits green fluorescence when excited by blue light of a specific wavelength. Through special filters and photographic equipment, doctors can take fluorescent images of the fundus, tracking the flow of dye in the retinal and choroidal blood vessels. This process usually records the entire process of dye from entering the eye to leaving it in several stages.

[0004] Through FFA technology, experts can observe the process of drug action in the retinal and choroidal blood vessels in detail, on the basis of which the state of the drug use area such as brightness and leakage is analyzed, the efficacy of the drug and potential side effects are evaluated, and the drug and its use method are optimized according to the effect.

[0005] However, detailed analysis of FFA images is required in this process, which usually takes a lot of time and is highly dependent on the experience and judgment ability of experts. Experts must carefully review each image to capture the subtle changes in the retinal and choroidal blood vessels, which is crucial for judging the efficacy of the drug, but also increases the complexity of diagnosis and workload. And the interpretation between experts may differ, and this subjectivity also poses a challenge to subsequent analysis.

[0006] In recent years, artificial intelligence technology, especially deep learning methods, has made significant progress in medical image processing. Deep learning methods have been widely applied to the analysis and processing of medical images such as MRI, CT, and endoscopy, significantly improving the accuracy and efficiency of medical diagnosis. In MRI image processing, deep learning technology is used in image reconstruction, segmentation, and disease prediction, among other aspects. By training deep learning networks, researchers can automatically segment brain tumors, spinal diseases, and other structures, helping doctors quickly obtain information about key lesions. In CT image analysis, deep learning technology is widely used in tasks such as tumor detection, lung disease identification, and fracture diagnosis. By training deep learning models, it can automatically identify and classify lesions in CT images, such as lung nodules or cerebral hemorrhage, helping radiologists quickly locate lesions in a large number of images and improve diagnostic efficiency. In endoscopic image processing, deep learning can be applied to image enhancement tasks to improve dark or poor-quality endoscopic images. Through enhancement algorithms, doctors can more clearly observe lesions, reducing the risk of misdiagnosis due to image quality problems. In addition, researchers have also used neural networks based on segmentation tasks to automatically identify and segment polyps and other lesions in endoscopic images. In summary, the application of deep learning technology in medical image processing significantly reduces the workload of doctors in the tedious image review process, improving the efficiency and accuracy of diagnosis.

[0007] Therefore, the use of deep learning technology in FFA image analysis has far-reaching significance, on the one hand, it can save a lot of manpower and material resources, on the other hand, it can help doctors improve the accuracy of analysis and reduce the impact of subjective judgment, ensuring the consistency of the analysis results.

[0008] In the drug area classification task of FFA images, there are often multiple key areas that need to be observed. The morphology, structure, and development process of these areas may differ significantly, so uniform classification of the entire image may result in the omission of important details. Therefore, to improve the accuracy of classification and the effectiveness of clinical application, the ideal solution is to classify each area of interest individually. This method not only allows more accurate identification of the type of lesion in each area, but also provides more detailed information about the disease.

[0009] To achieve this goal, target detection networks, a type of deep learning model, have become effective tools. Such networks not only automatically identify different target regions in images, but also provide accurate location coordinates for each region, enabling more accurate analysis. In the field of deep learning, there are two main types of target detection models: one-stage models represented by YOLO and two-stage models represented by R-CNN, Fast R-CNN, and Faster RCNN. One-stage models treat the target detection problem as a regression task, directly predicting classes and bounding boxes on the entire image without generating candidate regions. Two-stage models, on the other hand, generate candidate regions first and then perform accurate classification and bounding box regression. The advantage of one-stage models is speed, making them suitable for real-time detection tasks. However, their accuracy is slightly lower than that of two-stage models, especially when dealing with complex scenes and small targets. Two-stage models generally perform better in terms of detection accuracy and are suitable for tasks that require high accuracy. FFA images are very complex, mainly reflected in their rich vascular network and diverse morphological features of medication areas. The fundus contains a large number of small and dense blood vessels, and the shape, direction, and distribution of blood vessels are very complex. In addition, FFA images not only contain normal blood vessels, but also have subtle differences in shape and distribution of pathological areas. Therefore, two-stage target detection models are more suitable for FFA images.

[0010] Target detection models usually process single static images, while FFA image analysis involves not only the brightness, shape, and other information of the target area in the late image, but also the dynamic changes at different times. Therefore, relying solely on traditional target detection models may overlook important information in the time dimension. To solve this problem, a time processing module must be introduced into the model. Common time processing modules include recurrent neural networks and their variants, such as long short-term memory networks and gated recurrent units, which can effectively capture temporal dependencies in data. SUMMARY

[0011] (I) Technical problems solved

[0012] To address the shortcomings of the prior art, the present application provides a fundus blood vessel medication area classification method based on a time sequence fusion network, which solves the problems mentioned in the background art.

[0013] (II) Technical solutions

[0014] To achieve the above objectives, the present application is implemented by the following technical solutions: a fundus blood vessel medication area classification method based on a time sequence fusion network, specifically comprising the following steps:

[0015] Step 1: After obtaining the image sequence and corresponding labels, use an image annotation tool to convert the image and label information into a format suitable for neural network learning. Then, draw the bounding box of the medication area on the last image and label the corresponding category.

[0016] Step 2: Combine the temporal module to extract temporal information from a set of FFA images. This assists the target detection network in more accurately classifying the drug application areas based on the temporal information. The temporal module can classify multiple drug application areas in the FFA images.

[0017] Step 3: After the network is built, training begins. A set of images is input into the built network to generate predicted bounding boxes, categories, and confidence scores. Then, the coordinate loss between the predicted bounding boxes and the ground truth bounding boxes, as well as the classification loss between the predicted categories and confidence scores and the ground truth categories are calculated.

[0018] Step 4: The network processes the input image and generates the corresponding predicted bounding boxes, class labels, and confidence scores. During testing, the predicted results are compared with the true labels, and the average precision, accuracy, and recall metrics are calculated to evaluate the model performance.

[0019] Preferably, in order to reduce the amount of input in step one, a portion of images from the early, middle and late stages need to be selected to form an image sequence. The division between the early, middle and late stages is defined by professionals.

[0020] Preferably, in step two, when using time-series information to classify more accurately, dynamic changes are an important basis for classifying drug use regions.

[0021] Preferably, when the timing module is combined in step two, it can capture dynamic changes in a set of continuous images.

[0022] Preferably, in step three, after calculating the coordinate loss between the predicted bounding box and the true bounding box, and the classification loss between the predicted class and the confidence level and the true class, the gradient information of the loss function is propagated to each layer of the network through the backpropagation algorithm. Combined with the gradient descent method, the network parameters are optimized and updated to gradually reduce the loss value.

[0023] Preferably, when the gradient information of the loss function is propagated to each layer of the network through the backpropagation algorithm, and the network parameters are optimized and updated in combination with the gradient descent method to gradually reduce the loss value, this training process needs to be repeated until the loss value converges.

[0024] Preferably, in step four, when the network processes the input image, a set of test image sequences that have not been trained are prepared and these images are input into the trained network.

[0025] Preferably, in step four, the network parameters are not updated and the loss value is not calculated during the testing process.

[0026] (III) Beneficial Effects

[0027] This invention provides a method for classifying retinal vascular drug delivery regions based on a temporal fusion network. Compared with existing technologies, it has the following advantages:

[0028] This invention presents a temporal fusion network-based method for classifying drug-applied regions in fundus vessels. Taking into account the characteristics of drug-applied region classification in FFA images, it proposes a temporal fusion target detection network. Given a set of FFA images, the network not only locates the drug-applied region but also extracts its temporal information to provide a basis for classification. This not only overcomes the limitation of traditional target detection networks in processing sequential images but also, by introducing a temporal information extraction module, simulates the observation process of dynamic changes in the drug-applied region during manual classification, thereby improving classification accuracy. Attached Figure Description

[0029] Figure 1 A network structure diagram of drug-use region classification in fluorescein fundus angiography images based on a temporal fusion target detection network;

[0030] Figure 2 Regional pooling module structure diagram. Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] Please see Figures 1-2 The present invention provides a technical solution:

[0033] This invention proposes a temporal fusion target detection network for classifying drug application regions in FFA images, the framework of which is as follows: Figure 1As shown, a set of 10 FFA images is first acquired, including 5 early / mid-stage images and 5 late-stage images. These 10 images are input into the backbone network to obtain image features. Then, the image features of the last image are input into the region proposal network to obtain candidate boxes. The region pooling module maps the candidate boxes onto the image features and performs dimensionality reduction to obtain the region features of each image. The region features of the last image are fed into a fully connected layer to generate the coordinates of the predicted bounding boxes, and simultaneously fed into a temporal fusion module with the region features of other images to obtain the temporal features of the set of images. Next, the temporal features are fused with the region features of the last image to obtain fused temporal features. The fused temporal features are then fed into a fully connected layer to generate class scores. Post-processing removes redundant and overlapping predicted bounding boxes to facilitate the calculation of performance metrics and visualization.

[0034] Selecting a test platform

[0035] The temporal fusion object detection network of this invention is implemented using PyTorch, the most popular open-source deep learning framework, and trained and tested on a server equipped with 8 NVIDIA GeForce RTX 3090 GPUs and Intel Xeon Silver 4214R CPUs.

[0036] The specific technical solution includes the following steps:

[0037] 1. Dataset Acquisition

[0038] The data used in this invention consists of a sequence of images taken during the early, middle, and late stages of fluorescein fundus angiography. To reduce input, a subset of images from each stage is selected to form the image sequence. Each medication application area needs to be scored by a professional. After obtaining the image sequence and corresponding labels, an image annotation tool (such as LabelImg) is used to convert the image and label information into a format suitable for neural network learning. The bounding box of the medication application area needs to be drawn on the last image, and the corresponding category needs to be labeled.

[0039] The data used in the training and testing process of this invention came from 14 samples. For each sample, fluorescein fundus angiography (FFA) was performed at two, three, and four weeks after medication, with consecutive FFA images of both eyes. Subsequently, experts, based on their clinical experience, classified each medication-treated region, resulting in eight medication-treated regions per image, categorized into four groups: 1, 2, 3, and 4. Next, five early-to-mid-stage images and five late-stage images were selected from each image sequence. After obtaining the image sequences and category labels, the open-source image annotation tool LabelImg was used to mark the bounding boxes and corresponding categories of the medication-treated regions in the last image of each group, and the annotation files were saved. Ultimately, 840 image sequences and 84 annotation files were obtained, with a total of [number missing] annotations, which were used as the training and testing data for the network.

[0040] 2. Model Design

[0041] This invention proposes a temporal fusion target detection network capable of classifying multiple medication regions in FFA images. Traditional target detection methods can locate and classify multiple targets in a single image, but they cannot capture dynamic changes in a series of consecutive images, which are crucial for medication region classification. Therefore, this invention proposes an improved target detection network that incorporates a temporal module to extract temporal information from a set of FFA images. This assists the target detection network in more accurately classifying medication regions based on temporal information.

[0042] This invention proposes a temporal fusion target detection network for classifying drug-treated regions in fluorescein fundus angiography (FFA) images. The design motivation for this network is as follows: 1) In FFA images, classifying drug-treated regions relies on observing changes in target regions within a sequence of images. However, these regions are typically small, and some morphological changes are not obvious. Furthermore, the complex and numerous vascular structures in FFA images can easily lead to visual fatigue from prolonged observation. In addition, this process requires analysis by experienced experts, and fatigue and differences in experience among experts can lead to inconsistent classification results. Therefore, constructing a target detection network specifically for FFA images not only leverages the powerful capabilities of neural networks to accurately capture subtle changes in images, reducing human labor costs, but also improves the consistency of classification results. 2) Traditional target detection networks can achieve the localization and classification of multiple targets in static images. For example, the two-stage target detection network Faster R-CNN first generates a batch of candidate bounding boxes and then extracts feature information within these bounding boxes. Based on the extracted features, the class and probability of the target within the bounding box are output, and the coordinates of the bounding box are adjusted to improve the accuracy of target localization. In the task of classifying drug application regions in FFA images, it is necessary to both locate and classify the drug application regions. Naturally, it's considered to transfer object detection models to this task. However, the difference lies in the fact that classifying drug application regions in FFA images requires considering not only the brightness and morphology of the drug application regions in a single FFA image, but also the diffusion process of fluorescein within those regions. This necessitates examining multiple images, not just a single image, which for the network means acquiring additional temporal features. Therefore, this invention designs a temporal fusion object detection network that integrates temporal information into the object detection network for classifying drug application regions in FFA images.

[0043] Figure 1 The overall architecture of the target detection network that integrates temporal information designed in this invention is described. The network input is... to There are 10 images in total. The output is a set of bounding box coordinates for the last image and their corresponding categories. The composition of each part of the network is described below.

[0044] Backbone Network: Its main function is to extract image features. This network uses ResNet50 as its backbone, loaded with its pre-trained weights. This not only accelerates model convergence but also reduces the risk of overfitting due to limited training data. ResNet50 is a multi-layered convolutional neural network structure; higher layers have more abstract features, which are more conducive to recognizing large-scale targets in images, and vice versa. Since the area of ​​the drug application region in an FFA image is relatively small, this network does not use the top-level features of ResNet50 but instead uses features from its third stage (ResNet50 has four stages of features).

[0045] Region Proposal Network: Its main function is to generate corresponding candidate boxes based on image features.

[0046] Region pooling module: This module maps candidate boxes onto image features to obtain region features. Since candidate boxes vary in size, the mapped features will also have different scales. Therefore, the region pooling module uses region pooling layers to map these features of different sizes to a uniform scale. These features are then unfolded and fed into consecutive fully connected layers to reduce the dimensionality of the features and decrease the number of parameters.

[0047] Temporal fusion module: Its function is to extract temporal features from the sequence of images and fuse them with the regional features of the last image in the sequence to obtain fused temporal features, which are used as features for classification.

[0048] Fully connected layer 1: A fully connected layer used for regression to generate the original class scores for the corresponding predicted bounding boxes.

[0049] Fully connected layer 2: A fully connected layer used to adjust the coordinates of the predicted bounding box.

[0050] Specifically, for a set of FFA images ,in , and This represents the width and height of the image. The images are sequentially input into the backbone network to obtain their respective image features. Where c is the number of feature channels, and when using the third-stage features of ResNet50, c is 1024. and These represent the width and height of the feature, respectively. For the features of the 10th image... It is fed into the region proposal network to generate candidate boxes. , where n is the number of candidate boxes. Next, candidate boxes and image features The data is input into the region pooling module and first undergoes the operation shown in formula (1).

[0051] (1)

[0052] in, This indicates that the region pooling layer maps candidate boxes of different sizes onto image features and resizes them to the same size. ,in To ensure a uniform regional feature width, the operation in formula (2) will be performed next.

[0053] (2)

[0054] in, This indicates an unfolding operation, which expands the shape to... Feature expansion as , . , These are the parameters for fully connected layer a and fully connected layer b, respectively. and These represent the number of neurons in fully connected layer a and fully connected layer b, respectively. Fully connected layer a and fully connected layer b sequentially... Dimension mapping to , Finally, we got .

[0055] Regional features of the 10th image It is input into the fully connected layer 2 to adjust the coordinates of the predicted bounding box, as shown in Equation (3).

[0056] (3)

[0057] in, For the parameters of the fully connected layer 2, 20 represents the 4 position offsets corresponding to the 5 categories (4 categories and 1 background class), therefore .

[0058] at the same time to These features are first stacked in the time series fusion module and then generated into time series features through a gated loop unit. , as in formula (4).

[0059] (4)

[0060] in, Indicates a stacking operation. . Then, through a gated recurrent unit, the temporal features between features are captured, and the feature dimension of its hidden state is set to... ,but Obtaining temporal features Then, with Addition to obtain fusion features . The original category scores are generated by feeding them into the fully connected layer 1, as shown in formula (5).

[0061] (5)

[0062] in, For the parameters of fully connected layer 1, 5 represents 5 categories (4 categories and one background class), therefore The complete output of the network is then... , The predicted bounding box and the original class score have a one-to-one correspondence.

[0063] The post-processing operation involves a series of steps, such as removing background predicted boxes, removing low-confidence predicted boxes, and performing non-maximum suppression to eliminate redundant boxes for the same target, thereby obtaining clear and accurate detection results.

[0064] 3. Network training

[0065] The temporal fusion object detection network proposed in this invention is implemented using the widely used open-source deep learning framework PyTorch. After the network is built, training begins. First, a set of images is input into the network to generate predicted bounding boxes, categories, and confidence scores. Next, the coordinate loss between the predicted and ground truth bounding boxes, and the classification loss between the predicted and ground truth categories are calculated. Then, the gradient information of the loss function is propagated to each layer of the network through backpropagation, and the network parameters are optimized and updated using gradient descent to gradually reduce the loss value. This training process is repeated until the loss value converges.

[0066] The proposed object detection network, which integrates temporal information, is built using the open-source deep learning framework PyTorch. The dataset is divided into training and testing sets in an 8:2 ratio. First, after obtaining the training set, random horizontal flipping is performed for data augmentation. During this process, the images in each group are flipped simultaneously, and the coordinates of the candidate bounding boxes in the corresponding annotation files are also flipped. The augmented datasets are then fed into the network in batches to obtain the predicted bounding box coordinates, corresponding categories, and confidence scores. The loss is calculated, backpropagation training is performed, and then the parameters are adjusted using gradient descent to reduce the loss value until convergence.

[0067] Specifically, the loss for class prediction is calculated using binary cross-entropy, and the loss for bounding box coordinate prediction is calculated using smoothed L1. During network training, a stochastic gradient descent optimizer is used to adjust the parameters, with a learning rate of 0.005, momentum of 0.9, and weight decay of 1 × 10⁻⁴. The batch size is set to 8. The model is trained end-to-end for 100 training epochs.

[0068] 4. Network Testing

[0069] After training and optimizing the network, testing is necessary. The purpose of testing is to evaluate the model's performance and verify its generalization ability on unseen data. First, a sequence of test images not used in training is prepared and input into the trained network. The network processes the input images, generating corresponding predicted bounding boxes, class labels, and confidence scores. During testing, the network parameters are not updated, and no loss value is calculated. Instead, the predictions are compared with the true labels, and metrics such as mean precision, accuracy, and recall are calculated to evaluate model performance.

[0070] This invention uses mAP50 as a performance evaluation metric, which is a commonly used evaluation metric in object detection tasks. mAP50 refers to the average precision (AP) of the model calculated and averaged across all classes when the intersection-union ratio (IU) between the predicted and ground truth bounding boxes is 50%. Average precision (AP) primarily evaluates the model's precision and recall performance at different thresholds.

[0071] The intersection-union ratio (IUU) refers to the degree of overlap between the predicted bounding box and the actual bounding box, as shown in formula (6). (6)

[0072] in, Indicates intersection, union, and ratio. and These represent the predicted bounding box and the ground truth bounding box, respectively. When the intersection-union ratio (IU) exceeds the set threshold (50% in the mAP50 metric), the prediction is considered correct and is denoted as TP (True Positive). When the IU is less than the threshold, it is denoted as FP (False Positive). For ground truth bounding boxes that are not detected, they are denoted as FN (False Negative). Precision and recall are further calculated using TP, FP, and FN, as shown in formulas (7) and (8).

[0073] (7)

[0074] (8)

[0075] Where P is precision, also called accuracy, which is the proportion of correctly predicted bounding boxes out of all predicted bounding boxes. R is recall, also called recall, which is the proportion of correctly predicted bounding boxes out of the actual bounding boxes.

[0076] Subsequently, the P and R values ​​for pairs under different confidence thresholds are calculated. These P and R value pairs are sorted from high to low confidence and the AP is calculated using formula (9).

[0077] (9)

[0078] Where n is the index of the P, R value pair, in particular, . Then it is calculated using formula (10).

[0079] (10)

[0080] in, for The corresponding P-value.

[0081] After calculating AP for each category, the average value is used to calculate the mAP index of the model, as shown in formula (11).

[0082] (11)

[0083] in, The total number of categories is 4 in this invention, so... .

[0084] At the end of each training epoch, the network weights are saved, and these weights are then used to perform performance testing on the test set. For the test set, the mean precision (AP) for each class is first calculated, and then the average of these average precisions is taken as the mAP50, which is used as the performance metric for the current epoch. The network weight with the highest mAP50 over 100 training epochs is saved as the final network parameters.

[0085] The above description, in conjunction with relevant preferred embodiments, provides a detailed explanation of the implementation of the present invention. It should not be construed that the specific embodiments of the present invention are limited to these descriptions. The scope of protection of the present invention extends to technical means with the same performance and uses as the present invention, based on the concepts of the present invention, as can be learned by those skilled in the art.

[0086] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for classifying fundus vascular drug delivery regions based on temporal fusion networks, characterized in that, Specifically, the following steps are included: Step 1: After obtaining the image sequence and corresponding labels, use an image annotation tool to convert the image and label information into a format suitable for neural network learning. Then, draw the bounding box of the medication area on the last image and label the corresponding category. Step 2: Combine the temporal module to extract temporal information from a set of FFA images. This assists the target detection network in more accurately classifying the drug application areas based on the temporal information. The temporal module can classify multiple drug application areas in the FFA images. The process involves collecting an FFA image set, including early-stage and late-stage FFA images. This set is input into a backbone network to extract image features for each image. The image features corresponding to the last image in the FFA image set are selected and input into a region proposal network to generate target candidate boxes. A region pooling module maps these candidate boxes to the corresponding image features, and after dimensionality reduction, the region features of each FFA image are obtained. The region features of the last image are input into a fully connected layer to generate the coordinates of the predicted bounding boxes. Simultaneously, the region features of the last image are combined with those of the remaining images in the FFA image set and input into a temporal fusion module to extract the temporal features of the FFA image set. These temporal features are then fused with the region features of the last image to obtain fused temporal features. These fused temporal features are input into a fully connected layer to generate target category scores. Finally, the predicted bounding boxes are post-processed to remove redundant and overlapping bounding boxes, yielding the final detection result. Step 3: After the network is built, training begins. A set of images is input into the built network to generate predicted bounding boxes, categories, and confidence scores. Then, the coordinate loss between the predicted bounding boxes and the ground truth bounding boxes, as well as the classification loss between the predicted categories and confidence scores and the ground truth categories are calculated. Step 4: The network processes the input image and generates the corresponding predicted bounding boxes, class labels, and confidence scores. During testing, the predicted results are compared with the true labels, and the average precision, accuracy, and recall metrics are calculated to evaluate the model performance.

2. The method for classifying fundus vascular drug delivery regions based on temporal fusion networks according to claim 1, characterized in that: In step one, in order to reduce the amount of input, a portion of images from the early, middle and late stages need to be selected to form an image sequence. The division between the early, middle and late stages is defined by professionals.

3. The method for classifying fundus vascular drug delivery regions based on temporal fusion networks according to claim 1, characterized in that: In step two, when using time-series information for more accurate classification, dynamic changes are an important basis for classifying drug use regions.

4. The method for classifying fundus vascular drug delivery regions based on temporal fusion networks according to claim 1, characterized in that: When the timing module is combined in step two, dynamic changes in a set of continuous images can be captured.

5. The method for classifying fundus vascular drug delivery regions based on temporal fusion networks according to claim 1, characterized in that: In step three, after calculating the coordinate loss between the predicted bounding box and the true bounding box, as well as the classification loss between the predicted class and confidence and the true class, the gradient information of the loss function is propagated to each layer of the network through the backpropagation algorithm. Combined with the gradient descent method, the network parameters are optimized and updated to gradually reduce the loss value.

6. The method for classifying fundus vascular drug delivery regions based on a temporal fusion network according to claim 5, characterized in that: The process of propagating the gradient information of the loss function to each layer of the network through backpropagation and optimizing and updating the network parameters in combination with gradient descent to gradually reduce the loss value needs to be repeated until the loss value converges.

7. The method for classifying fundus vascular drug delivery regions based on temporal fusion networks according to claim 1, characterized in that: In step four, when the network processes the input image, a set of test image sequences that were not used in training are prepared and input into the trained network.

8. The method for classifying fundus vascular drug delivery regions based on temporal fusion networks according to claim 1, characterized in that: In step four, during the test, the network parameters will not be updated, and the loss value will not be calculated.

Images