A fundus atrophy arc region change prediction method based on a recurrent neural network

Abstract

The application discloses a kind of based on recurrent neural network's fundus atrophy arc area change prediction method, belong to image processing technical field.This method uses the feature that feature extractor outputs as input, network training is joined for the similarity constraint of input feature and output feature, solve the problem of missing input data when predicting time sequence, reduce the influence brought by data;By adding attention mechanism in the prediction network, make full use of historical input features, maintain input time causality, further improve the prediction performance of network;Before carrying out prediction network training, registration operation is carried out, the influence of image acquisition angle and resolution at different times is excluded, the area of atrophy arc at different times can be compared, the accuracy of fundus atrophy arc area change prediction is improved;The application is suitable for ophthalmic medical field etc., provides accurate and targeted myopia prediction and prevention and control through atrophy arc growth trend.

Description

TECHNICAL FIELD

[0001] The present application relates to a kind of based on recurrent neural network's fundus atrophy arc area change prediction method, belong to image processing technical field. BACKGROUND

[0002] Optic disc atrophy arc is a kind of reaction retinal layer and retinal pigment epithelium atrophy clinical findings, it is highly relevant with myopia of children and adolescents, glaucoma retinal detachment and dense cataract, so optic disc atrophy arc can be used as the biomarker of the above eye diseases.According to the research report of World Health Organization, the number of myopia patients in China is up to 600 million, and the myopia rate of adolescents is the highest in the world, and the proportion of high myopia in children and adolescents is on the rise.Currently, studies have shown that the area of optic disc atrophy arc is positively correlated with the degree of myopia, so if the future change of optic disc atrophy arc can be predicted, intervention can be implemented to prevent the rapid development of myopia.

[0003] The existing research on optic disc atrophy arc prediction is mainly to predict whether optic disc atrophy arc will appear in the future.The current research mainly performs classification operation and cannot be refined to pixel-level prediction.The existing prediction method mainly uses the features of the previous moment to predict the result of the current moment, and the features of the historical moment cannot be fully utilized.If the future optic disc atrophy arc can be predicted by fully utilizing the historical fundus images, and compared with the existing fundus image atrophy arc segmentation result, early detection, early intervention and early treatment can be achieved according to the change of atrophy arc area and width, which is of great significance for myopia prevention and control. SUMMARY

[0004] In view of the problems that the existing optic disc atrophy arc prediction method only performs classification and cannot be refined to pixel-level prediction, and the historical data features cannot be fully utilized, the main purpose of the present application is to propose a kind of based on recurrent neural network's fundus atrophy arc area change prediction method, adopt convolution long short-term memory recurrent neural network, and add attention mechanism in the network, to improve the prediction accuracy of fundus atrophy arc area change, so as to assist myopia prediction and prevention and control.

[0005] The purpose of the present application is realized by the following technical scheme:

[0006] The application discloses an eye fundus atrophy arc area change prediction method based on a recurrent neural network.

[0007] The application discloses an eye fundus atrophy arc area change prediction method based on a recurrent neural network, and comprises the following steps.

[0008] Step 1: registration operation is performed on historical images, a data set is constructed, and a region of interest is extracted.

[0009] Step 1.1, for the eye fundus images of the same patient and the same eye taken in different years, registration operation is performed with the initial examination image as a reference, and the registered images are output, so that the positions of the optic discs and the distributions of blood vessels of the eye fundus images of the same patient and the same eye in different years correspond to each other pixel by pixel.

[0010] Step 1.2, a training set, a verification set and a test set are constructed, wherein the data set is a plurality of groups of single-eye retinal fundus image sequences of patients collected in different years. For the training set and the verification set, in addition to the clinical images, the annotations of the optic discs and the atrophy arcs made by doctors are also included, and the test set data does not need to be annotated by doctors.

[0011] Step 1.3, the region of interest centered on the optic disc is intercepted, and the specific steps are as follows:

[0012] Step 1.3.1, the training set and the verification set constructed in step 1.2 are used to train and verify the optic disc and atrophy arc area segmentation model in units of pictures, wherein only the training set is used during training, and the model with the minimum loss on the verification set is selected as the final segmentation model according to the results on the verification set.

[0013] Step 1.3.2, the model obtained in step 1.3.1 is used to segment the optic disc and atrophy arc area of all images.

[0014] Step 1.3.3, the segmented optic disc results are taken and collected for a single eye of a patient, the minimum circumscribed circle is calculated, and the center and the radius are obtained.

[0015] Step 1.3.4, the center of the circle and the radius obtained according to step 1.3.3 is intercepted to obtain a region of interest, wherein the center of the region of interest is the center of the circle, and the length of the side of the rectangle is P times the radius, and if the intercepted region exceeds the image size, a zero padding operation is performed.

[0016] Step 1.4, the size of the intercepted image is transformed to a specified size, and the size of the scaled picture is H1*W1. Since the image size needs to be the same during subsequent network training, the image is scaled.

[0017] Step 1.5, the picture obtained in step 1.4 is grouped according to the data set of step 1.2 to obtain a data set for subsequent use.

[0018] Step 2: using the training set and the validation set obtained in step 1, the training and validation of the optic disc and atrophy arc region segmentation model are carried out in units of pictures, and according to the results on the validation set, the model with the lowest loss on the validation set is selected as the final segmentation model.

[0019] Step 3: according to the segmentation model obtained in step 2, an atrophy arc region prediction model is constructed, and the data set of step 1 is used for training;

[0020] Step 3.1, the segmentation model obtained in step 2 is removed from the last convolution layer as a feature extractor, and the feature extractor input is an image.

[0021] Step 3.2, a basic module in the time series prediction network is constructed;

[0022] The basic module structure is a convolutional long short-term memory recurrent unit, as shown in (1):

[0023]

[0024] t represents the time; X is the input of the basic module; H is the hidden state; C is the memory unit state; tan() and σ() represent different activation functions; wherein σ is a sigmoid function; * represents convolution operation; ° represents Hadamard product; W is a model parameter, which is obtained by network learning.

[0025] Step 3.3, an attention module is constructed to fully utilize the information of historical time;

[0026] For t = 1, no processing is performed. For t ≠ 1, time t is regarded as a query time, and the previous t-1 times are historical times. First, the features of the query are respectively passed through two convolutional layers, thereby obtaining two features, which are referred to as and The historical time features are respectively passed through two convolutional layers, and the outputs of the two convolutional layers are respectively concatenated along the time dimension to obtain a feature referred to as and Then dimension transformation is performed to transform K Q into K M into V M into By comparing each spatio-temporal position in K M with each spatial position in K Q for similarity matching. Then a softmax weighted sum is used to retrieve V M , and the resulting feature F out is dimension transformed back into and concatenated with V Q to obtain the final output feature.

[0027] Step 3.4, module combination of the time series prediction network is constructed;

[0028] The module combination is composed of Q basic modules, Q / 2-1 down-sampling layers, Q / 2-1 up-sampling layers, and one attention module. The first Q / 2 modules are connected through the down-sampling layers, the last Q / 2 modules are connected through the up-sampling layers, and the output of the Q / 2th module is input into the Q / 2+1th module after being processed by the attention module. For the last Q / 2 modules, the hidden state and the memory cell state are the sum of the state at the previous time and the hidden state and the memory cell state output by the corresponding module at the current time. Wherein Q is an even number.

[0029] Step 3.5, a spatial time series prediction model is constructed;

[0030] The spatial time series prediction model is composed of T module combinations, where T represents the number of historical time in the sequence group, and the parameters in the T module combinations are shared. For time 1, the hidden state and the memory cell state of the previous time are set to zero. For other time, the output of the hidden state and the memory cell state of the previous time is the input of the current time. For the first layer basic module, the input is the feature output by step 3.1. For other layer modules, the input is the feature map output by the previous layer after down-sampling or up-sampling or attention module.

[0031] Step 3.6, a segmentation port is constructed;

[0032] The segmentation port is composed of a convolution layer and a softmax activation layer. The input of the segmentation port includes the output feature of step 3.1 and the hidden state output by the last layer basic module in step 3.5, and the output is the segmentation result of the optic disc and the atrophy arc.

[0033] Step 3.7, the atrophy arc region prediction model is trained;

[0034] The network uses the data set obtained in step 1 for training, and the loss function used is composed of cross-entropy loss, foreground Dice loss and mean square error loss.

[0035] The cross-entropy loss is represented as As shown in (2):

[0036]

[0037] At this time The predicted results of the disc and the atrophy arc output by the segmentation port are y, the corresponding doctor's true value is y, the number of images is N, and the number of classes (including background) is Cls.

[0038] The Dice loss is represented as As shown in (3):

[0039]

[0040] At this time y, N and Cls are defined as before, and ∈ is a number approaching 0. When calculating the Dice loss, the background is not considered, and only the disc and the atrophy arc are calculated.

[0041] The mean square error is represented as As shown in (4):

[0042]

[0043] The features output by the feature extractor are The hidden state features output by the last layer of the spatial-temporal prediction model are

[0044] The final constructed loss function is shown in (5):

[0045]

[0046] The model with the minimum loss on the validation set is selected as the final prediction model.

[0047] Step 4: Apply the model obtained in step 3 to predict actual clinical data, improve the prediction accuracy of the future changes of the atrophy arc area of the fundus, and optimize the myopia prevention and treatment scheme;

[0048] Step 4.1, register and extract the region of interest operation on the clinical data. The clinical data may have missing data at a certain time of the input data.

[0049] Step 4.2, input the processed data into the prediction model obtained in step 3, output the future prediction result and the current time disc and atrophy arc segmentation result, and calculate the atrophy arc area at each time. For the case where there is missing input data, the output feature of the last time is used as the input of the missing time.

[0050] Step 4.3, draw a broken line graph in time sequence according to the predicted atrophy arc area at future time and the atrophy arc segmentation area change at historical time, and assist myopia prediction and prevention in a targeted manner.

[0051] Beneficial effects:

[0052] A fundus atrophy arc area change prediction method based on a recurrent neural network, compared with an existing fundus atrophy arc prediction network, has the following beneficial effects:

[0053] 1. The fundus atrophy arc area change prediction method based on the recurrent neural network disclosed in the present application uses the features output by the feature extractor as input, adds a similarity constraint for the input features and output features during network training, can solve the problem of missing input data in time sequence prediction, and reduces the influence of data.

[0054] 2. The fundus atrophy arc area change prediction method based on the recurrent neural network disclosed in the present application adds an attention mechanism in the prediction network, fully utilizes historical input features, maintains input time causality, and further improves network prediction performance.

[0055] 3. The fundus atrophy arc area change prediction method based on the recurrent neural network disclosed in the present application performs a registration operation before training the prediction network, eliminates the influence of different time image acquisition angles and resolutions, can compare atrophy arc areas at different times, and improves the accuracy of atrophy arc area change prediction.

[0056] 4. The fundus atrophy arc area change prediction method based on the recurrent neural network disclosed in the present application uses an atrophy arc area prediction model, which can not only predict the optic disc and atrophy arc area at future time, but also segment the optic disc and atrophy arc area at historical times, and draw a broken line graph of the atrophy arc area prediction results at historical times and future times, which is more conducive for doctors to predict myopia and prevention in a targeted manner through the atrophy arc growth trend. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 is a flowchart of the fundus atrophy arc area change prediction method based on the recurrent neural network disclosed in the present application;

[0058] Figure 2is a structure schematic diagram of the optic disc and atrophy arc segmentation model in the embodiment of the eye fundus atrophy arc area change prediction method based on a recurrent neural network in the embodiment;

[0059] Figure 3 is a structure schematic diagram of the future prediction model of the optic disc and atrophy arc area in the embodiment of the eye fundus atrophy arc area change prediction method based on a recurrent neural network in the embodiment;

[0060] Figure 4 is a structure schematic diagram of the attention module in the future prediction model of the optic disc and atrophy arc area in the embodiment of the eye fundus atrophy arc area change prediction method based on a recurrent neural network in the embodiment;

[0061] Figure 5 is a patient atrophy arc area change broken line chart in the embodiment of the eye fundus atrophy arc area change prediction method based on a recurrent neural network in the embodiment. DETAILED DESCRIPTION

[0062] The present application will be described in detail below with reference to the accompanying drawings and embodiments. At the same time, the technical problems solved by the technical scheme of the present application and the beneficial effects are described. It should be pointed out that the described embodiments are only intended to facilitate the understanding of the present application and do not have any limiting effect on it.

[0063] The present embodiment describes an eye fundus atrophy arc area change prediction method based on a recurrent neural network. The first grade to fifth grade eye fundus images are used to predict the sixth year of the optic disc and atrophy arc area, and the change curve is used to assist the doctor to optimize the myopia prevention and control and intervention treatment scheme.

[0064] The eye fundus atrophy arc area change prediction method based on a recurrent neural network disclosed in the embodiment, as shown in FIG. (1), specifically includes the following steps

[0065] Step 1: Perform registration operation on historical images, construct data set and extract region of interest;

[0066] Step 1.1. Registration is performed on the patient sequence data based on the initial examination fundus image, and the registered image is output. A fundus image registration method based on Harris-PIIFD feature points is used. First, the Harris detector is used to detect the feature corner points of the image, and the background points at the boundary between the retina and the black background are removed by retracting towards the center. Then the average square gradient method is used to assign the main direction to the feature corner points. The PIIFD feature vector in the neighborhood of the feature corner point is extracted with the feature corner point as the center. Next, the BBF algorithm is used to match the PIIFD feature vectors extracted from different feature corner points in the sequence data set. The feature corner points in the non-initial examination image are used as the reference points, and the feature corner points corresponding to the initial examination image are used as the matching points. The false matching points are removed by the main direction and geometric distribution of the feature corner points. The matching point position is fine-tuned, and all feature corner points within a 5*5 pixel range around the matching point are searched. The PIIFD feature vectors of the reference points and the matching points are compared, and the highest point of the matching point is selected to replace the original matching point. Finally, the non-initial examination data is geometrically transformed according to the number of matching points. At this point, the optic disc and blood vessels in the registered image and the reference image are pixel by pixel corresponding in position.

[0067] Step 1.2. Construct training set, validation set and test set. The 327 groups of single eye fundus images of all clinical examinations from first grade to sixth grade of the collected patients are the original data set, of which 250 groups of data are used as the training set, 27 groups of data are used as the validation set, and 50 groups of data are used as the test set. The training set and the validation set include the labels of the optic disc and the atrophy arc labeled by the doctors for the collected images.

[0068] Step 1.3. The region of interest centered on the optic disc is intercepted, and the specific steps are as follows:

[0069] Step 1.3.1. The training set (1500 images) and the validation set (162 images) constructed in step 1.2 are used to train and validate the optic disc and atrophy arc region segmentation model in units of pictures. In this example, the U-Net network is used for segmentation task, and the U-Net structure is as shown in Figure 2 Since the input of the segmentation network needs to be fixed in size, the input image is first resized to 512*512*3, and the aspect ratio of the image is kept unchanged during the resizing. For the blank area, a zero-filling operation is performed. The network performs 4 times of downsampling operation and 4 times of upsampling operation. The convolution kernel size in the network is 3*3 except for the last convolution operation where the convolution kernel size is 1*1. The input of the network is the registered image, and the output is the segmented optic disc and atrophy arc region. During training, only the training set is used, and the model with the smallest loss on the validation set is selected as the final segmentation model according to the results on the validation set.

[0070] Step 1.3.2. Use the model obtained in step 1.3.1 to perform optic disc and optic cup segmentation for all images, a total of 1962 images.

[0071] Step 1.3.3. Take the union of the segmented optic disc results for each patient's single eye, calculate the minimum circumscribed circle, and obtain the center and radius. Since the input of the segmentation network is scaled, the optic cup area is small in the original image, and information will be lost after scaling, resulting in poor optic cup segmentation results, but good optic disc segmentation results. Therefore, only the optic disc segmentation results are used here.

[0072] Step 1.3.4. According to the center and radius obtained in step 1.3.3, the region of interest is extracted, which is a square with the center as the center and the side length as 5 times the radius. If the extracted region exceeds the image size, zero padding is performed.

[0073] Step 1.4. Transform the size of the extracted image to the specified size, and the size of the scaled picture is 512*512. Since the input image size needs to be the same during subsequent network training, we perform scaling operation on the image.

[0074] Step 1.5: Group the images obtained in step 1.4 according to the data set in step 1.2 to obtain the data set for subsequent use.

[0075] Step 2: Use the training set and validation set in step 1 to train and validate the optic disc and optic cup segmentation model. Here, the segmentation model is the same as step 1.3.1, which is a U-Net structure, as shown in Figure 2 According to the results on the validation set, select the model with the smallest loss on the validation set as the final segmentation model.

[0076] Step 3: According to the segmentation model obtained in step 2, construct and train the optic cup area prediction model. This model uses the first five years of fundus images to predict the sixth year's optic disc and optic cup area, and performs optic disc and optic cup segmentation on the first five years of fundus images. The specific steps are as follows:

[0077] Step 3.1: Remove the last convolution layer of the segmentation model obtained in step 2 as a feature extractor, and fix the parameters in subsequent experiments. When training, the first six years of fundus images are all passed through this feature extractor, and when testing, only the first five years of fundus images are input.

[0078] Step 3.2: Construct a time series prediction network; as Figure 3The structure shown in the middle dashed line box, the features extracted by the feature extractor (features extracted from first grade to fifth grade) are input in time sequence, and the features at each time are input through 6 RNN modules, during which 2 times of down-sampling, 1 time of attention operation and 2 times of up-sampling operation are performed. Among them, the first 3 modules are connected through the down-sampling layer, the last 3 modules are connected through the up-sampling layer, and the output of the third module is input into the fourth module after being processed by the attention module. For the last three modules, the hidden state and the memory cell state are the state of the previous time and the hidden state and the memory cell state output by the corresponding module at the current time, which are added to form the corresponding module connection as shown in Figure 3 The preferred RNN module in the example is a convolutional long short-term memory recurrent unit (ConvLSTM), and the specific formula of ConvLSTM is shown as (1):

[0079]

[0080] t represents the time; X is the input of the basic module; H is the hidden state; C is the memory cell state; tan() and σ() represent different activation functions; wherein σ is a sigmoid function; * represents convolution operation; ° represents Hadamard product; W is a model parameter, which is obtained by network learning.

[0081] The attention module needs to record the features of each time input by the current sequence group, and the purpose of the attention module is to make full use of the information of the historical time when processing the current time. The specific operation is shown in Figure 4 For t = 1, no processing is performed, and for t ≠ 1, the time t is regarded as the query time, and the previous t-1 times are historical times. First, the query feature is respectively passed through two convolutional layers to obtain two features, which are called and The historical time features are respectively passed through two convolutional layers, and the outputs of the two convolutional layers are respectively concatenated along the time dimension to obtain the features, which are called and In order to reduce the calculation amount and improve the calculation speed, K M , V M and K Q are all down-sampled by 2, and then dimension transformation is performed. After the transformation, K M is compared with each spatial position of K Q to perform similarity matching, and then softmax weighted summation is used to retrieve V M , and the obtained feature F out is transformed back to The F is obtained by 2 times up-sampling operation out The V Q The final output feature is obtained by concatenation.

[0082] Step 3.3: Constructing segmentation port

[0083] The segmentation port is composed of a convolutional layer and a softmax activation layer. The input of the segmentation port is the output feature of step 2.1 (the feature after encoding of grades one to six) and the hidden state feature output by the last RNN module in step 2.2. The output is the prediction result of the optic disc and the atrophy arc.

[0084] Step 3.4: Training the atrophy arc region prediction model

[0085] The loss function used by the network is composed of cross-entropy loss, Dice loss of foreground and mean square error loss.

[0086] The cross-entropy loss is represented as is represented as (2):

[0087]

[0088] At this time is the prediction result of the optic disc and the atrophy arc output by the segmentation port, y is its corresponding doctor's labeled true value, N is the number of images, and Cls is the number of classes (including background).

[0089] The Dice loss is represented as is represented as (3):

[0090]

[0091] At this time y and The definitions are the same as before, and ∈ is a number approaching 0. When calculating the Dice loss, the background is not considered, only the optic disc and the atrophy arc are calculated.

[0092] The mean square error is represented as is represented as (4):

[0093]

[0094] is the feature output by the feature extractor, and is the hidden state feature output by the last layer of the spatial-temporal prediction model.

[0095] The final constructed loss function is represented as (5):

[0096]

[0097] The model with the minimum prediction loss on the validation set is taken as the final adopted prediction model.

[0098] Step 4: Apply the model obtained in step 3 to predict the actual clinical data, improve the prediction accuracy of the atrophic arc area change of the fundus, and assist in myopia prediction and prevention accordingly. The specific operation is as follows:

[0099] Step 4.1: Register and extract the region of interest of the clinical data.

[0100] Step 4.2: Input the processed data into the prediction model obtained in step 3, output the future prediction result and the current time disc and atrophic arc segmentation result, and calculate the atrophic arc area at each time.

[0101] Step 4.3: Draw a line graph of the predicted atrophic arc area at the future time and the atrophic arc segmentation area change at the historical time, as shown in Figure 5 , assist in myopia prediction and prevention accordingly according to the change curve.

[0102] The above specific description further details the purpose, technical solution and beneficial effects of the application. It should be understood that the above description is only a specific embodiment of the application and is not used to limit the protection scope of the application. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the application should be included in the protection scope of the application.

Claims

1. A method for predicting changes in fundus atrophy arc regions based on recurrent neural networks, characterized in that: Includes the following steps, Step 1: Perform registration operations on historical images, construct a dataset, and extract regions of interest; Step 2: Using the training set and validation set obtained in Step 1, train and validate the spectral disc and atrophic arc region segmentation model on an image-by-image basis. Based on the results on the validation set, select the model with the lowest loss on the validation set as the final segmentation model. Step 3: Based on the segmentation model obtained in Step 2, construct a shrinking arc region prediction model and train it using the dataset from Step 1; The implementation method for step 3 is as follows: Step 3.1: Remove the last convolutional layer from the segmentation model obtained in Step 2 and use it as a feature extractor. The input to the feature extractor is the image. Step 3.2: Construct the basic modules in the time series prediction network; The basic module structure is a convolutional long short-term memory recursive unit, as shown in (1): t represents time step; X is the input of the basic module; H is the hidden state; C is the memory unit state; tanh() and σ() represent different activation functions; where σ is the sigmoid function; * represents the convolution operation; denoted as Hadamard product; W represents the model parameters, obtained through network learning; Step 3.3: Construct an attention module to make full use of information from historical moments; No processing is performed when t=1; for times t≠1, time t is considered the query time, and the first t-1 times are all historical times; first, the query characteristics are... The data passes through two convolutional layers to obtain two features, which are called... and The historical time features are then passed through two convolutional layers, and the outputs of the two convolutional layers are concatenated along the time dimension to obtain the resulting features, which are called... and Then perform a dimensional transformation, transforming K... Q Transform into K M Transform into V M Transform into By K M Each spatiotemporal location in K Q Each spatial location is compared for similarity matching; then a softmax weighted sum is used to retrieve V. M The obtained feature F out Dimensional transformation back Again with V Q The final output features are obtained by cascading. Step 3.4: Construct the module combination of the time series prediction network; The module combination consists of Q basic modules, Q / 2-1 downsampling layers, Q / 2-1 upsampling layers, and an attention module. The first Q / 2 modules are connected through downsampling layers, and the last Q / 2 modules are connected through upsampling layers. The output of the Q / 2th module is processed by the attention module and then input into the Q / 2+1th module. For the last Q / 2 modules, their hidden state and memory unit state are the sum of the state at the previous time step and the hidden state and memory unit state output by the corresponding preceding module at the current time step; where Q is an even number. Step 3.5: Construct a spatial-temporal prediction model; The spatial temporal prediction model consists of T modules, where T represents the number of historical moments in the sequence group. The parameters are shared among the T modules. For moment 1, the hidden state and memory unit state of the previous moment are both set to zero. For other moments, the output of the hidden state and memory unit state of the previous moment is the input of the current moment. For the first basic module, the input is the feature output in step 3.

1. For other modules, the input is the feature map of the previous layer output after downsampling, upsampling, or the output of the attention module. Step 3.6: Construct the split port; The segmentation port consists of a convolutional layer and a softmax activation layer; the input to the segmentation port is the output features of step 3.1 and the hidden state features output from the last layer in step 3.5, and the output is the prediction results of the visual disk and the shrinkage arc. Step 3.7: Train the prediction model for the shrinking arc region; The network is trained using the dataset obtained in step 1, and the loss function used consists of cross-entropy loss, foreground Dice loss, and mean squared error loss. Cross-entropy loss is expressed as As shown in (2): at this time The output of the segmentation port is the prediction result of the optic disc and atrophic arc, y is the corresponding doctor's labeled ground truth value, N is the number of images, and Cls is the number of categories; Dice loss is expressed as As shown in (3): At this time, y, The definitions of N and Cls are the same as before, where ∈ is a number approaching 0. When calculating the Dice loss, the background is not considered, and only the visual disk and the shrinkage arc are calculated. Mean square error is expressed as As shown in (4): Remove the time step t=1 from the features output by the feature extractor. N represents the hidden state features output from the last layer of the spatial temporal prediction model, where N is the number of output features. The final loss function is shown in (5): The model with the minimum loss on the validation set is selected as the final prediction model. Step 4: Apply the model obtained in Step 3 to predict actual clinical data, improve the accuracy of predicting changes in the fundus atrophy arc region, and provide targeted assistance for myopia prediction and prevention.

2. The method for predicting changes in the fundus atrophy arc region based on a recurrent neural network as described in claim 1, characterized in that: The implementation method for step 1 is as follows: Step 1.1: For fundus images of the same eye taken over the years for the same patient, use the initial examination image as a reference to perform a registration operation and output the registered image to ensure that the optic disc position and blood vessel distribution in the fundus images of the same eye of the same patient over the years correspond pixel by pixel. Step 1.2: Construct training, validation, and test sets. The dataset consists of retinal fundus image sequences of multiple groups of patients collected clinically over the years. The training and validation sets include not only clinical images but also annotations of the optic disc and atrophic arc performed by doctors. The test set data does not require annotation by doctors. Step 1.3: Select the region of interest centered on the viewport. The specific steps are as follows: Step 1.3.1: Using the training set and validation set constructed in Step 1.2, train and validate the spectral disc and atrophic arc region segmentation model on an image-by-image basis. During training, only the training set is used, and the model with the smallest loss on the validation set is selected as the final segmentation model based on the results on the validation set. Step 1.3.2: Use the model obtained in Step 1.3.1 to segment the visual disk and atrophy arc regions in all images; Step 1.3.3: Group the patient's single eye, take the union of the segmented optic disc results, calculate the minimum circumscribed circle, and obtain the center and radius of the circle; Step 1.3.4: Based on the center and radius obtained in Step 1.3.3, extract the region of interest, where the center of the region of interest is the center of the circle and the side length is a rectangle with a radius P times the radius. If the extracted region exceeds the image size, zero padding is performed. Step 1.4: Transform the captured image to the specified size. The size of the scaled image is H1*W1. Since the input image size needs to be the same during subsequent network training, the image is scaled. Step 1.5: Group the images obtained in Step 1.4 according to the dataset grouping in Step 1.2 to obtain the dataset used later.

3. The method for predicting changes in the fundus atrophy arc region based on a recurrent neural network as described in claim 1, characterized in that: The implementation method for step 4 is as follows: Step 4.1: Register and extract regions of interest from the clinical data; the clinical data may contain missing data at a certain point in time. Step 4.2: Input the processed data into the prediction model obtained in Step 3, output the future prediction results and the current time disc and shrinkage arc segmentation results, and calculate the shrinkage arc area at each time step; for cases where there is missing input data, the output features of the previous time step are used as the input for the missing time step. Step 4.3: Plot the changes in the area of ​​the atrophic arc at the predicted future time and the area of ​​the atrophic arc at the historical time in chronological order into a line graph to specifically assist in myopia prediction and prevention.

Images