A method for automatically partitioning retinopathy of prematurity based on multi-task learning

Abstract

The present application relates to the field of eye medical image detection, more specifically, a kind of automatic partitioning method of premature infant retinopathy based on multi-task learning is disclosed, comprising the following steps: disc and macular coordinates in premature infant retinopathy image are marked;The marked premature infant retinopathy image is preprocessed and divided into training set and verification set, network model is established, data enhanced data is put into network to obtain output result, the error between the output result of training and real label is calculated, after training, the performance of model is evaluated on verification set, the average Euclidean distance error of macular and disc positioning is calculated, based on premature infant retinopathy partition definition, automatic partitioning of premature infant retinopathy is realized, the present application uses a kind of automatic partitioning method of premature infant retinopathy based on multi-task learning to realize automatic partitioning of premature infant retinopathy, the correlation between two tasks is used to realize the mutual guidance between disc and macular center positioning task, it is helpful to optimize the positioning task of disc and macular center, and then the positioning accuracy and the precision of partition are improved, the problem of computer-aided diagnosis and treatment of premature infant retinopathy is solved.

Description

Technical Field

[0001] This invention relates to the field of ocular medical image detection, and more specifically, to an automatic zoning method for retinopathy of prematurity based on multi-task learning. Background Technology

[0002] Retinopathy of prematurity (ROP) is an eye disease that occurs in premature infants and is the leading cause of blindness in premature babies. Without timely diagnosis and intervention, ROP can progress to retinal detachment within weeks, ultimately leading to blindness. ROP zoning is an important tool for dividing fundus images of premature infants into regions, helping doctors accurately assess the degree and extent of the retinal disease. In the definition of ROP zoning, Zone I is a circular area centered on the optic disc, with a radius twice the distance from the optic disc to the center of the macula. Zone II is the area surrounding the edge of Zone I, with a radius twice the diameter of the optic disc (equivalent to an outer radius 1.3 times the radius of Zone I). Zone III is the outer region outside of Zone II.

[0003] Therefore, the key to retinopathy of prematurity (ROP) zoning lies in using computer technology to analyze and process ROP images to jointly locate the optic disc and macula, obtain the zoning center and radius of ROP, and output ROP zoning reference lines for doctors to assist in diagnosis. Currently, methods for ROP zoning mainly fall into the following categories: one uses optic disc segmentation to obtain the optic disc center and diameter, and uses the optic disc diameter to estimate the macula position, thus achieving ROP zoning. However, the optic disc and macula may have positional shifts and shape changes in fundus images, so inferring the macula position solely based on the optic disc diameter may lead to inaccurate localization. Another method segments the optic disc and blood vessels, classifying the three regions based on the area of ​​the extended blood vessels and the area of ​​each ROP region. However, this method does not achieve zoning based on an objective definition of ROP zoning.

[0004] In summary, current methods for zoning retinopathy of prematurity (ROP) typically lack direct detection of the macula and do not zon ROP based on objective definitions. Traditional fundus imaging methods can only locate the optic disc or macula individually, failing to perform simultaneous combined localization. This increases the cumbersomeness and complexity of the ROP zoning process. Summary of the Invention

[0005] To overcome the problem that traditional fundus image localization methods described in the prior art can only locate the optic disc or macula individually and cannot perform joint localization at the same time, which also increases the cumbersomeness and complexity of the retinopathy of prematurity (ROP) zoning process, this invention provides an automatic zoning method for ROP based on multi-task learning.

[0006] The present invention aims to solve the above-mentioned technical problems to at least some extent.

[0007] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows: An automatic regionalization method for retinopathy of prematurity based on multi-task learning includes the following steps: S1: Collect images of retinopathy of prematurity and select images of retinopathy of prematurity that simultaneously exhibit both optic disc and macular structures; S2: Mark the coordinates of the optic disc and the macula in the retinal lesion image, and use the marked image as a dataset of retinopathy of prematurity partitions; S3: Preprocess the dataset and calculate the actual probability density map of the images in the preprocessed dataset; S4: Construct a deep neural network model and use the deep neural network model to output a predicted probability density map of the images in the preprocessed dataset; S5: Calculate the loss value between the actual probability density map and the predicted probability density map; S6: Calculate the predicted coordinates of the optic disc and macula based on the predicted probability density map; S7: Determine the error between the predicted coordinates and the coordinates of the optic disc and macula. If the error is greater than the first preset value, use multi-task learning to optimize the deep neural network model, calculate the probability density map, and return to step S5. If the error is less than the first preset value, save the optimized deep neural network model as the final deep neural network model. S8: Using the final deep neural network model obtained in step S7, predict the coordinates of the optic disc and macula in the retinopathy of prematurity (ROP) image to be partitioned, and partition the ROP based on the predicted coordinates of the optic disc and macula.

[0008] Further, in step S3, the preprocessing of the dataset, which involves calculating the actual probability density map from the images of the preprocessed dataset, includes: S3.1: Standardize and normalize the images of retinopathy of prematurity. S3.2: Based on the coordinates of the optic disc and macula, a probability density map centered on the coordinates of the optic disc and macula is generated using a multivariate normal distribution function.

[0009] In the formula, A two-dimensional vector representing the viewport coordinates. The mean of the visual coordinate distribution is denoted as . It is the covariance matrix; A two-dimensional vector representing the macular coordinates. This represents the mean of the macular coordinate distribution; S3.3: Perform synchronous rotation, translation, scaling, and boundary filling operations on the retinopathy of prematurity image after uniform size and normalization in step S3.1 and the actual probability density map obtained in step S3.2.

[0010] Further, in step S4, the deep neural network model includes a dense dilated convolution module, an output layer, and a multi-kernel pooling module. The preprocessed dataset is passed through the dense dilated convolution module and the multi-kernel pooling module to obtain four feature maps and one feature map, respectively. The four feature maps and one feature map are concatenated to obtain an output feature map. The output layer consists of two 1×1 convolutional blocks and a sigmoid function. The input of the output layer is the output feature map, and the output layer outputs a predicted probability density map.

[0011] Furthermore, in step S6, the predicted coordinates of the optic disc and macula are calculated based on the predicted probability density map, including the following steps: A deep neural network model is used to obtain a predicted probability density map, and the gray value of each pixel in the predicted probability density map is calculated. The grayscale value of each pixel is weighted and added to the coordinates of that pixel in the predicted probability density map to obtain the weighted coordinates of all pixels. Sum the weighted coordinates of all pixels to obtain a total weighted coordinate. Divide the total weighted coordinates by the sum of the gray values ​​of all pixels to obtain the predicted coordinates of the optic disc and macula.

[0012] Further, in step S5, the loss values ​​of the actual probability density map and the predicted probability density map are calculated, including:

[0013] In the formula, The loss function is used to predict the probability density map and the actual probability density map. The probability values ​​are the actual probability density map obtained from the coordinates of the visual disk and the macula. These are the probability values ​​of the probability density map predicted by the deep neural network model. This represents the total number of image samples.

[0014] Further, step S7, using multi-task learning to optimize the deep neural network model, includes the following steps: In multi-task learning, when multiple tasks share the hidden layer parameters of a deep neural network model, the features of multiple tasks are learned and the results of each task are output. In multi-task learning, the parameters of the hidden layers of a deep neural network model are shared. Therefore, task t in the deep neural network model... d Layer output for,

[0015] Where H is the input to the hidden layer in the deep neural network model. It is the ReLU activation function. For the first deep neural network model d Convolutional kernels of layers; The hidden layer parameters of a deep neural network model are partitioned using a binary parameter partitioning matrix:

[0016] In the formula, It is the first in the deep neural network model d Layers and Tasks The relevant column vectors, from which a subset of parameters is selected for each task.

[0017] Two Adam optimizers are used to update the shared parameters, employing stochastic gradient descent for the update process.

[0018] In the formula For data samples, For learning rate, A task-specific exponential moving average mechanism. The introduced memos only involve updates prior to task t. For shared parameters.

[0019] Furthermore, the binary parameter partitioning matrix It follows a Bernoulli distribution with parameter p:

[0020] Where p is the proportion of parameters shared by each layer. Let p be a Bernoulli distribution with parameter p.

[0021] Further, in step S7, the error between the predicted coordinates and the coordinates of the optic disc and macula is determined. If the error is less than a first preset value, the optimized deep neural network model is saved as the final deep neural network model. This includes: feeding the images of the dataset into the deep neural network model one by one to obtain the corresponding probability density map, calculating the predicted coordinates, calculating the average Euclidean distance error between the actual coordinates and the predicted coordinates, taking half of the sum of the positioning errors of the optic disc and macula as the total error, and if the current total error is less than the first preset value, the optimized deep neural network model is saved.

[0022] Furthermore, the average Euclidean distance error is calculated. ,include:

[0023] In the formula, The total number of images in the dataset. and These are the predicted coordinates and actual coordinates of the i-th input image, respectively.

[0024] Furthermore, in step S8, retinopathy of prematurity is divided into regions based on the predicted coordinates of the optic disc and macula, including: Based on the predicted coordinates from the deep neural network model, the predicted coordinates are restored to the coordinates of the original image to obtain the coordinates of the optic disc and macula. The optic disc coordinates are used as the center point of the retinopathy of prematurity (ROP) region. According to the definition of ROP region, the distance relationship between the optic disc coordinates and the macula coordinates is used to draw the ROP region reference line.

[0025] Compared with the prior art, the beneficial effects of the technical solution of the present invention are: The present invention proposes an automatic zoning method for retinopathy of prematurity (ROP) based on multi-task learning, which has the ability to accurately zonify ROP regions in preterm infants. Based on multi-task learning, it can simultaneously process and learn the localization tasks of the optic disc and fovea, without using multiple separate network models to process each task one by one, thereby reducing the number of network parameters and computational cost. In addition, by utilizing the network parameter sharing of the multi-task learning mechanism, the correlation between the two tasks is used to achieve mutual guidance between the optic disc and fovea localization tasks, which helps to optimize the localization tasks of the optic disc and fovea, thereby improving the accuracy of localization and the precision of zoning. Attached Figure Description

[0026] Figure 1 The flowchart is a method for automatic zoning of retinopathy of prematurity based on multi-task learning, as described in this invention. Figure 2 This is a schematic diagram of the retinopathy of prematurity (ROP) zoning described in this embodiment; Figure 3This is the probability density map described in this embodiment; Figure 4 This is a diagram of the deep learning model framework described in this invention; Figure 5 This is a diagram of the DAC module described in this invention; Figure 6 This is a diagram of the RMP module described in this invention; Figure 7 This is a zoning effect diagram of retinopathy of prematurity in this embodiment; Figure 8 This is a schematic diagram of the IOU calculation for retinopathy of prematurity in this embodiment; Figure 9 This is a flowchart of the deep learning model training process described in this embodiment. Detailed Implementation

[0027] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent. To better illustrate this embodiment, some parts in the accompanying drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions; It will be understood by those skilled in the art that certain well-known structures and their descriptions may be omitted in the accompanying drawings.

[0028] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0029] Example 1 An automatic regionalization method for retinopathy of prematurity based on multi-task learning, such as Figure 1 As shown, it includes the following steps: S1: Collect images of retinopathy of prematurity and select images of retinopathy of prematurity that simultaneously exhibit both optic disc and macular structures; S2: Mark the coordinates of the optic disc and the macula in the retinal lesion image, and use the marked image as a dataset of retinopathy of prematurity partitions; S3: Preprocess the dataset and calculate the actual probability density map of the images in the preprocessed dataset; S4: Construct a deep neural network model and use the deep neural network model to output a predicted probability density map of the images in the preprocessed dataset; S5: Calculate the loss value between the actual probability density map and the predicted probability density map; S6: Calculate the predicted coordinates of the optic disc and macula based on the predicted probability density map; S7: Determine the error between the predicted coordinates and the coordinates of the optic disc and macula. If the error is greater than the first preset value, use multi-task learning to optimize the deep neural network model, calculate the probability density map, and return to step S5. If the error is less than the first preset value, save the optimized deep neural network model as the final deep neural network model. S8: Using the final deep neural network model obtained in step S7, predict the coordinates of the optic disc and macula in the retinopathy of prematurity (ROP) image to be partitioned, and partition the ROP based on the predicted coordinates of the optic disc and macula.

[0030] In the specific implementation process, the first step is to collect images of retinopathy of prematurity (ROP) in premature infants, filter out unqualified images, and select qualified images that simultaneously exhibit both optic disc and macular structures. Corresponding optic disc center location annotations and macular center location annotations are then provided as the dataset for ROP partitioning. This dataset is then divided into a training set, a validation set, and a test set.

[0031] Step 2: Perform data preprocessing on the images and coordinates of retinopathy of prematurity.

[0032] Step 3: Construct a deep neural network model to automatically extract features of the optic disc and macula, and use data-enhanced images of retinopathy of prematurity to train a localization model of the optic disc and macula.

[0033] Step 4: Find the predictions of the visual disc and macula from the generated probability density map, obtain the coordinates, and save the model of the validation set using the mean Euclidean distance error as the evaluation index.

[0034] Step 5: Based on the definition of retinopathy of prematurity (ROP) zones, output zone reference lines for Zones I, II, and III of ROP, and use circular IOUs as evaluation indicators.

[0035] Example 2 This embodiment, based on Embodiment 1, continues to disclose the following content: Collect images of retinopathy of prematurity, such as Figure 2 As shown, images of retinopathy of prematurity (ROP) that do not meet the requirements are filtered out. Then, the optic disc and macula in the qualified ROP images are pixel-level labeled to obtain the corresponding coordinates. The labeled data is used as the ROP partition dataset, which is divided into training set and validation set. Images that do not meet the requirements include: images where the ROP lesions cannot be identified due to excessively strong or dim lighting; images where multiple doctors cannot identify the ROP lesions; and images that do not contain both the optic disc and macula.

[0036] In the specific implementation process, step (1) first collects at least 2,000 images of retinopathy of prematurity as a dataset of retinopathy of prematurity partitions; Step (2): Screen out unqualified images, and screen out images that do not contain both the optic disc and the macula at the same time, and for which the doctor cannot identify the specific location of the macula; Step (3) Create zonal labels for retinopathy of prematurity (ROP) and mark the positions of the optic disc and macula on the ROP image, such as... Figure 3 As shown, the labelme tool was used to annotate the images of retinopathy of prematurity, resulting in 525 .png images of retinopathy of prematurity with coordinate labels.

[0037] Example 3 Based on Examples 1 and 2, this embodiment continues to disclose the following content: In step S3, the preprocessing of the dataset includes: S3.1: Standardize and normalize the images of retinopathy of prematurity. S3.2: Based on the coordinates of the optic disc and macula, a probability density map centered on the coordinates of the optic disc and macula is generated using a multivariate normal distribution function.

[0038] In the formula, A two-dimensional vector representing the viewport coordinates. The mean of the visual coordinate distribution is denoted as . It is the covariance matrix; A two-dimensional vector representing the macular coordinates. This represents the mean of the macular coordinate distribution; S3.3: Perform synchronous rotation, translation, scaling, and boundary filling operations on the retinopathy of prematurity image after uniform size and normalization in step S3.1 and the actual probability density map obtained in step S3.2.

[0039] In step S4, the deep neural network model includes a dense dilated convolution module, an output layer, and a multi-kernel pooling module. The preprocessed dataset is passed through the dense dilated convolution module and the multi-kernel pooling module to obtain four feature maps and one feature map, respectively. The four feature maps and one feature map are concatenated to obtain an output feature map. The output layer consists of two 1×1 convolutional blocks and a sigmoid function. The input of the output layer is the output feature map, and the output layer outputs an output probability density map.

[0040] In the specific implementation process, such as Figure 4As shown, the deep neural network model is a U-Net network with VGG16 as the encoder. A DAC module and an RMP module are introduced in the bottleneck layer. The overall network framework of the deep neural network model is as follows: Figure 4 As shown, the image of retinopathy of prematurity X∈R256×256×3 is used as the input of the network. The optic disc localization and macular localization are regarded as two tasks respectively. The model outputs a probability density map Yt∈R256×256×1, where t∈[T] and T is the total number of tasks. Specifically, the model obtains a feature map of size 16×16×512 after passing through one convolutional layer and four downsampling layers. Each downsampling layer includes convolution (Conv), batch normalization (BN), rectified linear function activation (ReLU), and max pooling (MaxPool). The first convolutional layer has a 3×3 kernel, 3 input channels, 64 output channels, a stride of 1, and 1 padding pixel. The first downsampling layer has 64 input channels and 128 output channels, the second downsampling layer has 128 input channels and 256 output channels, the third downsampling layer has 256 input channels and 512 output channels, and the fourth downsampling layer has 512 input channels and 512 output channels.

[0041] The DAC module has four cascaded branches, which gradually increase with the number of natural convolutions, such as... Figure 5 As shown, the receptive fields are 1 through 1, 3, and 5, and then the receptive fields for each branch will be 3, 7, 9, and 19. It employs different receptive domains, similar to the Inception structure. In each branch, a 1×1 convolution is applied for rectified linear activation, and finally, the feature maps of each branch are fused to obtain a 16×16×512 feature map.

[0042] The RMP module uses four pooling branches of different sizes and upsampling, such as Figure 6 As shown, the feature maps are 2, 3, 5, and 6 respectively, resulting in 4 feature maps. These are then concatenated with the feature map from the previous module to obtain a feature map of size 16×16×516.

[0043] After the obtained feature map is passed through the encoder, it will be output through the decoder. The decoder contains four upsampling layers. The output of each upsampling layer will be concatenated with the output of the corresponding downsampling layer of the encoder, and then output to the next upsampling layer to gradually restore the spatial resolution of the original input image.

[0044] The output layer uses two 1×1 convolutions to parse the features and generate the final output, which consists of two feature maps representing the probability density maps of the optic disc and macula, respectively. Figure 3 As shown.

[0045] Example 4 Based on Examples 1, 2, and 3, this embodiment continues to disclose the following content: Step S5, the predicted coordinates calculated based on the probability density map, includes the following steps: A probability density map is obtained using a deep neural network model, and the gray value of each pixel in the probability density map is calculated. The grayscale value of each pixel is weighted and added to the coordinates of that pixel in the probability density map to obtain the weighted coordinates of all pixels; Sum the weighted coordinates of all pixels to obtain a total weighted coordinate. The predicted coordinates are obtained by dividing the total weighted coordinates by the sum of the gray values ​​of all pixels.

[0046] In step S5, the loss values ​​of the actual probability density map and the predicted probability density map are calculated, including:

[0047] In the formula, The loss function is used to predict the probability density map and the actual probability density map. The probability values ​​are the actual probability density map obtained from the coordinates of the visual disk and the macula. These are the probability values ​​of the probability density map predicted by the deep neural network model. This represents the total number of image samples.

[0048] Step S7, optimize the deep neural network model using multi-task learning, including the following steps: In multi-task learning, when multiple tasks share the hidden layer parameters of a deep neural network model, the features of multiple tasks are learned and the results of each task are output. In multi-task learning, the parameters of the hidden layers of a deep neural network model are shared. Therefore, task t in the deep neural network model... d Layer output for,

[0049] Where H is the input to the hidden layer in the deep neural network model. It is the ReLU activation function. For the first deep neural network model d Convolutional kernels of layers; The hidden layer parameters of a deep neural network model are partitioned using a binary parameter partitioning matrix:

[0050] In the formula, It is the first in the deep neural network model d Layers and Tasks The relevant column vectors, from which a subset of parameters is selected for each task.

[0051] Two Adam optimizers are used to update the shared parameters, employing stochastic gradient descent for the update process.

[0052] In the formula For data samples, For learning rate, A task-specific exponential moving average mechanism. The introduced memos only involve updates prior to task t. For shared parameters. When t=1, i.e., the first task, the parameter update uses parameters from the last task T. This means only the current task is considered. (Parameters) Based on the data of the current task t and functions related to task t Calculate the gradient and combine it with the learning rate. and task t The exponential moving average mechanism updates the weight parameters.

[0053] When t>1, the parameter update uses the parameters from the previous task t-1. .parameter Based on the data of the current task t and functions related to task t Calculate the gradient and combine it with the learning rate. Exponential moving average mechanism for task t The weight parameters are updated. Each task has its own parameter update rules, and an exponential moving average mechanism is used to introduce a memory term between tasks, so that the parameter updates of each task can benefit from its own historical information.

[0054] The binary parameter partitioning matrix It follows a Bernoulli distribution with parameter p:

[0055] Where p is the proportion of parameters shared by each layer. Let p be a Bernoulli distribution with parameter p.

[0056] The sharing rate determines whether each parameter will be shared by multiple tasks. A larger value for p indicates a higher probability of parameter sharing, leading to greater parameter overlap between tasks. Conversely, a smaller p indicates a higher probability of parameter independence, reducing parameter sharing between tasks. Therefore, adjusting the value of p controls the degree of parameter sharing in multi-task learning, thus affecting the model's training performance. This initialization method helps reduce interference between tasks during training and simplifies the optimization problem.

[0057] Example 5 This embodiment, based on embodiments 1, 2, 3, and 4, further discloses the following content: In step S7, the error between the predicted coordinates and the coordinates of the optic disc and macula is determined. If the error is less than a first preset value, the optimized deep neural network model is saved as the final deep neural network model. This includes: feeding the images of the dataset into the deep neural network model one by one to obtain the corresponding probability density map, calculating the predicted coordinates, calculating the average Euclidean distance error between the actual coordinates and the predicted coordinates, and taking half of the sum of the positioning errors of the optic disc and macula as the total error. If the current total error is less than the first preset value, the optimized deep neural network model is saved.

[0058] Calculate the mean Euclidean distance error ,include:

[0059] In the formula, For the number of tasks, and These are the predicted coordinates and actual coordinates of the i-th input image, respectively.

[0060] In step S8, retinopathy of prematurity is divided into regions based on the predicted coordinates of the optic disc and macula, including: Based on the predicted coordinates from the deep neural network model, the predicted coordinates are restored to the coordinates of the original image to obtain the coordinates of the optic disc and macula. The optic disc coordinates are used as the center point of the retinopathy of prematurity (ROP) region. According to the definition of ROP region, the distance relationship between the optic disc coordinates and the macula coordinates is used to draw the ROP region reference line.

[0061] Based on the definition of retinopathy of prematurity (ROP) zoning, the predicted coordinates from the model are restored to the original image's size coordinates. For example... Figure 7 As shown, using the optic disc coordinates as the center point of the retinopathy of prematurity (ROP) zones, and twice the distance from the optic disc to the macula as the radius, a circular reference line for zone I is drawn on the original image; a circular reference line for zone II is drawn with a radius of 2.6 times the distance from the optic disc to the macula; and a circular reference line for zone III is drawn with a radius of 4 times the distance from the optic disc to the macula. Figure 8 Compare these circular reference lines with the circular reference lines formed by the actual coordinates to obtain the IoU evaluation index for each partition.

[0062] The IoU calculation is shown in the figure. Assuming the areas of the two circles are C1 and C2, and the area of ​​their intersection is I, the formula for calculating IoU is as follows:

[0063] The experimental results of the present invention are shown in Tables 1 and 2 in the following specific embodiments.

[0064] Table 1. Partitioning Experiment of Retinopathy of Prematurity Based on Multi-Task Learning

[0065] Table 2 shows the macular and optic disc localization experiments on the IDRID dataset.

[0066] Multi-task learning refers to using a single network model to jointly localize the optic disc and macula simultaneously, meaning one network model can handle multiple different tasks at the same time. Single-task learning, on the other hand, refers to using two separate networks to localize the optic disc or macula separately, meaning that the number of network models required corresponds to the number of tasks.

[0067] In the specific implementation process, such as Figure 9 As shown, images of retinopathy of prematurity (ROP) in premature infants were collected; images of ROP containing both the optic disc and macula were selected; pixel-level annotations were performed on the locations of the optic disc and macula in the ROP images; the annotated ROP images underwent data preprocessing, dividing them into training, validation, and test sets; a network model was established; training was performed using the training set: online data augmentation was performed on the training set data, and the augmented data was fed into the network to obtain the output results. The error between the training output results and the true labels was calculated, and two backpropagation algorithms were used to update the parameters for the two tasks, completing the update of the weight parameters in the network; after training, the mean Euclidean distance error for macula and optic disc localization was calculated on the validation set using the currently trained model. If the performance of the current model was better than the previous model, the current network model was saved; the network model that performed best on the validation set was obtained by training N images through epochs; the test set was input into the optimal network model to obtain the automatic partitioning results of ROP in premature infants.

[0068] The same or similar labels correspond to the same or similar parts; The terms used to describe coordinate relationships in the accompanying drawings are for illustrative purposes only and should not be construed as limiting this patent. Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. An automatic regionalization method for retinopathy of prematurity based on multi-task learning, characterized in that, Includes the following steps: S1: Collect images of retinopathy of prematurity and select images of retinopathy of prematurity that simultaneously exhibit both optic disc and macular structures; S2: Mark the coordinates of the optic disc and the macula in the retinal lesion image, and use the marked image as a dataset of retinopathy of prematurity partitions; S3: Preprocess the dataset and calculate the actual probability density map of the images in the preprocessed dataset; S4: Construct a deep neural network model and use the deep neural network model to output a predicted probability density map of the images in the preprocessed dataset; S5: Calculate the loss value between the actual probability density map and the predicted probability density map; S6: Calculate the predicted coordinates of the optic disc and macula based on the predicted probability density map; S7: Determine the error between the predicted coordinates and the coordinates of the optic disc and macula. If the error is greater than the first preset value, use multi-task learning to optimize the deep neural network model, calculate the probability density map, and return to step S5. If the error is less than the first preset value, save the optimized deep neural network model as the final deep neural network model. S8: Use the final deep neural network model obtained in step S7 to predict the coordinates of the optic disc and macula in the retinopathy of prematurity image to be partitioned, and partition the retinopathy of prematurity based on the predicted coordinates of the optic disc and macula. In step S3, the preprocessing of the dataset, which involves calculating the actual probability density map from the images of the preprocessed dataset, includes: S3.1: Standardize and normalize the images of retinopathy of prematurity. S3.2: Based on the coordinates of the optic disc and macula, a probability density map centered on the coordinates of the optic disc and macula is generated using a multivariate normal distribution function. In the formula, A two-dimensional vector representing the viewport coordinates. The mean of the visual coordinate distribution is denoted as . It is the covariance matrix; A two-dimensional vector representing the macular coordinates. This represents the mean of the macular coordinate distribution; S3.3: Perform synchronous rotation, translation, scaling, and boundary filling operations on the retinopathy of prematurity image after uniform size and normalization in step S3.1 and the actual probability density map obtained in step S3.2; In step S4, the deep neural network model includes a dense dilated convolution module, an output layer, and a multi-kernel pooling module. The preprocessed dataset is passed through the dense dilated convolution module and the multi-kernel pooling module to obtain four feature maps and one feature map, respectively. The four feature maps and one feature map are concatenated to obtain an output feature map. The output layer consists of two 1×1 convolutional blocks and a sigmoid function. The input of the output layer is the output feature map, and the output layer outputs a predicted probability density map. In step S6, the predicted coordinates of the optic disc and macula are calculated based on the predicted probability density map, including the following steps: A deep neural network model is used to obtain a predicted probability density map, and the gray value of each pixel in the predicted probability density map is calculated. The grayscale value of each pixel is weighted and added to the coordinates of that pixel in the predicted probability density map to obtain the weighted coordinates of all pixels. Sum the weighted coordinates of all pixels to obtain a total weighted coordinate. Divide the total weighted coordinates by the sum of the gray values ​​of all pixels to obtain the predicted coordinates of the optic disc and macula. Step S7, optimize the deep neural network model using multi-task learning, including the following steps: In multi-task learning, when multiple tasks share the hidden layer parameters of a deep neural network model, the features of multiple tasks are learned and the results of each task are output. In multi-task learning, the parameters of the hidden layers of a deep neural network model are shared. Therefore, task t in the deep neural network model... d Layer output for, Where H is the input to the hidden layer in the deep neural network model. It is the ReLU activation function. For the first deep neural network model d Convolutional kernels of layers; The hidden layer parameters of a deep neural network model are partitioned using a binary parameter partitioning matrix: In the formula, It is the first in the deep neural network model d Layers and Tasks The relevant column vectors, from which a subset of parameters is selected for each task. Two Adam optimizers are used to update the shared parameters, employing stochastic gradient descent for the update process. In the formula For data samples, For learning rate, A task-specific exponential moving average mechanism. The introduced memos only involve updates prior to task t. For shared parameters.

2. The automatic regionalization method for retinopathy of prematurity based on multi-task learning according to claim 1, characterized in that, In step S5, the loss values ​​of the actual probability density map and the predicted probability density map are calculated. include: In the formula, The loss function is used to predict the probability density map and the actual probability density map. The probability values ​​are the actual probability density map obtained from the coordinates of the visual disk and the macula. These are the probability values ​​of the probability density map predicted by the deep neural network model. This represents the total number of image samples.

3. The automatic zoning method for retinopathy of prematurity based on multi-task learning according to claim 1, characterized in that, The binary parameter partitioning matrix It follows a Bernoulli distribution with parameter p: Where p is the proportion of parameters shared by each layer. Let p be a Bernoulli distribution with parameter p.

4. The automatic regionalization method for retinopathy of prematurity based on multi-task learning according to claim 1, characterized in that, In step S7, the error between the predicted coordinates and the coordinates of the optic disc and macula is determined. If the error is less than a first preset value, the optimized deep neural network model is saved as the final deep neural network model. This includes: feeding the images of the dataset into the deep neural network model one by one to obtain the corresponding probability density map, calculating the predicted coordinates, calculating the average Euclidean distance error between the actual coordinates and the predicted coordinates, and taking half of the sum of the positioning errors of the optic disc and macula as the total error. If the current total error is less than the first preset value, the optimized deep neural network model is saved.

5. The automatic zoning method for retinopathy of prematurity based on multi-task learning according to claim 4, characterized in that, Calculate the mean Euclidean distance error ,include: In the formula, The total number of images in the dataset. and These are the predicted coordinates and actual coordinates of the i-th input image, respectively.

6. The automatic zoning method for retinopathy of prematurity based on multi-task learning according to claim 4, characterized in that, In step S8, retinopathy of prematurity is divided into regions based on the predicted coordinates of the optic disc and macula, including: Based on the predicted coordinates from the deep neural network model, the predicted coordinates are restored to the coordinates of the original image to obtain the coordinates of the optic disc and macula. The optic disc coordinates are used as the center point of the retinopathy of prematurity (ROP) region. According to the definition of ROP region, the distance relationship between the optic disc coordinates and the macula coordinates is used to draw the ROP region reference line.

Images