Ai-based rapid diagnosis and triage system for retinopathy of prematurity in remote telemedicine

Abstract

The invention provides a remote screening system for diagnosing Retinopathy of Prematurity (ROP), using AI and deep learning. The system captures retinal images of premature infants via a RetCam device, transmits them to a diagnostic center, and processes them with a MobileNetV2 convolutional neural network (CNN). The AI identifies ROP stages and alerts medical professionals if treatment is necessary. The system features real-time triage and continuous improvement through feedback. Its performance is optimized with deep separable convolutions, Bottleneck blocks, and periodic updates. The system enhances diagnostic accuracy and ensures timely intervention for premature infants.

Description

[0001] AI-Based Rapid Diagnosis and Triage System for Retinopathy of Prematurity in Remote Telemedicine

[0002] Technical Field:

[0003] This disclosure pertains to the field of ophthalmology, specifically addressing remote screening and telemedicine systems for diagnosing retinopathy of prematurity (ROP) in preterm infants. The system incorporates artificial intelligence (Al) for real-time triage and prompt assessment of suspected cases by medical specialists. Operating on deep learning techniques and image processing, it facilitates efficient analysis and interpretation of retinal images.

[0004] Background:

[0005] Retinopathy of Prematurity (ROP) is a serious eye condition that impacts premature infants due to the abnormal development of retinal blood vessels. It is especially common among infants born before the 31st week of gestation or those with a birth weight of less than 1500 grams. Key risk factors for ROP include prolonged oxygen therapy and various complications associated with preterm birth. The abnormal growth of these vessels can result in severe consequences, including retinal detachment and blindness.

[0006] The critical window for diagnosing and treating this condition is between the fourth and sixth weeks after birth. During this period, timely medical intervention can help prevent further disease progression. Regular eye examinations for infants at risk are essential for early detection.

[0007] In the past, diagnosis was performed using ophthalmoscopy, which required significant expertise and specialized equipment. However, with advancements in technology, new methods like digital imaging and Optical Coherence Tomography (OCT) have been developed. These innovations have improved diagnostic accuracy and allowed for a more detailed assessment of the retina.In the field of treatment, earlier methods involved laser photocoagulation, which aimed to target and destroy abnormal blood vessels. In more advanced cases, injections of anti-VEGF drugs are utilized to manage the growth of these vessels. Furthermore, vitrectomy surgery may be necessary in situations involving retinal detachment.

[0008] Even with recent advancements, successful treatment of ROP relies heavily on early diagnosis and timely intervention. If the disease is not recognized quickly, it can lead to permanent retinal damage and vision impairment. This highlights the vital importance of closely monitoring infants who are at risk.

[0009] Summary:

[0010] This invention introduces a comprehensive remote medical system designed to screen and diagnose ROP. The system features Al-driven software that utilizes advanced deep learning models, specifically MobileNetV2, to analyze retinal images of infants and identify cases warranting treatment. The software is developed in C++, while the Al components are implemented in Python. To enhance its performance, the MobileNetV2 model has been customized with deep separable convolutions and Bottleneck blocks. Images captured by the RetCam imaging device are transmitted over the internet to a diagnostic center, where the Al processes the images and promptly alerts the relevant doctors if treatment is required.

[0011] Brief Description of the Drawings:

[0012] Figure 1 : Diagram of telemedicine system architecture.

[0013] Figure 2: Diagram of deep learning model for diagnosing retinopathy of prematurity cases requiring treatment.

[0014] Detailed Description:This technology initially involved the development of a comprehensive remote medical system capable of transmitting images over the Internet. Following that, software utilizing Al and deep learning was created to diagnose cases of ROP. The remote screening software, programmed in C++, is tailored specifically for this invention and is installed on compatible computers at designated centers. Images are transmitted via the internet to a specialized ROP diagnostic center for evaluation by an expert.

[0015] For real-time triage, images transmitted over the internet are processed by an Al program developed in Python and integrated into the screening software. The hardware used for programming, training, and testing the Al includes a T4 GPU alongside an Intel Xeon CPU featuring two virtual CPUs and 12GB of RAM. Any system meeting the above minimum specifications can train, test, and implement this artificial intelligence. Initially, the research team, in collaboration with computer engineers and relevant medical professionals, conducted a thorough review of sources and literature to achieve high accuracy and rapid processing speed. Based on their research findings, they selected a model for design and customization that best learns from patterns in disease data. The pre-trained model utilized in this innovation is MobileNetV2, a convolutional neural network with 53 layers and an input size of 224x224 pixels across three channels. This model employs convolutional layers to extract features from input images by applying filters.

[0016] Rather than relying on standard convolutions, which can be computationally intensive, a depthwise separable convolution is utilized. This approach consists of two distinct stages. The first stage, Depthwise Convolution, applies unique filters to each input channel separately, significantly reducing both computations and parameters. The second stage, Pointwise Convolution (1x1 convolution), integrates the channels after they have been individually filtered. This process allowsthe model to capture complex channel relationships while minimizing computational demands and optimizing memory usage.

[0017] The model employs Bottleneck blocks and residual connections. Initially, a 1x1 convolution layer is used to expand the number of input channels into a higher dimensional feature space. This is followed by a depth wise convolution and another 1x1 convolution to revert the number of channels to their original size. This sequence of operations within the Bottleneck blocks effectively reduces the number of parameters and computational burden while preserving the model's feature space. When a Bottleneck block's input and output dimensions are aligned, a residual connection is incorporated to help alleviate the vanishing gradient problem in deeper layers, thereby enhancing the model's learning capabilities.

[0018] Bottleneck blocks utilize Inverted Residuals, where the number of channels is first increased to expand the feature space and then decreased to compress it after processing. This method effectively extracts essential features while minimizing resource usage.

[0019] The architecture of the model comprises three essential sections:

[0020] 1. Input Layer: The model accepts an image of size 224x224 pixels, which includes three color channels. This layer serves to input the image into the network.

[0021] 2. Bottleneck Blocks with or without Residual Connections: Each bottleneck block employs a 1x1 convolution to increase the number of channels, followed by a depthwise separable convolution that processes each channel independently. Finally, another 1x1 convolution is applied to compress the output to the original input dimensions.

[0022] 3. Downsampling Layers: At various points in the network, the spatial dimensions of the feature space are reduced to minimize computational load and enable the model to leam higher-level features. This is typically achieved through convolutions using a stride of 2.The model concludes with a fully connected layer that adjusts the number of outputs to correspond with the number of classes.

[0023] The model underwent further optimization through hierarchical clustering, strategically placing convolutional layers at varying depths within the neural network to capture low- and high-level details from the images. Convolutional layers featuring 1x1 and 2x2 pixel strides and a Rectified Linear Unit (ReLU) activation function were incorporated. The ReLU function facilitates rapid convergence by converting negative values to zero while leaving positive values unchanged. Several normalization techniques were implemented to address issues related to gradient instability. Weight and activation normalization were employed to mitigate the risks of vanishing and exploding gradients. Additionally, batch normalization was utilized to stabilize the network by adjusting the distribution's center or scale, which enhances training speed and ensures a more consistent convergence.

[0024] For the loss function, binary cross-entropy was selected, as it effectively quantifies the difference between the predicted values and the true labels. The formula for the loss function is represented as follows, where y corresponds to the true label and y represents the network's prediction.

[0025]

[0026] Model training can be conducted utilizing a dataset from any clinic or hospital with sufficient patient data related to ROP. In this specific model, the data was meticulously gathered from the neonatal clinic at Khatam-ul-Anbiya Hospital in Mashhad. This institution provided a robust set of ROP patient records, facilitating a thorough analysis and training process for the model. Strict measures for confidentiality and data anonymization were upheld throughout the process. Two categories of images were selected: one set from infants whose doctors opted for intravitreal bevacizumab injections and another from infants whom doctors chose to monitor withouttreatment. In total, 50 sets of images were compiled and overseen by qualified medical professionals, consisting of 50 eye image sets requiring treatment and 50 image sets under observation (which included stages such as preparation, preprocessing, and data augmentation). No public datasets were employed, and all images were specifically sourced from ROP patients at Khatam-al-Anbia Hospital.

[0027] The images in the dataset were resized to align with the model's input dimensions, and the dataset was partitioned into training and testing sets, with approximately 80% designated for training. The pixel values of the images were adjusted to facilitate processing. The model utilizes a convolutional system to filter the images, extracting features passed to additional layers for more comprehensive analysis. The weights of the filters and classification parameters were fine-tuned based on the error function and the model's overall performance.

[0028] After training the model, 20% of the remaining images were utilized for testing. Sensitivity and specificity evaluations were performed, indicating that the model attained a sensitivity of 82% and a specificity of 86%. Furthermore, the area under the ROC curve was calculated to be 0.841, which signifies excellent diagnostic accuracy.

[0029] Appropriate imaging devices were supplied for imaging and diagnosis, and personnel were trained in their use and the remote screening system. Following thorough training, premature infants were imaged in the neonatal intensive care unit. The captured images were then transmitted over the Internet to the diagnostic center.

[0030] At the core of the operation, images were analyzed in real-time by an Al system. When the system identified a stage that required treatment, it promptly sent alerts to the attending physician and relevant personnel via SMS, email, or phone. This ensured timely image reviews and informed treatment decisions, preventing any lapse in the critical window for diagnosis and intervention.Subsequently, a specialist ophthalmologist reviews the prioritized images identified by the Al, affirming or disputing the diagnosis. Following this final assessment, a treatment plan is formulated, which may encompass options such as medication injections, surgical intervention, laser therapy, or ongoing monitoring. This information is then relayed back to the screening software to further enhance the ATs precision.

[0031] The Al is updated every two weeks using feedback and deep learning techniques. These techniques allow the system to continuously identify and correct errors, thereby improving its accuracy and efficiency.

Claims

Claims:What is claimed is:1: A remote screening system for diagnosing Retinopathy of Prematurity (ROP), comprising:• An image capturing device configured to capture retinal images of premature infants; • A communication module for transmitting the captured images over the Internet to a diagnostic center;• An artificial intelligence (Al) module programmed to process the transmitted images using a convolutional neural network (CNN) model to identify the stages of ROP;• A user interface for displaying the diagnostic results to medical professionals;• A notification system that alerts medical professionals when a critical stage of ROP is detected.2: The system of claim 1, wherein the Al module is trained using a dataset of retinal images collected from retinal image banks available in any hospital or clinic, with specific data sourced from a neonatal clinic or hospital.3: The system of claim 1, wherein the Al module employs the MobileNetV2 architecture, incorporating depthwise separable convolutions to optimize computational efficiency.4: The system of claim 3, wherein the Al module utilizes a Rectified Linear Unit (ReLU) activation function to facilitate rapid convergence and process the images effectively.5: The system of claim 1, further comprising a notification system that alerts medical professionals of critical ROP stages via various communication methods, including SMS, email, or software-based alerts.6: The system of claim 1, wherein the Al module's performance is periodically updated using feedback from medical professionals and newly collected image data, incorporating deep learning techniques to improve accuracy and reduce errors.7: The system of claim 1, wherein the user interface is configured to display the diagnostic results in realtime, allowing medical professionals to interact with the results and provide immediate feedback.8: A method for diagnosing Retinopathy of Prematurity (ROP) using a remote screening system, comprising:• Capturing retinal images of premature infants using an image capturing device;• Transmitting the captured images over the Internet to a diagnostic center;• Processing the transmitted images in real-time using an Al module that applies a convolutional neural network (CNN) model to identify ROP stages;Notifying medical professionals of the diagnostic results through the notification system;Providing a user interface for medical professionals to review and validate the AI- generated diagnosis.9: The method of claim 8, wherein the Al module processes the images using hierarchical clustering for enhanced feature extraction and improved accuracy in diagnosing ROP.10: The method of claim 8, further comprising the step of fine-tuning the neural network model based on feedback from medical professionals, allowing the system to improve its diagnostic accuracy over time.11 : The method of claim 8, wherein the captured retinal images are preprocessed to adjust their dimensions and pixel values to ensure optimal input into the Al module.12: The method of claim 8, wherein the neural network model includes Bottleneck blocks and residual connections for enhanced feature extraction and improved model performance.13: The method of claim 8, further comprising the step of using batch normalization to stabilize the training of the neural network model, ensuring faster and more consistent convergence. 14: The method of claim 8, wherein the loss function used by the Al module is binary crossentropy, which quantifies the difference between the predicted ROP stage and the true label for each image.15: The method of claim 8, wherein the sensitivity and specificity of the Al module are periodically evaluated using new test datasets to ensure ongoing improvement and diagnostic accuracy.16: The system of claim 1, wherein the system is capable of real-time triage of transmitted images as they are received over the Internet.17: The system of claim 1, wherein the Al module is programmed in Python, and a suitable computing system with a GPU T4 and Intel Xeon CPU with dual Virtual CPUs and 12 GB of RAM is used for training and testing.18: The system of claim 1, wherein the images are captured and processed in a neonatal intensive care unit (NICU) setting, and the diagnostic results are transmitted to the diagnostic center via the Internet.19: The method of claim 8, wherein the Al model is periodically updated every two weeks based on feedback and new data.20: The method of claim 8, wherein the feedback from medical professionals is integrated into the Al module to improve diagnostic performance.21: The system of claim 1, wherein the Al system is integrated into a remote screening software designed specifically for ROP diagnosis and is capable of receiving and processing retinal images from the diagnostic center.22: The system of claim 1, wherein the medical professional reviews the Al-identified images in priority for diagnosis, ensuring timely treatment and preventing delays in critical ROP diagnosis and intervention.23: The system of claim 21, wherein the medical professional provides feedback to the Al regarding the accuracy of identified cases, improving future performance.24: The system of claim 22, wherein the Al module improves its performance through deep learning techniques based on the feedback provided by the medical professional.

Images