FPGA-based fundus retinopathy real-time screening system

Abstract

This invention discloses a real-time screening system for fundus retinal lesions based on FPGA, belonging to the fields of medical image detection and artificial intelligence diagnostic technology. The system includes an image acquisition module, an FPGA hardware processing module, a DDR cache module, a UDP Ethernet transmission module, an ISP image optimization module, a host computer display module, and an AI lesion recognition module. It uses a MIPI camera to acquire high-resolution fundus images, implements 12 types of image processing algorithms in parallel through the FPGA, enhances blood vessel contrast and suppresses overexposure through a customized ISP module, utilizes dual-channel DDR to improve data access efficiency, and achieves synchronous transmission of original images, optimized images, and AI-labeled images via gigabit UDP Ethernet. It is equipped with a lightweight YOLOv8n network to achieve rapid detection of four types of fundus lesions, with a recognition accuracy of 93.2% and an inference latency of less than 60ms. This system features low processing latency, low cost, and a high degree of domestic production, making it suitable for real-time screening of fundus lesions in primary healthcare institutions.

Description

Technical Field

[0001] This invention relates to the fields of medical image detection, FPGA hardware acceleration and artificial intelligence diagnostic technology, specifically to a real-time screening system for fundus retinal lesions based on Anlu FPGA. Background Technology

[0002] Currently, primary healthcare institutions generally face problems such as a shortage of professional ophthalmologists, high costs of large-scale fundus examination equipment, and poor portability. Traditional fundus image screening relies on PC-based software processing, which suffers from high latency and weak parallel processing capabilities. Simultaneously, fundus images commonly exhibit low vascular contrast, overexposure of the optic disc, and unclear details, affecting the accuracy of lesion identification. Existing medical imaging equipment largely relies on imported chips, with low levels of domestic substitution, hindering large-scale deployment. Therefore, there is an urgent need for a low-cost, domestically produced, real-time processing fundus lesion screening system with AI-assisted diagnosis. Summary of the Invention

[0003] 1. Technical problems to be solved

[0004] This invention addresses the technical problems of existing fundus screening equipment, such as high cost, large processing delay, poor image quality, non-real-time AI diagnosis, and low degree of domestic production.

[0005] 2. Technical Solution

[0006] This invention discloses a real-time screening system for fundus retinal diseases based on FPGA, comprising:

[0007] The system includes an image acquisition module, an FPGA hardware processing module, a DDR cache module, a UDP Ethernet transmission module, an ISP image optimization module, a host computer display module, and an AI lesion recognition module.

[0008] The image acquisition module uses a MIPI camera to acquire high-resolution fundus images;

[0009] The FPGA hardware processing module uses Verilog to implement 12 types of parallel image processing algorithms;

[0010] The ISP module enhances fundus images, improving blood vessel contrast and suppressing overexposure;

[0011] The DDR module adopts a dual-channel independent read / write architecture to improve image access efficiency;

[0012] UDP Ethernet enables the creation of original graphs, optimized graphs, and AI tagging. Figure 3 Image synchronous transmission;

[0013] The AI ​​module uses a lightweight YOLOv8n network to achieve rapid lesion detection.

[0014] 3. Beneficial effects

[0015] This invention utilizes domestically produced Anlu FPGAs for hardware parallel acceleration, achieving a single-frame processing latency of ≤0.92ms; a customized ISP module ensures ≥90% preservation of vascular details; AI recognition accuracy reaches 93.2%, with an inference latency of <60ms; the total hardware cost of the system is less than 2500 yuan, making it suitable for deployment in primary healthcare settings; it integrates image acquisition, hardware-accelerated processing, real-time transmission, AI diagnosis, and data archiving, addressing the pain points of equipment shortages and low diagnostic efficiency in primary healthcare settings. Attached Figure Description

[0016] Figure 1 System Overall Block Diagram

[0017] Figure 2 Data Flow Diagram of MIPI Image Acquisition Circuit

[0018] Figure 3 Block diagram of FPGA image processing module

[0019] Figure 4. UDP Ethernet transmission architecture diagram

[0020] Figure 5. Flowchart of AI lesion recognition process Detailed Implementation

[0021] 1. Overall System Structure

[0022] The system is based on the Anlu MLK-H10-PH1A180 FPGA and connects to a MIPI camera, MIPI display, DDR3, Gigabit Ethernet PHY chip and PC host computer.

[0023] 2. Image Acquisition Module

[0024] The MLK-CAM001-CS500 MIPI camera is used to output RAW images with a resolution of 2592×1944; the images are transmitted to the FPGA via the MIPI CSI-2 interface; after unpacking, RAW10 decompression, and black level correction, they are converted into standard AXI4-Stream video streams.

[0025] 3. FPGA Hardware Image Processing Module

[0026] A parallel pipeline architecture is used to implement 12 types of algorithms: grayscale conversion, gamma transform, Gaussian filtering, Laplacian sharpening, embossing, binarization, Sobel edge detection, morphological edge extraction, erosion, dilation, and frame difference motion detection; algorithm switching is achieved through serial port commands; single frame processing latency is ≤0.92ms.

[0027] 4. Customized ISP image optimization module

[0028] Perform black level correction, depigmentation, automatic white balance, contrast enhancement, and brightness adjustment on fundus images; enhance the distinction between red blood vessels and yellow fundus, solve the problems of low blood vessel contrast and overexposure of optic disc, and preserve blood vessel details ≥90%.

[0029] 5. Dual-channel DDR3 cache architecture

[0030] The system sets up independent channels for raw image and grayscale data, and adopts an optimized arbitration algorithm to achieve concurrent read and write operations. It provides a stable frame buffer for frame difference method, motion detection, and real-time display, thereby improving bus utilization and system smoothness.

[0031] 6. UDP High-Speed ​​Ethernet Transmission Module

[0032] Based on 1000Mbps Ethernet, a custom frame header, packet reassembly, and multi-threaded receiving mechanism are adopted to achieve real-time parallel transmission of three images: "raw fundus image + ISP optimized image + AI labeled image", with a packet loss rate of ≤0.1% and an image transmission time of ≤50ms.

[0033] 7. Host Computer and AI Diagnostic Module

[0034] The host computer is developed based on Python, OpenCV, and Tkinter, enabling image reception, simultaneous display of three images, parameter adjustment, screen capture and recording, and data archiving. The AI ​​module loads a lightweight YOLOv8n model to detect four types of lesions: hemorrhagic microaneurysms, neovascular hemorrhage, hard exudates, and retinal effusion. Pathogens with a confidence level <85% are automatically marked as "recommended for manual review," achieving an accuracy of 93.2% and an inference latency of <60ms.

[0035] 8. Work Process

[0036] 1) The camera captures images of the fundus;

[0037] 2) The FPGA performs acquisition, decoding, and ISP optimization;

[0038] 3) Parallel execution of 12 types of image processing;

[0039] 4) Dual-channel DDR cache ensures real-time performance;

[0040] 5) Upload the three images to the host computer via UDP;

[0041] 6) AI automatically identifies and marks lesions;

[0042] 7) The screen displays synchronously with the PC, supporting archiving and review.

[0043] 9. Performance Indicators

[0044] Image processing latency ≤ 0.92ms;

[0045] AI recognition accuracy ≥92%;

[0046] Ethernet speed 1000Mbps, packet loss rate ≤0.1%;

[0047] Hardware cost ≤ 2536.6 yuan;

[0048] It runs continuously for 24 hours without any problems.

Claims

1. A real-time screening system for fundus retinal diseases based on FPGA, characterized in that, The system includes an image acquisition module, an FPGA hardware processing module, a DDR cache module, a UDP Ethernet transmission module, an ISP image optimization module, a host computer display module, and an AI lesion recognition module. The image acquisition module is used to acquire fundus images. The FPGA hardware processing module is used to execute multiple image processing algorithms in parallel. The ISP image optimization module is used to improve the contrast of fundus blood vessels and suppress overexposure of the optic disc. The UDP Ethernet transmission module is used to synchronously transmit the original image, the optimized image, and the AI-labeled image to the host computer. The AI ​​lesion recognition module is used to detect and label fundus lesions in real time.

2. The system according to claim 1, characterized in that, The image acquisition module uses an MLK-CAM001-CS500 MIPI camera, which outputs RAW format images with a resolution of 2592×1944 and transmits them to the FPGA via the MIPI CSI-2 interface.

3. The system according to claim 1, characterized in that, The FPGA hardware processing module uses Verilog hardware description language to implement a parallel pipeline architecture, supporting 12 types of algorithms, including grayscale, gamma transform, Gaussian filtering, Laplacian sharpening, embossing effect, binarization, Sobel edge detection, morphological edge extraction, erosion, dilation, and frame difference motion detection, with a single frame image processing latency of ≤0.92ms.

4. The system according to claim 1, characterized in that, The ISP image optimization module performs black level correction, depigmentation, automatic white balance, brightness and contrast adjustment to enhance the distinction between red blood vessels and yellow fundus, with a blood vessel detail retention rate of ≥90%.

5. The system according to claim 1, characterized in that, The DDR cache module adopts a dual-channel independent read / write architecture, which caches the original image data and grayscale image data respectively, and uses an optimized arbitration algorithm to improve image access efficiency and bus bandwidth utilization.

6. The system according to claim 1, characterized in that, The UDP Ethernet transmission module operates at a rate of 1000Mbps and employs a custom frame header, packet reassembly, and multi-threaded reception mechanism to achieve parallel transmission of "raw fundus image + ISP optimized image + AI labeled image" with a packet loss rate of ≤0.1%.

7. The system according to claim 1, characterized in that, The AI ​​lesion recognition module uses a lightweight YOLOv8n network to detect four types of lesions: hemorrhagic microaneurysms, neovascular hemorrhage, hard exudates, and retinal effusions. The recognition accuracy is ≥92%, and the inference latency is <60ms. Results with a confidence level <85% are automatically marked as "recommended for manual review".

8. The system according to claim 1, characterized in that, The host computer display module, based on Python and OpenCV, enables simultaneous display of three images, parameter adjustment, screenshotting, video recording, and data archiving. It supports the transmission and local saving of 5-megapixel high-resolution images.

Images