Premature infant retinopathy diagnosis system based on equivariant convolutional network

The preterm retinopathy of prematurity diagnostic system based on isovariant convolutional networks achieves high-precision segmentation and partitioning of the optic disc and lesion area. Combined with staging results, it solves the problem of low diagnostic consistency in existing technologies and provides an efficient and objective diagnostic tool.

CN121708368APending Publication Date: 2026-03-20WOMEN & CHILDRENS MEDICAL CENTER AFFILIATED WITH GUANGZHOU MEDICAL UNIVERSITY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202511880031.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-12-12
Publication Date
2026-03-20

AI Technical Summary

Technical Problem

Existing technologies for diagnosing retinopathy of prematurity suffer from high subjectivity and low diagnostic consistency. Furthermore, existing deep learning solutions lack a systematic framework that combines staging and partitioning, leading to inaccurate segmentation results, accumulated errors, and an impact on the overall reliability of the diagnosis.

Method used

A diagnostic system for retinopathy of prematurity based on isovariant convolutional networks is adopted, which includes modules for image acquisition, segmentation, partitioning, and staging. The system segments the optic disc and lesion area using isovariant convolutional networks, stages the lesion based on the partitioning results, and generates a comprehensive diagnostic report.

Benefits of technology

It improves the segmentation accuracy of the optic disc and lesion area, and collaboratively completes the zoning and staging, thereby enhancing the accuracy and robustness of ROP diagnosis and providing an efficient and objective auxiliary diagnostic tool.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121708368A_ABST
    Figure CN121708368A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of medical image processing, and discloses a premature infant retinopathy diagnosis system based on an isotropic convolutional network, which comprises an image acquisition module, a segmentation module, a partition module, a staging module and a result generation module. The segmentation module is used for processing the retina image by adopting an equivariant convolutional network, and accurately segmenting an optic disc and a lesion area; the partitioning module divides a retina region according to an optic disc segmentation result; the staging module integrates the lesion segmentation result and the zoning result to output lesion staging; and the result generation module integrates all information to generate a comprehensive diagnosis report. According to the premature infant retinopathy diagnosis method and system, the image segmentation precision is improved through the equivariant convolutional network, and the automatic and high-accuracy comprehensive diagnosis of the premature infant retinopathy is realized by utilizing the cooperative work of the modules, so that the clinical diagnosis and treatment decision is effectively assisted.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, specifically a diagnostic system for retinopathy of prematurity based on isovariant convolutional networks. Background Technology

[0002] Retinopathy of prematurity (ROP) is a leading cause of vision loss in children, and its accurate diagnosis relies on a comprehensive assessment of retinal images, including staging (disease severity) and zoning (lesion location). Currently, clinical diagnosis primarily depends on ophthalmologists manually analyzing retinal images, which suffers from high subjectivity and low diagnostic consistency. Although some studies have attempted to utilize deep learning technology for automated screening and staging of ROP, these methods still have significant limitations.

[0003] Existing technologies often focus on single tasks, such as lesion staging or plus disease detection, lacking a systematic framework that combines staging and zoning for comprehensive diagnosis. Furthermore, due to the irregular morphology and blurred boundaries of ROP lesions, and the limited availability of high-quality labeled data for training, existing segmentation models struggle to accurately identify the optic disc and lesion regions. Inaccurate segmentation results further propagate to the zoning and staging stages, causing error accumulation and ultimately affecting the reliability of the overall diagnosis. Therefore, there is an urgent need for an automated system capable of collaboratively performing high-precision segmentation, zoning, and staging, and providing comprehensive diagnostic results. Summary of the Invention

[0004] The purpose of this invention is to provide a diagnostic system for retinopathy of prematurity based on equivariant convolutional networks, so as to solve the technical problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention discloses the following technical solution: a diagnostic system for retinopathy of prematurity based on isovariant convolutional networks, comprising: The image acquisition module is configured to acquire retinal images of premature infants. The segmentation module, connected to the image acquisition module, is configured to: process the retinal image using an isovariant convolutional network to segment the optic disc and lesion region, and output the corresponding segmentation results; The partitioning module, connected to the segmentation module, is configured to: determine the center point and diameter of the optic disc based on the segmentation result of the optic disc, divide the retinal region based on the classification criteria for retinopathy of prematurity, and output the partitioning result; The staging module, which connects the segmentation module and the partitioning module, is configured to output the staging results of retinopathy of prematurity based on the segmentation results and the partitioning results of the lesion area. The result generation module, which connects the partitioning module and the phasing module, is configured to integrate the partitioning results and the phasing results to generate a comprehensive diagnostic report.

[0006] Optionally, the segmentation module includes a VGF-UNet network, whose encoder employs a VGG-16 network pre-trained on the ImageNet dataset, and whose decoder employs equivariant convolutional layers based on Fourier series expansion.

[0007] Optionally, the equivariant convolutional layer achieves rotational equivariance in the following manner: By enhancing the Fourier basis function set for the two-dimensional continuous filter function Parameterization is performed, and the expression is as follows: , in, For fixed basis functions, These are learnable coefficient parameters. Represents two-dimensional coordinates Grid; By adjusting the filter function Apply rotation transformation matrix Derive the rotated filter .

[0008] Optionally, the segmentation module employs a weighted Dice loss function during the training phase, the expression of which is: , in, For predicted values, For the true value, The category weights are calculated based on the pixel proportions of the optic disc and the lesion area. This is a term with minimal tolerance.

[0009] Optionally, it also includes a training module, configured to train the segmentation module using a phased transfer learning strategy; wherein the phased transfer learning strategy includes, in sequence: Freeze the pre-trained weights of the encoder of the VGF-UNet network and train only the equivariant convolutional layers of the decoder; After the training loss converges, the encoder weights are unfrozen, and the VGF-UNet network is jointly trained end-to-end with a lower learning rate.

[0010] Optionally, the staging module is further configured to: perform pixel-level fusion of the segmentation result of the lesion area with the retinal image to generate an enhanced input image; the staging result is obtained by processing the enhanced input image using the EfficientNetB4 classification network and then outputting the result.

[0011] Optionally, the phased module is further configured to: receive the partitioning results output by the partitioning module, and execute a partitioning guidance prompt strategy; the prompt strategy includes: When the partitioning result corresponds to a circular region centered on the optic disc with a radius of 3 times the diameter of the optic disc, or to an annular region centered on the optic disc with a radius from the center point of the optic disc to the serrated edge on the nasal side, the partitioning mask of the corresponding region and the segmentation result of the lesion region are superimposed on the retinal image to generate the enhanced input image.

[0012] Optionally, the EfficientNetB4 classification network uses a class weighting method during the training phase to handle the imbalance problem of period data, with the weight of each period category... The calculation formula is: , in, The total number of training samples, Installment categories The number of samples.

[0013] Optionally, the partitioning module outputs the partitioning results through the following steps: The segmentation result of the visual disk is fitted into a circle to determine the center point and radius of the visual disk; A circular area is drawn with the center point of the viewing disk as the center and a radius of 3 times the diameter of the viewing disk as the radius, which is designated as Zone I; A ring-shaped area is drawn with the center point of the optic disc as the center and the distance from the center point of the optic disc to the serrated edge on the nasal side as the radius, which is designated as area II. Area II does not include area I. The remaining retinal area outside of Zone I and Zone II is designated as Zone III.

[0014] Optionally, the retinal images include multi-view retinal images from a single examination; The staging module is further configured to: extract deep features from each of the multi-view retinal images after processing by the segmentation module, fuse the deep features of multiple multi-view retinal images using a proxy interactive attention mechanism, generate a comprehensive feature representation, and use the comprehensive feature representation, the segmentation result of the lesion area, and the partitioning result output by the partitioning module to output the staging result.

[0015] Beneficial Effects: The preterm infant retinopathy of prematurity (ROP) diagnostic system based on equivariant convolutional networks of this invention improves the segmentation accuracy of the optic disc and lesion regions by introducing an equivariant convolutional network based on Fourier series expansion, laying a reliable foundation for subsequent partitioning. This system integrates segmentation, partitioning, and staging modules, and uses lesion segmentation and partitioning results as prompts to guide staging diagnosis. It effectively utilizes the synergistic effect between modules, breaking the limitations of task isolation in traditional methods. This integrated design not only overcomes the problem of error accumulation caused by inaccurate segmentation but also significantly improves the accuracy and robustness of comprehensive ROP diagnosis through multi-source information fusion, providing a more efficient and objective auxiliary diagnostic tool for clinical practice. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a structural block diagram of a preterm infant retinopathy of prematurity diagnostic system based on equivariant convolutional networks, provided in an embodiment of the present invention. Detailed Implementation

[0018] To facilitate understanding of the technical solutions provided in the embodiments of this application, the background technology involved in the embodiments of this application will be described below.

[0019] Retinopathy of prematurity (ROP) is a highly dangerous eye disease in infancy and one of the leading causes of permanent vision loss in children, seriously threatening the visual health of premature infants. Accurate clinical diagnosis of ROP hinges on a comprehensive assessment of two key aspects: staging, which accurately determines the severity of the disease and clarifies its stage of development; and zoning, which precisely locates the specific position of the lesion on the retina. The results of these two assessments directly determine the subsequent treatment plan and the prognosis of the child's vision.

[0020] Currently, clinical diagnosis of ROP still relies primarily on ophthalmologists manually analyzing retinal images. This traditional method has inherent limitations. Because the identification of subtle features of retinal lesions is highly dependent on the doctor's clinical experience, the professional level, judgment criteria, and even the doctor's work condition on that day can all affect the diagnostic results. For example, an experienced doctor may accurately identify a minor lesion with indistinct borders, while a novice doctor may miss it; doctors in different hospitals may also give different conclusions regarding the severity rating of the same lesion. This strong subjectivity leads to extremely low consistency in clinical diagnosis, easily causing misdiagnosis and missed diagnosis, and potentially delaying the optimal treatment time, resulting in irreversible visual impairment for children.

[0021] Although recent studies have attempted to leverage deep learning technology to overcome the limitations of traditional diagnosis and achieve automated screening and staging of Retinal Occult Disease (ROD), these solutions still have significant shortcomings and fail to meet actual clinical needs. Firstly, most existing solutions focus on a single diagnostic task, such as designing algorithms only for lesion staging or focusing solely on the detection of "plus disease," lacking a systematic framework that deeply integrates the two core tasks of staging and zoning. The clinical diagnostic logic for ROP requires combining the location of the lesion (zoning results) and its severity (staging results) to comprehensively determine the condition; single-task diagnostic results cannot provide complete and reliable decision-making basis for clinicians. Secondly, the complexity of ROP lesions and data limitations pose significant challenges to diagnosis. ROP lesions are irregular in shape, have blurred boundaries, and due to the specificity of cases and the high difficulty of annotation, high-quality annotated retinal image data suitable for model training is extremely scarce. This makes it difficult for existing segmentation models to accurately identify the optic disc and lesion areas. Optic disc localization and lesion area segmentation are the foundational steps for subsequent zoning and staging. The accuracy of the segmentation results directly determines the reliability of subsequent diagnostic steps. Once a segmentation deviation occurs, the error will continue to propagate and accumulate in the zoning and staging stages, ultimately leading to a distorted overall diagnostic result that cannot provide a reliable reference for clinical practice.

[0022] In summary, there is an urgent clinical need for a technological solution that can overcome the limitations of existing diagnostic methods. This solution should be able to collaboratively achieve high-precision segmentation of the optic disc and lesion areas, accurate retinal partitioning, and scientific staging of lesions, and integrate all results to generate a comprehensive diagnostic report. This would solve problems such as the subjectivity of manual diagnosis, the limited scope of existing deep learning solutions, and the accumulation of diagnostic errors, providing clinicians with an efficient, objective, and reliable ROP-assisted diagnostic tool.

[0023] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. In this document, the term "comprising" is intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements, but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0024] This embodiment presents a diagnostic system for retinopathy of prematurity based on isovariant convolutional networks, such as... Figure 1 As shown, it includes an image acquisition module, a segmentation module, a partitioning module, a phased module, and a result generation module, as detailed below.

[0025] The image acquisition module is configured to acquire retinal images of premature infants. In practice, retinal images are acquired using the clinically common RetCam wide-angle digital fundus imaging system. Informed consent from the child's guardian is required before image acquisition, and the image resolution is fixed at 640×480 pixels. The number of images taken in a single examination ranges from 1 to 32 (clinically 6 is common, covering different angles of the retina to avoid missing local lesions in a single image).

[0026] The segmentation module, connected to the image acquisition module, is configured to use an isovariant convolutional network to process the retinal image, segment the optic disc and lesion areas, and output the corresponding segmentation results.

[0027] In practical implementation, the equivariant convolutional network achieves rotational equivariance based on Fourier series expansion. By parameterizing the two-dimensional filter through an enhanced set of Fourier basis functions, it can effectively capture the irregular and blurred features corresponding to ROP lesions (such as ridge lesions formed by retinal fibrovascular proliferation), avoiding the defect of traditional convolution being sensitive to rotational features. The segmentation result is output as a binary mask image (the pixel value of the optic disc / lesion region is 1, and the background is 0).

[0028] The partitioning module, connected to the segmentation module, is configured to: determine the center point and diameter of the optic disc based on the segmentation results of the optic disc, divide the retinal region based on the classification criteria for retinopathy of prematurity, and output the partitioning results.

[0029] In practice, the classification standard for retinopathy of prematurity adopts the International Classification of Retinopathy of Prematurity (ICROP), and the specific steps are as follows: ① The disk segmentation mask is fitted to a circle using the least squares method to determine the center point and radius r of the disk, and the diameter of the disk D=2r; ② Draw a circular area with the center of the optic disc as the center and 3D (i.e., 6r) as the radius, and define it as Zone I (a high-risk clinical area, where the condition is more serious when the lesion is in this area); ③ Draw a ring-shaped area with the center of the optic disc as the center, with an inner radius of 3D and an outer radius of R (R is the Euclidean distance from the center of the optic disc to the nasal serrated edge, which is determined by clinical experts by marking the fundus boundary). This area is defined as Zone II (excluding Zone I). ④The crescent-shaped region of the retina outside of zones I and II is defined as zone III; The partitioning results are output as a "color mask" (zone I is marked in red, zone II in blue, and zone III in green).

[0030] The staging module, which connects the segmentation module and the partitioning module, is configured to output the staging results of retinopathy of prematurity based on the segmentation and partitioning results of the lesion area.

[0031] In practice, staging is based on the ICROP criteria, focusing on stages 1-3 (because the incidence of ROP in stages 4-5 is extremely low, and clinical intervention can be achieved through early screening), specifically: ① Phase 1 corresponds to "the lesion in the segmentation result is a 'thin dividing line' and is located in any partition"; ② Stage 2 corresponds to "the lesions in the segmentation results are 'the boundary line evolves into a ridge', and most of them are located in Zone II"; ③ Stage 3 corresponds to "the lesion in the segmentation result is 'external retinal fibrovascular proliferation', and if it is located in Zone I, it should be given priority for intervention".

[0032] The phased results are output as "category labels" (e.g., "Phase 3 - Zone I").

[0033] The results generation module, which connects the partitioning module and the phased module, is configured to integrate the partitioning results and the phased results to generate a comprehensive diagnostic report.

[0034] In practice, the comprehensive diagnostic report includes three core components: ① Original retinal image and annotations (lesion areas are marked with white arrows); ②Segmentation-partition visualization results (images overlaid with visual disc / lesion mask and partition mask); ③ Written diagnostic conclusions (including staging results, zone results, and clinical recommendations, such as "Stage 3-Zone I: It is recommended to have a follow-up examination within 48 hours to assess whether laser treatment is necessary").

[0035] Based on the above, this embodiment overcomes the limitations of traditional manual diagnosis, which is characterized by strong subjectivity and low diagnostic consistency, through a modular collaborative design of image acquisition, segmentation, partitioning, staging, and result generation. It also addresses the single-task deficiencies of existing deep learning solutions. The modules are sequentially connected: the image acquisition module provides a high-quality data foundation for subsequent diagnosis; the segmentation module provides precise location benchmarks for partitioning and staging; and the partitioning and staging results are collaboratively integrated to generate a complete comprehensive diagnostic report. This approach aligns with the comprehensive diagnostic logic of clinical location and severity while avoiding information gaps from single-task results, significantly improving the objectivity, completeness, and efficiency of ROP diagnosis and providing a reliable basis for clinical treatment decisions.

[0036] As an optional implementation of this embodiment, the segmentation module includes a VGF-UNet network, whose encoder uses a VGG-16 network pre-trained on the imageNet dataset, and whose decoder uses equivariant convolutional layers based on Fourier series expansion.

[0037] In its implementation, the encoder (VGG-16 network) comprises 16 weight layers (13 convolutional layers + 3 fully connected layers). The convolutional layers use 3×3 kernels with a stride of 1 and padding of 1. The dimensions of the fully connected layers are 4096, 4096, and 1000, respectively. Since the ImageNet dataset contains 1.2 million natural images, the pre-trained weights have already learned common image features such as edges and textures. When transferring these features to the ROP image segmentation task, it avoids the overfitting problem caused by training from scratch due to the limited amount of labeled ROP data (only 2059 images).

[0038] Secondly, the decoder (equivariant convolutional layer) replaces the traditional U-Net convolutional layers, employing parameterized convolution based on Fourier series expansion (F-Conv), which uses an enhanced set of Fourier basis functions to apply to the two-dimensional continuous filter function. Parameterization is performed, and the expression is as follows: , in, For fixed basis functions (composed of combinations of trigonometric functions, such as...) , (for frequency parameters) These are learnable coefficient parameters (initialized randomly and updated via the Adam optimizer). Represents two-dimensional coordinates Grid, , For filter size, The grid size is specified. This design enables the decoder to be rotationally equivariant, adapting to the multi-directional characteristics of ROP lesions (such as horizontal / vertical boundaries).

[0039] Based on the above, the VGF-UNet network for the segmentation module combines a pre-trained VGG-16 encoder with a Fourier equivariant convolutional decoder. On the one hand, the encoder reuses imageNet pre-trained weights, eliminating the need for training from scratch on small sample ROP data, effectively avoiding overfitting, shortening the training cycle, and enhancing robustness to low-quality images. On the other hand, the equivariant convolutional layers of the decoder overcome the traditional convolutional layer's sensitivity to rotation, accurately capturing the irregular morphological features of ROP lesions. In addition, the U-shaped skip connections fuse high and low layer features, improving the segmentation accuracy of the optic disc and lesions, laying a reliable foundation for subsequent region division in the partitioning module and severity determination in the staging module.

[0040] Furthermore, the pre-trained VGG-16 network of the encoder can directly reuse general feature extraction capabilities without training from scratch on small sample ROP data, significantly shortening the training cycle (more than 60% shorter than the model without pre-training), while avoiding overfitting and improving the model's robustness to low-quality ROP images (such as images with noise interference). The isovariant convolutional layer of the decoder is parameterized through Fourier basis functions, overcoming the limitation of traditional convolution being translation-invariant but rotation-sensitive: traditional convolution requires data augmentation (rotation, flipping) to recognize rotated lesions (such as ridge lesions rotated 90°), while F-Conv can learn... Automatically adapting to lesions at different rotation angles, VGF-UNet improves the segmentation continuity of the optic disc and lesions without additional data augmentation (e.g., avoiding the fragmented segmentation of irregular lesions by traditional convolution). The U-shaped encoder-decoder structure of VGF-UNet, combined with skip connections, can fuse high-level semantic features extracted by the encoder (such as lesion category information) with low-level spatial features extracted by the decoder (such as lesion boundary information), further improving segmentation accuracy and providing more reliable optic disc / lesion location information for subsequent partitioning modules.

[0041] Furthermore, by adjusting the filter function Apply rotation transformation matrix Derive the rotated filter .

[0042] In practical implementation, the rotation transformation matrix The two-dimensional inverse rotation transformation matrix is ​​expressed as: , in The rotation angle (range 0°-360°, covering all possible rotation directions of ROP lesions); the rotated filter The derivation process is as follows: pass Performing coordinate transformation and substituting into the parametric formula, we get: This process is achieved through sharing. Parameters, to achieve different Consistent learning of the filter is used to avoid training the filter separately for each angle.

[0043] Based on the above, the equivariant convolutional layer achieves rotational equivariance of the filter through Fourier basis function parameterization and rotation transformation matrix: First, the enhanced Fourier basis function set can accurately represent any two-dimensional filter, avoiding aliasing effects caused by rotation and ensuring the integrity of segmentation details; second, the rotation transformation matrix enables the filter to synchronously adapt to the rotation angle of the lesion, solving the feature mismatch problem of traditional convolution for rotated lesions; third, by sharing learnable coefficient parameters, the number of model parameters is greatly reduced, balancing rotation adaptability and computational efficiency, enabling the model to efficiently handle multi-directional ROP lesions and improve segmentation robustness.

[0044] Furthermore, the enhanced Fourier basis function set possesses zero-error representation characteristics: it can accurately represent any two-dimensional filter without rotation (error approaches 0), and with rotation, it can preserve the frequency characteristics of the basis functions through coordinate transformation, avoiding the "aliasing effect" (such as blurred lesion edges) caused by rotation in traditional discrete convolution, thus ensuring the integrity of the segmentation results; the rotation transformation matrix... To make the filter possess rotational equivariance: when the lesion in the input image rotates... At that time, the filter Can rotate synchronously It maintains consistent feature responses, addressing the feature mismatch issue in traditional convolution for rotated lesions (e.g., traditional convolution might misclassify rotated ridge lesions as boundaries). It also shares features. The parameter design significantly reduces the number of model parameters: it eliminates the need to design independent filters for each rotation angle while ensuring rotation adaptability, enabling the model to run efficiently on low-configuration clinical hardware (such as ordinary GPUs), thus improving the feasibility of clinical deployment. Based on the aforementioned segmentation module, as a further optional implementation of this embodiment, the segmentation module employs a weighted Dice loss function during the training phase. The expression for the weighted Dice loss function is: , in, For predicted values, For the true value, The category weights are calculated based on the pixel proportions of the optic disc and the lesion area.

[0045] In practice, the predicted value is the probability that pixel i in the VGF-UNet output represents the optic disc / lesion, ranging from [0,1]. The ground truth value is the label of pixel i, where 1 represents the optic disc / lesion and 0 represents the background. When setting class weights, because the optic disc and lesions only account for 5%-10% of pixels in the ROP image, there is a serious class imbalance; therefore, a foreground weight is set. Background weight ,Right now (like )or (like ); This is a very small tolerance term used to avoid calculation anomalies caused by a zero denominator (such as calculations in normal images without lesions).

[0046] Based on the above, the weighted Dice loss function addresses the class imbalance problem between the optic disc, lesions, and background in ROP images. By assigning high weights to the foreground (optic disc, lesions), it forces the model to focus on learning a small number of foreground pixels, effectively improving the recall rate of the optic disc and lesions and avoiding the missed segmentation caused by the excessive focus on the background in traditional Dice loss. At the same time, the minimal tolerance term ensures the stability of the training process and avoids computational anomalies caused by extreme samples. This loss function fits the need for the integrity of the segmentation region of interest in medical images, improves segmentation accuracy, and reduces error propagation in subsequent partitioning and staging processes.

[0047] Based on the aforementioned segmentation module, as another further optional implementation of this embodiment, the system also includes a training module, configured to train the segmentation module using a staged transfer learning strategy. The staged transfer learning strategy includes, in sequence: Phase 1: Freeze the pre-trained weights of the encoder (VGG-16 network) of the VGF-UNet network (only retain the pre-trained parameters of the convolutional and fully connected layers, without updating them), and train only the isovariant convolutional layers of the decoder. The second stage: After the training loss converges, the encoder weights are unfrozen, and the VGF-UNet network is jointly trained end-to-end with a lower learning rate.

[0048] In practice, the training parameters for training the equivariant convolutional layer are set as follows: batch size = 8, initial learning rate = 10. -4 The optimizer is Adam, and the number of training epochs is 50. The purpose of this stage is to allow the decoder to focus on learning the specific segmentation features of the ROP image (such as the irregular shape of lesions) based on the general features extracted by the encoder, so as to avoid destroying the encoder's ability to extract general features in the early training.

[0049] Secondly, training loss convergence refers to the situation where the Dice loss on the validation set in the first stage does not decrease for five consecutive epochs (fluctuation less than 10). -6When the training loss converges, it is determined that the training loss has converged. In the second stage, a lower learning rate is used, which is one-tenth of that in the first stage. The training parameters are set as follows: training epochs = 30, with other parameters remaining unchanged. The purpose of this stage is to fine-tune the encoder parameters to adapt general features to the ROP image (such as adjusting the convolution kernel weights to focus more on retinal vascular features), while optimizing the feature fusion effect between the encoder and decoder.

[0050] Based on the above, the phased transfer learning strategy employs a two-stage operation: freezing the encoder to train the decoder and then unfreezing the end-to-end joint training. First, the encoder's pre-training weights are frozen initially to avoid compromising its general feature extraction capabilities, allowing the decoder to focus on learning the specific segmentation features of ROP images. Second, the encoder is unfrozen and fine-tuned later to adapt to ROP image features, optimizing the feature fusion effect between the encoder and decoder. This strategy effectively addresses the overfitting problem caused by limited ROP labeled data, improves model generalization ability, shortens the training cycle, ensures training stability, and guarantees that the segmentation module can still converge efficiently and output high-precision results even in small-sample scenarios.

[0051] As an optional implementation of this embodiment, the staging module is further configured to: perform pixel-level fusion of the segmentation results of the lesion area with the retinal image to generate an enhanced input image; the staging result is obtained by processing the enhanced input image using the EfficientNetB4 classification network.

[0052] In practice, pixel-level fusion uses a feature overlay method, fusing the lesion segmentation mask (1 representing lesion, 0 representing background) with the original retinal image according to the formula: , in, The original retinal image (RGB three channels, pixel values ​​normalized to [0,1]). For lesion segmentation mask, The fusion coefficients (determined through validation set tuning to balance the original image details with the lesion highlighting effect, such as the set value) are used. After fusion, the pixel value of the lesion area is amplified (e.g., a lesion area with an original pixel value of 0.5 becomes 0.5×(1+0.5)=0.75 after fusion), improving the contrast with the background and solving the problem of low contrast between the lesion and the background in the original image (e.g., small difference between exudate and retinal background).

[0053] Secondly, the EfficientNetB4 classification network adopts an MBConv inverted residual structure and optimizes the balance between accuracy and computation through multi-objective neural architecture search (NAS). Its input size is 380×380 pixels (the fused data will be used to further refine the classification). Adjust to this size while maintaining image aspect ratio), outputting 4-category probabilities (corresponding to normal, stage 1, stage 2, and stage 3), and taking the category with the highest probability as the stage classification result; network training uses the cross-entropy loss function, and the optimizer is AdamW (weight decay = 10). -4 (to prevent overfitting).

[0054] Based on the above, the pixel-level fusion + EfficientNetB4 classification design of the staging module has three main advantages: First, pixel-level fusion overlays the lesion segmentation results with the original image, improving the contrast between the lesion and the background, solving the problem of low recognition of subtle lesions in the original image, and reducing missed diagnoses. Second, EfficientNetB4, through MBConv structure and NAS optimization, reduces the number of parameters and computational load while ensuring classification accuracy, balancing diagnostic accuracy and real-time performance, and adapting to the needs of rapid clinical diagnosis. Third, the lightweight network structure is not prone to overfitting, adapting to small sample tasks in ROP staging, and ensuring stable prediction of each staging category.

[0055] Based on the aforementioned phased-out module configuration, as a further optional implementation of this embodiment, the phased-out module is further configured to: receive the partitioning results output by the partitioning module, and execute a partitioning boot prompt strategy; the prompt strategy includes: When the partitioning result corresponds to a circular region centered on the optic disc with a radius of 3 times the diameter of the optic disc (i.e., region I), or an annular region centered on the optic disc with a radius from the center point of the optic disc to the nasal serrated edge (i.e., region II), the partitioning mask of the corresponding region and the segmentation result of the lesion region are superimposed on the retinal image to generate an enhanced input image.

[0056] In practical implementation, the methods for generating the partition mask include: Partition I mask. (Pixels are 1 within area I and 0 outside), Area II mask (Pixels are 1 within zone II and 0 outside), directly output by the partitioning module.

[0057] Secondly, the formula for multi-mask overlay fusion of images in region I or region II is as follows: , in, Use the mask for zone I or zone II (select according to the partitioning results). and The weights are optimized using the validation set to avoid the partition mask overly covering the details of the original image, such as the value selection. , If the partitioning result is Zone III, the lesion mask is only merged according to the further configuration of the aforementioned staging module (because Zone III lesions have a low clinical risk, there is no need to highlight the partition).

[0058] In addition, the region determination logic automatically determines the region based on the partition label output by the partition module. If the label is “Region I” or “Region II”, multiple mask overlay is triggered; if it is “Region III”, single mask overlay is triggered.

[0059] Based on the above, the partition-guided prompting strategy addresses the diagnostic logic that zones I and II are high-risk lesion areas in clinical practice. By overlaying partition masks and lesion segmentation results in zone I and II scenarios to generate enhanced images, it guides EfficientNetB4 to prioritize high-risk area features, avoiding the model from overemphasizing low-risk zone III or background noise. This strategy enables the model to learn the correlation features between "partition location and lesion severity," improving the staging accuracy of lesions in zones I and II, while maintaining the rationality of single lesion mask fusion in zone III scenarios. It takes into account the diagnostic needs of different partition scenarios and reduces the interference of irrelevant areas on the staging results.

[0060] Building upon the aforementioned lesion mask fusion, as a further optional implementation of this embodiment, the EfficientNetB4 classification network employs a class-weighted method during the training phase to address the imbalance problem in staged data, with the weight of each stage category... The calculation formula is: , in, The total number of training samples, Installment categories The number of samples.

[0061] In practice, this means the normal training set contains 861 images, the first phase training set contains 180 images, the second phase training set contains 200 images, and the third phase training set contains 312 images. ). That is, This indicates that it is normal. ; Indicates period 1. ; Indicates period 2. ; Indicates 3 periods. .

[0062] Furthermore, the weights are applied to the cross-entropy loss function, resulting in the corrected loss function: ,in, The true label (one-hot encoded, such as the first-phase sample) ), Classes output by EfficientNetB4 The predicted probability.

[0063] Based on the above, the category-weighted method addresses the category imbalance problem in ROP staging data by assigning higher weights to staging categories with fewer samples, correcting the bias of the cross-entropy loss function, and forcing the model to focus on feature learning of the fewer sample categories. This effectively improves the recall rate of fewer sample stages (such as stage 1) and avoids missed diagnoses caused by the model's bias towards the more sample categories in traditional training. At the same time, the weighted processing balances the loss contribution of each staging category, reduces the performance differences between different staging categories, and ensures the consistency of the model's diagnosis of normal, stage 1, stage 2, and stage 3, meeting the clinical need to not miss any stage of the disease.

[0064] As an optional implementation of this embodiment, the retinal images include multi-view retinal images in a single examination; The staging module is further configured to: extract deep features from each multi-view retinal image after processing by the segmentation module; fuse the deep features of multiple multi-view retinal images using a proxy interactive attention mechanism to generate a comprehensive feature representation; and use the comprehensive feature representation, the segmentation results of the lesion region, and the partitioning results output by the partitioning module to output the staging result. Specifically, the comprehensive feature representation, along with the enhanced input image generated based on the segmentation results of the lesion region and the partitioning results output by the partitioning module, are input into the EfficientNetB4 classification network to output the staging result.

[0065] In practice, as mentioned above, multi-view retinal images refer to taking 6 images in a single examination (a common clinical number) to cover different angles of the premature infant's retina (such as nasal, temporal, superior, and inferior) to avoid missing local lesions (such as fibrovascular proliferation at the retinal margin) in a single image.

[0066] Secondly, deep feature extraction refers to extracting deep features from the last layer of the VGF-UNet encoder (the fc6 layer of VGG-16), with a dimension of 4096 (including semantic features of the optic disc / lesion, such as lesion type and optic disc size).

[0067] Furthermore, the process of integrating agent interaction attention mechanisms includes: ① Calculate the proxy vector: ,in Number of viewpoints (corresponding to, for example) ), Let be the deep features of the i-th image; ② Calculate attention weights: ,in Cosine similarity (used to measure) The similarity between the features and the proxy vector; a high similarity indicates that the viewpoint contains more important lesion features. ③ Generate a comprehensive feature representation: .

[0068] Furthermore, the multi-feature fusion input specifically involves... The vector is flattened to 4096 dimensions and concatenated with the features output from the 5th MBConv block of EfficientNetB4 (dimension 1792×12×12, flattened to 254016 dimensions) to obtain a fused feature of 258112 dimensions. This fused feature is then input into the fully connected layer of EfficientNetB4 for classification and outputs the classification result.

[0069] Based on the above, the staging module's multi-view feature fusion and multi-modal input design achieves two main benefits: First, multi-view images cover the entire retinal area, avoiding localized missed diagnoses from single images. Combined with a proxy interaction attention mechanism, it automatically identifies and focuses on key perspectives with significant lesion features, suppressing interference from low-quality or lesion-free perspectives and improving feature effectiveness. Second, the complementary fusion of deep semantic features and enhanced image visual features achieves multi-dimensional information integration of lesion type, location, and morphology, resulting in more accurate identification of complex lesions compared to single feature input. This design is suitable for real-world clinical multi-view examination scenarios, significantly improving the comprehensiveness and accuracy of staging results, especially reducing the risk of misdiagnosis and missed diagnosis of lesions with differences across multiple perspectives.

[0070] In the embodiments provided by this invention, it should be understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, code, or any suitable combination thereof. For hardware implementation, the processor can be implemented in one or more of the following: application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, other electronic units designed to implement the functions described herein, or combinations thereof. For software implementation, some or all of the processes of the embodiments can be performed by a computer program instructing the associated hardware. During implementation, the program can be stored in a computer-readable storage medium or transmitted as one or more instructions or code on a computer-readable storage medium. Computer-readable storage media include computer storage media and communication media, wherein communication media include any medium that facilitates the transmission of a computer program from one place to another. Storage media can be any available medium accessible to a computer. Computer-readable storage media can include, but are not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code having the form of instructions or data structures and accessible to a computer.

[0071] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A diagnostic system for retinopathy of prematurity based on isovariant convolutional networks, characterized in that, include: The image acquisition module is configured to acquire retinal images of premature infants. The segmentation module, connected to the image acquisition module, is configured to: process the retinal image using an isovariant convolutional network to segment the optic disc and lesion region, and output the corresponding segmentation results; The partitioning module, connected to the segmentation module, is configured to: determine the center point and diameter of the optic disc based on the segmentation result of the optic disc, divide the retinal region based on the classification criteria for retinopathy of prematurity, and output the partitioning result; The staging module, which connects the segmentation module and the partitioning module, is configured to output the staging results of retinopathy of prematurity based on the segmentation results and the partitioning results of the lesion area. The result generation module, which connects the partitioning module and the phasing module, is configured to integrate the partitioning results and the phasing results to generate a comprehensive diagnostic report.

2. The diagnostic system for retinopathy of prematurity based on isovariant convolutional networks according to claim 1, characterized in that, The segmentation module includes a VGF-UNet network, whose encoder uses a VGG-16 network pre-trained on the ImageNet dataset, and whose decoder uses equivariant convolutional layers based on Fourier series expansion.

3. The diagnostic system for retinopathy of prematurity based on isovariant convolutional networks according to claim 2, characterized in that, The equivariant convolutional layer achieves rotational equivariance in the following manner: By enhancing the Fourier basis function set for the two-dimensional continuous filter function Parameterization is performed, and the expression is as follows: , in, For fixed basis functions, For learnable coefficient parameters, Represents two-dimensional coordinates Grid; By adjusting the filter function Apply rotation transformation matrix Derive the rotated filter .

4. The diagnostic system for retinopathy of prematurity based on isovariant convolutional networks according to claim 2 or 3, characterized in that, The segmentation module employs a weighted Dice loss function during the training phase. The expression for the weighted Dice loss function is as follows: , in, For predicted values, For the true value, The category weights are calculated based on the pixel proportions of the optic disc and the lesion area. This is a term with minimal tolerance.

5. The diagnostic system for retinopathy of prematurity based on isovariant convolutional networks according to claim 2, characterized in that, It also includes a training module, which is configured to train the segmentation module using a phased transfer learning strategy; wherein the phased transfer learning strategy includes, in sequence: Freeze the pre-trained weights of the encoder of the VGF-UNet network and train only the equivariant convolutional layers of the decoder; After the training loss converges, the encoder weights are unfrozen, and the VGF-UNet network is jointly trained end-to-end with a lower learning rate.

6. The diagnostic system for retinopathy of prematurity based on isovariant convolutional networks according to claim 1, characterized in that, The staging module is further configured to: perform pixel-level fusion of the segmentation result of the lesion area with the retinal image to generate an enhanced input image; the staging result is obtained by processing the enhanced input image using the EfficientNetB4 classification network.

7. The diagnostic system for retinopathy of prematurity based on isovariant convolutional networks according to claim 6, characterized in that, The phased module is further configured to: receive the partitioning results output by the partitioning module, and execute a partitioning guidance prompt strategy; the prompt strategy includes: When the partitioning result corresponds to a circular region centered on the optic disc with a radius of 3 times the diameter of the optic disc, or to an annular region centered on the optic disc with a radius from the center point of the optic disc to the serrated edge on the nasal side, the partitioning mask of the corresponding region and the segmentation result of the lesion region are superimposed on the retinal image to generate the enhanced input image.

8. The diagnostic system for retinopathy of prematurity based on isovariant convolutional networks according to claim 6 or 7, characterized in that, The EfficientNetB4 classification network employs a class-weighted method during training to address the imbalance problem in periodic data, with each periodic category having a weighted weight. The calculation formula is: , in, The total number of training samples, Installment categories The number of samples.

9. The diagnostic system for retinopathy of prematurity based on isovariant convolutional networks according to claim 1, characterized in that, The partitioning module outputs the partitioning results through the following steps: The segmentation result of the visual disk is fitted into a circle to determine the center point and radius of the visual disk; A circular area is drawn with the center point of the viewing disk as the center and a radius of 3 times the diameter of the viewing disk as the radius, which is designated as Zone I; A ring-shaped area is drawn with the center point of the optic disc as the center and the distance from the center point of the optic disc to the serrated edge on the nasal side as the radius, which is designated as area II. Area II does not include area I. The remaining retinal area outside of Zone I and Zone II is designated as Zone III.

10. The diagnostic system for retinopathy of prematurity based on isovariant convolutional networks according to claim 1, characterized in that, The retinal images include multi-view retinal images from a single examination. The staging module is further configured to: extract deep features from each of the multi-view retinal images after processing by the segmentation module, fuse the deep features of multiple multi-view retinal images using a proxy interactive attention mechanism, generate a comprehensive feature representation, and use the comprehensive feature representation, the segmentation result of the lesion area, and the partitioning result output by the partitioning module to output the staging result.