Method for detecting and partitioning retinopathy of prematurity (ROP) lesions based on fundus color photograph

Abstract

The application provides an ROP lesion detection and partition positioning method based on fundus color photography. The method uses a double-branch cooperative detection architecture to independently process lesion area positioning and anatomical area positioning tasks to avoid abnormal problems in the proportion of the macula and the optic disc position caused by the difference in the retinal development of premature infants, and to ensure that the positioning of the circular I area accurately reflects the true anatomical structure. The feature fusion module strengthens the lesion area feature response and screens key feature channels to improve the target recognition capability in complex fundus images. The multi-shape joint loss function forces the network to learn the spatial topological relationship between the lesion area and the circular anatomical area to eliminate feature redundancy and improve detection sensitivity. Thus, the problem of low accuracy of ROP lesion detection and partition positioning is solved.

Description

Technical Field

[0001] This application relates to the interdisciplinary field of medical image processing and artificial intelligence, and in particular to a method for ROP lesion detection and regional localization based on fundus color images. Background Technology

[0002] Retinopathy of Prematurity (ROP) is a proliferative retinal angiopathy in premature and low birth weight infants. Its progression can be divided into five stages, with stages three and above requiring urgent intervention to prevent retinal detachment and blindness. In clinical diagnosis, ophthalmologists rely on manual assessment using binocular indirect ophthalmoscopy or fundus photography, which is highly subjective, inefficient, and lacks precise localization. Automated ROP detection technology focuses on staging and classification, but lacks definition of lesion area localization and anatomically related regions.

[0003] Specifically, the ROP automatic detection technology detects the coordinates of the macula center and the optic disc boundary in the fundus image, calculates the distance between the optic disc and the macula based on a predefined proportional relationship, draws a circular area with a fixed radius centered on the macula as circular zone I, marks the extent of the lesion area with a rectangular bounding box, and identifies abnormal vascular proliferation areas using a sliding window scan or feature classifier to output the lesion location and staging results marked by the rectangular box.

[0004] However, differences in retinal development in premature infants lead to deviations in the actual position of the macula and optic disc from the standard ratio, resulting in discrepancies between the indirectly calculated circular region I and the actual lesion area. When the macula or optic disc is obscured by blood vessels or the image is blurred, the localization process fails. The rectangular bounding box cannot match the irregular shape of ROP lesions, resulting in incomplete coverage and easy inclusion of normal tissue. The separation of circular region I localization from lesion detection tasks ignores spatial correlation, leading to feature redundancy. Blood vessel obscuration and image noise further reduce the robustness of recognition in complex scenes. In summary, the accuracy of ROP lesion detection and regional localization is low. Summary of the Invention

[0005] This application provides a method for ROP lesion detection and regional localization based on fundus color imaging to solve the problem of low accuracy in ROP lesion detection and regional localization.

[0006] This application provides a method for ROP lesion detection and regional localization based on fundus color imaging, including:

[0007] Obtain a training sample set, which includes multiple RGB images of the fundus enhanced with brightness.

[0008] A localization network is constructed using the training sample set, and the localization network includes at least a feature fusion module and a dual-branch detection module.

[0009] The image to be located is input into the feature fusion module to output a multi-scale feature map;

[0010] The multi-scale feature map is input into the dual-branch detection module to output a rectangular detection box through the first branch and a circular detection box through the second branch.

[0011] Based on the rectangular and circular detection boxes, the positioning result is obtained;

[0012] Based on the rectangular and circular detection boxes, a multi-shape joint loss function is constructed, and the localization network is updated using the loss value calculated by the multi-shape joint loss function.

[0013] In some feasible embodiments, obtaining the training sample set includes:

[0014] Obtain RGB images of the fundus;

[0015] Using the original brightness value of the fundus RGB image as input, a brightened image is output through nonlinear transformation;

[0016] Set the brightness adjustment intensity parameter to generate an enhanced sample image based on the brightened image;

[0017] The enhanced sample images are integrated into a training sample set.

[0018] In some feasible embodiments, inputting the image to be located to the feature extraction unit to output a feature map includes:

[0019] Obtain the image to be located;

[0020] Initial feature extraction is performed on the image to be located through convolutional layers, and a first feature map is generated through normalization and nonlinear activation functions;

[0021] Perform a downsampling operation on the first feature map to output a downsampled feature map;

[0022] The downsampled feature map is input into multiple cascaded residual stages to perform hierarchical feature extraction, so as to output multiple residual feature maps with different sizes.

[0023] The feature map is determined based on the residual feature map.

[0024] In some feasible embodiments, the first branch includes a feature enhancement module, which includes a multi-scale feature fusion unit, a spatial attention unit, and a channel attention unit;

[0025] The method further includes:

[0026] The multi-scale feature fusion unit performs cross-scale fusion operations on multiple residual feature maps and generates multi-scale fused feature maps by upsampling and lateral connection.

[0027] The spatial attention unit applies spatial attention weights to the multi-scale fused feature map, and outputs a spatially enhanced feature map.

[0028] The channel attention unit applies channel attention weights to the multi-scale fused feature map, and outputs a channel-enhanced feature map.

[0029] In some feasible embodiments, the first branch includes a first classification branch and a first regression branch; the method further includes:

[0030] The spatial dimension of the residual feature map is adjusted by a convolution kernel to generate a feature map of the first size.

[0031] In the first classification branch, feature compression is performed by adjusting the convolutional layer through continuous channels to output a probability map of the first target category;

[0032] In the first regression branch, feature transformation is performed by adjusting the convolutional layer through continuous channels to output a rectangular box boundary coordinate parameter map;

[0033] Based on the probability of the first target category and the coordinate parameters of the bounding box, a candidate rectangular detection box is output.

[0034] In some feasible embodiments, the second branch includes a second classification branch and a second regression branch; the method further includes:

[0035] The spatial dimension of the residual feature map is adjusted by a convolution kernel to generate a second-size feature map;

[0036] In the second classification branch, feature compression is performed by adjusting the convolutional layer through continuous channels to output the probability of the second target class;

[0037] In the second regression branch, feature transformation is performed by adjusting the convolutional layer through continuous channels to output the center coordinates and radius offset of the circular box;

[0038] Based on the probability of the second target category, the coordinates of the center of the circular box, and the radius offset, candidate circular detection boxes are output.

[0039] In some feasible embodiments, the method further includes:

[0040] Based on the rectangular and circular detection boxes, calculate the distance intersection-union ratio (DIU) loss term for the rectangular region and the distance intersection-union ratio (DIU) loss term for the circular region.

[0041] A multi-shape joint loss function is constructed based on the rectangular region distance intersection-union ratio loss term and the circular region distance intersection-union ratio loss term.

[0042] The localization network is updated using the loss value calculated by the multi-shape joint loss function.

[0043] In some feasible embodiments, constructing the multi-shape joint loss function includes:

[0044] Obtain candidate rectangular detection boxes;

[0045] Based on the coordinates of the candidate rectangular detection box and the coordinates of the rectangular labeled area, calculate the area of ​​the first intersecting region and the area of ​​the first merging region;

[0046] The intersection-union ratio is generated based on the ratio of the area of ​​the first intersecting region to the area of ​​the first merging region.

[0047] Calculate the Euclidean distance between the candidate rectangular detection box and the center point of the labeled area;

[0048] Calculate the diagonal length of the smallest bounding rectangle containing the candidate rectangular detection box and the rectangular annotation area;

[0049] The intersection-union ratio, the Euclidean distance of the center point, and the diagonal length are combined to generate a rectangular region distance intersection-union ratio loss term;

[0050] Based on the distance intersection-union ratio (DIU) loss term for the rectangular region and the distance intersection-union ratio (DIU) loss term for the circular region, a multi-shape joint loss function is constructed.

[0051] In some feasible embodiments, the method further includes:

[0052] Obtain candidate circular detection boxes;

[0053] Based on the center coordinates and radius parameters of the candidate circular detection box and the circular annotation area, calculate the Euclidean distance between the center of the candidate circular detection box and the center of the circular annotation area.

[0054] Based on the radius parameter and the Euclidean distance between the center of the circle, the projection component parameters of the intersecting common chord are determined;

[0055] Calculate the area of ​​the second intersection region and the area of ​​the second merger region based on the projection component parameters and the radius parameters;

[0056] The intersection-union ratio is generated based on the ratio of the area of ​​the second intersecting region to the area of ​​the second merging region.

[0057] By fusing the geometric penalty term related to the intersection-union ratio and the distance to the center of the circle, a circular region distance intersection-union ratio loss term is generated.

[0058] In some feasible embodiments, the first branch includes a feature enhancement module, which includes a multi-scale feature fusion unit, a spatial attention unit, and a channel attention unit;

[0059] The step of inputting the multi-scale feature map into the dual-branch detection module to output a rectangular detection box through the first branch and a circular detection box through the second branch includes:

[0060] The candidate rectangular detection boxes are processed by the multi-scale feature fusion unit, spatial attention unit, and channel attention unit to output rectangular detection boxes.

[0061] as well as,

[0062] The candidate circular detection boxes are processed by the multi-scale feature fusion unit, spatial attention unit, and channel attention unit to output circular detection boxes.

[0063] As can be seen from the above technical solutions, this application provides a method for ROP lesion detection and regional localization based on fundus color images. The method includes: acquiring a training sample set, which includes multiple fundus RGB images enhanced with brightness; constructing a localization network using the training sample set, which includes at least a feature fusion module and a dual-branch detection module; inputting the image to be localized into the feature fusion module to output a multi-scale feature map; inputting the multi-scale feature map into the dual-branch detection module to output a rectangular detection box through a first branch and a circular detection box through a second branch; and obtaining a localization result based on the rectangular and circular detection boxes.

[0064] The proposed method employs a dual-branch collaborative detection architecture to independently handle lesion region localization and anatomical region localization tasks. This avoids the problem of abnormal macular and optic disc position ratios caused by differences in retinal development in premature infants, ensuring that the localization of the circular region I accurately reflects the true anatomical structure. A feature fusion module enhances the feature response of the lesion region and filters key feature channels, jointly improving target recognition capabilities in complex fundus images. A multi-shape joint loss function forces the network to learn the spatial topological relationship between the lesion region and the circular anatomical region, eliminating feature redundancy and improving detection sensitivity. This ultimately solves the problem of low accuracy in ROP lesion detection and regional localization. Attached Figure Description

[0065] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0066] Figure 1A schematic diagram illustrating the process of locating the macular region of a fundus image using an edge detection algorithm, as provided in an embodiment of this application;

[0067] Figure 2 A schematic flowchart illustrating the method for ROP lesion detection and zonal localization based on fundus color imaging provided in this application embodiment;

[0068] Figure 3 This is a schematic diagram of the structure of the positioning network provided in an embodiment of this application;

[0069] Figure 4 A schematic diagram illustrating the positioning results provided in an embodiment of this application;

[0070] Figure 5 A schematic diagram of a rectangle for calculating the intersection-union ratio of rectangles provided in an embodiment of this application;

[0071] Figure 6 A rectangular schematic diagram for calculating the intersection-union ratio of circles provided in an embodiment of this application;

[0072] Figure 7 This is a schematic diagram of the location network update process provided in an embodiment of this application. Detailed Implementation

[0073] The embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described below do not represent all embodiments consistent with this application. They are merely examples of systems and methods consistent with some aspects of this application as detailed in the claims.

[0074] The progression of ROP includes multiple stages. Stage 1: A white demarcation line appears at the junction of the vascular and avascular areas of the retina. This is mild and can disappear spontaneously in about 80% of cases. Stage 2: The demarcation line thickens and becomes raised, resembling a ridge. This is moderate and can disappear spontaneously in some cases. Stage 3: Abnormal neovascularization and fibrous tissue proliferation threshold lesions appear at the ridge. Stage 4: Partial retinal detachment (divided into 4A / 4B subtypes). Stage 5: Total retinal detachment.

[0075] The lesion locations include zones I, II, and III. Zone I (the most central) is a circular area centered on the optic disc with a radius of twice the distance between the optic disc and the macula (the lesion is most dangerous in this area). Zone II is a ring-shaped area from the outer edge of zone I to the nasal ora serrata. Zone III is the remaining temporal peripheral retina outside zone II (the lowest risk).

[0076] Regarding the distinction between Zone II and Zone III, existing equipment has difficulty capturing the location of the serrated edge. Therefore, existing images can only distinguish Zone I and Zone II. In this embodiment, Zone I is distinguished, and the remaining area is Zone II.

[0077] The closer the ROP is to the optic disc, the more dangerous it is. Because different devices are used for shooting, the scale of the reference will be different if the distance is calculated directly. Therefore, it is necessary to use partitioning to make the reference twice the distance to the center of the optic disc macular center.

[0078] like Figure 1 As shown, existing technologies locate the macular region of fundus images through edge detection algorithms and identify the coordinates of the center point by combining color features (the macula is dark red) with morphological filtering. This process relies on manually setting thresholds to segment the image and requires that the macular region is not obscured by blood vessels and has clear boundaries. The detection results serve as the reference points for subsequent geometric calculations, and their accuracy affects the positioning accuracy of circular partitions.

[0079] Specifically, the optic disc boundary is determined using ellipse fitting technology. First, the candidate optic disc area is segmented by a brightness threshold, and then the ellipse boundary parameters (center point, major axis, minor axis) are fitted using the least squares method. This process assumes that the optic disc is a standard ellipse with significant contrast to the background, and has limited adaptability to uneven image exposure or optic disc tilt.

[0080] Based on clinical experience, a fixed ratio of optic disc to macular distance is set (usually 3 times the optic disc diameter). This ratio serves as the core parameter for geometric calculations, used to derive the radius of the circular zone I. This method ignores the differences in retinal development in preterm infants, forcibly matching individual anatomical variations to standardized parameters.

[0081] Circular region I is drawn with the center of the macula as the center and a fixed radius. The boundary of the circular region does not adjust with changes in fundus curvature, resulting in compression and deformation of the temporal region of the retina and insufficient coverage of the nasal region. This defect is particularly significant in the fundus of highly myopic eyes.

[0082] The lesion area was uniformly marked with a rectangular bounding box. However, the right angle characteristic of the rectangular bounding box is incompatible with the morphology of ROP lesions. Specifically, exudative lesions are distributed radially, and the rectangular bounding box includes a large amount of normal tissue. New blood vessels grow in a dendritic manner, and the rectangular bounding box cannot fit the bifurcation structure, resulting in false positive markings and insufficient lesion coverage.

[0083] A fixed-step sliding window is used to traverse fundus images, and a shallow classifier (such as SVM) is used to determine whether there are lesions within the window. However, the fixed window size cannot adapt to the size differences between microaneurysms and neovascularizations; if the step size is too large, small lesions will be missed, and if the step size is too small, the computational efficiency will be low; the classifier relies on manual feature engineering and has weak generalization ability.

[0084] Existing technologies treat the localization of the circular region I and lesion detection as separate tasks. The anatomical division is completed first, and then the lesion is detected within the division. There is no feature sharing mechanism between the two, which leads to the lesion detection ignoring anatomical constraints and the localization result exceeding the actual division range. Features are extracted repeatedly, resulting in wasted computational resources and increased noise sensitivity.

[0085] Detection methods that rely on low-level features (color / edges) suffer from problems such as macular regions blending into the background when the image is underexposed, blurred optic disc boundaries when there are motion artifacts, and confusion of lesion features when there is blood vessel projection. Traditional methods lack feature enhancement mechanisms and their accuracy drops sharply in noisy environments.

[0086] This application provides a method for ROP lesion detection and zonal localization based on fundus color imaging. The method outputs results synchronously through parallel rectangular or circular branches, ensuring that the failure of one target detection does not affect another. The lesion area is automatically constrained within anatomical zonal regions, resolving the issues of sequential breakdown and spatial fragmentation in existing technologies. Furthermore, a feature fusion module integrates microvascular details and lesion semantics, suppresses responses in areas obscured by blood vessels, and strengthens pathological feature channels to overcome detection bottlenecks in noisy environments.

[0087] like Figure 2 As shown, the method includes the following steps:

[0088] S100: Obtain the training sample set.

[0089] The training sample set includes multiple RGB images of the fundus enhanced with increased brightness, as well as multiple retinal image samples. Each sample undergoes a non-linear transformation to adjust pixel value distribution, enhancing illumination robustness. This sample set serves as the data foundation for model training, and its images cover different pathological stages and imaging conditions, providing diverse inputs for the localization network.

[0090] S200: Construct a localization network using a training sample set.

[0091] The localization network includes at least a feature fusion module and a dual-branch detection module. The feature fusion module is used to extract multi-scale visual features, and the dual-branch detection module simultaneously processes lesion localization and anatomical partitioning tasks.

[0092] The localization network learns retinal feature representations through end-to-end training to output more accurate localization results.

[0093] S300: Input the image to be located into the feature fusion module to output a multi-scale feature map.

[0094] The feature fusion module integrates local details and global semantics from retinal images, outputting a spatially hierarchical feature representation. The multi-scale feature map is the output representation of the feature fusion module, including visual features at different levels of abstraction. Shallow features preserve details such as vascular texture and exudate edges, while deep features encode the spatial association between lesion areas and anatomical landmarks. This multi-scale characteristic adapts to the detection needs of pathological structures of different sizes.

[0095] S400: Input the multi-scale feature map into the dual-branch detection module to output a rectangular detection box through the first branch and a circular detection box through the second branch.

[0096] In one implementation, the processes of the first branch outputting a rectangular detection box and the second branch outputting a circular detection box are executed synchronously. At this time, multi-scale feature maps are simultaneously input into both branches of the dual-branch detection module. The first branch outputs a rectangular detection box, and the second branch outputs a circular detection box. The two branches share the same input features but operate independently, ensuring the task specificity of lesion localization and anatomical region localization.

[0097] In another implementation, rectangular or circular detection boxes can be output separately. That is, the process of the first branch outputting rectangular detection boxes and the second branch outputting circular detection boxes are executed asynchronously. For example, during the process of the first branch outputting rectangular detection boxes, the multi-scale feature map is input to the second branch.

[0098] However, it is understandable that the output rectangular and / or circular detection boxes will remain unaffected regardless of the implementation method.

[0099] Among them, the rectangular detection box is the spatial localization result of the first branch output. The boundary of the lesion area is defined by the coordinates of the upper left corner and the lower right corner. This output represents the spatial distribution range of abnormal retinal tissue, covering pathological structures such as microaneurysm clusters and neovascularization areas, and provides geometric basis for lesion staging.

[0100] The circular detection box is the anatomical partitioning result output by the second branch. It defines key anatomical regions (such as circular region I) with the coordinates of the center and the radius. This output describes the spatial relationship between the optic disc and the macular axis, providing an anatomical reference framework for lesion localization.

[0101] The dual-branch detection module shares input features but outputs independently. The rectangular branch focuses on the boundary localization of the lesion area, while the circular branch focuses on the spatial partitioning of anatomical landmarks. The two are optimized by establishing a loss function, avoiding the accumulation of errors in the traditional serial process.

[0102] S500: Based on rectangular and circular detection boxes, the positioning results are obtained.

[0103] Specifically, non-maximum suppression is performed on the set of rectangular detection boxes and the set of circular detection boxes, and the final retained rectangular boxes (lesion areas) and circular boxes (anatomical regions) are output.

[0104] Non-maximum suppression (NMS) is performed on the feature vectors of candidate regions to eliminate redundant detection boxes with excessive overlap. NMS performs spatial redundancy elimination on the feature vectors of candidate regions, and the optimal localization result is selected based on confidence ranking and intersection-union (IU) threshold to ensure that the output detection boxes meet the requirements of spatial uniqueness and complete pathological coverage.

[0105] The validity of the candidate region is verified based on the preset spatial constraints, and the final positioning result is output. In this embodiment, the positioning result includes both the rectangular boundary of the lesion region and the circular parameters of the anatomically associated region.

[0106] For example, when a new fundus image of a premature infant is input, the trained localization network extracts multi-scale features through a feature fusion module. In the dual-branch detection module, the first branch predicts three candidate rectangular boxes (confidence levels of 0.92, 0.85, and 0.78) based on high-resolution features, and the second branch predicts two candidate circular boxes (confidence levels of 0.95 and 0.82) based on semantic features. After non-maximum suppression, two rectangular boxes (marking temporal and nasal lesions) and one circular box (marking the area of ​​region I) are retained to form the final localization result.

[0107] like Figure 4 The localization results are shown in figures 4a and 4b. 4a is the image to be localized, and 4b is the processed image. The circular detection box represents the circular region I, the posterior pole anatomical region, centered on the optic disc, with a radius equal to 2 × the optic disc-macula distance. This marks the high-risk area (region I) for ROP, where lesions require urgent intervention. The rectangular detection box marks the ROP lesion area, selecting areas of abnormal vascular proliferation or avascularity (such as neovascularization in stage III lesions). Position verification checks whether the lesion has invaded the high-risk region I (the spatial relationship between the red box and the yellow circle). The optic disc, the central dark area where retinal nerve fibers converge, has a diameter of approximately 1.5 mm and serves as the spatial localization reference point.

[0108] The proposed method employs an end-to-end deep learning framework to achieve automatic detection and localization of ROP lesions. First, the input fundus image is preprocessed, and multi-scale features are extracted through a feature fusion module. These multi-scale features are then input into a dual-branch detection module, which simultaneously outputs rectangular detection boxes for the lesion region and circular detection boxes for the anatomically related region. A multi-shape joint loss function constructed based on the dual outputs drives network optimization, and finally, precise localization results are obtained through candidate region selection.

[0109] Through a dual-branch collaborative detection architecture, the tasks of lesion area localization and anatomical area localization are handled independently. The circular detection box of the anatomically related area is directly generated by the network learning, rather than relying on geometric calculations for indirect derivation in traditional methods. This alleviates the problem of abnormal macular and optic disc position ratio caused by the differences in retinal development in premature infants, and ensures that the localization of circular area I accurately reflects the real anatomical structure.

[0110] In some embodiments, obtaining the training sample set includes:

[0111] Obtain RGB images of the fundus;

[0112] Using the original brightness value of the fundus RGB image as input, a brightened image is output through nonlinear transformation;

[0113] Set the brightness adjustment intensity parameter to generate an enhanced sample image based on the brightened image;

[0114] The enhanced sample images are integrated into a training sample set.

[0115] To obtain a training sample set, stored or real-time raw fundus RGB images can be accessed. Fundus RGB images are color images of the retina acquired through fundus camera equipment, including visual information from the red, green, and blue spectral channels. Fundus RGB images record key anatomical structures such as retinal vessel distribution, optic disc morphology, and the macula, while raw pixel values ​​characterize tissue reflectance properties under illumination conditions.

[0116] After reading the pixel matrix of the RGB image of the fundus, the illumination conditions are optimized by reconstructing the pixel value distribution through a mathematical mapping operation on the brightness values ​​of the image pixels using a nonlinear transformation.

[0117] In some embodiments, the nonlinear transformation is achieved through a power function relation:

[0118]

[0119] Where x is the input pixel value (0-255) and γ is the intensity parameter.

[0120] The transformation curve has a steep slope in the low brightness area (significantly brightening the dark areas) and is close to linear in the high brightness area (preserving the details of the bright areas). The RGB three channels are processed simultaneously to maintain color balance.

[0121] The brightened image is output after nonlinear transformation. The brightened image is the output result of nonlinear transformation, and its pixel value distribution is nonlinearly shifted relative to the original image. The brightened image retains all the anatomical details of the retina (such as microvascular bifurcation points and hemorrhage patch edges), but the visual discernibility of dark areas (such as the fovea of ​​the macula) is significantly enhanced, and the overexposure phenomenon in bright areas (such as optic disc reflection) is suppressed.

[0122] To ensure the illumination diversity of the generated samples, a brightness adjustment intensity parameter γ is set or adjusted. This parameter controls the degree of nonlinear transformation and ranges from [0.3, 0.7]. This parameter determines the shape of the mapping curve. When γ = 0.3, the curve is steep, producing a high-contrast brightening effect; when γ = 0.7, the curve is gentle, close to the original brightness distribution. Each enhancement operation randomly selects a γ value from a preset range to ensure the illumination diversity of the generated samples.

[0123] After adjusting the brightness and intensity parameters, the generated enhanced sample image retains the anatomical structure intact, meaning that the blood vessel direction and lesion boundary are consistent with the original image; the illumination distribution is reconstructed, meaning that the visibility of dark details is improved and there is no loss of information in bright details; and the pathological features are enhanced, meaning that the contrast between the bright areas of exudate and the background is increased.

[0124] After performing the above processing on a preset number of fundus RGB images, multiple enhanced sample images are generated. These multiple enhanced sample images are then integrated to generate a training sample set.

[0125] Nonlinear transformation overcomes common lighting defects in fundus images by reconstructing pixel value distribution curves, brightening dark areas, improving the visibility of capillaries in the macular region, and avoiding missed detection of microaneurysms; it also suppresses bright areas, preserves details in the reflective areas of the optic disc, and prevents overexposure of lesion information.

[0126] The random intensity parameter enables the model to adapt to diverse lighting scenarios. During the training phase, it is exposed to samples generated with different γ values ​​and learns to extract pathological features that are constant under lighting conditions. In practical applications, it can stably handle exposure variations caused by differences in equipment.

[0127] A localization network is constructed using a training sample set, where, for example... Figure 3 As shown, the localization network includes a feature fusion module and a dual-branch detection module. In some embodiments, inputting the image to be localized into the feature fusion module to output a multi-scale feature map includes:

[0128] Obtain the image to be located;

[0129] Initial feature extraction is performed on the image to be located through convolutional layers, and a first feature map is generated through normalization and nonlinear activation functions;

[0130] Perform a downsampling operation on the first feature map to output a downsampled feature map;

[0131] The downsampled feature map is input into multiple cascaded residual stages to perform hierarchical feature extraction, so as to output multiple residual feature maps with different sizes.

[0132] The multi-scale feature map is determined based on the residual feature map.

[0133] The image to be localized is an RGB image of the retina acquired through a fundus camera, including complete anatomical information of the optic disc, macula, and vascular network. Clinical inputs include samples from different pathological stages, such as typical ROP manifestations like stage I peripheral avascular zone and stage III neovascularization.

[0134] The image to be localized is the input image for ROP lesion and regional localization. This image is an unprocessed raw RGB color photo of the fundus, including retinal structural information to be analyzed.

[0135] In some embodiments, the feature fusion module consists of a 7×7 convolutional layer, a max-pooling layer, residual stage 1, residual stage 2, residual stage 3, and residual stage 4. The convolutional layer is the computational unit in the localization network that performs initial feature extraction. It consists of a set of learnable convolutional kernels and extracts basic visual features (such as edges, textures, and spots) of local regions by performing sliding window-style multiplication and addition operations on the input image or feature map. This transforms the raw pixel information into a more expressive low-level feature representation, outputting a feature map with a size of 512×512×64.

[0136] The max pooling layer is used for downsampling, and the output feature map has a size of 256×256×64. The residual stage 1, residual stage 2, residual stage 3, and residual stage 4 allow the network to learn the residual (i.e., the amount of change) between the input features and the output features. By introducing skip connections, the input features are directly passed to the output and added to the features after convolution transformation, which effectively alleviates the gradient vanishing problem in deep network training and enables the network to be built deeper to extract more abstract and higher-level features.

[0137] Among them, residual stage 1 outputs a feature map with a size of 256×256×256, residual stage 2 outputs a feature map with a size of 128×128×512, residual stage 3 outputs a feature map with a size of 64×64×1024, and residual stage 4 outputs a feature map with a size of 128×128×256.

[0138] For example, the image to be localized is input into a convolutional layer, where each convolutional kernel performs a sliding convolution operation on the image. Each kernel is used to detect specific types of local patterns in the image (such as edges in a specific direction or textures of a specific shape). Through this local awareness and weight sharing approach, the convolutional layer effectively extracts basic low-level visual features from the original pixels. After the convolution operation is complete, the output is an intermediate result whose spatial size may be the same as or slightly smaller than the input image (depending on the stride and padding), but whose number of channels equals the number of convolutional kernels.

[0139] To improve training stability and efficiency, normalization is applied to the output of the convolutional layers. Specifically, for all intermediate feature maps generated within a batch, the mean and standard deviation of all spatial location data in each channel are calculated. Then, the mean of each channel's data is subtracted and divided by its standard deviation to achieve data centering and scaling. Finally, two learnable parameters (scaling factor γ and offset factor β) are introduced to perform an affine transformation on the normalized data, giving the network the ability to recover the original distribution or learn a new distribution, thereby mitigating the internal covariate shift problem during network training.

[0140] After normalization, a nonlinear activation function is applied. In this embodiment, the ReLU function is used. The ReLU function performs an element-wise operation on the normalized data; if the input value is greater than zero, it outputs that value; if the input value is less than or equal to zero, it outputs zero. Nonlinear transformation introduces nonlinear expressive power into the network, enabling it to fit more complex functional relationships. After ReLU activation, negative responses in the feature map are suppressed, while positive responses are preserved, enhancing the sparsity and discriminativeness of the features. The feature representation obtained from the convolutional layer output, after normalization and ReLU activation, is the first feature map. The first feature map carries the low-level feature information of the image after initial transformation.

[0141] To retain key information while reducing subsequent computational complexity and expanding the receptive field, a downsampling operation needs to be performed on the first feature map. This embodiment uses max pooling to achieve downsampling. For example, a fixed-size sliding window (e.g., 2x2 pixels) is set, and the window slides on the first feature map with a fixed step size (e.g., 2). For each local region covered by the window, the maximum value of all pixels in this region is calculated, and this maximum value is used as the value of the output downsampled feature map at that location. Max pooling has the effect of selecting the most salient feature response and provides invariance to small spatial displacements. After this operation, the spatial dimensions (width and height) of the output downsampled feature map are reduced to half that of the first feature map (if a 2x2 window with a step size of 2 is used), while the number of channels remains unchanged.

[0142] Downsampled feature maps are fed into a series of cascaded residual stages for hierarchical feature extraction, each residual stage consisting of multiple residual blocks stacked sequentially. The input features are first transformed through one or more convolutional layers, during which normalization and ReLU activation are applied; simultaneously, the input features are directly passed through skip connections; finally, the convolutionally transformed features are element-wise added to the original input features passed through the shortcut (Add operation). If the number of channels or spatial dimensions of the input and output do not match, 1x1 convolutions are used in the skip connections for adjustment. This residual learning mechanism allows the localization network to easily learn small incremental changes (residuals) from input to output, alleviating the optimization challenges of deep networks.

[0143] Different residual stages have different spatial downsampling rates and channel expansion factors. For example, the first residual stage may maintain the spatial size and mainly increase the number of channels; subsequent residual stages may perform spatial downsampling upon entry (e.g., through convolution with a stride of 2) while significantly increasing the number of channels. The output of each residual stage is the residual feature map of that stage.

[0144] After multiple residual stages of progressive processing, the localization network outputs a set of residual feature maps with different sizes (width and height) and increasing channel depth. These feature maps constitute a multi-scale, multi-level feature representation of the image, from local details to global semantics. These residual feature maps of different sizes are called multi-scale feature maps.

[0145] The multi-scale feature maps will be fed into the dual-branch detection module for further processing, providing a rich feature base for subsequent dual-branch detection tasks. The dual-branch detection module includes a first branch and a second branch. The first branch is used to output rectangular detection boxes, and the second branch is used to output circular detection boxes.

[0146] In some embodiments, the method further includes:

[0147] The spatial dimension of the residual feature map is adjusted by a convolution kernel to generate a feature map of the first size.

[0148] In the first classification branch, feature compression is performed by adjusting the convolutional layer through continuous channels to output a probability map of the first target category;

[0149] In the first regression branch, feature transformation is performed by adjusting the convolutional layer through continuous channels to output a rectangular box boundary coordinate parameter map;

[0150] Based on the probability of the first target category and the coordinate parameters of the bounding box, a candidate rectangular detection box is output.

[0151] In this embodiment, the residual feature map is the feature map output by residual stage 3. The feature map is processed by a 3×3 convolution kernel, which slides on the input feature map and extracts specific patterns or performs feature transformations by multiplying and accumulating local regions.

[0152] In some embodiments, the first classification branch includes four 1×1 convolutions and one max pooling layer, and the first regression branch includes four 1×1 convolutions. That is, the first feature map is processed by four 1×1 convolutions and one max pooling layer to generate a feature map of the first size, namely a 128×128×128 feature map.

[0153] Spatial dimension adjustment refers to the operation of changing the spatial size (width and height) or number of channels of the input feature map using convolution operations. By setting the stride, padding, and number of kernels, downsampling, size-preserving compression, or channel expansion can be achieved.

[0154] The first classification branch is used to predict the probability of ROP lesions (or other specified categories) existing at each spatial location (or preset anchor point). The input first-size feature map first flows through a continuous channel adjustment convolutional layer. In the first 1×1 convolution, the input feature map (e.g., size 128×128×128) is received and, through a set number of convolutional kernels (e.g., 256), outputs a higher-dimensional intermediate feature map (128×128×256), improving the expressive power of the features.

[0155] In the second 1×1 convolutional layer, an intermediate feature map is received, with K+1 kernels (K being the number of target classes, such as ROP lesion staging classes; +1 representing the background class). This map is then passed through the third and fourth convolutional layers. The fourth convolutional layer outputs the first target class probability map (size 128×128×(K+1)). Each spatial location (i,j) in this probability map corresponds to a vector of length K+1. This vector is normalized using the Softmax function, and the value of each element in the vector represents the probability that the corresponding target class (or background) exists at that location.

[0156] The first regression branch then uses continuous channel adjustment convolutional layers, specifically four 1×1 convolutions, to predict the precise positional offset of the bounding box at each spatial location (or preset anchor point) relative to that preset anchor point. The input first-size feature map (similar to the classification branch) also flows through continuous channel adjustment convolutional layers for feature transformation. In the first 1×1 convolution, the structure is the same as the first layer of the classification branch (e.g., outputting a 128×128×256 feature map), sharing some low-level feature representations. In the second 1×1 convolution, the number of convolutional kernels is fixed at 4 (corresponding to the four boundary coordinate offsets of the bounding box: △x1, △y1, △x2, △y2). The third and fourth convolutional layers then output a bounding box boundary coordinate parameter map (size such as 128×128×4). Each spatial location (i,j) in this parameter map corresponds to a vector of length 4, and the vector element values ​​are the predicted coordinate offsets.

[0157] Finally, the generation of candidate rectangular detection boxes requires combining the outputs of the classification and regression branches. For each preset spatial location (or anchor point) on the feature map, the corresponding class probability vector is extracted from the first target class probability map. The class with the highest probability is selected (if the highest probability corresponds to the background, this anchor point is ignored).

[0158] Extract the four predicted coordinate offsets (△x1, △y1, △x2, △y2) from the bounding box coordinate parameter graph at the same location. Using the preset anchor box associated with this location (a rectangle with a default center position, width, and height) and the predicted offsets, apply the bounding box regression formula to calculate the final rectangular box coordinates. The predicted box center x-coordinate is the sum of the anchor box center x-coordinate and △cx × anchor box width; the predicted box center y-coordinate is the sum of the anchor box center y-coordinate and △cy × anchor box height; the predicted box width is the anchor box width × exp(△w); and the predicted box height is the anchor box height × exp(△h).

[0159] The offset is defined directly as the corner offset or center offset plus the logarithm of the width and height scaling. This embodiment uses general bounding box regression.

[0160] Each rectangle generated through this process, along with its corresponding category label and classification confidence (i.e., prediction probability), constitutes a candidate rectangle detection box. A large number of candidate boxes constitute the preliminary detection result set of the first branch.

[0161] In this embodiment, by adjusting the spatial dimension, the input residual features are further refined and adapted to form a first feature map optimized for rectangular target detection, strengthening the local feature representation related to irregular shapes such as lesion edges and texture heterogeneity. The first classification branch gradually compresses and transforms features through continuous channel adjustment convolutional layers, ultimately outputting a category probability map. This focuses on learning abstract features to distinguish different ROP lesion stages or differentiate lesions from the background, ensuring the accuracy of category judgment. Simultaneously, the first regression branch utilizes a similar network structure (sharing some low-level features) to focus on learning how to accurately adjust the preset anchor boxes to the boundary positions of the real lesion area, enabling the two subtasks to be performed independently.

[0162] Finally, by combining the classification prediction results (category label and confidence score) of each spatial location with the coordinate offset of its regression prediction, and calculating the final rectangular coordinates based on the preset anchor box, a candidate rectangular detection box is generated. This end-to-end prediction method enables the output candidate box to have complete information for subsequent filtering (such as non-maximum suppression) and final localization, simplifying the process and improving efficiency.

[0163] In the second branch, spatial dimension adjustment is performed on the residual feature map using a convolution kernel to generate a second-size feature map;

[0164] In the second classification branch, feature compression is performed by adjusting the convolutional layer through continuous channels to output the probability of the second target class;

[0165] In the second regression branch, feature transformation is performed by adjusting the convolutional layer through continuous channels to output the center coordinates and radius offset of the circular box;

[0166] Based on the probability of the second target category, the coordinates of the center of the circular box, and the radius offset, candidate circular detection boxes are output.

[0167] The structures of the convolution kernel, the second classification branch, and the second regression branch are the same as those of the convolution kernel, the first classification branch, and the first regression branch in the first branch, and the processing procedures are also the same. For details, please refer to the first branch, and we will not elaborate further here.

[0168] Unlike the first branch, in the second regression branch, the input features are mapped to circular geometric parameters through a continuous channel adjustment convolutional layer. Specifically, the second regression branch is used to predict the geometric parameter offset of the circular box corresponding to each spatial location or preset anchor point relative to the preset anchor circle at that location. The input second-size feature map (same as the classification branch) flows through the continuous channel adjustment convolutional layer for feature transformation.

[0169] In the first 1×1 convolutional layer, the process is the same as the first layer of the classification branch (e.g., outputting a 64×64×256 feature map); in the second 1×1 convolutional layer, the number of convolutional kernels is fixed at 3 (corresponding to the center coordinate offsets Δcx, Δcy, and radius scaling offset Δr, respectively). Then, the third and fourth convolutional layers output the center coordinates and radius offset of the circular bounding box (size such as 64×64×3). Each spatial location (i, j) in this parametric map corresponds to a vector of length 3 ([Δcx, Δcy, Δr]).

[0170] The generation of candidate circular detection boxes requires combining the outputs of the classification and regression branches. For each preset spatial location (or anchor point) on the feature map, the corresponding class probability vector is extracted from the second target class probability. The class with the highest probability is selected (if the highest probability corresponds to the background, this anchor point is ignored).

[0171] Extract the three predicted parameters (△cx, △cy, △r) from the coordinates of the center and radius offset of the circular frame at the same location. Using the preset anchor circle associated with this location (a circle with a default center position and radius) and the predicted offset, apply the circular frame regression formula to calculate the final circle parameters. The predicted circle center x-coordinate is the sum of the anchor circle center x-coordinate and △cx × anchor circle radius; the predicted circle center y-coordinate is the sum of the anchor circle center y-coordinate and △cy × anchor circle radius; and the predicted radius is the anchor circle radius × exp(△r).

[0172] Each circular box generated through this process consists of its center coordinates (cx, cy), radius r, corresponding category label, and classification confidence score, forming a candidate circular detection box. A large number of candidate boxes constitute the preliminary detection result set of the second branch.

[0173] The candidate rectangular detection boxes generated by the first branch and the candidate circular detection boxes generated by the second branch are further processed by the feature enhancement modules set in the first and second branches to generate rectangular and circular detection boxes, respectively.

[0174] In some embodiments, both the first branch and the second branch include a feature enhancement module, which includes a multi-scale feature fusion unit, a spatial attention unit, and a channel attention unit.

[0175] The feature enhancement module receives multiple residual feature maps of different sizes as input. Through processing by specific internal computational units, it outputs enhanced feature maps to improve the feature's representation ability of the target region and its robustness to interference factors. The multi-scale feature fusion unit integrates cross-scale information from the multiple input residual feature maps. By combining deep, high-semantic features with shallow, high-resolution features, it generates a fused feature map that includes both rich details and high-level semantic information, addressing the problem of limited expressive power of single-scale features.

[0176] The spatial attention unit learns and generates a spatial weight map, which is the same size (width and height) as the input feature map. Each value in the weight map indicates the importance of the corresponding spatial feature, used to emphasize the target region features and suppress irrelevant background or noise interference. The channel attention unit learns and generates a channel-dimensional weight vector, which has the same length as the number of channels in the input feature map. Each value indicates the importance of the corresponding feature channel, used to dynamically adjust the contribution of different channels.

[0177] The method further includes:

[0178] The multi-scale feature fusion unit performs cross-scale fusion operations on multiple residual feature maps and fuses them through upsampling and lateral connection to generate multi-scale fused feature maps.

[0179] The spatial attention unit applies spatial attention weights to the multi-scale fused feature map, and outputs a spatially enhanced feature map.

[0180] The channel attention unit applies channel attention weights to the multi-scale fused feature map, and outputs a channel-enhanced feature map.

[0181] The multi-scale feature fusion unit performs cross-layer fusion of the output features from residual stage 2, stage 3, and stage 4. Through upsampling, i.e. bilinear interpolation ROI and lateral connection, it generates fused feature maps at three scales with sizes of 128×128×256, 64×64×256, and 32×32×256, respectively, while preserving shallow details and deep semantic information.

[0182] The spatial attention unit applies 3×3 convolution and sigmoid activation to the scale-fused feature map to generate a spatial attention weight map (the same size as the feature map). The target region features are enhanced by element-wise multiplication, and background interference is suppressed. The channel attention unit performs global average pooling and global max pooling on the scale-fused feature map. After passing through a shared fully connected layer and sigmoid activation, channel weights are generated. The importance of each channel is dynamically adjusted to highlight key feature channels.

[0183] By upsampling and lateral connection operations of multi-scale feature fusion units, the high-level semantic understanding contained in deep features (such as the category attributes of lesions and the overall layout of partitions) is integrated with the fine spatial details preserved by shallow features (such as the boundary contours of lesions and the fine structure of blood vessels), enabling subsequent modules to simultaneously utilize global context and local precision information to accurately locate lesions and partitions of different sizes.

[0184] The spatial attention unit, through learning and generating a spatial attention weight map, can dynamically identify and emphasize the spatial locations in the image related to ROP lesions and key anatomical regions. It effectively suppresses background interference in fundus images (such as large uniform areas, non-lesion vessels, and unevenly illuminated areas) and concentrates computational resources on important areas. Especially when facing complex situations such as blurred edges of lesion areas or partial occlusion by blood vessels, the spatial enhancement feature map can provide clearer and more prominent target responses, improving the reliability of detection.

[0185] The channel attention unit can automatically evaluate and highlight the most discriminative feature channels for the current detection task (distinguishing between lesions and non-lesions, locating specific regions) by analyzing the global statistical response (such as average intensity and maximum response) of each feature channel, while suppressing the contribution of redundant or noisy channels.

[0186] After obtaining candidate rectangular and circular detection boxes, in some embodiments, the rectangular and circular detection results output by the dual branches are filtered by confidence (threshold 0.5), retaining high-confidence targets. These high-confidence targets are then input into the feature enhancement module. In some embodiments, the candidate rectangular detection boxes are processed by the multi-scale feature fusion unit, spatial attention unit, and channel attention unit to output rectangular detection boxes; and the candidate circular detection boxes are processed by the multi-scale feature fusion unit, spatial attention unit, and channel attention unit to output circular detection boxes.

[0187] The input to the feature enhancement module is the candidate detection box (including coordinates and confidence score) generated by the first or second branch and its corresponding original image features. For ease of explanation, the candidate detection box includes candidate rectangular detection boxes and candidate circular detection boxes.

[0188] For each candidate detection box covering the image region, precise region truncation is performed on the fused feature map output by the multi-scale feature fusion unit. Bilinear interpolation is used to solve coordinate quantization errors and ensure that the extracted features are spatially aligned with the candidate boxes.

[0189] The captured region features are input into a spatial attention unit, which calculates the weights of each spatial location within the region (high weights focus on the target, low weights suppress the background). The weight map is then multiplied element-wise with the region features to generate spatially enhanced region features.

[0190] The optimized region features are input into the classification and regression branches to calculate class confidence and coordinate fine-tuning. Then, the spatially enhanced features are input into the channel attention unit, which evaluates the importance of each feature channel and generates a channel weight vector. The weight vector is multiplied by the features along the channel dimension to output channel-enhanced region features. In some embodiments, these features are then sequentially passed through a Flatten layer, a first fully connected layer, a second fully connected layer, and a regression head to generate bounding boxes.

[0191] The Flatten layer converts the three-dimensional feature block (50×128×7×7) into a two-dimensional feature matrix, preserving all spatial information within the candidate box while eliminating dimensional barriers. The first FC layer is used for feature compression and primary abstraction, while the second FC layer is used for feature refinement and high-order expression. Through two layers of nonlinear transformation, deep semantic features are gradually extracted, and the original visual features are transformed into highly discriminative vectors. The 256-dimensional output vector carries the essential feature expression of the candidate box region.

[0192] In the first branch, the regression head is a rectangular branch (50×(4+1+1)), where 4 represents the coordinate offset of the rectangular bounding box, the first 1 represents the target category confidence, and the second 1 represents the localization quality score.

[0193] In the second branch, the regression head is a circular branch (50×(3+1+1)), where 4 represents the circular parameter offset, the first 1 represents the target class confidence, and the second 1 represents the localization quality score.

[0194] The regression head decodes the 256-dimensional feature vector into geometric parameters and confidence metrics that can be directly used for the final detection, achieving an end-to-end mapping from abstract features to specific detection results.

[0195] It should be noted that in this embodiment, both the first branch and the second branch are equipped with feature enhancement modules to simultaneously enhance the candidate rectangular detection boxes and the candidate circular detection boxes.

[0196] To improve the positioning accuracy of the positioning network, in some embodiments, based on the rectangular detection box and the circular detection box, the distance intersection ratio (DIP) loss term for the rectangular region and the distance intersection ratio (DIP) loss term for the circular region are calculated;

[0197] A multi-shape joint loss function is constructed based on the rectangular region distance intersection-union ratio loss term and the circular region distance intersection-union ratio loss term.

[0198] The localization network is updated using the loss value calculated by the multi-shape joint loss function.

[0199] A multi-shape joint loss function integrates the optimization objectives of rectangular and circular detection boxes, constructing a comprehensive evaluation index through geometric overlap and spatial location correlation factors. This function drives the network to simultaneously learn lesion localization accuracy and anatomical partitioning accuracy, establishing an intrinsic link between the two tasks. The calculation of the loss function considers both spatial overlap and location correlation factors, providing directional guidance for network parameter optimization.

[0200] The scalar value obtained from the loss function calculation represents the overall deviation between the current prediction and the ground truth annotation. The predicted rectangular or circular detection box is compared with the annotated ground truth region to calculate the rectangular DIoU loss and the circular IoU loss, and the weighted sum is used to obtain the total loss value.

[0201] The gradient of the loss function with respect to the localization network parameters is calculated by backpropagation, and the network weights are updated using the gradient descent algorithm. This process is repeated until the loss converges.

[0202] Based on the generated rectangular and circular detection boxes, a multi-shape joint loss function is constructed, which in some embodiments includes:

[0203] Obtain candidate rectangular detection boxes;

[0204] Based on the coordinates of the candidate rectangular detection box and the coordinates of the rectangular labeled area, calculate the area of ​​the first intersecting region and the area of ​​the first merging region;

[0205] The intersection-union ratio is generated based on the ratio of the area of ​​the first intersecting region to the area of ​​the first merging region.

[0206] Calculate the Euclidean distance between the candidate rectangular detection box and the center point of the labeled area;

[0207] Calculate the diagonal length of the smallest bounding rectangle that includes the candidate rectangular detection box and the rectangular annotation area;

[0208] The intersection-union ratio, the Euclidean distance of the center point, and the diagonal length are combined to generate a rectangular region distance intersection-union ratio loss term;

[0209] Based on the distance intersection-union ratio (DIU) loss term for the rectangular region and the distance intersection-union ratio (DIU) loss term for the circular region, a multi-shape joint loss function is constructed.

[0210] like Figure 5 As shown in the figure, the intersection-union ratio (IoU, DIoU) of rectangles A and B needs to be calculated according to the following steps. The core is to combine the intersection-union ratio (IoU) of the two rectangles, the distance between their center points, and the length of the diagonal of the smallest circumscribed rectangle to comprehensively measure the similarity.

[0211] First, the parameters of the rectangles are defined. The coordinates of rectangles A and B are represented by the top-left corner (x1, y1) and the bottom-right corner (x2, y2). Rectangle A: (x A1 y A1 x A2 y A2 ), with the center point being (C Ax C Ay ),in, Rectangle B: (x B1 y B1 x B2 y B2 ), with the center point being (C Bx C By ),in,

[0212] To calculate the area of ​​the intersecting region, first calculate the coordinates of the top-left and bottom-right corners of the intersecting region. The x-coordinate of the top-left corner of the intersection is as follows:

[0213] x ∩1 =max(x A1 x B1 );

[0214] The y-coordinate of the top left corner of the intersection is as follows:

[0215] y ∩1 =max(y A1 y B1 );

[0216] The x-coordinate of the lower right corner of the intersection is shown in the following formula:

[0217] x ∩2 =min(x A2 x B2 );

[0218] The y-coordinate of the lower right corner of the intersection is shown in the following formula:

[0219] y ∩2 =min(y A2 y B2 );

[0220] If x ∩1 ≥x ∩2 or y ∩1 ≥y ∩2 If the two rectangles do not intersect, then |A∩B|=0; otherwise, |A∩B|=(x ∩2 -x ∩1 )×(y ∩2 -y ∩1 ).

[0221] The area of ​​the contiguous region, the area of ​​rectangle A, is |A| = (x A2 -x A1 )×(y A2 -y A1 The area of ​​rectangle B is |B| = (x) B2 -x B1 )×(y B2 -y B1 If |A∪B| = |A| + |B| - |A∩B|, then |A∪B| = |A| + |B| - |A∩B|.

[0222] IoU is calculated using the following formula:

[0223]

[0224] If |A∪B|=0, then IOU=0.

[0225] The center-point distance d0, i.e., the Euclidean distance between the center points of the two rectangles, is calculated by the following formula:

[0226]

[0227] The smallest bounding rectangle whose diagonal length *d* can simultaneously encompass two rectangles has a top-left corner coordinate of (x, y). min =min(x A1 x B1 ), y min =min(y A1 y B1 The coordinates of the lower right corner are (x max =min(x A2 x B2 ), y max =min(y A2 y B2 If the length of the diagonal is given, then the length of the diagonal is given.

[0228] The DIoU calculation formula for two rectangles is the same as that for the bounding box, i.e., it is calculated using the following formula:

[0229]

[0230] Wherein, IoU represents the degree of overlap between the two rectangles. Represents the relative distance to the center point.

[0231] The combination of the two allows DIoU to more comprehensively measure the similarity between two rectangles.

[0232] For the distance intersection-union ratio loss term for circular regions, in some embodiments, candidate circular detection boxes are obtained;

[0233] Based on the center coordinates and radius parameters of the candidate circular detection box and the circular annotation area, calculate the Euclidean distance between the center of the candidate circular detection box and the center of the circular annotation area.

[0234] Based on the radius parameter and the Euclidean distance between the center of the circle, the projection component parameters of the intersecting common chord are determined;

[0235] Calculate the area of ​​the second intersection region and the area of ​​the second merger region based on the projection component parameters and the radius parameters;

[0236] The intersection-union ratio is generated based on the ratio of the area of ​​the second intersecting region to the area of ​​the second merging region.

[0237] By fusing the geometric penalty term related to the intersection-union ratio and the distance to the center of the circle, a circular region distance intersection-union ratio loss term is generated.

[0238] like Figure 6 As shown, first define the center position of the label as A(Ax, Ay) and the radius as r. A The predicted center position is B(Bx, By) and the radius is r. B Half of the common chord is L y The distance from the center of the circle to the common chord is L. x .

[0239] Next, we calculate the area of ​​the region where the two circles intersect. This region consists of the areas of two segments. The area of ​​each segment is equal to the area of ​​the corresponding sector minus the area of ​​the corresponding triangle (the triangle formed by the center of the circle and the two endpoints of the common chord), as shown in the following formula:

[0240] S=(S 扇形ADE -S ΔADE )+(S 扇形BDE -S ΔBDE );

[0241] The area of ​​a sector is derived from the fact that the central angle of sector ADE is α = ∠DAE. In right triangle ACD, Right now

[0242] According to the formula for the area of ​​a sector (where θ is the central angle in radians), the area of ​​sector ADE can be obtained as:

[0243]

[0244] Similarly, for circle B, the central angle of sector BDE is β = ∠DBE. The area of ​​sector BDE is:

[0245]

[0246] For ΔADE, this triangle is composed of two congruent right triangles ACD and ACE, and its area is the sum of the areas of the two right triangles, that is:

[0247]

[0248] Similarly, for ΔBDE, this triangle is composed of two congruent right triangles BCD and BCE, where BC = dL. x Its area is:

[0249]

[0250] Substituting the areas of the sectors and triangles above into the formula for the area of ​​the intersecting region, we get:

[0251]

[0252] Use L x Let d represent L, derived previously. x Relationship with d Simplifying, we obtain a quadratic equation in d:

[0253]

[0254] To solve this equation, consider that d > 0 and d = L x +CB, taking the positive root, we get:

[0255]

[0256] Will Substituting into the formula for the area of ​​the intersecting region derived in the preliminary step, we get:

[0257]

[0258] Let d be the distance between the centers of the two circles plus the length of the two radii.

[0259]

[0260] The final IOU is then calculated using the following formula:

[0261] IoU = IoU 矩形+IoU 圆形 .

[0262] like Figure 7 As shown, based on the final IOU, a joint loss function is constructed. The gradient of the loss function with respect to the localization network parameters is calculated through backpropagation. The network weights are updated using the gradient descent algorithm. The above process is repeated until the loss converges.

[0263] The method uses ResNet101 as the backbone feature extraction network for the localization network framework, introduces a feature enhancement module and dual detection branches, and achieves high-precision detection of rectangular, circular and near-circular targets through multi-scale feature fusion, shape-aware regression and attention mechanism.

[0264] Similar parts between the embodiments provided in this application can be referred to mutually. The specific implementation methods provided above are only a few examples under the overall concept of this application and do not constitute a limitation on the scope of protection of this application. For those skilled in the art, any other implementation methods extended from the solution of this application without creative effort shall fall within the scope of protection of this application.

Claims

1. A method for detecting and zonal localization of ROP lesions based on fundus color imaging, characterized in that, The method comprises the following steps: obtaining a training sample set comprising a plurality of fundus RGB images after brightness enhancement; constructing a positioning network through the training sample set, wherein the positioning network comprises at least a feature fusion module and a double-branch detection module; inputting an image to be positioned into the feature fusion module to output a multi-scale feature map; inputting the multi-scale feature map into the double-branch detection module to output a rectangular detection box through a first branch and a circular detection box through a second branch; the first branch comprises a first classification branch and a first regression branch; the second branch comprises a second classification branch and a second regression branch; wherein, through a convolution kernel, spatial dimension adjustment is performed on the residual feature map to generate a first size feature map; in the first classification branch, feature compression is performed through a continuous channel adjustment convolution layer to output a first target class probability map; in the first regression branch, feature conversion is performed through a continuous channel adjustment convolution layer to output a rectangular frame boundary coordinate parameter map; based on the first target class probability and the rectangular frame boundary coordinate parameter, a candidate rectangular detection box is outputted; through a convolution kernel, spatial dimension adjustment is performed on the residual feature map to generate a second size feature map; in the second classification branch, feature compression is performed through a continuous channel adjustment convolution layer to output a second target class probability; in the second regression branch, feature conversion is performed through a continuous channel adjustment convolution layer to output a circular frame center coordinate and a radius offset; based on the second target class probability, the circular frame center coordinate and the radius offset, a candidate circular detection box is outputted; based on the rectangular detection box and the circular detection box, a rectangular region distance intersection and union loss term and a circular region distance intersection and union loss term are calculated; a multi-shape joint loss function is constructed based on the rectangular region distance intersection and union loss term and the circular region distance intersection and union loss term; the loss value calculated through the multi-shape joint loss function is used to update the positioning network; based on the rectangular detection box and the circular detection box, a positioning result is obtained.

2. The method of claim 1, wherein the method further comprises: The method comprises the following steps: obtaining a fundus RGB image; through a non-linear transformation, an image is outputted based on the original brightness value of the fundus RGB image; setting a brightness adjustment intensity parameter to generate an enhanced sample image based on the brightened image; the enhanced sample image is integrated into a training sample set.

3. The method of claim 1, wherein the method further comprises: The method comprises the following steps: obtaining an image to be positioned; through a convolution layer, initial feature extraction is performed on the image to be positioned, and a first feature map is generated through normalization processing and a non-linear activation function; a down-sampling operation is performed on the first feature map to output a down-sampled feature map; the down-sampled feature map is inputted into a plurality of serially connected residual stages to perform hierarchical feature extraction to output a plurality of residual feature maps, wherein the sizes of the plurality of residual feature maps are different; based on the residual feature maps, the multi-scale feature map is determined.

4. The method of claim 3, wherein the method further comprises: The first branch includes a feature enhancement module, and the feature enhancement module includes a multi-scale feature fusion unit, a spatial attention unit, and a channel attention unit. The method further includes: The multi-scale feature fusion unit performs a cross-scale fusion operation on the plurality of residual feature maps and fuses through upsampling and lateral connection to generate a plurality of scale fusion feature maps; The spatial attention unit applies spatial attention weights to the multi-scale fusion feature maps to output spatial enhancement feature maps; The channel attention unit applies channel attention weights to the multi-scale fusion feature maps to output channel enhancement feature maps.

5. The method of claim 1, wherein the method further comprises: The second branch includes a second classification branch and a second regression branch. The method further includes: The convolution kernel performs spatial dimension adjustment on the residual feature map to generate a second size feature map; In the second classification branch, the feature compression is performed by the consecutive channel adjustment convolution layer to output the second target category probability; In the second regression branch, the feature conversion is performed by the consecutive channel adjustment convolution layer to output the circular frame center coordinates and the radius offset; Based on the second target category probability, the circular frame center coordinates, and the radius offset, the candidate circular detection frame is output.

6. The method of claim 1, wherein the method further comprises: The multi-shape joint loss function is constructed, including: Obtain a candidate rectangular detection frame; Based on the coordinates of the candidate rectangular detection frame and the coordinates of the rectangular labeled region, calculate the first intersection area and the first union area; Based on the ratio of the first intersection area to the first union area, generate the intersection-over-union ratio; Calculate the Euclidean distance between the candidate rectangular detection frame and the center point of the labeled region; Calculate the diagonal line length of the minimum circumscribed rectangle containing the candidate rectangular detection frame and the rectangular labeled region; Fuse the intersection-over-union ratio, the center point Euclidean distance, and the diagonal line length to generate the rectangular region distance intersection-over-union ratio loss term; Based on the rectangular region distance intersection-over-union ratio loss term and the circular region distance intersection-over-union ratio loss term, construct the multi-shape joint loss function.

7. The method of claim 1, wherein the method further comprises: determining a location of the ROP lesion based on the color photograph of the fundus. The method further includes: Obtain a candidate circular detection frame; Based on the center coordinates and the radius parameters of the candidate circular detection frame and the circular labeled region, calculate the Euclidean distance between the center points of the candidate circular detection frame and the circular labeled region; Based on the radius parameter and the center Euclidean distance, determine the projection component parameter of the intersection common chord; According to the projection component parameter and the radius parameter, calculate the second intersection area and the second union area; Based on the ratio of the second intersection area to the second union area, generate the intersection-over-union ratio; Fuse the intersection-over-union ratio and the geometric penalty term related to the center distance to generate the circular region distance intersection-over-union ratio loss term.

8. The method of claim 1, wherein the method further comprises: determining a location of the ROP lesion based on the color photograph of the fundus. The first branch includes a feature enhancement module, and the feature enhancement module includes a multi-scale feature fusion unit, a spatial attention unit, and a channel attention unit. The input of the multi-scale feature map into the double-branch detection module to output the rectangular detection frame through the first branch and the circular detection frame through the second branch includes: The candidate rectangular detection frame is processed by the multi-scale feature fusion unit, the spatial attention unit and the channel attention unit to output a rectangular detection frame. and, The candidate circular detection frame is processed by the multi-scale feature fusion unit, the spatial attention unit and the channel attention unit to output a circular detection frame.

Images