Fundus disease image segmentation method and system based on multi-mode focus simulation

Abstract

The invention relates to the field of image processing, in particular to a fundus disease image segmentation method and system based on multi-mode focus simulation. According to the method, for a simulation focus image of an OCT mode, image denoising, OCT layer information detection, layering abnormity judgment, RPE layer inclination angle determination, to-be-implanted OCT focus-free eye fundus image and focus implantation position determination, simulation focus size adjustment and simulation focus pixel value reconstruction are carried out in sequence, and an OCT multi-class synthetic eye fundus disease image and a segmentation label thereof are generated; and for the simulation focus image of the CFP mode, a CFP multi-class synthetic fundus disease image and a segmentation label thereof are generated by selecting a proper background image, adjusting the brightness of the background image, cutting off the edge of the simulation focus and reconstructing the pixel value of the simulation focus in sequence. Clinical pathology constraints are followed, fundus disease image data diversity is increased, meta-task enhancement is realized, and segmentation precision of a focus segmentation network is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a method and system for segmenting fundus disease images based on multimodal lesion simulation. Background Art

[0002] Intelligent medical image analysis is an important research direction at the intersection of computer vision and clinical medicine. Its core goal is to achieve disease detection, lesion localization, and pathology quantification through automated algorithms. In ophthalmology, fundus images (such as optical coherence tomography (OCT) and color fundus photography (CFP)) are key to diagnosing diseases such as diabetic retinopathy (DR), age-related macular degeneration (AMD), choroidal neovascularization (CNV), and retinitis pigmentosa (RP). In recent years, deep learning technology, with its powerful feature representation capabilities, has made significant progress in fundus image classification and segmentation tasks.

[0003] Traditional medical image analysis methods primarily rely on handcrafted features such as texture, shape, and grayscale distribution to segment fundus disease images using machine learning models, such as support vector machines and random forests. However, these models have limited feature representation capabilities and struggle to adapt to complex and diverse pathological morphologies. For example, threshold-based segmentation algorithms are insensitive to low-contrast lesions (such as pigmentation in RP), while region-growing algorithms are susceptible to noise, leading to over-segmentation or under-segmentation. With the rise of convolutional neural networks (CNNs), encoder-decoder architectures, such as UNet, fuse multi-scale features through skip connections, significantly improving pixel-level segmentation accuracy. Furthermore, the Visual Transformer (ViT), which models long-range dependencies through a multi-head attention mechanism, has demonstrated superior performance to CNNs in medical image analysis. Meanwhile, large models based on prompt-based segmentation (such as the Segment Anything Model (SAM)) have garnered attention due to their versatility.

[0004] Although deep learning-based methods have achieved efficient and accurate automated analysis for fundus disease segmentation through end-to-end optimization and multimodal feature fusion, the scarcity of fundus disease image samples, especially for rare diseases such as retinitis pigmentosa (RP), leads to insufficient segmentation accuracy of lesion segmentation networks for fundus disease images with relatively few training samples, making it difficult to meet the clinical diagnostic requirements for lesion localization and classification. Existing approaches to address the scarcity of annotated data have alleviated the challenges of medical image segmentation to some extent, but they still have significant drawbacks. For example, conventional data augmentation methods (such as rotation and noise injection) can only induce low-level feature changes and cannot generate anatomically plausible lesion morphologies, resulting in insufficient generalization of the model to real lesions. Generative models (such as GANs and diffusion models) can synthesize fundus images, but the generation process is uncontrollable and it is difficult to simultaneously generate pixel-level labels. Furthermore, the generation of small-scale lesions (such as punctate inner choroidal lesions) is prone to topological distortion, which affects segmentation accuracy. Transfer learning relies on the distribution similarity between the source domain and the target domain, while the domain differences between natural images and medical images result in inefficient feature transfer. Domain adaptation methods require the source domain and the target domain to share similar pathological features, and the high cost of collecting source domain data limits its application in rare disease scenarios. Incremental learning requires continuous acquisition of multi-category data, but the scarcity of samples of rare fundus diseases makes it difficult to accumulate effective prior knowledge. Multi-task learning relies on the completeness of multi-task datasets, and the design of task relevance is complex, making clinical deployment difficult. Existing meta-learning methods are mostly based on shallow networks and simplified task assumptions, making it difficult to adapt to the complex characteristics of fundus images (such as OCT layered structure and CFP vascular distribution). In addition, the single category nature of medical datasets leads to insufficient meta-task diversity, limiting the generalization ability of the model. Therefore, a solution to the scarcity of fundus disease image samples is still needed. Summary of the Invention

[0005] To this end, the technical problem to be solved by the present invention is to overcome the defect that the existing fundus disease image samples are scarce, resulting in insufficient segmentation accuracy of the lesion segmentation network for fundus disease types with a small number of samples, making it difficult to accurately locate and classify lesions.

[0006] To solve the above technical problems, the present invention provides a fundus disease image segmentation method based on multimodal lesion simulation, comprising:

[0007] Based on the contour features, size features, and position features of the lesions in the real fundus disease images and their segmentation labels, the lesions are geometrically transformed to generate simulated lesion images and their segmentation labels;

[0008] Determine the lesion-free fundus image and implantation position of the simulated lesion image;

[0009] According to the characteristics of the target simulated lesion category, spatial constraints are set, and the simulated lesion image and its segmentation label are processed synchronously to make the simulated lesion image conform to the fundus anatomical structure;

[0010] The pixel values ​​in the processed simulated lesion image and its segmentation label are reconstructed to match the characteristics of the target simulated lesion category, generating a synthetic fundus disease image and its segmentation label; the synthetic fundus disease image and its segmentation label are used to train the lesion segmentation network, and the actual fundus disease image is segmented using the trained lesion segmentation network.

[0011] Preferably, based on the contour features, size features, and position features of the lesions in the real OCT fundus disease image and its segmentation labels, the lesions are geometrically transformed to generate an OCT simulated lesion image and its segmentation labels;

[0012] Based on the contour features, size features, and position features of the lesions in the CFP real fundus disease images and their segmentation labels, the lesions are geometrically transformed to generate CFP simulated lesion images and their segmentation labels.

[0013] Preferably, determining the OCT fundus image without lesions to be implanted and the implantation position of the OCT simulated lesion image includes:

[0014] The current OCT simulated lesion image and its segmentation label are denoised, and it is determined whether the denoised current OCT simulated lesion image has layered abnormalities. If so, the current OCT simulated lesion image and its segmentation label are discarded.

[0015] Preferably, the denoising of the current OCT simulated lesion image and its segmentation labels, and determining whether the denoised current OCT simulated lesion image has layered abnormalities, includes:

[0016] Obtaining the segmentation mask of each retinal layer in the current denoised OCT simulated lesion image, extracting pixel sequences along the boundaries of each retinal layer, and determining whether there is an interlayer discontinuity based on the spatial correlation between adjacent pixels. If so, determining that the current denoised OCT simulated lesion image has layered abnormalities;

[0017] Obtain the segmentation mask area of ​​each retinal layer. Based on the ratio of the mask area of ​​each retinal layer to the reference area of ​​that retinal layer, determine whether there is structural deficiency. If so, determine that the current OCT simulated lesion image after denoising has layered abnormalities.

[0018] If the denoised current OCT simulated lesion image does not have any interlayer discontinuity or structural loss, it is determined that the denoised current OCT simulated lesion image does not have any layer abnormality.

[0019] Preferably, determining the OCT lesion-free fundus image to be implanted and the implantation position of the image based on the contour features of the OCT simulated lesion image and the underlying anatomical layer in the interlayer region to which the target simulated lesion category belongs includes:

[0020] Generate an initial set of candidate coordinate points on the contour line of the underlying anatomical layer in the current OCT fundus image to be implanted, based on the underlying anatomical layer in the interlayer region to which the target simulated lesion category belongs;

[0021] Determine the size of the analysis window based on the contour features of the OCT simulated lesion image; construct an analysis window for each initial candidate coordinate point based on the size of the analysis window, taking each initial candidate coordinate point as the center;

[0022] Calculating the geometric inclination of the underlying anatomical layer in the analysis window of each initial candidate coordinate point, and determining whether the geometric inclination of the underlying anatomical layer in the analysis window of the current initial candidate coordinate point is greater than a set inclination threshold; if so, taking the initial candidate coordinate point as the target candidate coordinate point;

[0023] Determine whether the number of target candidate coordinate points is greater than 0. If so, use the current lesion-free fundus image as the OCT lesion-free fundus image into which the OCT simulation lesion image is to be implanted, and select any target candidate coordinate point as the mapping position of the geometric center of the OCT simulation lesion image in the current lesion-free fundus image; if equal to 0, replace the OCT lesion-free fundus image until an OCT lesion-free fundus image with a number of target candidate coordinate points greater than 0 is matched, and use it as the OCT lesion-free fundus image into which the OCT simulation lesion image is to be implanted.

[0024] Preferably, according to the characteristics of the target simulated lesion category, spatial constraints are set, and the OCT simulated lesion image and its segmentation label are synchronously processed to make the OCT simulated lesion image conform to the fundus anatomical structure, including:

[0025] According to the interlayer region to which the target simulated lesion category belongs, the size of the OCT simulated lesion image and its segmentation label are synchronously adjusted until the maximum vertical span of the lesion is less than or equal to the maximum interlayer distance of the interlayer region to which the target simulated lesion category belongs in the OCT lesion-free fundus image to be implanted, and the total number of pixels is less than the total number of pixels in the interlayer region.

[0026] Preferably, the step of synchronously adjusting the size of the OCT simulated lesion image and its segmentation label according to the inter-layer region to which the target simulated lesion category belongs includes:

[0027] Determine whether the maximum vertical span of the lesion in the OCT simulated lesion image is less than or equal to the maximum inter-layer distance of the inter-layer region to which the target simulated lesion category belongs in the OCT lesion-free fundus image to be implanted, and whether the total number of pixels is less than the total number of pixels in the inter-layer region;

[0028] If so, there is no need to adjust the size of the OCT simulated lesion image and its segmentation labels;

[0029] If not, the size of the current OCT simulated lesion image and its segmentation label is adjusted using the scaling formula. The scaling formula is:

[0030]

[0031] Wherein, W′ is the width of the image after adjustment, W is the width of the image, H′ is the height of the image after adjustment, H is the height of the image, d′ is the maximum inter-layer distance of the inter-layer area belonging to the target simulated lesion category in the OCT lesion-free fundus image to be implanted into the OCT simulated lesion image, e1 is the first compensated discretization error value, and e2 is the second compensated discretization error value.

[0032] Preferably, when the target simulated lesion category is choroidal neovascularization, reconstructing each pixel value in the processed OCT simulated lesion image and its segmentation label includes:

[0033] For each pixel point inside the lesion in the processed OCT simulated lesion image, two pixel points on the lesion outline in the OCT simulated lesion image with the same horizontal coordinate as the point are taken and recorded as the upper boundary point and the lower boundary point respectively;

[0034] Select any pixel point on the boundary line between the outer plexiform layer and the inner plexiform layer of the OCT fundus image without lesions to be implanted as a position reference;

[0035] Calculate the vertical relative position ratio of each pixel point within the lesion in the processed OCT simulated lesion image, and use the ratio and position reference coordinates to map each pixel point within the lesion in the processed OCT simulated lesion image to the corresponding position in the choroid layer;

[0036] Add a random horizontal offset to the mapped position of each pixel inside the lesion in the processed OCT simulated lesion image as the target position of each pixel;

[0037] The pixel value of the target position of each pixel point inside the lesion in the processed OCT simulated lesion image is obtained, and random perturbation is added as the reconstructed pixel value of each pixel point inside the lesion in the OCT simulated lesion image, and the segmentation label of the current OCT simulated lesion image is synchronously adjusted.

[0038] Preferably, determining the lesion-free fundus image to be implanted with the CFP simulated lesion image includes:

[0039] The structural similarity between the CFP simulated lesion image and each CFP non-lesion fundus image is calculated, and the CFP non-lesion fundus image with the highest structural similarity to the CFP simulated lesion image is used as the CFP non-lesion fundus image to be implanted into the current CFP simulated lesion image.

[0040] Preferably, after determining the lesion-free fundus image to be implanted with the simulated lesion image and the implantation position, the method further includes:

[0041] According to the set proportional coefficient, the brightness of the CFP non-lesion fundus image to be implanted with the CFP simulated lesion image is adjusted until the pixel mean of the CFP non-lesion fundus image is within the distribution range of the CFP image pixel mean.

[0042] Preferably, according to the characteristics of the target simulated lesion category, spatial constraints are set, and the CFP simulated lesion image and its segmentation label are synchronously processed to make the CFP simulated lesion image conform to the fundus anatomical structure, including:

[0043] The CFP simulated lesion image is projected onto the anatomical coordinate system of the CFP lesion-free fundus image to be implanted. Based on the distance between the geometric center of the lesion and the center of the optic disc, it is determined whether the lesion has invaded the optic disc protection zone. If so, the portion of the CFP simulated lesion image and its segmentation label whose distance from the optic disc center is less than the protection radius is truncated.

[0044] At the same time, it is determined whether there is overlap between the lesion and the central area of ​​the macula. If so, the overlapping part in the CFP simulated lesion image and its segmentation label is removed.

[0045] Preferably, when the target simulated lesion category is retinitis pigmentosa, reconstructing each pixel value in the processed CFP simulated lesion image and its segmentation label includes:

[0046] Extracting a block containing the lesion from the processed CFP simulated lesion image, and extracting a background block of the same size at the same anatomical position in the CFP fundus image without lesions to be implanted;

[0047] According to the sum Z of all pixel values ​​of the background tile b The sum of all pixel values ​​Z in the patch containing the lesion a The ratio ratio is used to adjust the pixel values ​​in the processed CFP simulated lesion image and its segmentation label. The adjustment formula is: in, The pixel coordinates in the reconstructed image are The pixel value of The coordinates of the pixel points in the image are The pixel value of .

[0048] The present invention also provides a fundus disease image segmentation system based on multimodal lesion simulation, comprising:

[0049] A simulated lesion image generation module is used to perform geometric transformation on the lesions based on the contour features, size features, and position features of the lesions in the real fundus disease image and its segmentation labels, and generate a simulated lesion image and its segmentation labels;

[0050] An implantation selection module is used to determine the lesion-free fundus image to be implanted into the simulated lesion image and the implantation position;

[0051] The spatial constraint module is used to set spatial constraints based on the characteristics of the target simulated lesion category and synchronously process the simulated lesion image and its segmentation label to make the simulated lesion image conform to the fundus anatomical structure;

[0052] A reconstruction module is used to reconstruct the pixel values ​​in the processed simulated lesion image and its segmentation label to match the characteristics of the target simulated lesion category and generate a synthetic fundus disease image and its segmentation label;

[0053] The segmentation module is used to train the lesion segmentation network using the synthetic fundus disease images and their segmentation labels, and to segment the actual fundus disease images using the trained lesion segmentation network.

[0054] The above technical solution of the present invention has the following beneficial effects compared with the prior art:

[0055] The present invention describes a fundus disease image segmentation method and system based on multimodal lesion simulation, which performs geometric transformation based on the contour, size, position and other features of the real fundus disease image and its segmentation label to generate a simulated lesion image and its segmentation label, thereby expanding the lesion diversity while ensuring the morphological authenticity, spatial rationality and pathological diversity of the simulated lesions. De-noising can effectively remove the inherent interference factors of speckle noise in OCT imaging, thereby improving the detection accuracy of layered abnormalities in OCT simulated lesion images; by detecting layered abnormalities in OCT simulated lesion images, the integrity of the retinal layered structure can be verified, and abnormal OCT simulated lesion images can be eliminated to ensure the anatomical rationality of the lesion simulation; based on the contour features of the OCT simulated lesion image, the OCT lesion-free fundus image and the implantation position of the image are determined, thereby ensuring that the lesion is adapted to the anatomical structure of the OCT lesion-free fundus image when it is implanted; and by adjusting the OCT The size of the T simulated lesion image is adjusted until the maximum vertical span of the image is less than or equal to the maximum inter-layer distance of the inter-layer region to which the target simulated lesion category belongs in the OCT lesion-free fundus image to be implanted, and the total number of pixels is less than the total number of pixels in the inter-layer region, so as to avoid the appearance of lesion sizes beyond the physiological and anatomical scope; the pixel values ​​in the resized OCT simulated lesion image and its segmentation label are reconstructed to match the characteristics of the target simulated lesion category, and the pixel values ​​are reconstructed according to different simulated lesion categories to ensure that the optical properties of the synthetic lesions are consistent with the real pathological characteristics. The present invention designs a transparent simulation process through mathematical modeling and clinical pathology rules to ensure the clinical credibility of the generated data, effectively solves the problem of scarcity of fundus disease image samples, and effectively improves the segmentation accuracy of the lesion segmentation network for fundus disease types with a small number of samples.

[0056] The present invention also proposes a method for generating a synthetic fundus disease image of the CFP modality. By comparing the structural similarity of the CFP simulated lesion image with each CFP lesion-free fundus image, the most suitable CFP lesion-free fundus image is selected as the CFP lesion-free fundus image to be implanted. Considering that the brightness uniformity of the CFP synthetic fundus disease image directly affects the visual effect of lesion simulation and the accuracy of segmentation labels, the CFP lesion-free fundus image may have overexposure or underexposure problems. The pixel values ​​in the CFP lesion-free fundus image are corrected so that the adjusted CFP lesion-free fundus image is integrated with the simulated lesion. The fusion is more natural; CFP simulated lesion images may cover the black background area or the optic cup and optic disc due to improper random implantation, resulting in the generated CFP synthetic fundus disease image being inconsistent with the actual pathological distribution. By detecting whether the lesion's circumscribed rectangle involves the black background area or the optic cup and optic disc area, if the lesion center enters the optic disc protection radius or covers the macula center, the truncation mechanism is triggered; and based on the target simulated lesion category characteristics, the pixel values ​​in the processed CFP simulated lesion image and its segmentation label are reconstructed to ensure that the lesion and background images in the generated CFP synthetic fundus disease image and its segmentation label can be seamlessly integrated. By generating synthetic fundus disease image samples of different modalities, the dataset of the lesion segmentation network can be effectively expanded, and the segmentation accuracy of the lesion segmentation network can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below based on specific embodiments of the present invention in conjunction with the accompanying drawings, wherein:

[0058] Figure 1 This is a flowchart of the steps of a fundus disease image segmentation method based on multimodal lesion simulation of the present invention.

[0059] Figure 2 It is a flow chart of a method for segmenting fundus disease images based on multimodal lesion simulation according to the present invention.

[0060] Figure 3 It is a schematic diagram of generating simulated lesion images and their segmentation labels.

[0061] Figure 4 It is a flowchart of the steps for generating OCT synthetic fundus disease images and their segmentation labels.

[0062] Figure 5 This is a schematic diagram of CNV lesion pixel value reconstruction.

[0063] Figure 6 It is a flowchart of the steps for generating CPF synthetic fundus disease images and their segmentation labels.

[0064] Figure 7 This is a schematic diagram of RP lesion pixel value reconstruction.

[0065] Figure 8 This is a schematic diagram of the meta-training task construction.

[0066] Figure 9 It is a schematic diagram of generating synthetic fundus disease images and their segmentation labels when the target simulated lesion categories are CNV and RP respectively.

[0067] Figure 10 This is a visualization of the segmentation results of the lesion segmentation network on the BV1000 and RPHS test sets.

[0068] Figure 11 This is a schematic diagram of generating synthetic fundus disease images and their segmentation labels when the target simulated lesion categories are pigment epithelial detachment and drusen respectively. Figure 11 (A) is a schematic diagram of generating a synthetic fundus disease image and its segmentation label when the target simulated lesion category is pigment epithelial detachment. Figure 11 (B) is a schematic diagram of generating a synthetic fundus disease image and its segmentation label when the target simulated lesion category is drusen. DETAILED DESCRIPTION

[0069] The present invention will be further described below with reference to the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it. However, the embodiments are not intended to limit the present invention.

[0070] refer to Figure 1 、 Figure 2 As shown, this embodiment 1 provides a fundus disease image segmentation method based on multimodal lesion simulation, including the following steps:

[0071] This method extracts morphological features (such as contour, size, and location) from real fundus disease images and enhances them using affine transformation and elastic deformation to ensure the diversity and authenticity of simulated lesions. Subsequently, a simulation function library is used to customize lesion placement rules based on fundus anatomical structures (such as OCT layered tissue or CFP optic disc boundaries) to generate high-resolution synthetic fundus disease images. The specific steps are as follows:

[0072] Step S11: Based on the contour features, size features, and position features of the lesions in the real fundus disease image and its segmentation labels, a geometric transformation is performed on the lesions to generate a simulated lesion image and its segmentation labels; the simulated lesion image includes: an OCT simulated lesion image and a CFP simulated lesion image;

[0073] Among them, the OCT simulated lesion image is a simulated lesion image based on optical coherence tomography (OCT) technology, that is, a simulated lesion image of the OCT modality; the CFP simulated lesion image is a simulated lesion image based on color fundus photography (CFP) technology, that is, a simulated lesion image of the CFP modality.

[0074] In this embodiment, specifically, based on the contour features, size features, and position features of the lesions in the real OCT fundus disease image and its segmentation labels, the lesions are geometrically transformed to generate an OCT simulated lesion image and its segmentation labels;

[0075] Based on the contour features, size features, and position features of the lesions in the CFP real fundus disease images and their segmentation labels, the lesions are geometrically transformed to generate CFP simulated lesion images and their segmentation labels.

[0076] like Figure 3 As shown, Figure 3 Schematic diagram of generating simulated lesion images and their segmentation labels. Figure 3 The left side shows the schematic diagram of generating an OCT simulated lesion image and its segmentation label. The OCT real fundus disease image is subjected to feature extraction and feature enhancement to generate a simulated lesion image with CNV as the lesion type and its segmentation label. Figure 3 The right side shows the schematic diagram of generating CFP simulated lesion images and their segmentation labels. The CFP real fundus disease images are subjected to feature extraction and feature enhancement to generate simulated lesion images with lesion type RP and their segmentation labels.

[0077] The first stage is feature extraction, which aims to quantify the geometric and spatial properties of lesions from annotated real fundus disease images. Specifically, it includes the following three key dimensions:

[0078] (1) Contour features, including the closed boundary contour, geometric shape, and local curvature changes of the edge of the lesion;

[0079] The closed boundary contour of the lesion can accurately define the scope of the lesion. This embodiment extracts the closed boundary contour of the lesion through a pixel-level annotation mask. The pixel-level annotation mask is based on the accurate classification of each pixel in the image and can clearly and accurately outline the boundary of the lesion, providing an accurate basis for subsequent analysis.

[0080] The geometric shape of the lesion reflects the morphological changes during the development of the disease. This embodiment uses the Canny morphological edge detection algorithm to extract the geometric shape of the lesion.

[0081] At the same time, contour features not only include the overall shape of the lesion (such as an irregular polygon or a quasi-circular shape), but also record the local curvature changes at the edge. This local curvature change reflects the degree of curvature of the lesion boundary. Different diseases may cause the lesion boundary to exhibit different curvature characteristics. For example, in choroidal neovascularization (CNV) lesions, the edges may have multiple areas of large curvature changes due to the irregular growth of new blood vessels; while the edge curvature changes of retinitis pigmentosa lesions are relatively gentle. This local curvature change information provides a key spatial constraint for subsequent elastic deformation. During image simulation, the lesion can be reasonably deformed based on these real curvature characteristics to make it more consistent with the actual pathological condition.

[0082] (2) Size characteristics (Size), including the physical size of the lesion;

[0083] In this embodiment, the number of pixels in the lesion area is counted based on a binary mask. This binary mask classifies the pixels in the image into two categories: lesions and non-lesions. By counting the number of pixels belonging to the lesion category, the area of ​​the lesion in the image can be determined. Combined with the image resolution, the physical size of the lesion can be further accurately calculated, for example, by converting the number of pixels into a micron-level diameter.

[0084] This feature limits the scale distribution of simulated lesions by setting a threshold range of the lesion's physical size (e.g., 50-500 pixels), thus avoiding the generation of abnormal sizes that are inconsistent with clinical pathology.

[0085] (3) Position features (Position), based on the image coordinate system, include the coordinates of the upper left corner and the lower right corner of the lesion bounding box, and the spatial offset of the lesion relative to the anatomical structure (such as the OCT retinal layer or the CFP optic disc area);

[0086] Positional features ensure that the simulated lesions are implanted in an anatomical manner. For example, choroidal neovascularization (CNV) is usually located at the site of Bruch's membrane rupture, while retinitis pigmentosa (RP) lesions are mostly distributed in the peripheral retina.

[0087] Secondly, to overcome the constraint of limited real data samples, the feature enhancement stage (Aug) performs geometric perturbations on the extracted lesion features through affine transformation and elastic deformation to generate a set of simulated lesions with diverse morphologies; the simulated lesion set includes different simulated lesion images and their segmentation labels;

[0088] Affine Transformation: Apply linear transformations such as translation, rotation, scaling, and shearing to lesions in real fundus disease images.

[0089] Elastic Deformation: A Gaussian kernel-based displacement field is used to perturb the lesion contour, simulating the irregular deformation of biological tissue. Elastic deformation is particularly suitable for simulating the "osteocyte-like" pigmentation of retinitis pigmentosa (RP) lesions or the fuzzy boundaries of choroidal neovascularization (CNV) leakage areas, enhancing the pathological complexity of the synthesized data.

[0090] In this embodiment, preferably, the step of performing geometric transformation on the lesion based on the contour features, size features, and position features of the lesion in the real fundus disease image and its segmentation label to generate the simulated lesion image and its segmentation label includes:

[0091] After performing a translation operation on the lesion in the real fundus disease image, determining whether the lesion in the translated real fundus disease image exceeds the boundary of the real fundus disease image; if so, re-determining the translation distance and performing the translation operation again; if not, retaining the current translation result;

[0092] In this embodiment, the translation distance range is limited to ±15% of the size of the real fundus disease image to avoid crossing the boundary;

[0093] The lesions in real fundus disease images were rotated, with the rotation angle randomly sampled in the interval (-30°, 30°) to simulate the orientation of the lesions under different imaging perspectives.

[0094] The lesions in the real fundus disease images were scaled, and the scaling factor was sampled uniformly according to the logarithm (0.8-1.2 times) to cover the scale changes of the natural growth or degeneration of the lesions;

[0095] A shearing transformation operation is performed on the lesions in real fundus disease images by randomly generating a 2×2 affine matrix with a maximum shear amplitude of 0.1 to simulate the deformation effect of tissue stress.

[0096] Batch processing of feature enhancement is achieved through parallel computing. A single enhancement can generate K derivative lesions (e.g., K = 5). Each derivative lesion inherits the semantic information of the original features, and sample independence is ensured by randomly combining transformation parameters. This strategy improves data diversity while strictly adhering to clinical pathological constraints, such as avoiding CNV lesions overlapping the optic disc or RP plaques that are out of bounds. Ultimately, the enhanced feature set serves as input to drive the subsequent lesion implantation and image optimization processes.

[0097] The present invention uses the feature extraction stage and feature enhancement stage as the lesion feature generation module. Its goal is to extract key pathological features from real fundus disease images and generate diverse simulated lesions through controllable enhancement strategies, aiming to ensure the morphological authenticity, spatial rationality and pathological diversity of the simulated lesions, while providing an explainable basis for subsequent pixel-level label generation.

[0098] In this embodiment, the method further includes obtaining an anatomical structure segmentation map of the lesion-free fundus image;

[0099] During the lesion simulation process, the anatomical structure information of the background image is the key to ensuring the rationality of the simulated lesion to be implanted. Using the lesion-free fundus image (normal fundus image) as the background carrier, it is necessary to obtain features such as retinal layer organization and optic disc boundary through accurate anatomical structure segmentation, thereby providing constraints for the spatial positioning and morphological adaptation of the lesion. To this end, the anatomical structure segmentation of the lesion-free fundus image includes:

[0100] For OCT fundus images without lesions, key layer structures such as retinal nerve fiber layer (RNFL), inner nuclear layer (INL), outer nuclear layer (ONL) and Bruch's membrane are segmented;

[0101] For CFP fundus images without lesions, the optic disc outline and the distribution characteristics of its surrounding blood vessels are obtained by segmentation;

[0102] Among them, the target lesions (such as CNV or RP plaques) must be strictly excluded from the lesion-free fundus images to avoid confusion between real pathological features and synthetic lesions during the simulation process.

[0103] This example uses a UNet model with a symmetrical encoder-decoder structure to perform anatomical structure segmentation on lesion-free fundus images. The core modules of the model include:

[0104] Encoder: It consists of a 4-level downsampling module. Each level contains two 3×3 convolutional layers, ReLU activation function and 2×2 maximum pooling layer to gradually extract multi-scale features.

[0105] Decoder: Upsampling is achieved through transposed convolution, and each level is skipped with the corresponding feature map of the encoder to fuse low-resolution semantic information with high-resolution spatial details.

[0106] Output layer: Use 1×1 convolution and Softmax activation function to generate pixel-level segmentation probability map.

[0107] In this embodiment, the training data of the UNet model mainly comes from the public dataset Drive and a self-built set of normal fundus images. For the training of the OCT layer segmentation model, the present invention uses a private dataset containing 2,000 accurately annotated images. All annotations are completed by a team of senior ophthalmologists to ensure the accuracy and reliability of the data. For the training of the optic cup, optic disc and boundary segmentation model of CFP images, it is based on the widely used public dataset Drive, which has been strictly standardized and has high annotation quality and consistency. Thanks to the relatively consistent anatomical structure features of normal fundus images and the low annotation complexity, the difficulty of training such UNet models is significantly reduced, and the model can converge quickly and achieve ideal segmentation accuracy.

[0108] Step S12: determining the lesion-free fundus image and the implantation position where the simulated lesion image is to be implanted;

[0109] Step S13: according to the characteristics of the target simulated lesion category, set spatial constraints, and synchronously process the simulated lesion image and its segmentation label to make the simulated lesion image conform to the fundus anatomical structure;

[0110] In this embodiment, specifically, when synthesizing OCT fundus disease images and their segmentation labels, the spatial constraint is to constrain the size of the lesions in the OCT simulated lesion images; when synthesizing CFP fundus disease images and their segmentation labels, the spatial constraint is that the lesions in the CFP simulated lesion images cannot invade the optic disc protection area and cannot overlap with the central area of ​​the macula;

[0111] Step S14: reconstructing each pixel value in the processed simulated lesion image and its segmentation label to match the characteristics of the target simulated lesion category, generating a synthetic fundus disease image and its segmentation label; using the synthetic fundus disease image and its segmentation label, training a lesion segmentation network, and using the trained lesion segmentation network to segment the actual fundus disease image;

[0112] Based on the first embodiment, the second embodiment constructs an OCT synthetic fundus disease image and its segmentation label based on the OCT simulated lesion image and its segmentation label, the OCT lesion-free fundus image and its anatomical structure segmentation map, and the simulation function library, including:

[0113] like Figure 4 As shown, Figure 4 Flowchart of the steps for generating OCT synthetic fundus disease images and their segmentation labels. In this second embodiment, the functions in the simulation function library are used to construct OCT synthetic fundus disease images and their segmentation labels. The function sequence is: Func.2->Func.3->Func.4->Func.5->Func.8;

[0114] Step S21: performing image denoising on the current OCT simulated lesion image and its segmentation labels (Func.2);

[0115] Speckle noise is an inherent interference factor in OCT imaging and can significantly reduce the reliability of lesion simulation algorithms and the performance of downstream segmentation models. In this embodiment, a contrast-enhanced adaptive denoising method (CEAD) is preferably used. The specific process is as follows:

[0116] The sliding window method was used to calculate the average pixel value of the OCT simulated lesion image. The formula is:

[0117]

[0118] Wherein, μ is the average pixel value of the OCT simulated lesion image, H is the height of the OCT simulated lesion image, W is the width of the OCT simulated lesion image, h is the height index of the OCT simulated lesion image, w is the width index of the OCT simulated lesion image, and I(h,w) is the pixel value of the pixel point with height h and width w in the OCT simulated lesion image.

[0119] Based on the brightness compensation offset and the average pixel value of the OCT simulated lesion image, the contrast of each pixel value in the OCT simulated lesion image is enhanced. The formula is:

[0120] I′(h,w)=1.1×I(h,w)-(μ+β),

[0121] Where I′(h,w) is the pixel value of the pixel with a height of h and a width of w in the OCT simulated lesion image after contrast enhancement, I(h,w) is the pixel value of the pixel with a height of h and a width of w in the OCT simulated lesion image, and β is the brightness compensation offset, which is the set value.

[0122] Determine whether the value of each contrast-enhanced pixel in the OCT simulated lesion image is less than 0. If so, set the current pixel value to 0; otherwise, leave it unchanged. The formula is:

[0123]

[0124] Where I″(h,w) is the pixel value of the pixel with a height of h and a width of w in the OCT simulated lesion image after eliminating negative values, and max(·) is the maximum value operation.

[0125] After mapping each processed pixel value in the OCT simulated lesion image to the standard grayscale space, the CLAHE algorithm is used to enhance the local details of each pixel value to obtain the denoised OCT simulated lesion image;

[0126] The formula for mapping each processed pixel value in the OCT simulated lesion image to the standard grayscale space is:

[0127]

[0128] in, is the pixel value after the pixel with height h and width w in the OCT simulated lesion image is mapped to the standard grayscale space, min(I″) is the minimum pixel value in the entire OCT simulated lesion image I″ after eliminating negative values, and max(I″) is the maximum pixel value in the entire OCT simulated lesion image I″ after eliminating negative values.

[0129] The formula for enhancing the local details of each pixel value using the CLAHE algorithm is:

[0130]

[0131] Where O is the denoised OCT simulated lesion image, CLAHE(·) is the CLAHE algorithm, clip_limit is the histogram clipping threshold (in percentage), which is used to control the upper limit of local contrast enhancement to avoid noise amplification, and tile_grid_Size = (8,8) divides the image into 8 × 8 grids, with each grid undergoing independent histogram equalization to preserve local details.

[0132] The CEAD algorithm has dual advantages over traditional BM3D and wavelet denoising methods. First, it suppresses noise while preserving the edge sharpness of the retinal layered structure through a dynamic balance between the contrast enhancement factor (1.1) and the brightness compensation term (μ+β). Second, its efficient implementation based on OpenCV (cv2.meanStdDev / cv2.normalize / cv2.createCLAHE) greatly reduces the processing time of a single OCT image.

[0133] Step S22: After denoising the current OCT simulated lesion image and its segmentation labels, determine whether there is a layered anomaly (Func.3). If so, remove the current OCT simulated lesion image and its segmentation labels.

[0134] Func.3 detects OCT layer information to determine whether there are layered abnormalities in the OCT simulated lesion image. It aims to verify the integrity of the retinal layered structure and provide reliable anatomical constraints for lesion simulation.

[0135] In this embodiment, preferably, after denoising the current OCT simulated lesion image and its segmentation labels, determining whether there is a layered abnormality includes:

[0136] Obtain the segmentation mask of each retinal layer in the current denoised OCT simulated lesion image, extract pixel sequences along the boundaries of each retinal layer, and determine whether there is an interlayer discontinuity (such as nerve fiber layer rupture) based on the spatial correlation between adjacent pixels. If so, the denoised current OCT simulated lesion image is determined to have a layered abnormality;

[0137] In this embodiment, a pre-trained semantic segmentation model, such as a U-Net or DeepLab segmentation model, is used to perform pixel-level classification on the denoised OCT simulated lesion image to obtain segmentation masks for each retinal layer (including the nerve fiber layer, ganglion cell layer, inner plexiform layer, inner nuclear layer, outer plexiform layer, outer nuclear layer, retinal pigment epithelium, and choroid). The segmentation mask is a binary image, where each pixel has a value of 0 or 1, representing that the location belongs to the background or a specific retinal layer, respectively.

[0138] In this embodiment, specifically, any one of the Euclidean distance and Manhattan distance between adjacent pixels can be calculated as the spatial correlation between adjacent pixels; for example, if the Euclidean distance between adjacent pixels exceeds a preset Euclidean distance threshold, such as a spacing of 3-5 pixels, it is determined that there is an interlayer discontinuity at that location; if an interlayer discontinuity is detected at the boundary of any retinal layer, it is determined that the denoised current OCT simulated lesion image has a layered abnormality, and the image and its segmentation label need to be discarded; conversely, if all retinal layer boundaries remain continuous without discontinuity, it is determined that the image layered structure is normal and can be used for subsequent processing.

[0139] Obtain the segmentation mask area of ​​each retinal layer. Based on the ratio of the mask area of ​​each retinal layer to the reference area of ​​that retinal layer, determine whether there is structural deficiency. If so, determine that the current OCT simulated lesion image after denoising has layered abnormalities.

[0140] The reference area is the area benchmark of each retinal layer under normal physiological conditions. For example, for the RPE layer, when the ratio of the mask area of ​​the RPE layer to the reference area of ​​the RPE layer is less than 5%, the RPE layer is judged to have structural loss. If any retinal layer is judged to have structural loss according to the above rules, or a key layer is missing, the denoised current OCT simulated lesion image is deemed to have layered abnormalities, and the image and its segmentation label need to be removed. If the area ratios of all retinal layers are within a reasonable range, the image layered structure is considered complete and can be used for subsequent analysis.

[0141] If the denoised current OCT simulated lesion image does not have any interlayer discontinuity or structural loss, it is determined that the denoised current OCT simulated lesion image does not have any layer abnormality.

[0142] Traditional retinal layering methods rely on fixed thresholds or manual rules, making them difficult to adapt to complex medical images. In this embodiment, a pre-trained UNet layered model is used to generate a segmentation mask, known as a layer mask, to identify possible layer anomalies. By detecting mask boundaries and marking abnormal areas, subsequent simulation tasks are ensured to be performed only on structurally intact normal images, thereby improving the pathological credibility of the synthesized data and the robustness of the algorithm. For example, if the RPE layer mask is missing, the CNV simulation area cannot be accurately determined, resulting in a decrease in the anatomical rationality of the lesion simulation.

[0143] Step S23: If not, the OCT fundus image without lesions to be implanted and the implantation position are determined based on the contour features of the current OCT simulated lesion image and the underlying anatomical layer in the interlayer region to which the target simulated lesion category belongs, and the position of the current OCT simulated lesion segmentation label is adjusted simultaneously (Func.4);

[0144] Func.4 is based on the contour features of the current OCT simulated lesion image. By calculating the inclination of the underlying anatomical layer in the interlayer area to which the target simulated lesion category belongs, the OCT lesion-free fundus image to be implanted and the implantation position are determined. This function incorporates the anatomical rationality of the simulated lesion into the generation process through strict geometric constraints.

[0145] In this embodiment, preferably, based on the contour features of the current OCT simulated lesion image and the underlying anatomical layer in the interlayer region to which the target simulated lesion category belongs, determining the OCT lesion-free fundus image to be implanted and the implantation position of the image includes:

[0146] Generate an initial set of candidate coordinate points on the contour line of the underlying anatomical layer in the current OCT fundus image to be implanted, based on the underlying anatomical layer in the interlayer region to which the target simulated lesion category belongs;

[0147] In this embodiment, optionally, a uniform sampling strategy is used to uniformly sample at fixed intervals to generate an initial set of candidate coordinate points on the contour line of the underlying anatomical layer in the current OCT fundus image without lesions to be implanted;

[0148] Determine the size of the analysis window based on the contour features of the current OCT simulated lesion image;

[0149] In this embodiment, optionally, if the width of the lesion outline circumscribed rectangle in the current OCT simulated lesion image is W l , height H l , then the width of the analysis window size is Height is is the magnification factor used to ensure that the analysis window contains enough underlying anatomical layer neighborhood structures to assess the anatomical suitability of the lesion to be implanted.

[0150] Centered on each initial candidate coordinate point, an analysis window for each initial candidate coordinate point is constructed based on the size of the analysis window.

[0151] Calculate the geometric inclination angle of the underlying anatomical layer in the analysis window of each initial candidate coordinate point, and determine whether the geometric inclination angle of the underlying anatomical layer in the analysis window of the current initial candidate coordinate point is greater than the set inclination threshold (such as 45°). If it is less than or equal to, the current initial candidate coordinate point is used as the target candidate coordinate point.

[0152] In this embodiment, specifically, calculating the geometric inclination angle of the underlying anatomical layer in the analysis window of each initial candidate coordinate point includes:

[0153] Obtain the coordinates (x1, y1) and (x2, y2) of the two intersection points of the analysis window of the current initial candidate coordinate point and the underlying anatomical layer in the OCT lesion-free fundus image to be implanted currently.

[0154] If x1 = x2 and y1 = y2, then the geometric inclination angle angle of the underlying anatomical layer in the analysis window of the current initial candidate coordinate point is 0.0.

[0155] If x1 = x2 and y2 < y1, then the geometric inclination angle of the underlying anatomical layer in the analysis window of the current initial candidate coordinate point

[0156] If x1 = x2 and y2 > y1, then the geometric inclination angle of the underlying anatomical layer in the analysis window of the current initial candidate coordinate point

[0157] If x2 > x1 and y2 > y1, then the geometric inclination angle angle of the underlying anatomical layer in the analysis window of the current initial candidate coordinate point is arctan((x2 - x1) / (y2 - y1)).

[0158] If x2 > x1 and y2 < y1, then the geometric inclination angle of the underlying anatomical layer in the analysis window of the current initial candidate coordinate point

[0159] If x2 < x1 and y2 < y1, then the geometric inclination angle angle of the underlying anatomical layer in the analysis window of the current initial candidate coordinate point is π + arctan((x2 - x1) / (y2 - y1)).

[0160] If x2 < x1 and y2 > y1, then the geometric inclination angle of the underlying anatomical layer in the analysis window of the current initial candidate coordinate point

[0161] Convert angle to radians, angle′=angle×(180 / π), to obtain the target geometric inclination of the underlying anatomical layer in the analysis window of the current initial candidate coordinate point;

[0162] For example, when the target simulated lesion category is CNV, the underlying anatomical layer in the interlayer area to which the target simulated lesion category belongs is the retinal pigment epithelium (RPE layer); the continuity, local protrusions or depressions of the RPE layer will directly determine the starting position and expansion direction of the CNV, and its morphology is closely related to the local interlayer inclination angle.

[0163] The specific algorithm flow is as follows:

[0164]

[0165]

[0166] Determine whether the number of target candidate coordinate points of the current OCT lesion-free fundus image is greater than 0. If so, use the current lesion-free fundus image as the OCT lesion-free fundus image to be implanted into the image, and select any target candidate coordinate point as the mapping position of the geometric center of the OCT simulated lesion image in the current lesion-free fundus image; if equal to 0, replace the OCT lesion-free fundus image until an OCT lesion-free fundus image with the number of target candidate coordinate points greater than 0 is matched, and use it as the OCT lesion-free fundus image to be implanted into the current OCT lesion-free fundus image.

[0167] Step S24: Synchronously adjust the size of the current OCT simulated lesion image and its segmentation label according to the inter-layer region to which the target simulated lesion category belongs, until the maximum vertical span of the lesion is less than or equal to the maximum inter-layer distance of the inter-layer region to which the target simulated lesion category belongs in the OCT fundus image without lesions to be implanted, and the total number of pixels is less than the total number of pixels in the inter-layer region (Func.5);

[0168] During lesion simulation, the enhanced lesion outline may be mismatched in size due to geometric transformation, for example, exceeding the anatomical layer spacing of the lesion-free fundus image or being out of proportion with the surrounding tissue. To this end, the present invention dynamically adjusts the size of the simulated lesion through Func.5 to ensure its spatial compatibility with the normal retinal structure while retaining the significance of the pathological features. The specific process is as follows:

[0169] Determine whether the maximum vertical span of the lesion in the current OCT simulated lesion image after denoising is less than or equal to the maximum inter-layer distance of the inter-layer region to which the target simulated lesion category belongs in the OCT lesion-free fundus image to be implanted, and whether the total number of pixels is less than the total number of pixels in the inter-layer region;

[0170] In this embodiment, specifically, the maximum vertical span d of the current OCT simulated lesion image reflects the actual coverage of the lesion in the vertical direction, and the acquisition process is: counting the range of the vertical coordinates of all pixels within the lesion outline in the OCT simulated lesion image, m is the number of vertical coordinates of all pixels within the lesion outline in the current OCT simulation lesion image, max(·) is the maximum value operation, and y j is the value of the ordinate of the jth pixel point within the lesion outline in the current OCT simulation lesion image, y k is the value of the ordinate of the kth pixel point within the lesion outline in the current OCT simulated lesion image, j is the first ordinate index, and k is the second ordinate index.

[0171] In this embodiment, specifically, the maximum inter-layer distance d′ of the maximum inter-layer distance of the inter-layer region of the target simulated lesion category in the OCT fundus image without lesions to be implanted into the current OCT simulated lesion image is:

[0172]

[0173] in, is the value of the εth pixel ordinate in the interlayer region of the target simulated lesion category in the OCT fundus image without lesions to be implanted into the current OCT simulated lesion image, is the value of the ε′th pixel ordinate in the interlayer region of the target simulated lesion category in the OCT lesion-free fundus image to be implanted with the current OCT simulated lesion image, ε is the first pixel index in the interlayer region of the target simulated lesion category in the OCT lesion-free fundus image to be implanted with the current OCT simulated lesion image, and ε′ is the second pixel index in the interlayer region of the target simulated lesion category in the OCT lesion-free fundus image to be implanted with the current OCT simulated lesion image, It is the number of all pixels in the interlayer area of ​​the target simulated lesion category in the OCT lesion-free fundus image to be implanted into the current OCT simulated lesion image.

[0174] If yes, it means that the inter-slice spacing is sufficient to accommodate the lesion and the pixel density is reasonable, so the current size remains unchanged. If no, the size of the current OCT simulated lesion image and its segmentation label is adjusted using the scaling formula:

[0175]

[0176] Wherein, W′ is the adjusted width of the image, W is the width of the image, H′ is the adjusted height of the image, H is the height of the image, d′ is the maximum inter-layer distance of the inter-layer area belonging to the target simulated lesion category in the OCT lesion-free fundus image to be implanted into the OCT simulated lesion image, e1 is the first compensated discretization error value, e2 is the second compensated discretization error value, and the image is an OCT simulated lesion image or a segmentation label of the OCT simulated lesion image.

[0177] In this embodiment, e1=5 and e2=3 are set as empirical values ​​to compensate for discretization errors and ensure smooth transition of the lesion edge.

[0178] Taking the target simulated lesion category as CNV as an example, the interlayer area to which the target simulated lesion category belongs is the area between the retinal pigment epithelium and the choroid layer. The maximum interlayer distance between the retinal pigment epithelium and the choroid layer represents the maximum accommodating space between the retinal pigment epithelium and the choroid layer in the vertical direction. This parameter determines the maximum allowable size of the CNV simulated lesion in the vertical direction, avoiding the generation of abnormal lesions beyond the anatomical structure.

[0179] The specific algorithm flow is as follows:

[0180]

[0181] Func.5 maintains the integrity of the retinal layer by constraining the lesion size to not exceed the RPE-choroidal distance, avoiding invasion of the inner nuclear layer (INL) or vitreous cavity. It also prevents lesions from being too small, resulting in blurred features, ensuring that the synthetic data has recognizable pathological landmarks in the segmentation model. It also uses a scaling formula to compensate for discretization errors and enhance edge smoothness. Ultimately, combining strict geometric constraints with a dynamic scaling strategy solves the problem of matching simulated lesions with background structures. Func.5 provides a key technical guarantee for the generation of synthetic data across multiple centers and devices. Its personalized adjustment capabilities significantly improve the generalization performance of the segmentation model for complex anatomical scenarios.

[0182] Step S25: Reconstruct the pixel values ​​in the resized current OCT simulated lesion image and its segmentation label to match the characteristics of the target simulated lesion category (Func.8), and generate an OCT synthetic fundus disease image and its segmentation label corresponding to the current OCT simulated lesion image;

[0183] Recalculating the pixel values ​​of simulated lesions is a critical step in lesion simulation, ensuring that the optical properties of the synthesized lesions are consistent with the true pathological features. Func.8 is used to adjust the pixel values ​​of each pixel in the OCT simulated lesion image based on the target simulated lesion category, that is, the desired lesion type.

[0184] like Figure 5 As shown, Figure 5 Figure 2 shows a schematic diagram of pixel value reconstruction of CNV lesions. In OCT, taking CNV lesions as an example, CNV lesions appear as high-brightness areas near the retinal pigment epithelium (RPE) in OCT images, and their pixel values ​​are closely correlated with the choroid layer.

[0185] In this embodiment, specifically, when the target simulated lesion category is choroidal neovascularization (CNV), the pixel values ​​in the processed current OCT simulated lesion image and its segmentation label are reconstructed, including:

[0186] For each pixel point (w a ,h a ), take two pixel points on the current OCT simulation lesion image contour with the same horizontal coordinate as the point, and record them as the upper boundary points Lower boundary point Among them, w a is the horizontal coordinate of the ath pixel point inside the lesion in the processed current OCT simulated lesion image, h a is the vertical coordinate of the ath pixel inside the lesion in the processed current OCT simulated lesion image, is the vertical coordinate of the upper boundary point corresponding to the a-th pixel point inside the lesion in the processed current OCT simulated lesion image, is the vertical coordinate of the lower boundary point corresponding to the a-th pixel point inside the lesion in the processed current OCT simulated lesion image;

[0187] Select any pixel point (w q ,h q ), as a position reference, the boundary between the outer plexiform layer and the inner plexiform layer is a key structure for retinal neural signal transmission. Its position is relatively stable and can be used as the "horizontal zero scale line" inside the retina to determine the relative position of the choroid layer;

[0188] Calculate the vertical relative position ratio of each pixel point in the lesion in the current OCT simulated lesion image after processing, and use the ratio and position reference coordinates to convert each pixel point (w a ,h a ) is mapped to the corresponding position of the choroid layer (w a ,h′ a ), the formula is:

[0189]

[0190] Add a horizontal random offset σ∈int(-3,3) to the mapped position of each pixel inside the lesion in the processed current OCT simulation lesion image as the target position of each pixel (w a +σ,h′ a ), used to simulate the irregularity of the lesion margin;

[0191] Obtain the pixel value P(w) of the target position of each pixel point inside the lesion in the processed current OCT simulation lesion image. a +σ,h′ a ), and add random perturbation e∈int(-10,10) as the reconstructed pixel value P(w a +σ,h′ a )+e, synchronously adjust the segmentation label of the current OCT simulated lesion image to simulate the microstructural heterogeneity of the CNV leakage area and enhance the pathological authenticity of the synthetic image.

[0192] The specific algorithm flow is as follows:

[0193]

[0194] Based on the first embodiment, the third embodiment constructs a CFP synthetic fundus disease image and its segmentation label based on the CFP simulated lesion image and its segmentation label, the CFP lesion-free fundus image and its anatomical structure segmentation map, and the simulation function library, including:

[0195] like Figure 6 As shown, Figure 6 This is a flowchart of the steps for generating a CPF synthetic fundus disease image and its segmentation labels. This embodiment sequentially uses functions in the simulation function library to construct a CFP synthetic fundus disease image and its segmentation labels. The function sequence is: Func.1->Func.6->Func.7->Func.8;

[0196] Step S31: using the CFP non-lesion fundus image with the highest structural similarity to the current CFP simulated lesion image as the CFP non-lesion fundus image to be implanted into the current CFP simulated lesion image (Func.1);

[0197] Func.1 is background image selection, which is a key preprocessing step in the lesion simulation process. Its goal is to screen out candidate images that best match the anatomical structure of the CFP simulated lesion image from the normal fundus image set (fundus images without lesions).

[0198] In this embodiment, the Structural Similarity Index (SSIM) is used as a quantitative evaluation standard for structural similarity, and multi-dimensional feature matching is used to ensure the spatial rationality and pathological consistency of the synthesized image. This method has significant advantages over traditional pixel-level matching (such as MSE). By simulating the perceptual characteristics of the human visual system (HVS), SSIM can effectively capture the topological similarity of anatomical structures in fundus images and avoid mismatching due to local brightness differences.

[0199] The specific algorithm flow is as follows:

[0200]

[0201] Step S32: adjusting each pixel value of the CFP non-lesion fundus image to be implanted into the current CFP simulated lesion image until the pixel mean of the CFP non-lesion fundus image is within the distribution interval of the pixel mean of the CFP image (Func.6);

[0202] The brightness uniformity of the CFP image directly affects the visual effect of lesion simulation and the accuracy of the segmentation label. To address the possible overexposure or underexposure problem of the background image, the present invention implements adaptive brightness adjustment of the CFP lesion-free fundus image through Func.6.

[0203] In this embodiment, specifically, adjusting each pixel value of the CFP lesion-free fundus image to be implanted into the current CFP simulated lesion image until the pixel mean of the CFP lesion-free fundus image is within the distribution interval of the CFP image pixel mean includes:

[0204] The CFP fundus image without lesions to be implanted in the current CFP simulated lesion image is converted from RGB space to HSV space. The HSV model decouples brightness (Value) from hue (Hue) and saturation (Saturation), making brightness adjustment independent of color information and avoiding color cast. The CFP fundus image without lesions in HSV space is obtained.

[0205] Calculate the pixel mean of the brightness channel (V channel) in the CFP lesion-free fundus image in HSV space to quantify the overall brightness and darkness level of the image;

[0206] The distribution interval of the mean value of the CFP image pixels is (110, 220). If the mean is greater than the maximum endpoint threshold of the preset range (220), the brightness is reduced according to the ratio γ, that is, O = U × (1-γ), γ>0, until the mean is within the preset range (110, 220) to suppress the loss of details in the highlight area; where O is the CFP fundus image without lesions after brightness adjustment, U is the CFP fundus image without lesions in the RGB space, and γ is the brightness adjustment ratio coefficient;

[0207] If mean is less than the minimum endpoint threshold (110) of the preset range, the brightness is increased in proportion to γ, that is, O=U×(1+γ), to enhance the visibility of the dark area lesion.

[0208] The specific algorithm flow is as follows:

[0209]

[0210] The parameter selection and technical implementation of Func.6 were based on statistical analysis and experimental optimization of a clinical CFP dataset to balance the effectiveness of brightness adjustment with image fidelity. A mean interval of (110, 200) was used to avoid redundant adjustments to moderately bright images; a scaling factor of γ = 0.1 was set to balance the brightness correction magnitude with the risk of distortion. The Func.6 algorithm was implemented using the OpenCV "cv2.cvtColor" and "cv2.meanStdDev" functions.

[0211] Brightness normalization effectively eliminates brightness deviations caused by device differences or imaging conditions, ensuring that the optical properties of the synthesized lesions are consistent with the real data. It also enhances the segmentation model's robustness to brightness variations by suppressing edge blurring in overly bright areas and detail loss in underexposed areas. This adjustment results in a more natural fusion of the CFP lesion-free fundus image and the CFP simulated lesion image, significantly reducing domain shift across centers and devices. This module utilizes a lightweight adaptive strategy to improve the visual quality of the synthesized images while ensuring the fidelity of pathological features.

[0212] Step S33: Synchronously projecting the adjusted current CFP simulated lesion image and its segmentation label onto the anatomical coordinate system of the CFP lesion-free fundus image to be implanted, determining whether the lesion has invaded the optic disc protection area based on the distance between the geometric center of the lesion and the center of the optic disc; if so, truncating the portion of the adjusted current CFP simulated lesion image and its segmentation label whose distance from the optic disc center is less than the protection radius; and simultaneously determining whether the lesion overlaps with the central area of ​​the macula; if so, removing the overlapping portion of the adjusted current CFP simulated lesion image and its segmentation label to obtain a processed current CFP simulated lesion image;

[0213] During the lesion simulation process, the CFP simulated lesion image may cover the black background area or the optic cup and disc due to improper random implantation position, resulting in the generated CFP synthetic fundus disease image being inconsistent with the actual pathological distribution. To solve this problem, the present invention uses Func.7 to achieve intelligent truncation of lesions in abnormal positions.

[0214] In this embodiment, the core of Func.7 is to use the trained UNet model to extract the anatomical structure mask of the CFP lesion-free fundus image and dynamically correct the lesion distribution based on the mask information. The adjusted current CFP simulated lesion image and its segmentation labels are synchronously projected into the anatomical coordinate system of the CFP lesion-free fundus image to be implanted. A mapping relationship between the lesion pixels and the anatomical structure mask is established through bilinear interpolation. The lesion circumscribed rectangle is detected to see if it involves the black background area or the optic cup and optic disc area. If the lesion center enters the optic disc protection radius or covers the macula center, the truncation mechanism is triggered.

[0215] In this embodiment, specifically, determining whether the lesion has invaded the optic disc protection zone based on the distance between the geometric center of the lesion and the center of the optic disc includes: calculating the straight-line distance between the geometric center of the lesion and the center of the optic disc; if the distance is less than the radius of the optic disc protection zone, determining that the lesion has invaded the optic disc protection zone; if the distance is greater than or equal to the radius of the optic disc protection zone, determining that the lesion has not invaded and no treatment is required;

[0216] When the lesion is determined to have invaded the optic disc protection area, all pixels in the lesion that are within the optic disc protection range (i.e., pixels whose straight-line distance to the optic disc center is less than the protection radius) are found, and the color values ​​of these pixels are modified to the background color (such as the normal background color of the fundus image). At the same time, in the segmentation label, the labels of these pixels are changed from "lesion" to "background".

[0217] In this embodiment, specifically, determining whether the lesion overlaps with the central macula includes: checking whether the actual coverage of the lesion intersects with the macular protection area; if any part of the lesion falls within the macular protection area (i.e., pixels of the lesion overlap with pixels within the macular protection area), it is determined that the lesion overlaps with the central macula; if the lesion is completely outside the macular protection area, no processing is required;

[0218] When the lesion is determined to overlap with the central macula, the pixels in the lesion that overlap with the macular protection zone (i.e., pixels belonging to both the lesion and the macular protection zone) are determined; the color values ​​of these overlapping pixels are changed to the background color, and the corresponding lesion label is deleted in the segmentation label, restored to the background label, and only the part of the lesion that is outside the macular protection zone is retained.

[0219] Through Func.7, the present invention avoids the simulated lesions from irrationally covering key structures such as the optic disc or macula, ensures that the generated fundus images conform to the anatomical laws of real clinical scenarios, and improves data quality and the reliability of model training.

[0220] Step S34: reconstructing each pixel value in the processed current CFP simulated lesion image and its segmentation label to match the characteristics of the target simulated lesion category, and generating a CFP synthetic fundus disease image and its segmentation label corresponding to the CFP simulated lesion image;

[0221] like Figure 7 As shown, Figure 7 Schematic diagram of RP lesion pixel value reconstruction.

[0222] In the CFP modality, taking RP lesions as an example, RP lesions appear as widely distributed dark pigment deposits in CFP images, and their pixel values ​​need to be dynamically adapted to the background brightness.

[0223] When the target simulated lesion category is retinitis pigmentosa (RP), the pixel values ​​in the processed current CFP simulated lesion image and its segmentation label are reconstructed, including:

[0224] Extracting a block containing the lesion from the processed current CFP simulated lesion image, and extracting a background block of the same size at the same anatomical position in the CFP fundus image without lesions to be implanted;

[0225] According to the sum Z of all pixel values ​​of the background tile b The sum of all pixel values ​​Z in the patch containing the lesion a Ratio Adjust the pixel values ​​in the processed current CFP simulated lesion image and its segmentation label in, Indicates that the pixel coordinates in the reconstructed current image are It can preserve the dark area characteristics of RP lesions while adapting to the local brightness distribution of the background image.

[0226] Func.8 solves the optical compatibility issue between synthetic lesions and background images through a pathology-driven and data-adaptive pixel value reconstruction strategy. For CNV lesions, choroidal pixel sampling and noise injection are used to accurately simulate fluid accumulation and fibrous proliferation textures in high-brightness leakage areas. Anatomical layer boundary (OPL-IPL) mapping is used to avoid sampling offset caused by retinal deformation. For RP lesions, a scaling mechanism is used to dynamically adapt to background brightness, preserving the low reflective properties of pigment deposits. A tile matching mechanism is used to support multi-scale pigment patch morphology adaptation.

[0227] In this embodiment, specifically, the lesion segmentation network is any one of a U-Net network, a SegNet model, and a PSPNet model.

[0228] In this embodiment, specifically, the Dice loss and binary cross entropy loss between the segmentation probability map output by the lesion segmentation network and the pixel-level true label are used to construct a total loss function to train the lesion segmentation network, including:

[0229]

[0230] in, is the total loss function, is the Dice loss between the segmentation probability map output by the lesion segmentation network and the pixel-level true label, is the binary cross entropy loss between the segmentation probability map output by the lesion segmentation network and the pixel-level true label, and λ is the balancing weight (λ = 0.5).

[0231] The Dice loss is designed based on the Dice Similarity Coefficient (DSC) to optimize the structural overlap between the segmentation result and the true label at the image level. Its expression is:

[0232]

[0233] Where,∈,=1e -5 is a smoothing term to avoid the denominator being zero. Dice loss is highly robust to the problem of class imbalance and is especially suitable for scenes where the lesion area in medical images accounts for a small proportion (such as CNV or RP plaques). t is the pixel index, X t Y is the value of the t-th pixel in the segmentation probability map output by the lesion segmentation network, t is the value of the t-th pixel in the pixel-level true label.

[0234] The binary cross entropy loss optimizes the pixel-by-pixel probability distribution to enhance the model’s ability to capture edge details. It is defined as:

[0235]

[0236] Where N is the total number of pixels. BCE Loss provides fine-grained supervision by penalizing misclassified pixels, which is especially suitable for areas with blurred boundaries or severe noise interference.

[0237] During model training, to balance global consistency optimization at the image level with local detail supervision at the pixel level, this paper uses a combined loss function of Dice loss and Binary Cross Entropy Loss (BCE Loss). Dice loss focuses on global structural similarity, while BCE loss focuses on local pixel classification accuracy. Dice loss can suppress background dominance, while BCE loss accelerates model convergence. The combination of the two can alleviate the limitations of a single loss function.

[0238] Based on the OCT layered structure (such as the inclination of the RPE layer and the distance between retinal layers) and the CFP optic disc boundary, the present invention customizes the lesion location, size and morphological rules to ensure the biological rationality of the synthesized data; through an interpretable simulation function library (Func.1 to Func.8), the lesion brightness, contrast and edge characteristics are dynamically adjusted during the image synthesis process, and accurate segmentation labels are automatically output to avoid the label missing problem of traditional generation models; and it supports OCT and CFP dual-modal data generation, and designs differentiated simulation strategies for different imaging characteristics (such as the high-resolution layered structure of OCT and the wide-angle vascular distribution of CFP), which significantly improves the segmentation performance of the lesion segmentation network in few-sample, multi-modal and multi-device scenarios.

[0239] like Figure 8 As shown, Figure 8 Schematic diagram for constructing a meta-training task. In this embodiment, each real fundus disease image and its segmented image, the OTP synthetic fundus disease image generated based on the image and its segmentation label, and the CFP synthetic fundus disease image generated based on the image and its segmentation label are taken as a category to obtain fundus image datasets of multiple categories.

[0240] Based on multiple categories of fundus image datasets, we construct a meta-task instance of M-way-S-shot-Q-query.

[0241] Based on the constructed meta-task instance, the lesion segmentation network is trained to obtain a trained lesion segmentation network.

[0242] For example, in a 2-way-5-shot-5-query configuration, each task consists of two categories, with each category providing five support set samples for rapid model adaptation and five query set samples for evaluating generalization performance. By dynamically adjusting category combinations and lesion simulation parameters (such as location, size, and contrast), a large number of heterogeneous meta-tasks are generated, achieving meta-task enhancement. LSA is remarkably data-efficient, constructing high-quality meta-tasks by synthesizing multi-category data, significantly reducing the model's reliance on real-world labeled data.

[0243] The test task is constructed using real fundus disease data, and its goal is to drive the lesion segmentation network (base learner) to quickly adapt to real clinical scenarios through a meta-learner trained based on the distribution of synthetic data tasks. To achieve this goal, the present invention integrates two classic meta-learning algorithms, MAML and iMAML, into a fundus disease image segmentation method based on multimodal lesion simulation proposed by the present invention, which are called LS-mTAMS and LS-iTAMS methods respectively. Among them, MAML optimizes the base learner parameters through explicit gradient updates, while iMAML improves the optimization efficiency through implicit gradient calculations. Both of them improve the model's robustness to the diversity of fundus lesion morphology and cross-domain distribution differences through meta-training task enhancement strategies. Experiments show that the Reptile algorithm performs poorly in this research task. Its shallow network structure and fixed update step size strategy are difficult to adapt to the complex lesion segmentation requirements, so it was not included in the final solution. Based on heterogeneous meta-tasks to construct model-independent meta-learning optimization (MAML / iMAML), the model can quickly adapt to the distribution differences of multi-center data. By simulating the imaging characteristics of different devices (such as resolution and contrast), the synthetic data covers multi-center and multi-device scenarios. Through the three-stage training paradigm (pre-training-meta-training-fast adaptation), the model's fine-tuning time on the target device data is shortened, the dependence on real cross-domain data is reduced, and the adaptability to clinical deployment is significantly enhanced.

[0244] The present invention provides a fundus disease image segmentation method (LS-TAMS) based on multimodal lesion simulation. Through the lesion simulation algorithm (LSA), i.e., steps S1-S4, high-fidelity synthetic data is generated, and the pixel-level label synchronous output of OCT and CFP dual modalities is supported, thereby solving the label missing and structural distortion problems of traditional generative models (such as GAN). Experiments show that only 30% to 40% of real labeled data are required to achieve the performance of the fully supervised model, and the labeling cost is reduced by more than 60%. In addition, the lesion simulation algorithm generates diversified lesions (such as CNV leakage areas and RP pigment patches) through affine transformation, elastic deformation and anatomical constraints, which significantly improves data diversity and effectively alleviates the scarcity of rare disease samples. For the rare fundus disease RP, LS-TAMS optimizes the model parameters through a few-sample task enhancement strategy. The introduction of the joint loss function (Dice+BCE) further refines the lesion edge segmentation effect, significantly improving the segmentation performance of the model for rare fundus diseases.

[0245] Unlike black-box generative models, the Lesion Simulation Algorithm (LSA)'s interpretable simulation function library (e.g., Func.4 and Func.6) strictly adheres to clinical pathology rules to ensure the anatomical plausibility of the synthesized data. For example, constraints such as CNV lesions are distributed along the RPE layer and RP pigment deposition avoids the optic disc region. Furthermore, dynamic quality control mechanisms (size adaptation and edge truncation) avoid generating out-of-bounds or distorted lesions, ensuring the pathological fidelity of the synthesized data.

[0246] like Figure 9 As shown, Figure 9 Schematic diagram of generating synthetic fundus disease images and their segmentation labels when the target simulated lesion categories are CNV and RP respectively. Figure 9 The first and second rows correspond to CNV, and the third and fourth rows correspond to RP. Each section shows, from left to right, the real image, real label, background image, background label, synthesized image, and synthesized label. The real image is a real fundus disease image, and the real label is the segmentation label of the real fundus disease image; the background image is a lesion-free fundus image, and the background label corresponds to it; the synthesized image is a synthesized fundus disease image, and the synthesized label is the annotation of the synthesized image.

[0247] LSA automatically generates pixel-level labeled synthetic data based on real lesion images and multiple normal images, covering various CNV and RP lesion morphologies. The synthesized images are highly consistent with the real pathological features in terms of brightness, texture, and anatomical location (such as near the RPE layer). For example, the high-brightness leakage areas of CNV lesions and the dark pigmented spots of RP lesions are accurately simulated, demonstrating the effectiveness of the simulation library in feature customization and optical adaptation.

[0248] like Figure 10 As shown, Figure 10 The following figure shows the segmentation results of the lesion segmentation network on the BV1000 and RPHS test sets. Compared with the suboptimal model D-UNet, the proposed method significantly reduces the probability of over-segmentation (green area) and missed segmentation (red area).

[0249] In addition, the present invention also systematically expands the LSA algorithm to adapt it to the segmentation requirements of various retinal lesions. Based on the parameter constraint optimization of the simulation function library, this study achieved the generation of synthetic images of two types of lesions: pigment epithelial detachment (PED) and drusen. Figure 11 As shown, Figure 11 This is a schematic diagram of generating synthetic fundus disease images and their segmentation labels when the target simulated lesion categories are pigment epithelial detachment and drusen respectively. Figure 11 (A) is a schematic diagram of generating a synthetic fundus disease image and its segmentation label when the target simulated lesion category is pigment epithelial detachment. Figure 11 (B) is a schematic diagram of generating a synthetic fundus disease image and its segmentation label when the target simulated lesion category is drusen.

[0250] Under the premise of keeping the original function chain topology unchanged, the present invention implements differentiated parameter control according to the pathological characteristics of different lesions:

[0251] For PED lesions, the pixel value recalculation parameters of Func.8 were reconstructed to reduce the brightness value of the lesion core area to 60% of the CNV baseline value, thereby accurately simulating its low reflectivity characteristics.

[0252] For Drusen lesions, by optimizing the morphological parameters of Func.5, the average geometric size of the simulated lesions was compressed to 40% of the original CNV lesions, reproducing their tiny speckled morphology and effectively simulating their typical submillimeter speckled morphological characteristics.

[0253] It can be seen that the simulation function library architecture proposed in the present invention has good scalability and can achieve customized expansion for specific pathological characteristics by integrating new function modules.

[0254] This paper uses a real fundus disease image dataset covering two imaging modalities, OCT and CFP, and focuses on the segmentation tasks of two typical fundus diseases, CNV and RP (rare fundus diseases).

[0255] As shown in Table 1, Table 1 is the information of the real fundus disease image dataset.

[0256] Table 1

[0257]

[0258] The core target datasets used in this paper include D1 (BV1000), D2 (RPHS) and D3 (RIPS), which provide OCT images of CNV lesions and CFP images of RP lesions, respectively, covering imaging data from multiple devices such as BV1000, Topcon and Canon, ensuring the robustness of the method in cross-device scenarios.

[0259] As shown in Table 2, Table 2 is a diagram showing the division of the training set and test set in the real fundus disease image dataset.

[0260]

[0261] The target dataset is strictly divided into training and test sets by eye, with 20% of the training set used as a validation set to avoid data leakage and ensure the clinical reliability of the evaluation results. To support LSA lesion simulation, the present invention introduces an auxiliary dataset D4 (ODIR-5K) to supplement the normal CFP images missing in D3 for the synthesis and implantation of RP lesions; D5 (DRIVE)

[24] Provides segmentation and annotation of optic disc and blood vessel boundaries, and trains UNet models to support lesion localization under anatomical constraints; D6, D7, D8, D9 and D10 cover segmentation data from multiple fields such as colon polyps and skin lesions, construct five types of meta-tasks, and conduct comparative analysis with meta-tasks constructed based on LSA synthetic data to evaluate the effectiveness of the LSA task enhancement strategy. Through the systematic integration of multi-source heterogeneous datasets, this chapter not only verifies the effectiveness of the method in a single task, but also provides an empirical basis for the meta-learning framework of multiple modalities and fundus diseases. By integrating multi-source heterogeneous datasets (covering two modalities, OCT and CFP, and two types of diseases, CNV and RP), the present invention systematically verifies the effectiveness and robustness of the method proposed in this invention in multiple task scenarios, providing experimental support for the application of the method in complex clinical environments.

[0262] In order to comprehensively and fairly evaluate the performance of different fundus disease image segmentation methods, this paper uses four indicators: Dice similarity coefficient (DSC), intersection over union (IoU), recall rate (Recall), and precision rate (Precision) to quantitatively analyze the experimental results. The formula is as follows:

[0263]

[0264] Among them, TP represents the number of true positive samples, FP represents the number of false positive samples, FN represents the number of true negative samples, and FN represents the number of false negative samples.

[0265] In this paper, a four-fold cross-validation experiment was conducted on the training sets of three target segmentation data (D1, D2, and D3) to test the proposed LS-iTAMS and LS-mTAMS, and their performance was compared with the current mainstream segmentation models (UNet, UNet++, TransUNet, MANet, nnUNet, SwinUNetr, and LiteMedSAM).

[0266] As shown in Table 3, Table 3 shows the experimental results of four-fold cross validation of different segmentation methods using the training set of a real fundus disease image dataset.

[0267]

[0268]

[0269] On the D1 (BV1000 OCT) dataset, LS-mTAMS achieved recall (REC), precision (PRE), intersection over union (IoU), and Dice coefficient (DSC) of 75.38%, 73.2%, 59.21%, and 72.95%, respectively, representing improvements of 11.84%, 5.7%, 12.66%, and 9.81% over the next-best model, D-UNet. While LS-iTAMS was slightly lower than LS-mTAMS in REC (71.65%), IoU (55.43%), and DSC (72.24%), it outperformed LS-mTAMS in PRE (77.25%).

[0270] LS-iTAMS outperforms LSA-mTAM on the D2 (RPHS) and D3 (RIPS) datasets for the CFP modality. On D2, its REC (53.47%), IoU (45.93%), and DSC (62.81%) improve by 5.41%, 2.46%, and 2.64%, respectively, compared to D-UNet. LS-mTAMS also excels on PRE (63.98%). On D3, LS-iTAMS's DSC (55.65%) improves by 2.65% compared to D-UNet.

[0271] It is worth noting that the classic UNet model outperforms other advanced networks (such as TransUNet and SwinUNetr) on all three datasets, with its DSC reaching 63.14%, 60.17% and 53.00% on D1, D2 and D3 respectively. This result shows that although the Transformer and attention mechanism perform well in natural image tasks, their performance in medical image segmentation is limited by their ability to capture local details and the scale of annotated data. In contrast, this method significantly improves the potential of UNet through a meta-task enhancement strategy based on lesion simulation, and its DSC on D1, D2 and D3 are improved by 9.81%, 2.64% and 2.65% respectively compared with the native UNet, verifying the effectiveness of the framework.

[0272] The present invention also systematically evaluates the performance advantages of the proposed LS-mTAMS and LS-iTAMS by comparing deep learning methods (D-UNet, D-TransUNet), TL methods, and the method of Khadka et al.

[0273] As shown in Table 4, Table 4 is the experimental results comparing the present invention with the TL and Khadka methods.

[0274]

[0275] On the D1 (BV1000) and D2 (RPHS) test sets, the Dice coefficient (DSC) of LS-iTAMS reached 74.95% and 58.60%, respectively, which are 3.95% and 1.23% higher than the suboptimal methods TL-Unet and Khadka-MAML, respectively, verifying the effectiveness of the method in multiple modalities and multiple disease scenarios.

[0276] On the D1 dataset, transfer learning (TL-UNet) was pre-trained on LSA synthetic data, and although the DSC was improved to 71.58%, it was still lower than LS-iTAMS (74.95%). This gap stems from the fact that traditional transfer learning only relies on single-stage feature transfer, while the meta-learning framework uses a multi-task optimization strategy to explicitly model common knowledge between tasks, thereby enhancing the ability to characterize the layered structure and leakage boundaries of CNV lesions. In addition, the Khadka method constructs meta-tasks based on D6 to D10, and its DSC (61.79%, 63.09%) is even lower than that of native D-UNet (64.21%), highlighting the negative impact of domain differences in cross-domain data (such as colon polyps and retinal images) on meta-task construction. In contrast, the synthetic data generated by LSA ensures the distribution consistency between the meta-training task and the target data through anatomical constraints and lesion simulation, reducing the domain shift error by 11.86% and 12.45% (LS-iTAMS vs Khadka-iMAML, LS-mTAMS vs Khadka-MAML), respectively, significantly improving the generalization efficiency of meta-learning.

[0277] On the D2 dataset, the TL method had limited transfer effectiveness against TransUNet (DSC 47.99%), while Khadka's MAML method (DSC 57.37%) only achieved a slight improvement over the baseline model (D-UNet 57.33%). This suggests that traditional transfer learning and meta-tasks constructed from multimodal medical images struggle to adapt to the dispersed distribution and low-contrast characteristics of RP lesions.

[0278] In summary, the synthetic data generated by the proposed multimodal lesion simulation-based fundus disease image segmentation method is not only rich in categories, but also ensures optimal inter-domain differentiation between the multi-category data and the target segmentation data during the meta-training phase through a cross-domain difference regulation mechanism. The resulting meta-task enhancement (TAMS) framework effectively addresses the domain transfer bottleneck of traditional meta-learning algorithms in lesion segmentation tasks, significantly enhancing the model's generalization ability to new tasks.

[0279] This fourth embodiment provides a fundus disease image segmentation system based on multimodal lesion simulation, including:

[0280] A simulated lesion image generation module is used to perform geometric transformation on the lesions based on the contour features, size features, and position features of the lesions in the real fundus disease image and its segmentation labels, and generate a simulated lesion image and its segmentation labels;

[0281] An implantation selection module is used to determine the lesion-free fundus image to be implanted into the simulated lesion image and the implantation position;

[0282] The spatial constraint module is used to set spatial constraints based on the characteristics of the target simulated lesion category and synchronously process the simulated lesion image and its segmentation label to make the simulated lesion image conform to the fundus anatomical structure;

[0283] A reconstruction module is used to reconstruct the pixel values ​​in the processed simulated lesion image and its segmentation label to match the characteristics of the target simulated lesion category and generate a synthetic fundus disease image and its segmentation label;

[0284] The segmentation module is used to train the lesion segmentation network using the synthetic fundus disease images and their segmentation labels, and to segment the actual fundus disease images using the trained lesion segmentation network.

[0285] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0286] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0287] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0288] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0289] Obviously, the above embodiments are merely examples for clarity of explanation and are not intended to limit the implementation methods. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all implementation methods here. Obvious variations or modifications arising therefrom remain within the scope of protection of the present invention.

Claims

1. A fundus disease image segmentation method based on multimodal lesion simulation, characterized in that: include: Based on the contour features, size features, and position features of the lesions in the real fundus disease images and their segmentation labels, the lesions are geometrically transformed to generate simulated lesion images and their segmentation labels; Determine the lesion-free fundus image and implantation position of the simulated lesion image; According to the characteristics of the target simulated lesion category, spatial constraints are set, and the simulated lesion image and its segmentation label are processed synchronously to make the simulated lesion image conform to the fundus anatomical structure; The pixel values ​​in the processed simulated lesion image and its segmentation label are reconstructed to match the characteristics of the target simulated lesion category, generating a synthetic fundus disease image and its segmentation label; the synthetic fundus disease image and its segmentation label are used to train the lesion segmentation network, and the actual fundus disease image is segmented using the trained lesion segmentation network.

2. The method for segmenting fundus disease images based on multimodal lesion simulation according to claim 1, wherein: Based on the contour features, size features, and position features of the lesions in the real OCT fundus disease images and their segmentation labels, the lesions are geometrically transformed to generate OCT simulated lesion images and their segmentation labels. Based on the contour features, size features, and position features of the lesions in the CFP real fundus disease images and their segmentation labels, the lesions are geometrically transformed to generate CFP simulated lesion images and their segmentation labels.

3. The method for segmenting fundus disease images based on multimodal lesion simulation according to claim 2, wherein: Determine the OCT simulated lesion image, the OCT lesion-free fundus image to be implanted, and the implantation location, including: The current OCT simulated lesion image and its segmentation label are denoised, and it is determined whether the denoised current OCT simulated lesion image has layered abnormalities. If so, the current OCT simulated lesion image and its segmentation label are discarded.

4. The method for segmenting fundus disease images based on multimodal lesion simulation according to claim 3, wherein: Denoising the current OCT simulated lesion image and its segmentation labels, and determining whether the denoised current OCT simulated lesion image has layered abnormalities, includes: Obtaining the segmentation mask of each retinal layer in the current denoised OCT simulated lesion image, extracting pixel sequences along the boundaries of each retinal layer, and determining whether there is an interlayer discontinuity based on the spatial correlation between adjacent pixels. If so, determining that the current denoised OCT simulated lesion image has layered abnormalities; Obtain the segmentation mask area of ​​each retinal layer. Based on the ratio of the mask area of ​​each retinal layer to the reference area of ​​that retinal layer, determine whether there is structural deficiency. If so, determine that the current OCT simulated lesion image after denoising has layered abnormalities. If the denoised current OCT simulated lesion image does not have any interlayer discontinuity or structural loss, it is determined that the denoised current OCT simulated lesion image does not have any layer abnormality.

5. The method for segmenting fundus disease images based on multimodal lesion simulation according to claim 2, wherein: Based on the contour features of the OCT simulated lesion image and the underlying anatomical layer in the interlayer region to which the target simulated lesion category belongs, the OCT lesion-free fundus image to be implanted and the implantation position are determined, including: Generate an initial set of candidate coordinate points on the contour line of the underlying anatomical layer in the current OCT fundus image to be implanted, based on the underlying anatomical layer in the interlayer region to which the target simulated lesion category belongs; Determine the size of the analysis window based on the contour features of the OCT simulated lesion image; construct an analysis window for each initial candidate coordinate point based on the size of the analysis window, taking each initial candidate coordinate point as the center; Calculating the geometric inclination of the underlying anatomical layer in the analysis window of each initial candidate coordinate point, and determining whether the geometric inclination of the underlying anatomical layer in the analysis window of the current initial candidate coordinate point is greater than a set inclination threshold; if so, taking the initial candidate coordinate point as the target candidate coordinate point; Determine whether the number of target candidate coordinate points is greater than 0. If so, use the current lesion-free fundus image as the OCT lesion-free fundus image into which the OCT simulation lesion image is to be implanted, and select any target candidate coordinate point as the mapping position of the geometric center of the OCT simulation lesion image in the current lesion-free fundus image; if equal to 0, replace the OCT lesion-free fundus image until an OCT lesion-free fundus image with a number of target candidate coordinate points greater than 0 is matched, and use it as the OCT lesion-free fundus image into which the OCT simulation lesion image is to be implanted.

6. The method for segmenting fundus disease images based on multimodal lesion simulation according to claim 2, wherein: According to the characteristics of the target simulated lesion category, spatial constraints are set and the OCT simulated lesion image and its segmentation labels are processed synchronously to make the OCT simulated lesion image conform to the fundus anatomical structure, including: According to the interlayer region to which the target simulated lesion category belongs, the size of the OCT simulated lesion image and its segmentation label are synchronously adjusted until the maximum vertical span of the lesion is less than or equal to the maximum interlayer distance of the interlayer region to which the target simulated lesion category belongs in the OCT lesion-free fundus image to be implanted, and the total number of pixels is less than the total number of pixels in the interlayer region.

7. The method for segmenting fundus disease images based on multimodal lesion simulation according to claim 6, characterized in that: The step of synchronously adjusting the size of the OCT simulated lesion image and its segmentation label according to the inter-layer region to which the target simulated lesion category belongs includes: Determine whether the maximum vertical span of the lesion in the current OCT simulated lesion image after denoising is less than or equal to the maximum inter-layer distance of the inter-layer region to which the target simulated lesion category belongs in the OCT lesion-free fundus image to be implanted, and whether the total number of pixels is less than the total number of pixels in the inter-layer region; If so, there is no need to adjust the size of the OCT simulated lesion image and its segmentation labels; If not, the size of the current OCT simulated lesion image and its segmentation label is adjusted using the scaling formula. The scaling formula is: Among them, W ′ is the width of the image after adjustment, W is the width of the image, H ′ is the height of the image after adjustment, H is the height of the image, d ′ is the maximum inter-layer distance of the inter-layer area of ​​the target simulated lesion category in the OCT lesion-free fundus image to be implanted into the OCT simulated lesion image, e1 is the first compensated discretization error value, and e2 is the second compensated discretization error value.

8. The method for segmenting fundus disease images based on multimodal lesion simulation according to claim 2, wherein: When the target simulated lesion category is choroidal neovascularization, the pixel values ​​in the processed OCT simulated lesion image and its segmentation label are reconstructed, including: For each pixel point inside the lesion in the processed OCT simulated lesion image, two pixel points on the lesion outline in the OCT simulated lesion image with the same horizontal coordinate as the point are taken and recorded as the upper boundary point and the lower boundary point respectively; Select any pixel point on the boundary line between the outer plexiform layer and the inner plexiform layer of the OCT fundus image without lesions to be implanted as a position reference; Calculate the vertical relative position ratio of each pixel point within the lesion in the processed OCT simulated lesion image, and use the ratio and position reference coordinates to map each pixel point within the lesion in the processed OCT simulated lesion image to the corresponding position in the choroid layer; Add a random horizontal offset to the mapped position of each pixel inside the lesion in the processed OCT simulated lesion image as the target position of each pixel; The pixel value of the target position of each pixel point inside the lesion in the processed OCT simulated lesion image is obtained, and random perturbation is added as the reconstructed pixel value of each pixel point inside the lesion in the OCT simulated lesion image, and the segmentation label of the current OCT simulated lesion image is synchronously adjusted.

9. The method for segmenting fundus disease images based on multimodal lesion simulation according to claim 2, wherein: Determine the lesion-free fundus image to be implanted into the CFP simulated lesion image, including: The structural similarity between the CFP simulated lesion image and each CFP non-lesion fundus image is calculated, and the CFP non-lesion fundus image with the highest structural similarity to the CFP simulated lesion image is used as the CFP non-lesion fundus image to be implanted into the current CFP simulated lesion image.

10. The method for fundus disease image segmentation based on multimodal lesion simulation according to claim 2, characterized in that: After determining the lesion-free fundus image to be implanted and the implantation position of the simulated lesion image, the following steps are also included: According to the set proportional coefficient, the brightness of the CFP non-lesion fundus image to be implanted with the CFP simulated lesion image is adjusted until the pixel mean of the CFP non-lesion fundus image is within the distribution range of the CFP image pixel mean.

11. The method for segmenting fundus disease images based on multimodal lesion simulation according to claim 2, characterized in that: According to the characteristics of the target simulated lesion category, spatial constraints are set and the CFP simulated lesion image and its segmentation labels are processed synchronously to make the CFP simulated lesion image conform to the fundus anatomical structure, including: The CFP simulated lesion image is projected onto the anatomical coordinate system of the CFP lesion-free fundus image to be implanted. Based on the distance between the geometric center of the lesion and the center of the optic disc, it is determined whether the lesion has invaded the optic disc protection zone. If so, the portion of the CFP simulated lesion image and its segmentation label whose distance from the optic disc center is less than the protection radius is truncated. At the same time, it is determined whether there is overlap between the lesion and the central area of ​​the macula. If so, the overlapping part in the CFP simulated lesion image and its segmentation label is removed.

12. The method for fundus disease image segmentation based on multimodal lesion simulation according to claim 2, characterized in that: When the target simulated lesion category is retinitis pigmentosa, the pixel values ​​in the processed CFP simulated lesion image and its segmentation label are reconstructed, including: Extracting a block containing the lesion from the processed CFP simulated lesion image, and extracting a background block of the same size at the same anatomical position in the CFP fundus image without lesions to be implanted; According to the sum Z of all pixel values ​​of the background tile b The sum of all pixel values ​​Z in the patch containing the lesion a The ratio ratio is used to adjust the pixel values ​​in the processed CFP simulated lesion image and its segmentation label. The adjustment formula is: in, The pixel coordinates in the reconstructed image are The pixel value of The coordinates of the pixel points in the image are The pixel value of .

13. A fundus disease image segmentation system based on multimodal lesion simulation, characterized in that: include: A simulated lesion image generation module is used to perform geometric transformation on the lesions based on the contour features, size features, and position features of the lesions in the real fundus disease image and its segmentation labels, and generate a simulated lesion image and its segmentation labels; An implantation selection module is used to determine the lesion-free fundus image to be implanted into the simulated lesion image and the implantation position; The spatial constraint module is used to set spatial constraints based on the characteristics of the target simulated lesion category and synchronously process the simulated lesion image and its segmentation label to make the simulated lesion image conform to the fundus anatomical structure; A reconstruction module is used to reconstruct the pixel values ​​in the processed simulated lesion image and its segmentation label to match the characteristics of the target simulated lesion category and generate a synthetic fundus disease image and its segmentation label; The segmentation module is used to train the lesion segmentation network using the synthetic fundus disease images and their segmentation labels, and to segment the actual fundus disease images using the trained lesion segmentation network.

Images