A three-dimensional fundus OCT image lesion segmentation method and device

Abstract

This invention discloses a method and apparatus for lesion segmentation in three-dimensional fundus OCT images, relating to the field of OCT medical image segmentation technology. The method includes the following steps: inputting an initial fundus OCT image into a pre-trained three-dimensional lesion segmentation deep learning model; the three-dimensional segmentation deep learning model performs three-dimensional segmentation on the initial fundus OCT image to obtain a three-dimensional segmentation result of the lesion area; the three-dimensional lesion segmentation deep learning model is a 3D U-Net model incorporating a BiFormer module; the three-dimensional segmentation method in this invention, combined with a deep learning model, can achieve high-precision, automated three-dimensional segmentation of lesion areas, effectively generating three-dimensional lesion area segmentation results with clear structure and accurate boundaries, thereby providing reliable technical support for clinical diagnosis and disease progression assessment.

Description

Technical Field

[0001] This application relates to the field of OCT medical image segmentation technology, and more specifically, to a method and apparatus for segmenting lesions in three-dimensional fundus OCT images. Background Technology

[0002] Age-related macular degeneration (AMD) and diabetic macular edema (DME) are common fundus diseases. AMD primarily affects the macula, leading to progressive central vision loss, which can cause blindness in severe cases. AMD is more common in the elderly and is divided into dry and wet types, with wet AMD causing rapid vision loss due to abnormal neovascularization. Pigment epithelial detachment (PED) is one of the common clinical manifestations of AMD, and its size, morphology, and evolution are closely related to the clinical assessment and monitoring of AMD. DME is one of the main complications of diabetic retinopathy. Diabetic retinal vascular leakage causes macular edema, resulting in blurred vision and central vision loss. DME is one of the main causes of vision impairment in diabetic patients. Intraretinal fluid (IRF) refers to the accumulation of fluid within the layers of the retina, usually caused by leakage of fluid from damaged retinal vessels into the retinal tissue. It is one of the most representative pathological features for diagnosing DME and has a high diagnostic indicative value.

[0003] Optical coherence tomography (OCT) is an advanced non-contact, high-resolution imaging technique. Based on the principle of low-coherence interference, it can acquire cross-sectional and three-dimensional structural images of the retina and other ocular tissues with micron-level spatial resolution. OCT can clearly present the tissue structure and pathological features of each layer of the retina and is widely used in the diagnosis and follow-up of various ophthalmic diseases, including AMD and DME.

[0004] Three-dimensional medical image segmentation can automatically identify and extract three-dimensional structures or lesion regions from medical images. Leveraging the advantages of OCT three-dimensional imaging, three-dimensional segmentation of fundus lesions in AMD and DME can obtain quantitative indicators such as lesion volume and surface area, enabling a more comprehensive analysis of the condition. Compared to two-dimensional image segmentation, three-dimensional segmentation can maintain the continuity and integrity of image structure in space, demonstrating a significant advantage in processing OCT volumetric data. Therefore, developing an efficient and accurate three-dimensional OCT image segmentation method is of great significance for the clinical auxiliary diagnosis of AMD and DME. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this application provides a method and apparatus for segmenting lesions in three-dimensional fundus OCT images, which can achieve automatic and high-precision three-dimensional segmentation of lesions in human fundus OCT images.

[0006] To achieve the above objectives, the technical solution adopted in this application is as follows:

[0007] A method for lesion segmentation in three-dimensional fundus OCT images includes the following steps:

[0008] The initial fundus OCT image is input into a pre-trained three-dimensional lesion segmentation deep learning model, which performs three-dimensional segmentation on the initial fundus OCT image to obtain the three-dimensional segmentation result of the lesion area.

[0009] The 3D lesion segmentation deep learning model is a 3D U-Net model that integrates the BiFormer module.

[0010] Specifically, the 3D U-Net model that integrates the BiFormer module includes an encoder and a decoder;

[0011] The encoder includes four encoding modules: the first encoding module, the second encoding module, the third encoding module, and the fourth encoding module. Each encoding module includes multiple BiFormer modules.

[0012] The decoder includes four decoding modules and one output module. The four decoding modules are the first decoding module, the second decoding module, the third decoding module and the fourth decoding module. Each decoding module is connected to the corresponding encoding module in the encoder through skip connections to perform feature splicing and fusion, and the spatial resolution is gradually restored by using layer-by-layer upsampling operation.

[0013] The decoder has four decoding modules, each including transposed convolution and a hybrid attention mechanism. The output module includes transposed convolution and convolution blocks. The output module is followed by a 1×1×1 convolution, which is the output of the entire 3D U-Net model. The hybrid attention mechanism includes channel attention and spatial attention mechanisms connected in sequence.

[0014] Optionally, a void space pyramid pooling module is provided in the jump connection between the first decoding module and the fourth encoding module;

[0015] Convolutional blocks are provided in the skip connections between the second decoding module and the third encoding module, between the third decoding module and the second encoding module, and between the fourth decoding module and the first encoding module.

[0016] Optionally, the first encoding module includes one Patch Embedding module and two BiFormer modules, the second encoding module includes one Patch Merging module and two BiFormer modules, the third encoding module includes one Patch Merging module and eight BiFormer modules, and the fourth encoding module includes one Patch Merging module and two BiFormer modules.

[0017] Furthermore, before inputting the initial fundus OCT image into the pre-trained three-dimensional lesion segmentation deep learning model, the method further includes:

[0018] Construct the three-dimensional lesion segmentation deep learning model;

[0019] The initial three-dimensional lesion segmentation deep learning model is trained based on the sample dataset to obtain the three-dimensional lesion segmentation deep learning model.

[0020] Furthermore, before inputting the initial fundus OCT image into the pre-trained three-dimensional lesion segmentation deep learning model, the process also includes processing the sample dataset, including:

[0021] The dataset is divided into training, testing, and validation sets; preprocessing of the dataset includes data augmentation, image parameter adjustment, and size cropping.

[0022] Preferably, the loss function of the three-dimensional lesion segmentation deep learning model is:

[0023]

[0024]

[0025]

[0026] in, This represents the weighting coefficient used to balance the contributions of BCE and Tversky loss, for example, It can be set to 0.4. This refers to the Tversky function. This represents the BCE function. This indicates the model's prediction results. This represents the manually labeled ground truth label corresponding to the prediction result, with a value of 0 or 1. (Parameter) and These are used to adjust the weights for false positives and false negatives, respectively, for example, The value is 0.2. The value is 0.8.

[0027] Specifically, the training process of the three-dimensional lesion segmentation deep learning model includes the following steps:

[0028] A. Data preprocessing and encoder feature extraction:

[0029] Preprocessed volumetric image data from the training set is input into the network model in a 224×224×64 format. The encoder part of the model consists of four stages. In the first stage, Patch Embedding is used to perform preliminary image patching and feature embedding on the input image. In the second to fourth stages, Patch Merging is used to downsample the feature map, gradually halving the spatial dimension while increasing the channel dimension to enhance feature expressiveness. After each downsampling, the features are input to multiple BiFormer modules. The BiFormer module introduces a two-layer routing attention mechanism, which first divides the input features into multiple non-overlapping local regions, then calculates the global semantic importance of each region, and only retains the top k tokens most relevant to the current region for attention calculation. This mechanism can significantly reduce the computational cost when processing high-resolution 3D medical images while maintaining modeling capabilities.

[0030] Optionally, a dilated spatial pyramid pooling module can be introduced at the end of the encoder. This module consists of structures such as 1×1×1 convolution, dilated convolution with different dilation rates, and global average pooling. It extracts contextual information through multi-scale receptive fields and fuses global feature representations to further enhance the model's feature representation capabilities.

[0031] B. Decoder reconstruction and segmentation output:

[0032] The multi-scale features extracted by the encoder are sequentially input into the decoder. Each decoder layer is constructed layer by layer by transposed convolutional layers and convolutional blocks. The feature maps are upsampled through transposed convolution, gradually restoring spatial resolution while reducing the number of channels. The output of each decoder stage is skip-connected with the output of the corresponding stage in the encoder to fuse multi-scale contextual information. The features after skip connections are further processed by convolutional blocks to improve semantic consistency. Finally, a 3D voxel-level segmentation result is output through a 1×1×1 convolutional layer and a sigmoid activation function.

[0033] Optionally, the convolutional module can be replaced with a hybrid attention module, which integrates channel attention and spatial attention mechanisms to enhance the model's perceptual capabilities. Channel attention performs global average pooling and max pooling on the input feature map to obtain the global context description of each channel, and then uses two shared 1×1×1 convolutional layers for non-linear transformation to generate channel weights. Spatial attention performs average pooling and max pooling operations along the channel dimension to obtain spatial attention-guided feature maps. These two maps are then concatenated along the channel dimension and input into a convolutional layer with a large receptive field to model spatial relationships. These two attention mechanisms are typically combined in a cascade manner, i.e., channel attention is applied first, followed by spatial attention adjustment, to strengthen useful features, suppress redundant information, and improve segmentation accuracy and robustness.

[0034] C. Loss Function and Training Optimization:

[0035] During training, a weighted combination of binary cross-entropy loss (BCE) and Tversky loss function is used as a supervision signal to alleviate class imbalance and enhance the model's ability to segment small lesion regions. The model optimizes the network weights using backpropagation by calculating the segmentation loss between the predicted output and the true label.

[0036] D. Iterative training and model convergence:

[0037] Repeat steps A–C above, continuously performing forward inference and backward parameter updates until the model segmentation loss meets the preset convergence condition or reaches the maximum number of training epochs. Finally, save the optimized network weights; the resulting model is the deep learning network model used for segmentation of AMD and DME lesions in the 3D fundus.

[0038] Optionally, the loss information between the segmentation result and the corresponding ground truth image can be determined in the following way:

[0039] Based on the prediction results generated by the segmentation network and the corresponding ground truth labeled images, the loss function of the model is calculated, specifically including:

[0040]

[0041]

[0042] in, This refers to the Tversky function. This represents the BCE function. This indicates the model's prediction results. This represents the manually labeled ground truth label corresponding to the prediction result, and its value is either 0 or 1. Parameter and These are used to adjust the weights for false positives and false negatives, respectively, for example, The value is 0.2. The value is 0.8.

[0043] Optionally, the total model loss can be calculated using the formula above:

[0044]

[0045] in, These are weighting coefficients used to balance the contributions of BCE and Tversky losses, for example, It can be set to 0.4.

[0046] Optionally, the parameters of the initial network model are optimized using an adaptive gradient descent (Adam) algorithm. The optimization process is based on backpropagation of the total loss result after the weighted combination of the BCE loss and Tversky loss to achieve iterative update of the model training parameters.

[0047] Secondly, embodiments of this application also provide an apparatus, including a processor, a storage medium, and a bus. The storage medium stores program instructions executable by the processor. When the program instructions are executed, the processor communicates with the storage medium through the bus and executes the steps in the three-dimensional fundus OCT image lesion segmentation method described in the first aspect of this application to achieve three-dimensional segmentation processing of the fundus lesion region.

[0048] Thirdly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when read and executed by a processor, can implement the steps in the three-dimensional fundus OCT image lesion segmentation method described in the second aspect of this application.

[0049] The beneficial effects of this application are as follows:

[0050] A method and apparatus for lesion segmentation in three-dimensional fundus OCT images are provided. The three-dimensional segmentation method, combined with a deep learning model, can achieve high-precision and automated three-dimensional segmentation of lesion areas, effectively generating three-dimensional lesion area segmentation results with clear structure and accurate boundaries, thereby providing reliable technical support for clinical diagnosis and disease progression assessment. Attached Figure Description

[0051] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 A schematic diagram illustrating the application scenario and structure of a three-dimensional segmentation method for retinal AMD or DME optical coherence tomography images provided in this application embodiment;

[0053] Figure 2 This is a flowchart illustrating a three-dimensional segmentation method for retinal AMD or DME optical coherence tomography images provided in an embodiment of this application.

[0054] Figure 3 This is a flowchart illustrating the method for constructing a three-dimensional OCT image dataset of fundus AMD and DME provided in an embodiment of this application.

[0055] Figure 4 This is a schematic diagram of the overall network structure of a three-dimensional fundus OCT image lesion segmentation method based on 3D U-Net and BiFormer modules, provided for an embodiment of this application.

[0056] Figure 5 A flowchart illustrating the initial model training method provided in this application embodiment;

[0057] Figure 6 A schematic diagram of the fundus AMD and DME three-dimensional OCT datasets provided in the embodiments of this application.

[0058] Figure 7 A schematic diagram showing the comparison of the segmentation effects of the fundus AMD and DME three-dimensional segmentation methods provided in the embodiments of this application on the test set.

[0059] Figure 8 This is a schematic diagram of the initial model training system structure provided in an embodiment of this application;

[0060] Figure 9 This is a structural block diagram of an apparatus provided in an embodiment of this application. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0062] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0063] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0064] Figure 1 This diagram illustrates an application scenario for a three-dimensional fundus OCT image lesion segmentation method provided in this application embodiment. Specifically, the lesion can be retinal AMD or DME. Figure 1 As shown, this application is applicable to scenarios where OCT images of any sample are acquired and three-dimensional lesion segmentation is performed. The sample may include biological samples, such as the fundus of the human eye. In this application scenario, an optical coherence tomography (OCT) system and apparatus are involved. First, the fundus region of the target patient is scanned using the OCT system to obtain the corresponding initial three-dimensional OCT image. Then, the initial diseased fundus OCT image is input into the apparatus, where a pre-trained three-dimensional segmentation network model performs three-dimensional segmentation processing on the image, thereby outputting a three-dimensional target lesion region image of fundus AMD or DME. This three-dimensional segmentation process can achieve automatic identification and precise localization of typical lesion regions in the retina (such as retinal pigment epithelium detachment (PED) or intraretinal fluid accumulation (IRF)), possessing good clinical application prospects and promotional value.

[0065] Figure 2 This is a flowchart illustrating a method for segmenting lesions in three-dimensional fundus OCT images, provided as an embodiment of this application. Figure 2 As shown, this method can be applied to the aforementioned device, which can be a terminal device with computing and display capabilities (such as a desktop computer, laptop computer, etc.) or a server device.

[0066] A method for lesion segmentation in three-dimensional fundus OCT images includes the following steps:

[0067] The initial fundus OCT image is input into a pre-trained three-dimensional lesion segmentation deep learning model, which performs three-dimensional segmentation on the initial fundus OCT image to obtain the three-dimensional segmentation result of the lesion area.

[0068] The 3D lesion segmentation deep learning model is a 3D U-Net model that integrates the BiFormer module.

[0069] Specifically, the 3D U-Net model that integrates the BiFormer module includes an encoder and a decoder;

[0070] The encoder includes four encoding modules: the first encoding module, the second encoding module, the third encoding module, and the fourth encoding module. Each encoding module includes multiple BiFormer modules.

[0071] The decoder includes four decoding modules and one output module. The four decoding modules are the first decoding module, the second decoding module, the third decoding module and the fourth decoding module. Each decoding module is connected to the corresponding encoding module in the encoder through skip connections to perform feature splicing and fusion, and the spatial resolution is gradually restored by using layer-by-layer upsampling operation.

[0072] The decoder has four decoding modules, each including transposed convolution and hybrid attention mechanism. The output module includes transposed convolution and convolutional blocks. The output module is followed by a 1×1×1 convolution, which is the output of the entire 3D U-Net model.

[0073] The BiFormer module is based on a dynamic sparse attention mechanism, which can effectively model long-distance dependencies in images while maintaining computational efficiency, thereby improving the network's ability to identify complex structures and fine-grained lesion regions.

[0074] A void space pyramid pooling module is provided in the jump connection between the first decoding module and the fourth encoding module;

[0075] Convolutional blocks are provided in the skip connections between the second decoding module and the third encoding module, between the third decoding module and the second encoding module, and between the fourth decoding module and the first encoding module. Specifically, the third encoding module, after passing through the convolutional block, performs feature concatenation and fusion with the output of the first decoding module before inputting it to the second decoding module; the second encoding module, after passing through the convolutional block, performs feature concatenation and fusion with the output of the second decoding module before inputting it to the third decoding module; the first encoding module, after passing through the convolutional block, performs feature concatenation and fusion with the output of the third decoding module before inputting it to the fourth decoding module; and the original image input to the encoder is processed by performing feature concatenation and fusion with the output of the fourth decoding module before inputting it to the output module.

[0076] To enhance the model's ability to identify lesions at multiple scales, a Spatial Pyramid Pooling (ASPP) module is introduced at the end of the encoder. This module applies multiple convolutions with different dilation rates in parallel to the feature map to extract rich contextual semantic information. In addition, the features upsampled at each stage of the decoder can be input into a hybrid attention module. This module integrates channel attention and spatial attention mechanisms to enhance the model's ability to focus on key feature regions, thereby further improving the model's perception capabilities and segmentation accuracy.

[0077] The first encoding module includes one Patch Embedding module and two BiFormer modules; the second encoding module includes one Patch Merging module and two BiFormer modules; the third encoding module includes one Patch Merging module and eight BiFormer modules; and the fourth encoding module includes one Patch Merging module and two BiFormer modules.

[0078] Each layer encoder uses the Patch Merging mechanism to achieve layer-by-layer spatial downsampling and channel dimension enhancement in order to extract image features at different levels.

[0079] In this specific implementation, assuming the input image has dimensions H×W×D×1, the dimensions after passing through the PatchEmbedding module in the first encoding module are H / 2×W / 2×D / 2×32, the dimensions after passing through the Patch Merging module in the second encoding module are H / 4×W / 4×D / 4×64, the dimensions after passing through the Patch Merging module in the third encoding module are H / 8×W / 8×D / 8×128, the dimensions after passing through the Patch Merging module in the fourth encoding module are H / 16×W / 16×D / 16×256, the dimensions after passing through the Spatial Pyramid Pooling (ASPP) module are H / 16×W / 16×D / 16×256, the dimensions of the convolutional blocks in the corresponding skip connections after passing through each encoding module are the same as the dimensions of the corresponding encoding modules, the dimensions after passing through the fourth decoding module are H×W×D×16, and finally the output dimension after passing through a 1×1×1 convolution returns to H×W×D×1.

[0080] Furthermore, before inputting the initial fundus OCT image into the pre-trained three-dimensional lesion segmentation deep learning model, the method further includes:

[0081] Construct the three-dimensional lesion segmentation deep learning model;

[0082] The initial three-dimensional lesion segmentation deep learning model is trained based on the sample dataset to obtain the three-dimensional lesion segmentation deep learning model.

[0083] Furthermore, before inputting the initial fundus OCT image into the pre-trained three-dimensional lesion segmentation deep learning model, the process also includes processing the sample dataset, including:

[0084] The dataset is divided into training, testing, and validation sets; preprocessing of the dataset includes data augmentation, image parameter adjustment, and size cropping.

[0085] Preferably, the loss function of the three-dimensional lesion segmentation deep learning model is:

[0086]

[0087]

[0088]

[0089] in, i represents the weighting coefficient used to balance the contributions of BCE and Tversky loss, for example, It can be set to 0.4. This refers to the Tversky function. This represents the BCE function. This indicates the model's prediction results. This represents the manually labeled ground truth label corresponding to the prediction result, with a value of 0 or 1. (Parameter) and These are used to adjust the weights for false positives and false negatives, respectively, for example, The value is 0.2. The value is 0.8.

[0090] Specifically, the training process of the three-dimensional lesion segmentation deep learning model includes the following steps:

[0091] A. Data preprocessing and encoder feature extraction:

[0092] Preprocessed volumetric image data from the training set is input into the network model in a 224×224×64 format. The encoder part of the model consists of four stages. In the first stage, Patch Embedding is used to perform preliminary image patching and feature embedding on the input image. In the second to fourth stages, Patch Merging is used to downsample the feature map, gradually halving the spatial dimension while increasing the channel dimension to enhance feature expressiveness. After each downsampling, the features are input to multiple BiFormer modules. The biformer module introduces a two-layer routing attention mechanism, which first divides the input features into multiple non-overlapping local regions, then calculates the global semantic importance of each region, and only retains the top k tokens most relevant to the current region for attention calculation. This mechanism can significantly reduce the computational cost when processing high-resolution 3D medical images while maintaining modeling capabilities. Optionally, a hollow spatial pyramid pooling module can be introduced at the end of the encoder. This module consists of structures such as 1×1×1 convolution, dilated convolution with different dilation rates, and global average pooling. It extracts contextual information through multi-scale receptive fields and integrates global feature representations to further enhance the model's feature expression capabilities.

[0093] B. Decoder reconstruction and segmentation output:

[0094] The multi-scale features extracted by the encoder are sequentially input into the decoder. The decoder is constructed layer by layer from transposed convolutional layers and convolutional blocks. Transposed convolutions upsample the feature maps, gradually restoring spatial resolution while reducing the number of channels. The output of each decoder stage is skip-connected to the output of the corresponding stage in the encoder to fuse multi-scale contextual information. The features after skip connections are further processed by convolutional blocks to improve semantic consistency. Finally, a 3D voxel-level segmentation result is output through a 1×1×1 convolutional layer and a sigmoid activation function. Optionally, the convolutional module of the decoding layer can be replaced with a hybrid attention module that integrates channel attention and spatial attention mechanisms. Channel attention uses global average pooling and max pooling to obtain global contextual information for each channel, and combines this with two shared 1×1×1 convolutional layers to generate channel weights. Spatial attention generates a spatial weight map based on the channel-dimensional average and max-pooled features, enhancing the model's focus on key spatial regions. These two attention mechanisms are combined in series to enhance attention to key regions.

[0095] C. Loss Function and Training Optimization:

[0096] During training, a weighted combination of binary cross-entropy loss (BCE) and Tversky loss function is used as a supervision signal to alleviate class imbalance and enhance the model's ability to segment small lesion regions. The model optimizes the network weights using backpropagation by calculating the segmentation loss between the predicted output and the true label.

[0097] D. Iterative training and model convergence:

[0098] Repeat steps A–C above, continuously performing forward inference and backward parameter updates until the model segmentation loss meets the preset convergence condition or reaches the maximum number of training epochs. Finally, save the optimized network weights; the resulting model is the deep learning network model used for segmentation of AMD and DME lesions in the 3D fundus.

[0099] Optionally, the loss information between the segmentation result and the corresponding ground truth image can be determined in the following way:

[0100] Based on the prediction results generated by the segmentation network and the corresponding ground truth labeled images, the loss function of the model is calculated, specifically including:

[0101]

[0102]

[0103] in, This refers to the Tversky function. This represents the BCE function. This indicates the model's prediction results. This represents the manually labeled ground truth label corresponding to the prediction result, and its value is either 0 or 1. Parameter and These are used to adjust the weights for false positives and false negatives, respectively, for example, The value is 0.2. The value is 0.8.

[0104] Optionally, the total model loss can be calculated using formulas (i) and (ii) above:

[0105]

[0106] in, These are weighting coefficients used to balance the contributions of BCE and Tversky losses, for example, It can be set to 0.4.

[0107] Optionally, the parameters of the initial network model are optimized using an adaptive gradient descent (Adam) algorithm. The optimization process is based on backpropagation of the total loss result after the weighted combination of the BCE loss and Tversky loss to achieve iterative update of the model training parameters.

[0108] In this specific implementation, the fundus lesions can be selected as age-related macular degeneration (AMD) or diabetic macular edema (DME). During the dataset acquisition phase, the images were collected from patients clinically diagnosed with AMD or DME, excluding case data with poor image quality due to factors such as refractive media opacity or image motion artifacts. The dataset in this specific implementation includes multiple pairs of 3D OCT images. Each image pair includes one 3D OCT image and a corresponding 3D OCT image of the manually annotated lesion region, which is either an AMD or DME lesion region.

[0109] The 3D OCT dataset used in this specific implementation contains a total of 224 sets of 3D OCT images, including 122 sets of AMD images and 102 sets of DME images: among the 122 sets of AMD images, there are 62 sets of images manually annotated by professional ophthalmologists and 60 sets of unannotated images, with the annotated area being pigment epithelial detachment (PED); among the 102 sets of DME images, there are 42 sets of images annotated by doctors and 60 sets of unannotated images, with the annotated area being intraretinal cystic effusion (IRF).

[0110] 1. Prior to image acquisition, subjects underwent a standard ophthalmological examination procedure, including refractive error and best-corrected visual acuity assessment, non-contact intraocular pressure measurement, axial length measurement, slit-lamp examination, wide-angle fundus imaging, and OCT scanning, to ensure the integrity and clinical usability of the acquired images.

[0111] 2. Select 3D OCT images from the acquired images that show clear lesion areas and contain PED or IRF structures;

[0112] 3. The original image size is 512×512×1044 pixels. A cube image with a size of 512×512×512 pixels is generated through cropping preprocessing to ensure that the central concave area is in the center of the image. All images are uniformly converted to 8-bit integer (uint8) format for easy processing.

[0113] 4. The images are independently annotated by three junior ophthalmologists. If there are any discrepancies in the annotation process, the images are discussed and re-annotated. The annotation results are then reviewed and corrected by a senior ophthalmologist with extensive experience to ensure that the final annotations meet clinical standards and diagnostic practices.

[0114] This dataset is characterized by high quality, high consistency, and standardized structure, and can provide highly reliable training and validation support for subsequent deep learning-based 3D image segmentation models.

[0115] The method includes the following steps:

[0116] S101. Obtain the initial three-dimensional OCT image of the fundus AMD or DME output by the OCT system.

[0117] Optionally, the OCT system may include components such as a light source, a coupler, a reference mirror, a sample arm, and a spectrometer. The specific principle is as follows: the system can use a broadband light source, such as a broadband light source with a center wavelength of 850 nm and a full width at half maximum (FWHM) of 165 nm. The light emitted by the light source is transmitted via the coupler to the sample arm and the reference arm, respectively, illuminating the sample under test and the reference mirror. The reflected light from the two optical paths converges in the coupler and forms an interference signal under certain conditions. The interference signal is output to the spectrometer, which transmits the detected signal to a computer. The computer processes the interference data and generates a grayscale image, thereby outputting the initial OCT image.

[0118] S102. Input the initial three-dimensional OCT image of fundus AMD or DME into the pre-trained three-dimensional segmentation network model. The network model performs three-dimensional segmentation on the initial OCT image to obtain the corresponding three-dimensional segmentation result image of the AMD or DME lesion area.

[0119] Optionally, the three-dimensional segmentation network model is a model trained according to the method provided in this application. The trained three-dimensional segmentation deep learning network model can be used in conjunction with the OCT system in step S101 above, that is, the initial three-dimensional OCT image obtained in S101 is input into the trained three-dimensional network model, and the model performs three-dimensional segmentation on the image and outputs the three-dimensional result image of the AMD or DME lesion area.

[0120] Through the above implementation method, it is possible to acquire initial three-dimensional fundus OCT images from the OCT system and use a deep learning model to achieve automatic three-dimensional segmentation of the lesion area, thereby obtaining high-quality three-dimensional segmentation results of AMD or DME lesion areas with high accuracy and automation.

[0121] Optionally, the 3D AMD and DME segmentation network model is a 3D U-Net structure that integrates BiFormer modules. Wherein:

[0122] The encoder employs a hierarchical BiFormer module structure for efficient extraction of multi-scale features from medical images. This encoder comprises four stages, each consisting of multiple BiFormer modules. A two-layer routing attention mechanism effectively combines local detail modeling with long-distance dependency information interaction, thereby enhancing global context modeling capabilities.

[0123] Spatial downsampling and channel dimension are achieved through Patch Merging operations between different stages, gradually extracting image features with deeper semantic layers and stronger expressive power.

[0124] The feature maps extracted at each stage are passed to the decoder through skip connections for layer-by-layer recovery of image information, improving the ability to restore image details and segmentation accuracy.

[0125] This network structure combines the modeling capabilities of BiFormer with the hierarchical feature representation mechanism of U-Net, making it particularly suitable for complex 3D medical image segmentation tasks, such as segmentation of AMD and DME lesion regions in the fundus.

[0126] Figure 3 This is a flowchart illustrating the method for constructing a three-dimensional OCT image dataset of fundus AMD and DME provided in an embodiment of this application. Figure 3 As shown, the process of building a dataset includes:

[0127] S201. Using the modified optical coherence tomography system, the fundus of patients diagnosed with AMD or DME is scanned to obtain multiple initial three-dimensional OCT images corresponding to each sample object.

[0128] Optionally, the initial 3D OCT images can be screened for quality, prioritizing images with clear lesion boundaries, good image quality, and no ambiguity as subsequent annotation objects;

[0129] S202. Each initial image group is used as the original image in the image pair, and manual annotation is performed according to the case type. Specifically, for AMD images, the PED region is annotated; for DME images, the IRF region is annotated.

[0130] PED is a highly characteristic pathological feature of AMD, presenting as a focal bulge between the retinal pigment epithelium (RPE) and Bruch's membrane. In OCT images, PED has a distinct structural morphology, clear boundaries, and is easily identifiable. Because its size, shape, and location are closely related to disease progression, marking PED not only aids in the diagnosis of AMD but also provides quantitative indicators for assessing subsequent treatment response, thus becoming an important target area in AMD segmentation tasks. IRF is one of the key pathological features of DME. In OCT images, IRF typically appears as a dark area with low reflectivity between retinal layers, exhibiting a typical cystic appearance. The presence of IRF is directly related to the degree of macular edema and visual function impairment; therefore, accurate marking and measurement of IRF areas have significant clinical value for assessing the severity of DME and monitoring treatment efficacy.

[0131] S203. After the annotations are completed, multiple professionals will check and verify the annotation results one by one. For annotation areas with discrepancies or uncertainties, discussions will be organized and revisions will be made. Finally, experienced ophthalmologists will conduct a final review to ensure the accuracy and clinical reliability of the annotations.

[0132] S204. Pair the initial image corresponding to each sample object with its labeled image to form an image pair; then classify and number all image pairs according to category to form a complete initial fundus OCT sample dataset.

[0133] Figure 4 This is a schematic diagram of the overall network structure of a three-dimensional fundus OCT image lesion segmentation method provided in an embodiment of this application.

[0134] Figure 5 This is a flowchart illustrating the initial model training method provided in an embodiment of this application. Figure 5 As shown, this method trains an initial model based on a pre-constructed sample dataset to obtain a network model capable of performing 3D AMD and DME lesion region segmentation. The method includes the following steps:

[0135] S301. Input the data from the preprocessed sample training set into the initial model according to the size of 224×224×64.

[0136] Optionally, image preprocessing operations may include conventional data augmentation strategies such as cropping, scaling, rotation, and flipping to improve sample diversity and model generalization ability. The dataset is divided into training, validation, and test sets in an 8:1:1 ratio to form a complete model training sample set.

[0137] For example, in a sample of 100 patient image pairs, 80 can be divided into 10 for training, 10 for validation, and the remaining 10 for testing the practical performance of the network model.

[0138] S302. The input data is first downsampled through four stages of the model encoder. Each stage contains multiple BiFormer modules. Through the attention mechanism, local structural information and long-distance dependencies across regions are effectively captured, thereby extracting multi-scale semantic features.

[0139] Spatial downsampling is achieved through Patch Merging, extracting 3D image features layer by layer while increasing channel dimensions. The BiFormer module integrates a multilayer perceptron (MLP) and a two-layer routing attention mechanism, which can enhance global context modeling capabilities while capturing local details, improving the model's recognition performance on complex medical structures. This module also has dynamic routing capabilities, which can adaptively select key region features, effectively reducing redundant information interference and enhancing the robustness and discriminative power of feature representation.

[0140] S303. The multi-scale features output by the encoder are sequentially fed into the decoder for layer-by-layer spatial resolution restoration. At each stage, skip connections are used to fuse the features from the corresponding stage in the encoder with the current features in the decoder before upsampling. Finally, a 1×1×1 convolutional layer is used to map to the number of class channels, and a 3D voxel-level segmentation prediction result is output through a sigmoid activation function.

[0141] The upsampling process is achieved through transposed convolution. At the same time, an additional convolutional module is introduced after the skip connections and transposed convolution to extract richer and more refined semantic information, thereby improving the accuracy and consistency of the segmentation results.

[0142] S304. During training, a weighted combination of BCE and Tversky loss functions is used to calculate the error between the prediction result and the true label.

[0143] S305. The model performs backpropagation and optimization updates of parameters based on the loss function, and after satisfying the preset convergence, saves the current network weights as the final model for 3D medical image segmentation.

[0144] Steps S301 to S305 described above will be executed iteratively throughout the training set. Specifically, image pairs are selected sequentially from the training set as input, and steps S301 to S305 are executed to continuously adjust the network parameters and optimize the segmentation performance. After each iteration, the model performance can be evaluated using validation set data. When the segmentation loss meets the set convergence criteria, the final weight parameters of the network model are saved, and the resulting model is the target network model capable of performing 3D segmentation tasks for fundus AMD and DME lesion areas.

[0145] Figure 6A schematic diagram of the fundus AMD and DME three-dimensional OCT dataset provided in an embodiment of this application. Figure 6 As shown, Figures (a1) and (b1) are raw fundus AMD 3D OCT images containing a large number of B-Scan slices, respectively; Figures (c1) and (d1) are raw fundus DME 3D OCT images containing a large number of B-Scan slices, respectively. Figures (a2)-(d2) are 3D lesion images and their 2D B-Scan slice images manually annotated by ophthalmologists corresponding to the above images, used to represent the annotation results of different lesion areas (such as PED and IRF).

[0146] Figure 7 A schematic diagram comparing the segmentation effects of the fundus AMD and DME three-dimensional segmentation methods provided in the embodiments of this application on a test set. Figure 7 As shown, Figures (a1) and (b1) are the original fundus AMD 3D OCT images, and Figures (c1) and (d1) are the original fundus DME 3D OCT images; Figures (a2)-(d2) show the 3D lesion regions labeled by ophthalmologists corresponding to the above images; Figures (a3)-(d3) show the 3D lesion regions predicted by the segmentation network model trained based on the method of this application. It can be intuitively observed from the figures that the predicted 3D segmentation results not only closely approximate the morphological boundaries of the actual lesions but also preserve the surface details of the lesions well, verifying the accuracy and reliability of the model in complex medical image segmentation tasks.

[0147] Figure 8 This is a schematic diagram of the initial model training system structure provided in an embodiment of this application;

[0148] Figure 9 This is a structural block diagram of a device provided in an embodiment of this application. Figure 9 As shown, the device 300 may include: a processor 301 and a memory 302.

[0149] Optionally, the device may further include a bus 303. The memory 302 stores machine-readable instructions executable by the processor 301. When the device 300 is running, the processor 301 communicates with the memory 302 via the bus 303 and executes the machine-readable instructions to complete the steps described in the foregoing method embodiments of this application.

[0150] Furthermore, embodiments of this application also provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, is used to implement all the steps in the method embodiments of three-dimensional segmentation of fundus AMD and DME described in this application.

[0151] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some communication interfaces; the indirect coupling or communication connection of devices or modules can be electrical, mechanical, or other forms.

[0152] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0153] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for lesion segmentation in three-dimensional fundus OCT images, characterized in that, Includes the following steps: The initial fundus OCT image is input into a pre-trained three-dimensional lesion segmentation deep learning model, which performs three-dimensional segmentation on the initial fundus OCT image to obtain the three-dimensional segmentation result of the lesion area. The 3D lesion segmentation deep learning model is a 3D U-Net model that integrates the BiFormer module.

2. A method for lesion segmentation in three-dimensional fundus OCT images according to claim 1, characterized in that: The 3D U-Net model that integrates the BiFormer module includes an encoder and a decoder; The encoder includes four encoding modules: the first encoding module, the second encoding module, the third encoding module, and the fourth encoding module. Each encoding module includes multiple BiFormer modules. The decoder includes four decoding modules and one output module. The four decoding modules are the first decoding module, the second decoding module, the third decoding module and the fourth decoding module. Each decoding module is connected to the corresponding encoding module in the encoder through skip connections to perform feature splicing and fusion, and the spatial resolution is gradually restored by using layer-by-layer upsampling operation. The decoder has four decoding modules, each including transposed convolution and a hybrid attention mechanism. The output module includes transposed convolution and convolution blocks. The output module is followed by a 1×1×1 convolution, which is the output of the entire 3D U-Net model. The hybrid attention mechanism includes channel attention and spatial attention mechanisms connected in sequence.

3. The method for segmenting lesions in three-dimensional fundus OCT images according to claim 2, characterized in that: A void space pyramid pooling module is provided in the jump connection between the first decoding module and the fourth encoding module; Convolutional blocks are provided in the skip connections between the second decoding module and the third encoding module, between the third decoding module and the second encoding module, and between the fourth decoding module and the first encoding module.

4. The method for segmenting lesions in three-dimensional fundus OCT images according to claim 2, characterized in that: The first encoding module includes one Patch Embedding module and two BiFormer modules; the second encoding module includes one Patch Merging module and two BiFormer modules; the third encoding module includes one Patch Merging module and eight BiFormer modules; and the fourth encoding module includes one Patch Merging module and two BiFormer modules.

5. A method for lesion segmentation in three-dimensional fundus OCT images according to claim 1, characterized in that, Before inputting the initial fundus OCT image into the pre-trained three-dimensional lesion segmentation deep learning model, the process also includes: Construct the three-dimensional lesion segmentation deep learning model; The initial three-dimensional lesion segmentation deep learning model is trained based on the sample dataset to obtain the three-dimensional lesion segmentation deep learning model.

6. A method for lesion segmentation in three-dimensional fundus OCT images according to claim 1, characterized in that, Before inputting the initial fundus OCT image into the pre-trained three-dimensional lesion segmentation deep learning model, the process also includes processing the sample dataset, including: The dataset is divided into training, testing, and validation sets; preprocessing of the dataset includes data augmentation, image parameter adjustment, and size cropping.

7. A method for segmenting lesions in three-dimensional fundus OCT images according to claim 4, characterized in that, The loss function of the three-dimensional lesion segmentation deep learning model is: ； ； ； in, This represents the weighting coefficient, used to balance the contributions of BCE and Tversky losses. This refers to the Tversky function. This represents the BCE function. This indicates the model's prediction results. This represents the manually labeled ground truth label corresponding to the prediction result, with a value of 0 or 1. (Parameter) and These are used to adjust the weights for false positives and false negatives, respectively.

8. A three-dimensional fundus OCT image lesion segmentation device, characterized in that, The device includes a processor, memory, and a bus. The memory stores machine-readable instructions that the processor can execute. When the device is running, the processor communicates with the memory via the bus and executes the machine-readable instructions to perform the following methods: The initial fundus OCT image is input into a pre-trained three-dimensional lesion segmentation deep learning model, which performs three-dimensional segmentation on the initial fundus OCT image to obtain the three-dimensional segmentation result of the lesion area. The deep learning model for 3D lesion segmentation is a 3D U-Net model that integrates the BiFormer module; The 3D U-Net model that integrates the BiFormer module includes an encoder and a decoder; The encoder includes four encoding modules: the first encoding module, the second encoding module, the third encoding module, and the fourth encoding module. Each encoding module includes multiple BiFormer modules. The decoder includes four decoding modules and one output module. The four decoding modules are the first decoding module, the second decoding module, the third decoding module and the fourth decoding module. Each decoding module is connected to the corresponding encoding module in the encoder through skip connections to perform feature splicing and fusion, and the spatial resolution is gradually restored by using layer-by-layer upsampling operation. The decoder has four decoding modules, each including transposed convolution and hybrid attention mechanism. The output module includes transposed convolution and convolutional blocks. The output module is followed by a 1×1×1 convolution, which is the output of the entire 3D U-Net model.

Images