A method and system for retinal image segmentation

Abstract

The application relates to a retinal image segmentation method and system, which comprises the following steps: acquiring a multi-modal retinal image dataset and performing pretreatment; projecting the retinal image data; dividing the projected data into multiple groups of data; extracting the intra-group shallow features of each group of data, and performing splicing and transformation to obtain shallow structure features; performing feature enhancement on the reweighted features and fusing the feature enhancement branch to obtain enhanced features; splicing and fusing the enhanced features with the structure maintaining branch to obtain deep semantic features; performing feature mapping and feature enhancement on the deep semantic features to obtain decoding features; based on a spatial attention graph, performing element-by-element modulation on the second fusion features, and adding and convolving the modulated features with the second fusion features to obtain final fusion features; and based on the final fusion features, outputting a target retinal image segmentation result. Compared with the prior art, the application can realize high-precision segmentation of complex target structures.

Description

Technical Field

[0001] This invention relates to the field of medical image segmentation technology, and in particular to a method and system for retinal image segmentation. Background Technology

[0002] Medical image segmentation is a crucial foundation for disease analysis and assisted diagnosis. Existing medical image segmentation techniques primarily include methods based on 3D volumetric data and methods based on 2D images. While 3D volumetric data-based methods can utilize spatial structural information, they typically suffer from high computational costs. Traditional 2D methods, although computationally cheaper, have limited ability to represent complex structures, fine-grained targets, and boundary regions due to spatial information compression and detail loss, easily leading to structural breaks, missed details, and blurred boundaries. Furthermore, many existing deep learning segmentation models still rely heavily on traditional feature mapping mechanisms such as convolution, Transformer, or Multilayer Perceptron (MLP). MLPs, often consisting of alternating linear transformations and fixed activation functions, lack intuitive visualization of the internal feature mapping process, making the model's decision-making mechanism difficult to understand. In medical image segmentation, this black-box nature weakens the model's interpretability and credibility, thus limiting its potential application in assisted diagnosis.

[0003] The invention disclosed in CN114119638A presents a medical image segmentation method that integrates multi-scale features and an attention mechanism. First, a medical image segmentation model is constructed based on UNet, comprising an encoder-decoder, a multi-scale feature extraction module, and an attention module. This model takes the image to be segmented as input. Then, the high-level feature map obtained from the encoder and the low-level feature map, which integrates multi-scale information from the multi-scale feature extraction module, are input into the attention module to obtain a feature map that integrates the relationship between local and global information. Finally, the feature map is input into the decoder for upsampling to restore the image resolution, resulting in the corresponding segmentation result. This medical image segmentation method utilizes the multi-scale feature extraction module to integrate multi-scale information of the image, while employing an attention mechanism to focus on the most correlated information, improving the model's ability to model long-distance dependencies and thus enhancing the segmentation accuracy of medical images. However, this scheme has weak feature representation capabilities and fine-grained information representation capabilities for retinal images with complex structures. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a retinal image segmentation method and system. It constructs multi-channel features through statistical projection representation strategy to enhance the expression of input features, and combines group feature modeling, multi-scale structure modeling and cross-layer feature reconstruction mechanism to improve the model's ability to segment complex target structures.

[0005] This invention aims to address the following technical problems: how to improve the ability of two-dimensional input to represent spatial structural information while maintaining low computational overhead; how to enhance the model's ability to segment and preserve the structure of complex structures, fine-grained targets, and boundary regions; and how to construct a medical image segmentation framework that combines high segmentation performance with interpretability. In a specific embodiment, a nonlinear modeling mechanism based on the Kolmogorov-Arnold Network (KAN) can be used to enhance feature representation and model interpretability. Combined with the interpretability quantification index Plausibility IoU (PIoU), the consistency between the model's region of interest and the real structure can be analyzed, thereby improving the model's credibility and application value in medical image segmentation tasks.

[0006] The objective of this invention can be achieved through the following technical solutions: A retinal image segmentation method, characterized in that it includes: A multimodal retinal image dataset is acquired and preprocessed; the retinal image data is subjected to maximum intensity projection, vessel count projection, and standard deviation projection to obtain projected data; the projected data is divided into multiple groups; the shallow features within each group are extracted and spliced ​​and transformed in the channel dimension to obtain shallow structural features; The shallow structural features are transformed and then divided into a structure-preserving branch and a feature-enhancing branch along the channel dimension. The feature-enhancing branch is subjected to channel mapping and multi-scale feature fusion to obtain a first fused feature. The first fused feature is subjected to dimensionality reduction transformation and orientation-aware convolution to generate attention weights. The attention weights are multiplied element-wise with the first fused feature to obtain a reweighted feature. The reweighted features are enhanced and fused with the feature enhancement branch to obtain enhanced features; the enhanced features are concatenated and fused with the structure-preserving branch to obtain deep semantic features; the deep semantic features are then subjected to feature mapping and feature enhancement to obtain decoded features. The shallow structural features and deep semantic features are respectively channel aligned and feature remodeled, and then the remodeled features are fused to obtain a second fused feature; the decoded features are used as semantic guidance signals to guide the generation of a spatial attention map; based on the spatial attention map, the second fused feature is modulated element-wise, and the modulated features are summed and convolved with the second fused feature to obtain the final fused feature; based on the final fused feature, the target retinal image segmentation result is output.

[0007] Furthermore, the retinal image data is subjected to maximum intensity projection, vessel count projection, and standard deviation projection, with the corresponding projection expressions being as follows: in, Indicates the maximum intensity projection. This indicates the search range along the y-direction. Represents OCT volume data, Represents the projection of blood vessel count. Represents the standard deviation projection. This represents the total number of voxels contained in the anatomical region S at the projection position (x,z). Let represent the intensity value of the k-th voxel in the anatomical region S at the projection position (x,z). This indicates the intensity threshold used when generating the vessel count projection. This represents the standard deviation of the voxel signal intensity along the depth direction within the anatomical region S at the projection position (x,z).

[0008] Furthermore, the expressions for obtaining shallow structural features include: in, This represents the output features obtained after cross-group feature fusion and channel compression, i.e., shallow structure features. This represents the GroupNorm operation, and PReLU() represents the parameterized linear rectified activation function. This represents a 1×1 projective convolution. This represents the sets of features concatenated along the channel dimension. This represents the features extracted from the i-th channel group through independent learning branches, and Concat() represents the concatenation operation. B represents the real space where the feature tensor resides, C represents the batch size, H represents the number of output channels for each independent learning branch, and H and W represent the height and width of the feature map, respectively. This represents a convolutional feature extraction module consisting of two convolutional layers, GroupNorm, and PReLU. This indicates that all channel features corresponding to the channel index set are selected from the input feature X. This represents a group of channel indexes with similar semantic attributes.

[0009] Furthermore, channel mapping and multi-scale feature fusion are performed on the feature enhancement branch to obtain the first fused feature, and the corresponding fusion expression is: in, Indicates the first fusion feature, This represents a fusion operation consisting of GroupNorm, PreLU, and a 1×1 convolution. To maintain branches in the structure, For the feature enhancement branch, For downsampling features, This represents a depthwise convolution with a kernel size of k. This represents the features obtained after performing a pre-projection transformation on the feature enhancement branch. Represents the characteristic tensor of real numbers. Indicates the output channel.

[0010] Furthermore, the enhanced features are concatenated and fused with the structure-preserving branch to obtain deep semantic features. The corresponding concatenation and fusion expression is as follows: in, This represents the fusion operation consisting of convolution, normalization, and activation. This represents the combined features after concatenation; Concat() indicates the concatenation operation. To maintain branches in the structure, For the feature enhancement branch, Indicates the first fusion feature, This represents the fused features obtained after feature fusion transformation. The final output features obtained after recalibration via the channel attention mechanism are the subsequent deep semantic features. For the enhanced features obtained through residual connections, For channel recalibration features, This represents the Sigmoid function. This represents element-wise multiplication. For attention weights, Expand() represents the channel-dimensional convolution recovery. For heavily weighted features, SE(·) represents the compression-excitation channel attention mechanism, StripDWConv(·) represents the strip depth convolution operation, and Reduce(·) represents the dimensionality reduction transformation.

[0011] Furthermore, feature mapping and feature enhancement are performed on the deep semantic features to obtain decoded features. The corresponding feature enhancement expressions and feature mapping expressions are as follows: in, Indicates the characteristics of the initial token. This indicates a tokenization operation. Indicates the first Each stage inputs features of the ToK-KAN module. Indicates the first The token characteristics output by each ToK-KAN module This represents a depthwise convolution operation. express Network nonlinear feature mapping, This represents the output features obtained after ToK-KAN enhancement. For the decoded features of the output, This indicates a feature normalization operation.

[0012] Furthermore, the shallow structural features and deep semantic features are respectively channel aligned and feature reshaping performed, and then the reshaping features are fused to obtain the second fused feature. The corresponding fusion expression is: in, This is the second fusion feature. (·) represents the PReLU activation function. These are the deep semantic features after feature remodeling. These are the shallow structural features after feature remodeling. This indicates a GroupNorm operation. For convolutional kernels used to fuse and refine reconstructed features, This is the convolutional kernel used for channel mapping of the upsampled deep semantic features. For deep semantic features, This is a shallow structural feature. This is a convolutional kernel used for channel mapping of shallow structural features. This indicates bilinear interpolation upsampling.

[0013] Furthermore, the decoded features are used as semantic guidance signals to guide the generation of a spatial attention map, and the corresponding guidance expression is: in, This is a spatial attention map. For the convolutional kernel used to generate spatial attention responses, (·) denotes the PReLU activation function, and G denotes the decoding semantic guidance feature obtained from the decoding feature map. This represents the fusion mapping feature obtained from the second fusion feature mapping. This indicates a GroupNorm operation. The convolution kernel used for channel mapping of the second fused feature. Indicates the second fusion feature, Indicates decoding features, The convolution kernel used for channel mapping of decoded features. This represents a two-dimensional convolution operation.

[0014] Furthermore, based on the spatial attention map, the second fused feature is modulated element-wise, and the corresponding modulation expression is: in, This is the modulated second fusion feature; The modulated features are summed and convolved with the second fused feature to obtain the final fused feature. The corresponding summation and convolution expressions are as follows: in, This is the convolution kernel used for output projection and channel mapping of the additive residual features. This indicates the final fusion feature.

[0015] The present invention also provides a system for implementing the above-described retinal image segmentation method, comprising a memory and a processor, wherein the memory stores a computer program, and the processor invokes the computer program to execute the steps of the above-described method.

[0016] Compared with the prior art, the present invention has the following advantages: (1) This invention divides the initially extracted shallow structural features into a structure-preserving branch and a feature enhancement branch, performs channel mapping and multi-scale feature fusion on the feature enhancement branch, and performs element-wise reweighting by combining the attention weights generated by dimensionality reduction transformation and orientation-aware convolution. This can adaptively highlight key vascular regions and suppress background noise, significantly improving the sensitivity of features to small blood vessels and blurred boundary regions. At the same time, the reweighted feature enhancement results are fused with the feature enhancement branch and then spliced ​​with the structure-preserving branch. This preserves the integrity of the original spatial structure and integrates deep semantic information, so that the deep semantic features have both high-resolution spatial details and strong semantic discrimination ability. Finally, by using the decoded features as semantic guidance signals to generate spatial attention maps and perform cross-layer feature interaction and semantic guidance, the fine-grained structure recovery capability and spatial semantic consistency of the final fused features are improved, thereby improving the overall segmentation quality of complex retinal images.

[0017] (2) This invention projects and groups retinal image data separately, extracts shallow features within each group, and splices and transforms them in the channel dimension. This preserves the differences between features from different sources and reduces early feature aliasing. By performing further nonlinear feature mapping and feature enhancement on the obtained deep semantic features, the final decoded features have a strong ability to express complex structures, which is beneficial for the subsequent model to analyze the consistency between the image data's focus area and the real structure, thereby improving the credibility and interpretability of the model's prediction results.

[0018] (3) By introducing a statistical projection strategy, the present invention performs maximum intensity projection, blood vessel count projection and standard deviation projection on retinal image data respectively. This can fully utilize the structural and signal change information of different depth regions while compressing three-dimensional volume data, forming a multi-channel input representation with richer information expression, thereby improving the ability of two-dimensional input to express complex target structures and fine-grained information. Attached Figure Description

[0019] Figure 1 This is a flowchart of a retinal image segmentation method provided in an embodiment of the present invention; Figure 2 This is a diagram of the VesselKANNet network framework for a retinal image segmentation method provided in an embodiment of the present invention. Figure 3 This is a schematic diagram of the SPR multi-statistical projection strategy for a retinal image segmentation method provided in an embodiment of the present invention; Figure 4 This is a GMS grouping modality flow graph of a retinal image segmentation method provided in an embodiment of the present invention; Figure 5This is a schematic diagram of the MSD multi-scale downsampling module of a retinal image segmentation method provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the CAR cross-layer attention reconstruction module of a retinal image segmentation method provided in an embodiment of the present invention; Figure 7 This is a visual comparison example of the activation interpretability of arterial and venous structures with and without KAN, provided in an embodiment of the present invention. Figure 8 This is a comparative experimental visualization diagram of a retinal image segmentation method provided in an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0021] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0022] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0023] Example 1 like Figure 1 As shown, this embodiment provides a retinal image segmentation method, which includes the following steps: S1: Acquire and preprocess the multimodal retinal image dataset; perform maximum intensity projection, vessel count projection, and standard deviation projection on the retinal image data to obtain the projected data; divide the projected data into multiple groups; extract the shallow features within each group and stitch and transform them in the channel dimension to obtain the shallow structural features; S101: Data Acquisition and Input: Acquire and preprocess medical image datasets. Generate multi-channel input representations using feature construction strategies to enhance the ability of two-dimensional inputs to express spatial structure and target information. Taking retinal images from the OCTA-500 dataset as a specific example, multi-channel features can be generated using statistical projection, and combined with existing two-dimensional projections B1-B6 as network input.

[0024] Preferred, In terms of input construction, this embodiment uses the 6-channel 2D projection provided by the OCTA-500 dataset as the basic input, and further utilizes the original OCT / OCTA 3D volume data and retinal layer structure segmentation annotation to generate 9-channel projection features using a multi-statistical projection strategy, thus forming a total of 15-channel multimodal input representation. (Maximum Intensity Projection (MIP), Vessel Count Projection, and Standard Deviation Projection are collectively referred to as multi-channel projection features generated by the Statistical Projection Representation strategy).

[0025] Preferred, like Figure 2 As shown, for each OCT volume data V OCT Using retinal layer-by-layer annotation information, including the inner limiting membrane (ILM), outer plexiform layer (OPL), and Bruch's membrane (BM), the volume data was divided into two regions along the depth direction: the surface ILM-OPL and the deep OPL-BM. For these two anatomical sub-regions and the entire ILM-BM region, a corresponding one-dimensional depth sequence was extracted at each lateral position (x, z). Subsequently, to preserve high-intensity response information along the depth direction, the maximum intensity projection operator (MIP) was applied across each interlayer depth range. The calculation process is shown in the following equation, where... The results are dynamically determined by the layer segmentation results (ILM, OPL, BM) at the corresponding positions. For each OCT volume data, its projected layering region is consistent with that of the OCT. Based on the retinal layer segmentation results (ILM, OPL, and BM), at each lateral position... The corresponding interlayer depth range is dynamically determined, and a one-dimensional intensity sequence within this range is extracted. We construct vessel count projection and standard deviation projection from two perspectives: voxel distribution exceeding the threshold and intensity statistical characteristics. Vessel count projection (VC) is constructed by statistically analyzing the intensity of voxels exceeding a fixed threshold in the depth sequence. The number of voxels (this threshold parameter was determined experimentally, and experimental results show that this threshold can achieve optimal performance), and based on the total number of voxels within this interlayer range. Normalization is performed, and the result is then mapped to a grayscale range of [0, 255] to characterize the proportion of high-response voxels along the depth direction at that location. The calculation process is defined as follows: This characterizes the distribution characteristics of high-response voxels within each layer region. The standard deviation projection (STD) represents the one-dimensional signal standard deviation along the depth direction at that location: To enhance grayscale contrast, it is linearly scaled up (scale factor of 5) to fully utilize the 8-bit grayscale representation range and limit the result to the range of 0-255. Here, 255 represents the maximum value of the grayscale image. S102: Grouped Projection Stem (GPS): To address the issues of large modal differences and premature feature aliasing in multimodal data, Grouped Projection Stem (GPS) is used to semantically group the input channels. Shallow features of each group are extracted through independent convolutional branches. After concatenating the grouped features, 1×1 convolution is used to complete channel fusion and compression, preserving the differences between multi-source features and outputting structured shallow features.

[0026] Preferred, To address the differences in modal sources and statistical characteristics of multi-projection OCT / OCTA data, this embodiment designs a grouped projection stream (GPS), such as... Figure 3 As shown, this is used for early feature extraction for modality awareness before the encoder. Different projection channels are generated by different 3D structures or statistical transformations. If all channels are directly input into the same convolutional layer, features from different sources may be mixed prematurely in the initial stage. To address this, GPS first divides the input channels into several groups according to modality semantics and extracts shallow features within each group through independent convolutional branches.

[0027] Let the input be The channel index is divided into Each of them This represents a set of channel indices with similar semantic attributes. For the i-th modality group, GPS uses a separate convolutional submodule for feature extraction: in This represents a convolutional feature extraction module consisting of two convolutional layers, GroupNorm, and PReLU. Subsequently, the features from each group are concatenated along the channel dimension. To achieve inter-group information fusion and channel compression, this paper employs a fusion module based on 1×1 convolution to perform linear mapping, normalization, and nonlinear transformation on the concatenated features, obtaining the final output: in, For 1×1 projective convolution, GroupNorm (GPS) provides a more structured input representation for subsequent encoders through independent modeling within groups and unified fusion between groups, and helps to preserve the differences in features from different sources in the early stages.

[0028] S2: Perform feature transformation on the shallow structure features, and then divide them into structure-preserving branches and feature enhancement branches along the channel dimension; perform channel mapping and multi-scale feature fusion on the feature enhancement branches to obtain the first fused feature; perform dimensionality reduction transformation and orientation-aware convolution on the first fused feature to generate attention weights; multiply the attention weights element-wise with the first fused feature to obtain the reweighted feature; Multi-Scale Structure Downsampling (MSD): To address the issue of fine-grained structural information loss caused by traditional pooling operations, a multi-scale structure downsampling module is employed to achieve spatial dimensionality reduction. This module works collaboratively with structure-preserving branches and feature-enhancing branches, combining multi-scale convolution, orientation-aware attention, and channel attention mechanisms to preserve key structural information and boundary features while reducing spatial resolution.

[0029] Preferred, In the encoder downsampling stage, traditional pooling can easily lead to the loss of fine-grained structural information during spatial compression, thus affecting the modeling ability of slender structures such as retinal vessels. To address this, this paper proposes a downsampling structure that integrates multi-scale depthwise separable convolution and orientation-aware attention mechanisms to enhance structural representation capabilities after feature map downsampling. Figure 4 As shown, given input features First, a 3×3 convolution with a stride of 2 is used for spatial downsampling, and then GroupNorm and PReLU are combined to complete the feature transformation and generate the output channel. ; Subsequently, the feature Y is divided into two sub-branches along the channel dimension: the structure-preserving branch. With feature enhancement branch : in, Used to retain basic structural information. This is then used to further enhance the ability to express features.

[0030] After completing the basic spatial compression and channel reshaping, to further improve the scale adaptability of downsampled features, we introduced a multi-scale deep convolution mechanism in the feature enhancement branch. First, a 1×1 convolution combined with GroupNorm and PReLU is used to... Perform channel mapping, denoted as Subsequently, a set of parallel depthwise convolutions is used to model the local context at different scales, with the following kernel set: Multi-scale features are fused along the channel dimension and further integrated through normalization and nonlinear transformation to obtain: in, Indicates the kernel size as Depth convolution, This represents a fusion operation consisting of GroupNorm, PreLU, and a 1×1 convolution.

[0031] While multi-scale deep convolutions can effectively integrate local contextual information at different scales, their modeling ability for directional, elongated retinal vascular structures remains limited. Therefore, a direction-aware attention mechanism is further introduced into this branch to enhance the modeling ability for directional structures. First, the features are reduced in dimension. Then, spatial dependencies are modeled using direction-aware deep convolutions (StripConv) (1×K DWConv and K×1 DWConv), and channel dimensions are restored using 1×1 convolutions. Finally, attention weights are generated using the Sigmoid function and multiplied element-wise with the original feature map to achieve feature reweighting. in, This represents the Sigmoid function. This indicates element-wise multiplication.

[0032] Furthermore, to further highlight important channel information and suppress redundant features, a compressed-excitation channel attention mechanism is introduced: Subsequently, the enhanced features are fused with the original branch features through residual connections to stabilize the training process and preserve the original information: Finally, the enhanced feature Z is combined with the structure-preserving branch. The channels are concatenated and then fused using a 1×1 convolution to obtain the output features. in, This represents a fusion operation consisting of convolution, normalization, and activation. Compared to traditional downsampling methods, this module effectively preserves key structural information and enhances feature expressiveness during the reduction of spatial resolution through the synergistic effects of multi-scale spatial modeling, orientation-aware enhancement, and channel reweighting.

[0033] S3: Enhance the reweighted features and fuse them with the feature enhancement branch to obtain enhanced features; concatenate and fuse the enhanced features with the structure-preserving branch to obtain deep semantic features; perform feature mapping and feature enhancement on the deep semantic features to obtain decoded features; Nonlinear Interpretable Modeling and Quantitative Evaluation: Nonlinear modeling units are introduced during the network feature modeling stage. Feature mapping is achieved through learnable nonlinear functions to enhance the model's ability to express complex structures and nonlinear relationships, and to improve the interpretability of the model's feature mapping process. Furthermore, interpretability quantitative evaluation metrics are used to analyze the consistency between the model's focus area and the real structure. Taking retinal multi-object segmentation as a specific example, Tok-KAN can be used as the nonlinear modeling unit, and the Plausibility IoU (PIoU) metric can be used to achieve interpretability quantitative analysis of the multi-object structure.

[0034] Preferred, This embodiment introduces a KAN-based nonlinear modeling mechanism for two-dimensional retinal vessel and multi-object segmentation tasks, and uses the Tok-KAN module from UKAN as the core feature modeling unit. This module achieves feature mapping through a learnable one-dimensional nonlinear function, which enhances the network's ability to express complex vascular structures nonlinearly and improves the model's interpretability. The Tok-KAN module is placed in the network bottleneck layer and decoding stage for high-level semantic feature extraction and feature refinement, thereby improving the segmentation ability of microvessels, arteries, veins, and other structures.

[0035] Unlike traditional multilayer perceptrons (MLPs) that rely on linear weights and fixed activation functions, KAN is based on the Kolmogorov-Arnold representation theorem and achieves feature mapping through a learnable one-dimensional nonlinear function. The nonlinear feature transformation of KAN can be expressed as: in, This represents the matrix of learnable nonlinear functions in the Kth layer.

[0036] It also combines depthwise convolution, residual connections, and normalization mechanisms to achieve feature enhancement: In this context, the enhanced coded features output from the deepest layer in the encoding stage serve as the initial input to the deepest layer in the decoding stage. The connection between the deepest enhanced coded features and the input in the decoding stage can be represented as follows: Specifically, First, in the encoding stage, each deep encoding feature is processed. Tokenization is performed, and the ToK-AN module is used for nonlinear feature mapping and spatial feature enhancement to obtain enhanced encoded features. ; Encode the deepest enhanced features The bottleneck semantic features are input into the decoding stage. Subsequently, in the decoding stage, the bottleneck semantics... The process involves progressive upsampling, and the resulting decoded features are input into the ToK-KAN module for tokenization, nonlinear feature mapping, and spatial feature enhancement to obtain enhanced decoded features. Enhance decoding features This continues until the final segmentation result is obtained, at the next decoding stage. This represents the hierarchical index of the network stage. This indicates the sequence number of the ToK-KAN block in the current network phase; Indicates the first The features input to the ToK-KAN module at each stage are encoded as features in the encoding stage. In the decoding stage, it is a decoding feature. ; Indicates a tokenization operation; This represents the initial token feature obtained after the input features have been tokenized; Indicates the first The token characteristics output by each ToK-KAN module; This represents the nonlinear feature map of the Kolmogorov-Arnold network; Indicates a depthwise convolution operation; Presentation layer normalization operation; This indicates the total number of ToK-KAN modules in the current stage; This indicates a feature normalization operation; This represents the operation of restoring the token features to a spatial feature map; This represents the output features obtained after ToK-KAN enhancement, which are then used as enhanced encoded features during the encoding stage. To enhance decoding features during the decoding stage .

[0037] Furthermore, a multi-objective Plausibility IoU (PIoU) interpretability evaluation mechanism was constructed. This mechanism measures the consistency between the network's attention region and the anatomical structure by calculating the overlap between the thresholded attention maps of KAN and MLP and their corresponding ground truth (GT). Experimental results show that the present invention with the KAN mechanism can generate more continuous and stable responses in complex vascular structures and exhibits superior feature representation capabilities for microvessels and complex topologies. Compared to traditional MLP models, it demonstrates better interpretability and structural consistency, thereby improving the model's credibility and application value in medical auxiliary diagnosis and treatment scenarios.

[0038] S4: Channel alignment and feature reshaping are performed on the shallow structural features and deep semantic features respectively. Then, the reshaping features are fused to obtain the second fused feature. The decoded features are used as semantic guidance signals to guide the generation of a spatial attention map. Based on the spatial attention map, the second fused feature is modulated element-wise, and the modulated features are summed and convolved with the second fused feature to obtain the final fused feature. Based on the final fused feature, the target retinal image segmentation result is output.

[0039] S401: Cross-layer Attention Reconstruction (CAR) Module: To address the issue of insufficient information exchange between the encoder and decoder, a cross-layer attention reconstruction module is employed to optimize skip connections. By fusing shallow structural features, deep semantic features, and decoding features, spatial alignment is achieved through upsampling, and attention weights are generated based on decoding semantics. Adaptive modulation and residual compensation are then applied to the features to achieve fine-grained structure restoration.

[0040] Preferred, Unlike MSD, CAR aims to optimize the cross-layer information interaction between the encoder and decoder, enabling skip connections to not only transmit local details but also selectively reconstruct information guided by decoding semantics. In existing methods, multi-scale feature aggregation and decoding modulation are often modeled separately, resulting in a lack of a unified interaction mechanism between structural and semantic information.

[0041] To address this, this embodiment incorporates a cross-layer attention reconstruction module (CAR) to enhance skip connections in VesselKANNet. For example... Figure 5 As shown, this module simultaneously inputs shallow encoded features. Deep coding features With decoding features Collaborative modeling of cross-layer information is achieved through structural fusion and semantic guidance.

[0042] First, the deep-encoded features are... Aligned to spatial resolution (H, W) via bilinear interpolation and with After channel alignment and feature reshaping through 3×3 convolution, GroupNorm, and PReLU activation respectively, the following results are obtained: and : Subsequently, the two are added element-wise and refined through convolution to obtain the fused features. Then, the decoded features are introduced as semantic guidance signals. The fused features and decoded features are mapped to the intermediate channel space through 1×1 convolution and normalization, respectively. Then, the two are summed and nonlinearly transformed to generate a spatial attention map. This attention map is used for fused features. Element-wise modulation is performed to selectively enhance the skip connection information: To ensure the stability and integrity of information transmission, residual connections are introduced to add the original fused features to the gating result, and the final output is obtained through a 1×1 convolution: in, This represents a two-dimensional convolution operation. Upsampling for bilinear interpolation For GroupNorm, For PReLU, For the Sigmoid function, This indicates element-wise multiplication.

[0043] S402: Output Results: Based on the decoded fusion features, the target structure segmentation results are output. Taking retinal image segmentation as a specific example, the segmentation results of multiple target structures such as capillaries, arteries, veins and FAZ can be output.

[0044] Preferred, The final output includes multi-task segmentation results for capillaries, arteries, veins, and the FAZ region. The final segmentation results have clear boundaries and complete structure, meeting the needs of clinical analysis and assisted diagnosis.

[0045] Specifically, This invention addresses the limitations of existing medical image segmentation techniques, such as the limited structural representation capabilities of two-dimensional methods, the high computational cost of three-dimensional methods, the aliasing of multimodal features, the loss of fine-grained structural information, insufficient segmentation of complex target structures, and inadequate model interpretability. It proposes a new medical image segmentation scheme. First, in the input construction stage, to address the insufficient representation of two-dimensional projection information, this invention constructs multi-channel input features through a statistical projection strategy and combines grouped modal flows for early feature extraction based on modality awareness. Through grouped independent modeling and fusion mechanisms, the differences between features from different sources are effectively preserved, improving the quality of input features. Second, in the encoding stage, to address the problem of fine-grained structural information loss caused by traditional pooling downsampling, this invention introduces a multi-scale structural downsampling module. Utilizing multi-scale convolution, orientation-aware attention, and channel reweighting mechanisms, it enhances the representation ability of slender structures and complex boundaries while compressing spatial resolution. Furthermore, in the high-level feature modeling stage, this invention introduces a nonlinear modeling mechanism based on the Kolmogorov-Arnold Network (KAN). This mechanism enhances the expressive power of complex structures and nonlinear relationships through learnable one-dimensional nonlinear functions, and combines this with the interpretability quantification evaluation index Plausibility IoU (PIoU) to analyze the consistency between the model's region of interest and the real structure. In the retinal image example, PIoU is extended to multi-target structures. Finally, addressing the problem of insufficient cross-layer information interaction between the encoder and decoder, this invention designs a cross-layer attention reconstruction module. This module achieves selective reconstruction of cross-layer features through decoding semantic guidance, improving fine-grained structure recovery capability and spatial semantic consistency. Therefore, this invention, through the synergistic effect of modules such as statistical projection representation strategy, grouped projection feature extraction, structure enhancement, nonlinear interpretable modeling, and cross-layer semantic reconstruction, forms a complete medical image segmentation processing chain. This avoids the high computational overhead of three-dimensional volume data methods while improving the segmentation accuracy and model interpretability of complex target structures. Using OCT / OCTA retinal images as an example, high-precision segmentation of multiple target structures such as capillaries, arteries, veins, and FAZ can be achieved.

[0046] Specifically, Using the OCTA-500 dataset as an example, the method of the present invention is verified and compared with existing two-dimensional segmentation methods. Experimental results show that the present invention achieves good segmentation performance on both subsets of the OCTA-500 dataset. On the OCTA_6mm and OCTA_3mm datasets, the average crossover ratio (mIoU) for capillaries, arteries, veins, and FAZs reached 80.84% ​​and 86.76%, respectively, and the average Dice coefficients reached 89.13% and 92.77%, respectively. On the OCTA_6mm dataset, the IoUs for each target reached 76.88%, 78.78%, 79.63%, and 88.05%, respectively, and the Dice coefficients reached 86.85%, 88.04%, 88.51%, and 93.12%, respectively. On the OCTA_3mm dataset, the IoUs for each target reached 83.01%, 84.29%, 83.34%, and 96.39%, respectively, and the Dice coefficients reached 90.68%, 91.41%, 90.87%, and 98.14%, respectively. Compared with existing two-dimensional segmentation methods, this invention achieves the best performance in terms of average Dice and average IoU. Meanwhile, while maintaining the low computational overhead of two-dimensional methods, this invention achieves performance comparable to some three-dimensional methods in arteriovenous and FAZ segmentation tasks.

[0047] This invention improves the input information representation capability, complex target structure segmentation performance, and model interpretability in medical image segmentation tasks through collaborative design including multi-channel feature construction, multi-channel feature collaborative representation, and interpretable modeling. Specific technical effects are as follows: First, this invention utilizes a statistical projection feature construction mechanism to fully leverage structural and signal variation information from different depth regions while compressing 3D volume data, forming a more information-rich multi-channel input representation, thereby improving the ability of 2D input to express complex target structures and fine-grained information. Second, this invention employs collaborative modeling using Grouped Projection Stream (GPS), Multi-Scale Structural Downsampling (MSD), and Cross-Layer Attention Reconstruction (CAR). GPS preserves the differences between features from different sources, reducing early feature aliasing; MSD effectively maintains fine-grained structural and boundary information during downsampling through multi-scale convolution and orientation-aware mechanisms; and CAR achieves fine restoration of complex structural regions through cross-layer feature interaction and semantic guidance, thereby improving overall segmentation quality. Furthermore, this invention introduces a nonlinear modeling mechanism based on the Kolmogorov-Arnold Network (KAN), which enhances the expressive power of complex structures through a learnable one-dimensional nonlinear function. Combined with an interpretability quantification evaluation mechanism, it enables the analysis of the consistency between the model's focus area and the real structure, thereby improving the credibility and interpretability of the model's prediction results. Figure 6 As shown. Taking OCT / OCTA retinal image segmentation as a specific example, as... Figure 7As shown, on the OCTA-500 dataset, the average intersection-over-union (mIoU) of this invention reaches 80.84% ​​and 86.76%, respectively, and the average Dice coefficient reaches 89.13% and 92.77%, respectively. Furthermore, it achieves performance comparable to some 3D methods in arteriovenous and FAZ segmentation tasks. Therefore, this invention improves the segmentation capability and model interpretability of complex target structures while avoiding the high computational overhead of 3D methods, demonstrating significant application value.

[0048] Example 2 This embodiment provides a system for implementing a retinal image segmentation method as described in Embodiment 1, including a memory and a processor. The memory stores a computer program, and the processor calls the computer program to execute the steps of the method as described in Embodiment 1.

[0049] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A retinal image segmentation method, characterized in that, include: Acquire a multimodal retinal image dataset and perform preprocessing; The retinal image data is subjected to maximum intensity projection, vessel count projection, and standard deviation projection to obtain the projected data. The projected data is divided into multiple groups; shallow features within each group are extracted and then spliced ​​and transformed along the channel dimension to obtain shallow structural features. The shallow structural features are transformed and then divided into a structure-preserving branch and a feature-enhancing branch along the channel dimension; the feature-enhancing branch is then subjected to channel mapping and multi-scale feature fusion to obtain the first fused feature. The first fused feature is subjected to dimensionality reduction transformation and orientation-aware convolution to generate attention weights; the attention weights are multiplied element-wise with the first fused feature to obtain the reweighted feature; The reweighted features are enhanced and fused with the feature enhancement branch to obtain the enhanced features; The enhanced features are spliced ​​and fused with the structure-preserving branches to obtain deep semantic features; The deep semantic features are subjected to feature mapping and feature enhancement to obtain decoded features; The shallow structural features and deep semantic features are respectively channel aligned and feature remodeled, and then the remodeled features are fused to obtain a second fused feature; the decoded features are used as semantic guidance signals to guide the generation of a spatial attention map; based on the spatial attention map, the second fused feature is modulated element by element, and the modulated features are summed and convolved with the second fused feature to obtain the final fused feature; Based on the final fusion features, the target retinal image segmentation result is output.

2. The retinal image segmentation method according to claim 1, characterized in that, The projection expressions for maximum intensity projection, vessel count projection, and standard deviation projection of retinal image data are as follows: in, Indicates the maximum intensity projection. This indicates the search range along the y-direction. Represents OCT volume data, Represents the projection of blood vessel count. Represents the standard deviation projection. This represents the total number of voxels contained in the anatomical region S at the projection position (x,z). Let represent the intensity value of the k-th voxel in the anatomical region S at the projection position (x,z). This indicates the intensity threshold used when generating the vessel count projection. This represents the standard deviation of the voxel signal intensity along the depth direction within the anatomical region S at the projection position (x,z).

3. The retinal image segmentation method according to claim 1, characterized in that, The expressions for obtaining the shallow structural features include: in, This represents the output features obtained after cross-group feature fusion and channel compression, i.e., shallow structure features. This represents the GroupNorm operation, and PReLU() represents the parameterized linear rectified activation function. Represents a 1×1 projective convolution. This represents the sets of features concatenated along the channel dimension. This represents the features extracted from the i-th channel group through independent learning branches, and Concat() represents the concatenation operation. B represents the real space where the feature tensor resides, C represents the batch size, H represents the number of output channels for each independent learning branch, and H and W represent the height and width of the feature map, respectively. This represents a convolutional feature extraction module consisting of two convolutional layers, GroupNorm, and PReLU. This indicates that all channel features corresponding to the channel index set are selected from the input feature X. This represents a group of channel indexes with similar semantic attributes.

4. The retinal image segmentation method according to claim 1, characterized in that, Channel mapping and multi-scale feature fusion are performed on the feature enhancement branch to obtain the first fused feature, and the corresponding fusion expression is: in, Indicates the first fusion feature, This represents a fusion operation consisting of GroupNorm, PreLU, and a 1×1 convolution. To maintain branches in the structure, For the feature enhancement branch, For downsampling features, This represents a depthwise convolution with a kernel size of k. This represents the features obtained after performing a pre-projection transformation on the feature enhancement branch. Represents the characteristic tensor of real numbers. Indicates the output channel.

5. The retinal image segmentation method according to claim 1, characterized in that, The enhanced features are concatenated and fused with the structure-preserving branch to obtain deep semantic features. The corresponding concatenation and fusion expression is as follows: in, This represents the fusion operation consisting of convolution, normalization, and activation. This represents the combined features after concatenation; Concat() indicates the concatenation operation. To maintain branches in the structure, For the feature enhancement branch, Indicates the first fusion feature, This represents the fused features obtained after feature fusion transformation. The final output features obtained after recalibration via the channel attention mechanism are the subsequent deep semantic features. For the enhanced features obtained through residual connections, For channel recalibration features, This represents the Sigmoid function. This represents element-wise multiplication. For attention weights, Expand() represents the channel-dimensional convolution recovery. For heavily weighted features, SE(·) represents the compression-excitation channel attention mechanism, StripDWConv(·) represents the strip depth convolution operation, and Reduce(·) represents the dimensionality reduction transformation.

6. The retinal image segmentation method according to claim 5, characterized in that, The deep semantic features are subjected to feature mapping and feature enhancement to obtain decoded features. The corresponding feature enhancement expressions and feature mapping expressions are as follows: in, Indicates the characteristics of the initial token. This indicates a tokenization operation. Indicates the first Each stage inputs features of the ToK-KAN module. Indicates the first The token characteristics output by each ToK-KAN module This represents a depthwise convolution operation. express Network nonlinear feature mapping, This represents the output features obtained after ToK-KAN enhancement. For the decoded features of the output, This indicates a feature normalization operation.

7. The retinal image segmentation method according to claim 1, characterized in that, The shallow structural features and deep semantic features are respectively channel aligned and reshaped. Then, the reshaped features are fused to obtain the second fused feature. The corresponding fusion expression is: in, This is the second fusion feature. (·) represents the PReLU activation function. These are the deep semantic features after feature remodeling. These are the shallow structural features after feature remodeling. This indicates a GroupNorm operation. For convolutional kernels used to fuse and refine reconstructed features, This is the convolutional kernel used for channel mapping of the upsampled deep semantic features. For deep semantic features, This is a shallow structural feature. This is a convolutional kernel used for channel mapping of shallow structural features. This indicates bilinear interpolation upsampling.

8. A retinal image segmentation method according to claim 6, characterized in that, The decoded features are used as semantic guidance signals to guide the generation of a spatial attention map. The corresponding guidance expression is: in, This is a spatial attention map. For the convolutional kernel used to generate spatial attention responses, (·) denotes the PReLU activation function, and G denotes the decoding semantic guidance feature obtained from the decoding feature map. This represents the fusion mapping feature obtained from the second fusion feature mapping. This indicates a GroupNorm operation. The convolution kernel used for channel mapping of the second fused feature. Indicates the second fusion feature, Indicates decoding features, The convolution kernel used for channel mapping of decoded features. This represents a two-dimensional convolution operation.

9. A retinal image segmentation method according to claim 8, characterized in that, Based on the spatial attention map, the second fused feature is modulated element-wise, and the corresponding modulation expression is: in, This is the modulated second fusion feature; The modulated features are summed and convolved with the second fused feature to obtain the final fused feature. The corresponding summation and convolution expressions are as follows: in, This is the convolution kernel used for output projection and channel mapping of the additive residual features. This indicates the final fusion feature.

10. A system for implementing a retinal image segmentation method as described in any one of claims 1-9, characterized in that, It includes a memory and a processor, the memory storing a computer program, the processor invoking the computer program to perform the steps of the method as described in any one of claims 1 to 9.

Images