An OCT image choroidal neovascularization segmentation method and system

Abstract

The present application relates to the technical field of medical image processing, and particularly relates to an OCT image choroidal neovascularization segmentation method and system. The present application takes TransUNet as a benchmark network, designs an embedded heterogeneous feature alignment module, a curvature KAN guided attention module and a deep-shallow dual-path decoder to build a choroidal neovascularization segmentation model. Through the heterogeneous feature alignment module, the local features extracted by the convolutional neural network and the global context features extracted by the visual extractor are integrated to enable the network to comprehensively extract the features of the OCT image. Through the curvature KAN guided attention module, the attention mechanism is optimized to better obtain the local geometric features of the image. The deep-shallow dual-path decoder processes deep features and shallow features through two parallel decoding paths respectively, thereby significantly enhancing the network's ability to capture small target lesions. The present application effectively improves the segmentation accuracy of the OCT image choroidal neovascularization.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a method and system for segmenting choroidal neovascularization in OCT images. Background Technology

[0002] Choroidal neovascularization (CNV) is a pathological feature of various fundus diseases, commonly seen in age-related macular degeneration (AMD) and pathological myopia (PM). In patients under 50 years of age, pathological myopia is a common cause of CNV formation. It is estimated that 5.2%–11.3% of patients with pathological myopia will develop myopic CNV. In ophthalmology, optical coherence tomography (OCT) is a widely used detection technique. Based on the principle of optical interference, OCT can non-invasively detect reflected signals at different depths of fundus tissue, generating high-resolution three-dimensional fundus images. Widely used in ophthalmology, OCT can effectively visualize pathological features such as choroidal neovascularization, laying the foundation for quantitative analysis of choroidal neovascularization.

[0003] Traditional image processing methods primarily rely on manual feature extraction and classic algorithms such as edge detection and thresholding. While these methods played a crucial role in early medical image analysis, their reliance on manual feature extraction makes them ill-suited for complex medical image data, resulting in significant limitations in accuracy and efficiency. In recent years, with the rapid development of deep learning technology, Convolutional Neural Networks (CNNs) and Transformer architectures have gradually become mainstream methods for medical image processing and analysis. CNNs, through a combination of convolutional and pooling layers, can automatically learn local image features and have achieved significant results in tasks such as medical image classification, segmentation, and detection. The Transformer architecture, through its self-attention mechanism, can capture global dependencies in images, further improving the performance of medical image processing.

[0004] However, existing deep learning image segmentation methods, when processing targets, rely on learning statistical patterns in the dataset to determine pixel categories, neglecting the geometric features of choroidal neovascularization in OCT images. Therefore, when faced with targets like choroidal neovascularization with complex and diverse geometric shapes in OCT images, they often lack insight into the target's geometric features and struggle to extract the inherent geometric topological features from pixel associations. This results in the inability to accurately identify the true boundaries of choroidal neovascularization. Furthermore, when facing new or unseen geometric shapes, the lack of prior knowledge about these shapes often prevents accurate segmentation prediction, leading to poor segmentation accuracy for choroidal neovascularization. Summary of the Invention

[0005] Therefore, the technical problem to be solved by the present invention is to overcome the defect that the existing deep learning-based image segmentation methods ignore the geometric features of choroidal neovascularization in OCT images, resulting in the inability to accurately identify the true boundary of choroidal neovascularization and thus poor segmentation accuracy of choroidal neovascularization.

[0006] To address the aforementioned technical problems, this invention provides a method for segmenting choroidal neovascularization in OCT images, comprising:

[0007] The OCT image to be segmented is input into the choroidal neovascularization segmentation model, and the segmentation result of the OCT image to be segmented is output. The choroidal neovascularization segmentation model is obtained by setting a curvature KAN guided attention module between each convolutional neural network stage of the TransUNet model and its corresponding decoder layer.

[0008] Specifically, the input feature map of the curvature KAN-guided attention module is processed through its corresponding curvature KAN-guided attention module to extract its geometric feature map, including:

[0009] The curvature KAN guides the input feature map of the attention module, and extracts its corresponding local curvature feature map through a 3×3 curvature convolution kernel;

[0010] For each channel of the local curvature feature map corresponding to the input feature map of the curvature KAN guided attention module, the feature values ​​of all spatial locations within that channel are summed to obtain the channel curvature weight vector composed of the sum of the feature values ​​of each channel.

[0011] After normalizing the channel curvature weight vector corresponding to the input feature map of the curvature KAN-guided attention module, the channel curvature weight vector is optimized through the KAN network.

[0012] The geometric feature map of the curvature KAN-guided attention module is obtained by multiplying the input feature map of the curvature KAN-guided attention module element by element with its corresponding optimized channel curvature weight vector.

[0013] Preferably, the choroidal neovascularization segmentation model further includes:

[0014] A heterogeneous feature alignment module is set after each stage of the convolutional neural network to align the first... The local feature map of the first convolutional neural network stage and the first... The global feature map of the first visual converter stage is obtained through the first... The heterogeneous feature alignment module is obtained to achieve the first... The fused feature map of the heterogeneous feature alignment modules includes:

[0015] The first The global feature map in the first visual converter stage is sequentially processed through deformable convolution, multi-level transposed convolution, batch normalization, and activation functions to obtain the result that is consistent with the first visual converter stage. The local feature maps of the same size in each stage of the convolutional neural network are used as the first... Alignment feature maps for each visual converter stage; where... , This represents the number of stages in the convolutional neural network.

[0016] The first The alignment feature map of the first visual converter stage and the first... After concatenating the local feature maps from the first stage of the convolutional neural network, a 1×1 convolution is performed to obtain the second stage. Fusion feature map of heterogeneous feature alignment modules;

[0017] The first The fused feature map of the heterogeneous feature alignment module is used as the first... The curvature KAN corresponding to each stage of the convolutional neural network guides the input of the attention module.

[0018] Preferably, the OCT image to be segmented is input into the choroidal neovascularization segmentation model, and the segmentation result of the OCT image to be segmented is output, including:

[0019] The OCT image to be segmented is processed through a convolutional neural network to extract... Local feature maps of each stage of a convolutional neural network;

[0020] The first After downsampling the local feature maps of each convolutional neural network stage, they are divided into multiple patch blocks. After linear mapping of each patch block, position information is embedded to obtain a patch sequence with embedded position information.

[0021] The patch sequence with embedded location information is extracted using a visual transformer. Global feature maps for each visual converter stage;

[0022] The first The local feature map of the first convolutional neural network stage and the first... The global feature map of the first visual converter stage is obtained through the first... The heterogeneous feature alignment module is obtained to achieve the first... The fused feature map of three heterogeneous feature alignment modules; among them... ;

[0023] The fused feature map of each heterogeneous feature alignment module is used to extract the geometric feature map of each curvature KAN guided attention module through its corresponding curvature KAN guided attention module.

[0024] The first The global feature map of each visual converter stage and the geometric feature map of each curvature KAN-guided attention module are input into the decoder, and the segmentation result of the OCT image to be segmented is output.

[0025] Preferably, the step of placing the first The global feature maps of each visual converter stage and the geometric feature maps of each curvature KAN-guided attention module are input into the decoder, and the segmentation results of the OCT image to be segmented are output, including:

[0026] The first The global feature map of the first visual converter stage is sequentially passed through transposed convolution, batch normalization layer, and ReLU activation function to obtain the first... Global feature map after upsampling at each visual converter stage;

[0027] The first The global feature map upsampled after the first visual converter stage and the first... After concatenating the geometric feature maps of the curvature KAN-guided attention module, the result is obtained by passing two cascaded 3×3 convolutions, batch normalization layers, and ReLU activation functions. Output feature maps of each deep decoder layer;

[0028] The first The output feature map of the deep decoder layer is sequentially passed through transposed convolution, batch normalization layer, and ReLU activation function to obtain the upsampled feature map of the th layer. Output feature maps of each deep decoder layer;

[0029] After upsampling, the first The output feature map of the deep decoder layer and the first After concatenating the geometric feature maps of the curvature KAN-guided attention module, the result is obtained by passing two cascaded 3×3 convolutions, batch normalization layers, and ReLU activation functions. The target feature map of each deep decoder layer; among which... ;

[0030] The target feature map of the first deep decoder layer is passed through the segmentation head to obtain the segmentation result of the OCT image to be segmented.

[0031] Preferably, the decoder further includes a shallow decoder layer, into which the geometric feature maps of the first curvature KAN-guided attention module and the second curvature KAN-guided attention module are input to obtain a shallow feature map, including:

[0032] The geometric feature map of the second curvature KAN guided attention module is passed through bilinear interpolation, batch normalization layer and ReLU activation function in sequence to obtain the upsampled geometric feature map of the second curvature KAN guided attention module;

[0033] After concatenating the geometric feature map of the second curvature KAN guided attention module with the geometric feature map of the first curvature KAN guided attention module, a shallow feature map is obtained by passing two cascaded 3×3 convolutions, batch normalization layers, and ReLU activation functions.

[0034] After concatenating the shallow feature map with the feature map output from the first deep decoder layer along the channel dimension, the segmentation head is used to obtain the segmentation result of the OCT image to be segmented.

[0035] Preferably, the segmentation head includes a 3×3 convolution and a bilinear interpolation connected in sequence.

[0036] Preferably, when At that time, the OCT image to be segmented is passed through a convolutional neural network, and local feature maps of each stage of the convolutional neural network are extracted, including:

[0037] The OCT image to be segmented is passed through a series of convolutions with a kernel size of 7×7 and a stride of 2, followed by group normalization and ReLU activation function to obtain the local feature map of the first convolutional neural network stage.

[0038] The local feature maps of the first convolutional neural network stage are processed sequentially through max pooling with a kernel size of 3×3 and a stride of 2, group normalization, ReLU activation function, 3×3 convolution with a stride of 1, group normalization, and ReLU activation function to obtain the first stage residual transformation feature map.

[0039] After adding the feature map of the first stage residual transformation to the local feature map of the first convolutional neural network stage element by element, the local feature map of the second convolutional neural network stage is obtained by passing the ReLU activation function.

[0040] The local feature map of the second convolutional neural network stage is processed sequentially through 1×1 convolution, group normalization, ReLU activation function, 3×3 convolution with stride of 1, group normalization, and ReLU activation function to obtain the residual transformation feature map of the second stage.

[0041] After adding the feature map of the second-stage residual transformation to the local feature map of the second convolutional neural network stage element by element, the local feature map of the third convolutional neural network stage is obtained by applying the ReLU activation function.

[0042] Preferably, each visual converter stage consists of three stacked Transformer layers, each Transformer layer including a multi-head self-attention module and a feedforward neural network connected in sequence.

[0043] Preferably, the weight parameters of the 3×3 curvature convolution kernel are:

[0044] ,

[0045] in, The weight parameters are for a 3×3 curvature convolution kernel.

[0046] The present invention also provides an OCT image choroidal neovascularization segmentation system, comprising:

[0047] A memory for storing computer programs; a processor for executing the computer programs to implement the steps of the above-described OCT image choroidal neovascularization segmentation method.

[0048] Compared with the prior art, the above-described technical solution of the present invention has the following advantages:

[0049] This invention discloses an OCT image choroidal neovascularization segmentation method and system. It utilizes a 3×3 curvature convolution kernel to calculate curvature information on the output feature map of each convolutional neural network stage, transforming key geometric features such as the degree of boundary curvature and morphological changes of choroidal neovascularization into quantifiable local curvature feature maps, accurately capturing the geometric morphology of the true vessel boundary. Subsequently, by summing and normalizing the curvature feature values ​​of each channel, the spatial geometric information is aggregated into statistical features of the channel dimension. Then, a KAN network dynamically optimizes the local curvature feature map, enabling the network to adaptively focus on regions with significant geometric features and assign them higher weights. Finally, by element-wise multiplying the optimized weight map with the original feature map, the focus on geometric features is strengthened while suppressing irrelevant background. This process not only fills the gap in choroidal neovascularization segmentation models' insufficient utilization of geometric feature constraints but also, guided by the KAN network, allows the model to more accurately identify the true vessel boundary, effectively overcoming the problem of poor segmentation accuracy caused by ignoring geometric features and improving the segmentation accuracy of choroidal neovascularization.

[0050] The existing TransUNet model only uses the feature map output from the last stage of the visual converter, ignoring the features of the early and middle stages. This results in the loss of detailed morphology and mid-level spatial relationship information of the target, making it difficult for the model to accurately capture the subtle boundaries and complex branching structures of choroidal neovascularization. Furthermore, the global features of a single stage cannot cover semantic information at different scales, leading to insufficient cross-scale feature interaction. This, in turn, affects the matching degree with the local features of each stage of the convolutional neural network, causing scale misalignment and information discontinuity during feature fusion. Ultimately, this results in problems such as blurred boundaries and missed detection of small blood vessels in the segmentation results. This invention breaks through the limitations of single-stage feature reuse by setting independent heterogeneous feature alignment modules for each convolutional neural network stage and its corresponding visual converter stage, achieving full interaction of multi-scale features. By introducing deformable convolutions, the sampling position of the convolution kernel can be dynamically adjusted according to the input features. Combined with the progressive upsampling of multi-level transposed convolutions, the global feature map of the visual converter accurately adapts to the spatial size of the local feature maps of convolutional neural networks at different scales, significantly improving the alignment accuracy of cross-modal features in both space and scale. Through the integration of feature concatenation and 1×1 convolutions, not only are the local details of the convolutional neural network and the global semantics of the visual converter preserved, but the representational ability of the fused features is also enhanced through cross-level feature complementarity, improving the accuracy and robustness of feature alignment. Furthermore, it enhances the model's adaptability to complex scenes, ensuring that features from different sources achieve high consistency in both space and scale, and improving the segmentation accuracy of choroidal neovascularization.

[0051] Furthermore, this invention designs a dual-path decoder for deep and shallow layers. By constructing two parallel decoding paths, deep and shallow features are processed separately. The decoding path for deep feature processing follows the existing TransUNet model, while its upsampling method is specifically improved. Upsampling is performed using transposed convolution, batch normalization layers, and ReLU activation functions, which can preserve the global semantic information and overall structural features contained in deep features, ensuring accurate extraction of the overall morphology and spatial distribution of choroidal neovascularization. The decoding path for shallow feature processing uses bilinear interpolation, batch normalization layers, and ReLU activation functions for upsampling, which can better preserve the subtle texture and edge details in shallow features, accurately capturing local features such as small vessel branches and boundary contours. This significantly enhances the ability of the choroidal neovascularization segmentation model to perceive the edge details of small targets, taking into account the accurate restoration of global structure and local details, and improving the performance and accuracy of choroidal neovascularization segmentation results. Attached Figure Description

[0052] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings, wherein:

[0053] Figure 1 This is a structural diagram of the curvature KAN-guided attention module.

[0054] Figure 2 This is a structural diagram of the heterogeneous feature alignment module.

[0055] Figure 3 This is a structural diagram of a segmentation model of choroidal neovascularization.

[0056] Figure 4 This is a visualization of the comparative experimental results of the method of this invention with other segmentation methods on dataset I. Figure 4 In the image (a), the OCT image to be segmented is from dataset I. Figure 4 In (b), the CNV segmentation gold standard for the OCT image to be segmented in dataset I is... Figure 4 (c) in the image shows the segmentation result using UNet. Figure 4 (d) in the figure represents the segmentation result using CPFNet. Figure 4 (e) in the figure represents the segmentation result using TransUNet. Figure 4 In the diagram, (f) represents the segmentation result using TransAttUnet. Figure 4 In the diagram, (g) represents the segmentation result using UCTransNet. Figure 4 (h) in the figure represents the segmentation result using the method of the present invention.

[0057] Figure 5This is a visualization of the comparative experimental results of the method of this invention with other segmentation methods on dataset II. Figure 5 In the image (a), the OCT image to be segmented is from dataset II. Figure 5 In (b), CNV segmentation is the gold standard for the OCT images to be segmented in dataset II. Figure 5 (c) in the image shows the segmentation result using UNet. Figure 5 (d) in the figure represents the segmentation result using CPFNet. Figure 5 (e) in the figure represents the segmentation result using TransUNet. Figure 5 In the diagram, (f) represents the segmentation result using TransAttUnet. Figure 5 In the diagram, (g) represents the segmentation result using UCTransNet. Figure 5 (h) in the figure represents the segmentation result using the method of the present invention. Detailed Implementation

[0058] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0059] This embodiment provides a method for segmenting choroidal neovascularization in OCT images, including:

[0060] The OCT image to be segmented is input into the choroidal neovascularization segmentation model, and the segmentation result of the OCT image to be segmented is output. The choroidal neovascularization segmentation model is obtained by setting a curvature KAN guided attention module between each convolutional neural network stage of the TransUNet model and its corresponding decoder layer.

[0061] Specifically, the input feature map of the curvature KAN-guided attention module is processed through its corresponding curvature KAN-guided attention module to extract its geometric feature map, including:

[0062] The curvature KAN guides the input feature map of the attention module, and extracts its corresponding local curvature feature map through a 3×3 curvature convolution kernel;

[0063] For each channel of the local curvature feature map corresponding to the input feature map of the curvature KAN guided attention module, the feature values ​​of all spatial locations within that channel are summed to obtain the channel curvature weight vector composed of the sum of the feature values ​​of each channel.

[0064] After normalizing the channel curvature weight vector corresponding to the input feature map of the curvature KAN-guided attention module, the channel curvature weight vector is optimized through the KAN network.

[0065] The geometric feature map of the curvature KAN-guided attention module is obtained by multiplying the input feature map of the curvature KAN-guided attention module element by element with its corresponding optimized channel curvature weight vector.

[0066] In this embodiment, the input feature map of the curvature KAN-guided attention module is either a local feature map from the convolutional neural network stage or a fused feature map from the heterogeneous feature alignment module.

[0067] In this embodiment, specifically, the OCT image to be segmented is input into the choroidal neovascularization segmentation model, and the segmentation result of the OCT image to be segmented is output, including:

[0068] Step S1: Extract the OCT image to be segmented using a convolutional neural network. Local feature maps of each stage of a convolutional neural network; where... This represents the number of stages in the convolutional neural network.

[0069] In this embodiment, preferably, when At that time, the OCT image to be segmented is passed through a convolutional neural network, and local feature maps of each stage of the convolutional neural network are extracted, including:

[0070] The OCT image to be segmented is passed through a series of convolutions with a kernel size of 7×7 and a stride of 2, followed by group normalization and ReLU activation function to obtain the local feature map of the first convolutional neural network stage.

[0071] The local feature maps of the first convolutional neural network stage are processed sequentially through max pooling with a kernel size of 3×3 and a stride of 2, group normalization, ReLU activation function, 3×3 convolution with a stride of 1, group normalization, and ReLU activation function to obtain the first stage residual transformation feature map.

[0072] After adding the feature map of the first stage residual transformation to the local feature map of the first convolutional neural network stage element by element, the local feature map of the second convolutional neural network stage is obtained by passing the ReLU activation function.

[0073] The local feature map of the second convolutional neural network stage is processed sequentially through 1×1 convolution, group normalization, ReLU activation function, 3×3 convolution with stride of 1, group normalization, and ReLU activation function to obtain the residual transformation feature map of the second stage.

[0074] After adding the feature map of the second-stage residual transformation to the local feature map of the second convolutional neural network stage element by element, the local feature map of the third convolutional neural network stage is obtained by applying the ReLU activation function.

[0075] when At the same time, this convolutional neural network achieves a precise balance between feature extraction efficiency and adaptability to segmentation tasks. Firstly, the three-stage hierarchical division matches the multi-scale feature requirements of choroidal neovascularization in OCT images—the first stage rapidly extracts the basic texture and initial edges of the image through a large 7×7 convolution, laying the foundation for subsequent feature refinement; the second stage uses residual connections to fuse initial and transformed features, enhancing the capture of the mid-layer structure of the blood vessel (such as branch nodes and regional contours); the third stage further adjusts the channel dimension and deepens feature interaction through 1×1 convolution, focusing on the global spatial relationship between the blood vessel and surrounding tissues. This step-by-step, progressive three-stage structure avoids the need for fewer stages (such as...). or This addresses the issue of insufficient feature levels leading to loss of details or lack of semantic information, while also preventing excessive stages (such as...). The parameter redundancy and computational burden introduced by OCT images can be efficiently adapted to the feature span of choroidal neovascularization from fine boundaries to overall morphology. Simultaneously, each stage stabilizes the training process through group normalization and alleviates gradient vanishing through residual connections, ensuring that the three-stage features can be progressively transmitted and reinforce effective information. Ultimately, this provides hierarchical and fully represented local feature support for subsequent heterogeneous feature alignment and geometric feature extraction, improving the efficiency and accuracy of the segmentation model.

[0076] Step S2: Place the first After downsampling the local feature maps of each convolutional neural network stage, they are divided into multiple patch blocks. After linear mapping of each patch block, position information is embedded to obtain a patch sequence with embedded position information.

[0077] Step S3: Extract the location information from the patch sequence embedded with location information using a visual converter. Global feature maps for each visual converter stage;

[0078] In this embodiment, specifically, each visual converter stage consists of three stacked Transformer layers, each Transformer layer including a multi-head self-attention module and a feedforward neural network connected in sequence.

[0079] The geometry of choroidal neovascularization in optical coherence tomography (OCT) images is complex and diverse. Current deep learning image segmentation methods often lack insight into the geometric features of targets with complex geometries. Because the models fail to effectively utilize the geometric properties of the choroidal neovascularization to guide segmentation decisions, they cannot accurately identify the true boundaries of the neovascularization. Furthermore, when faced with new or unseen geometric shapes, existing segmentation models often fail to make accurate segmentation predictions due to a lack of prior knowledge about these shapes. This not only limits the model's generalization ability in diverse scenarios but also increases the difficulty of training and applying the segmentation model. Therefore, this invention designs a Curvature KAN-guided attention module (CKANA), such as... Figure 1 As shown, Figure 1 This is a structural diagram of the curvature KAN-guided attention module. The specific design is as described in step S4.

[0080] Step S4: Extract the geometric feature map of each curvature KAN-guided attention module from the local feature map of each convolutional neural network stage, including:

[0081] Step S41: Extract the local curvature feature map of the curvature KAN-guided attention module corresponding to each stage of the convolutional neural network by passing the local feature map of each stage through a 3×3 curvature convolution kernel.

[0082] In this embodiment, preferably, the weight parameters of the 3×3 curvature convolution kernel are:

[0083] ,

[0084] in, The weight parameters are for a 3×3 curvature convolution kernel.

[0085] This convolutional kernel, through a weighted comparison of the central pixel (-1) and its surrounding neighboring pixels (5 / 16), and weak suppression of the corner pixels (-1 / 16), accurately captures the curvature changes of choroidal neovascularization in OCT images. It strongly responds to the convex and concave morphology of vessel boundaries and abrupt curvature changes at branching points, while weakening interference from flat areas, thus generating a local curvature feature map focused on geometric morphological features. This customized weight distribution enhances the extraction of key geometric information such as vessel edges and balances the contributions of the central and neighboring pixels through numerical proportion design, avoiding feature noise caused by unidirectional or oversensitive approaches. This provides a precise curvature feature foundation for subsequent KAN network optimization of the local curvature feature map, improving the model's accuracy in recognizing the true boundaries of choroidal neovascularization.

[0086] Step S42: For each channel of the local curvature feature map of each curvature KAN-guided attention module, sum the feature values ​​of all spatial locations within that channel to obtain the feature value sum of each channel. Based on the feature value sum of each channel, construct the channel curvature weight vector; each element in the channel curvature weight vector represents the feature value sum of a channel.

[0087] Step S43: After normalizing the channel curvature weight vector of each curvature KAN-guided attention module, optimize the channel curvature weight vector of each curvature KAN-guided attention module through the Kolmogorov-Arnold Network (KAN).

[0088] Step S44: Multiply the feature map output by each convolutional neural network stage element by element with the channel curvature weight vector optimized by the corresponding curvature KAN guided attention module to obtain the geometric feature map of each curvature KAN guided attention module.

[0089] The curvature KAN-guided attention module designed in this invention calculates the curvature information of the input feature map by introducing prior knowledge of average curvature, and then guides the curvature of the channel feature map through the Kolmogorov-Arnold Network (KAN). Curvature KAN-guided attention (CKANA) solves the problem of insufficient utilization of geometric feature constraints by the model and enhances the model's ability to capture geometric features.

[0090] Step S5: Place the first The global feature map of each visual converter stage and the geometric feature map of each curvature KAN-guided attention module are input into the decoder, and the segmentation result of the OCT image to be segmented is output.

[0091] Medical images often suffer from class imbalance, where small objects (such as CNV regions) occupy a much smaller proportion of pixels than the background. Accurate detection and segmentation of small objects in lesion areas are crucial for the early diagnosis and precise treatment of related diseases. However, class imbalance causes existing deep learning models to tend to focus on the background regions that occupy more pixels during training, while ignoring small objects, leading to the problem of missing small object CNVs.

[0092] Therefore, this invention designs a deep and shallow dual-path decoder, as shown in steps S51-S59:

[0093] In this embodiment, preferably, the step of placing the first The global feature maps of each visual converter stage and the geometric feature maps of each curvature KAN-guided attention module are input into the decoder, and the segmentation results of the OCT image to be segmented are output, including:

[0094] Step S51: Place the first The global feature map of the first visual converter stage is sequentially passed through transposed convolution, batch normalization layer, and ReLU activation function to obtain the first... Global feature map after upsampling at each visual converter stage;

[0095] Step S52: Place the first The global feature map upsampled after the first visual converter stage and the first... After concatenating the geometric feature maps of the curvature KAN-guided attention module, the result is obtained by passing two cascaded 3×3 convolutions, batch normalization layers, and ReLU activation functions. Output feature maps of each deep decoder layer;

[0096] Step S53: Place the first The output feature map of the deep decoder layer is sequentially passed through transposed convolution, batch normalization layer, and ReLU activation function to obtain the upsampled feature map of the th layer. Output feature maps of each deep decoder layer;

[0097] Step S54: After upsampling, the first... The output feature map of the deep decoder layer and the first After concatenating the geometric feature maps of the curvature KAN-guided attention module, the result is obtained by passing two cascaded 3×3 convolutions, batch normalization layers, and ReLU activation functions. The target feature map of each deep decoder layer; among which... ;

[0098] Step S55: Pass the target feature map of the first deep decoder layer through the segmentation head to obtain the segmentation result of the OCT image to be segmented.

[0099] In this embodiment, preferably, the decoder further includes a shallow decoder layer;

[0100] Step S56: Input the geometric feature maps of the first curvature KAN-guided attention module and the second curvature KAN-guided attention module into the shallow decoder layer to obtain shallow feature maps, including:

[0101] Step S561: Pass the geometric feature map of the second curvature KAN guided attention module through bilinear interpolation, batch normalization layer and ReLU activation function in sequence to obtain the upsampled geometric feature map of the second curvature KAN guided attention module;

[0102] Step S562: After concatenating the geometric feature map of the second curvature KAN guided attention module with the geometric feature map of the first curvature KAN guided attention module, a shallow feature map is obtained by passing two cascaded 3×3 convolutions, batch normalization layers, and ReLU activation functions.

[0103] Step S57: After concatenating the shallow feature map with the feature map output by the first deep decoder layer along the channel dimension, the segmentation result of the OCT image to be segmented is obtained through the segmentation head.

[0104] In this embodiment, preferably, the segmentation head includes a 3×3 convolution and a bilinear interpolation connected in sequence.

[0105] The Deep-Shallow Dual-Path Decoding (DSDPD) addresses the model's insufficient ability to capture small target lesions. By constructing two parallel decoding paths to process deep and shallow features respectively, it significantly enhances the network's ability to perceive the edge details of small targets.

[0106] In recent years, both the Vision Transformer (ViT) and the Convolutional Neural Network (CNN) have demonstrated powerful performance in OCT image CNV segmentation tasks. ViT can capture global features of an image through a self-attention mechanism, while CNN excels at extracting local features. Combining the features extracted by both allows the model to simultaneously perceive local details and global context, thereby improving CNV segmentation performance.

[0107] Due to the significant differences between CNN and ViT features, existing deep learning models often perform poorly when fusing these two types of features. For example, some models, such as TransUNet, simply concatenate ViT and CNN features without fully considering the differences in the representation and distribution of the two features. This results in the model failing to fully utilize the global and local features of the OCT image, leading to low CNV segmentation performance.

[0108] Therefore, this invention addresses the problem of insufficient alignment of heterogeneous CNV features in existing choroidal neovascularization CNV segmentation methods by designing a heterogeneous feature alignment module, such as... Figure 2 As shown, Figure 2 The structural diagram of the heterogeneous feature alignment module is shown below, with the specific design as follows:

[0109] In this embodiment, preferably, a heterogeneous feature alignment module is set after each stage of the convolutional neural network to align the first feature with the second feature. The local feature map of the first convolutional neural network stage and the first... The global feature map of the first visual converter stage is obtained through the first... The heterogeneous feature alignment module is obtained to achieve the first... The fused feature map of the heterogeneous feature alignment modules includes:

[0110] The first The global feature map in the first visual converter stage is sequentially processed through deformable convolution, multi-level transposed convolution, batch normalization, and activation functions to obtain the result that is consistent with the first visual converter stage. The local feature maps of the same size in each stage of the convolutional neural network are used as the first... Alignment feature maps for each visual converter stage; where... ;

[0111] The first The alignment feature map of the first visual converter stage and the first... After concatenating the local feature maps from the first stage of the convolutional neural network, a 1×1 convolution is performed to obtain the second stage. Fusion feature map of heterogeneous feature alignment modules;

[0112] The first The fused feature map of the heterogeneous feature alignment module is used as the first... The curvature KAN corresponding to the nth convolutional neural network stage guides the input of the attention module, i.e., the nth... The curvature KAN guides the input of the attention module.

[0113] The heterogeneous feature alignment module MSHFAM addresses the issue of aligning discrepancies between ViT and CNN features. Since ViT and CNN extract features with different dimensions, directly fusing these two types of features leads to poor performance. MSHFAM designs a specific alignment mechanism that allows the sequential features of ViT and the spatial features of CNN to be effectively fused in a unified feature space. This enables the model to fully utilize both global and local features, improving the overall segmentation performance.

[0114] Based on Example 1, this Example 2 sets the number of stages in the convolutional neural network (CNN encoder). ,like Figure 3 As shown, Figure 3 This is a structural diagram of the choroidal neovascularization segmentation model. The choroidal neovascularization segmentation model (Heterogeneous Feature Alignment and Curvature KAN Attention Network, HFCKANet) uses TransUNet as the base network and incorporates a heterogeneous feature alignment module (MSHFAM), a curvature KAN-guided attention module (CKANA), and a deep and shallow dual-path decoder (DSDPD). This effectively solves problems such as insufficient feature alignment between ViT and CNN, inadequate utilization of geometric feature constraints, and insufficient ability to capture small target lesions, thus improving CNV segmentation performance.

[0115] The choroidal neovascularization segmentation model designed in this invention includes: (1) a feature encoder; (2) a heterogeneous feature alignment module MSHFAM; (3) a curvature KAN-guided attention module CKANA as a skip connection; and (4) a deep and superficial dual-path decoder DSDPD. Detailed descriptions are as follows:

[0116] Batch quantity is OCT image to be segmented Local feature maps from three stages of a convolutional neural network are extracted. and These represent the height and width of the OCT image to be segmented, respectively, and 3 represents the number of channels in the OCT image to be segmented. Indicates the batch quantity;

[0117] The main body of a CNN encoder consists of three stages of residual blocks, each corresponding to the extraction of features at different levels. For the first stage of a convolutional neural network:

[0118] OCT image to be segmented By sequentially connecting 7×7 convolutional kernels with a stride of 2, followed by group normalization and ReLU activation, features are extracted and spatial resolution is reduced to obtain the local feature map of the first stage of the convolutional neural network. ;

[0119] The local feature map of the first stage of the convolutional neural network The feature map of the first stage residual transformation is obtained by sequentially processing the feature map through max pooling with a sliding window (kernel size) of 3×3 and a stride of 2, group normalization, ReLU activation function, 3×3 convolution with a stride of 1, group normalization, and ReLU activation function.

[0120] Combine the feature map of the first-stage residual transformation with the local feature map of the first convolutional neural network stage. The residual connection, which involves element-wise summation followed by non-linear processing using the ReLU activation function, yields the local feature map for the second convolutional neural network stage. ;

[0121] The local feature map of the second convolutional neural network stage The residual transformation feature map of the second stage is obtained by sequentially processing the data through 1×1 convolution, group normalization, ReLU activation function, 3×3 convolution with stride of 1, group normalization, and ReLU activation function.

[0122] The residual transformation feature map of the second stage is concatenated with the local feature map residual of the second convolutional neural network stage, i.e., they are added element-wise, and then non-linearly processed using the ReLU activation function to obtain the local feature map of the third convolutional neural network stage. .

[0123] Local feature maps of the third stage of the convolutional neural network As an intermediate feature representation of the model, since the Transformer layer requires serialized input, it is necessary to... Convert to a serialization format suitable for Transformer processing.

[0124] Local feature maps of the third stage of the convolutional neural network After downsampling, the local feature map after downsampling in the third convolutional neural network stage is obtained. ,Will Divided into A patch block, resulting in , in, Indicates the dimension of embedding. This refers to the number of patch blocks. Then, the patch blocks... Flatten and swap the last two dimensions, and embed position information to obtain a patch sequence with embedded position information. , which serves as the input to the ViT encoder.

[0125] The ViT encoder consists of four stages, each consisting of three stacked Transformer layers. Each Transformer layer mainly contains two modules: a Multi-Head Self-Attention (MHSA) module and a Feed-Forward Neural Network (FFN).

[0126] In the four stages of the ViT encoder, the Transformer layer is repeatedly used and progressively deepened. The phased feature extraction method enables the model to better process feature information at different levels, achieving fine extraction and high-level abstraction of features, and improving the model's ability to understand and process complex image data.

[0127] The patch sequence with embedded location information is processed by a visual converter to extract global feature maps for four visual converter stages. The global feature map for the first visual converter stage is... The global feature map for the second visual converter stage is The global feature map of the third visual converter stage is The global feature map of the fourth visual converter stage is .

[0128] Taking the first stage of the ViT encoder as an example, the input is... The output is , After feature extraction through three Transformer layers, we obtain It can be described by the following formula:

[0129] ,

[0130] in, It is a feedforward layer. For layer normalization, It is a multi-head self-attention module.

[0131] To further combine the advantages of convolutional neural networks and visual converters, this invention designs a heterogeneous feature alignment module (MSHFAM), which aims to integrate the local feature maps extracted by the convolutional neural network and the global feature maps extracted by the visual converter, enabling the network to comprehensively extract features from OCT images.

[0132] Taking the first heterogeneous feature alignment module as an example, the local feature map of the first convolutional neural network stage is... Global feature map with the first visual converter stage The first heterogeneous feature alignment module is used to obtain the... The fused feature map of the heterogeneous feature alignment modules includes:

[0133] To better align the local feature maps of the first convolutional neural network stage Global feature map with the first visual converter stage First, deformable convolution pairs are used. Dynamic adjustments are made to obtain the feature map. ; feature map Through multi-level transposed convolution (ConvTranspose2d), batch normalization (BatchNorm2d), and ReLU activation function processing, an aligned feature map of the first visual transformer stage with the same size as the local feature map of the first convolutional neural network stage is obtained. ;

[0134] Align the feature map in the first visual converter stage. Local feature maps of the first stage of the convolutional neural network After concatenation, a 1×1 convolution is performed to obtain the fused feature map of the first heterogeneous feature alignment module. ;

[0135] The process of obtaining the fused feature map of the first heterogeneous feature alignment module is shown in the following formula:

[0136] ,

[0137] in, Represents deformable convolution. This represents transposed convolution. Indicates batch normalization. Represents the ReLU activation function. Indicates feature splicing, This represents a 1 × 1 convolution.

[0138] The curvature KAN-guided attention module optimizes the attention mechanism to better capture local geometric features of an image. For example, the curvature KAN-guided attention module is placed between the first convolutional neural network stage and its corresponding decoder layer.

[0139] Align the first heterogeneous feature with the fused feature map of the module. After passing through a 3×3 curvature convolution kernel Local curvature feature extraction is performed to obtain the local curvature response feature map of the first curvature KAN-guided attention module. The curvature convolution kernel weights are represented as follows:

[0140] ,

[0141] For the local curvature response feature map of the first curvature KAN-guided attention module Each image within a batch is processed individually, including the images it contains. For each channel in a single image, the individual channels are aggregated along the spatial channel dimension to obtain the channel curvature weight vector corresponding to that image. The channel curvature weight vectors of all images in the batch are then combined to obtain the total channel curvature weight, as shown in the formula:

[0142] ,

[0143] Total channel curvature weights for the first curvature KAN-guided attention module After normalization, to avoid non-convergence during training, the normalized curvature weights are obtained. The formula is:

[0144] ,

[0145] in, This indicates the position of the total channel curvature weights in the first curvature KAN-guided attention module. The element value at that position, and They are The maximum and minimum values, It is a small constant close to zero (e.g., 1e-8) used to avoid division by zero.

[0146] Normalized curvature weights A nonlinear mapping is performed using a KAN network with 64 hidden layers to optimize the total channel curvature weights of the first curvature KAN-guided attention module. The feature map output from the first convolutional neural network stage is then multiplied element-wise with the optimized total channel curvature weights of the first curvature KAN-guided attention module to obtain the geometric feature map of each curvature KAN-guided attention module. The formula is as follows:

[0147] ,

[0148] in, The geometric feature map for the first curvature KAN-guided attention module. For normalization, For KAN networks, This is for element-wise multiplication.

[0149] The global feature map from the fourth visual converter stage and the geometric feature maps from the three curvature KAN guided attention modules are input into the decoder to output the segmentation result of the OCT image to be segmented.

[0150] In this embodiment, the decoder adopts the Deep Shallow DualPath Decoder (DSDPD) designed in this invention. This decoder constructs two parallel decoding paths to process deep features and shallow features respectively, thereby significantly enhancing the model's ability to capture small target lesions.

[0151] The deep decoding path includes:

[0152] The global feature map of the fourth visual converter stage The resolution is increased by sequentially passing the transposed convolution with a stride of 2, a batch normalization layer, and a ReLU activation function, resulting in the upsampled global feature map in the fourth visual converter stage. ;

[0153] Upsample the global feature map after the fourth visual converter stage. Geometric feature map of the third curvature KAN-guided attention module After concatenation, features are further extracted and the number of channels is reduced through two cascaded 3×3 convolutions, batch normalization layers, and ReLU activation functions, resulting in the output feature map of the third deep decoder layer. ;

[0154] This process can be described by the following formula:

[0155] ,

[0156] ,

[0157] The output feature map of the third deep decoder layer The resolution is increased by sequentially passing the transposed convolution with a stride of 2, the batch normalization layer, and the ReLU activation function to obtain the output feature map of the third deep decoder layer after upsampling;

[0158] After upsampling, the output feature map of the third deep decoder layer is concatenated with the geometric feature map of the second curvature KAN-guided attention module. Then, through two cascaded 3×3 convolutions, batch normalization layers, and ReLU activation functions, features are further extracted and the number of channels is reduced to obtain the output feature map of the second deep decoder layer. ;

[0159] The output feature map of the second deep decoder layer The resolution is increased by sequentially passing the transposed convolution with a stride of 2, the batch normalization layer, and the ReLU activation function to obtain the output feature map of the second deep decoder layer after upsampling;

[0160] After upsampling, the output feature map of the second deep decoder layer is concatenated with the geometric feature map of the first curvature KAN-guided attention module. Then, through two cascaded 3×3 convolutions, batch normalization layers, and ReLU activation functions, features are further extracted and the number of channels is reduced to obtain the output feature map of the first deep decoder layer. ;

[0161] Shallow decoding paths include:

[0162] Guide the geometric feature map of the second curvature KAN to the attention module The resolution is increased by sequentially applying bilinear interpolation, batch normalization layers, and ReLU activation functions to obtain the geometric feature map after upsampling by the second curvature KAN-guided attention module. The formula is:

[0163] ,

[0164] in, This is a bilinear interpolation.

[0165] The geometric feature map after upsampling by the second curvature KAN is used to guide the attention module. Geometric feature map of the first curvature KAN-guided attention module After concatenation, features are further extracted and the number of channels is reduced through two cascaded 3×3 convolutions, batch normalization layers, and ReLU activation functions to obtain a shallow feature map. The formula is:

[0166] ,

[0167] shallow feature map Feature map output from the first deep decoder layer By stitching along the channel dimension, the fused feature map is obtained. The formula is:

[0168] ,

[0169] in, This is for splicing along the channel dimension.

[0170] The fused feature map The segmentation result of the OCT image to be segmented is obtained by using a segmentation head consisting of 3×3 convolution and bilinear upsampling. The formula is:

[0171] ,

[0172] in, This is bilinear upsampling.

[0173] The loss function of the choroidal neovascularization segmentation model in this invention adopts a joint segmentation function based on cross-entropy loss and Dice loss. The formula is:

[0174] ,

[0175] in, For joint partitioning functions, For cross-entropy loss, This is a loss for Dice.

[0176] Cross-entropy loss It is one of the most widely used loss functions in convolutional neural networks, and its definition is shown in the following formula:

[0177] ,

[0178] in, For the number of pixels, For pixel index, To the number of categories of the target segment, For category indexing, each pixel typically belongs to only one category. One of the categories, For pixels Category The true segmentation label, Pixel segmentation model for choroidal neovascularization Category The predicted probability, .

[0179] This invention uses the softmax function to process the final output of the choroidal neovascularization segmentation model, so that the sum of the predicted probabilities for all categories is 1.

[0180] Dice loss function After calculating the overlap between the target and the gold standard for each category, the category average is calculated using the following formula:

[0181] ,

[0182] in, It is a smoothing factor to prevent the numerator or denominator from being zero, and its value is 1e-6.

[0183] When this invention is applied to CNV segmentation tasks, it achieves good segmentation performance on multiple datasets, demonstrating the effectiveness of the heterogeneous feature alignment module MSHFAM, the curvature KAN guided attention CKANA module, and the deep and shallow dual-path decoder DSDPD designed in this invention, as well as the good generalization of the HFCKANet network as a whole.

[0184] To verify the effectiveness of the OCT image choroidal neovascularization segmentation method of the present invention, this embodiment uses two OCT datasets of CNV lesions for experimental verification. Details are as follows:

[0185] This study used two independent datasets for experiments. Dataset 1 contains 1230 OCT images from 68 affected eyes, divided into training, validation, and test sets in a 6:2:2 ratio, with 747 images for training, 240 for validation, and 243 for testing. Dataset 2 contains 1483 OCT images from 64 affected eyes, also split in a 6:2:2 ratio, with 984 images for training, 244 for validation, and 255 for testing. To standardize input specifications and improve network training efficiency, all OCT images were resampled to 512×512 pixels. Segmentation labels were annotated under the guidance of experienced ophthalmologists to ensure accuracy and reliability. To avoid overfitting during model training and enhance the model's generalization ability, online random data augmentation strategies were employed, including flipping, sharpening, blurring (including Gaussian blur, mean blur, and median blur), and contrast adjustment. The model performance was evaluated using Dice coefficient, intersection-over-union ratio (IoU), sensitivity (Sen), and accuracy (Acc) as objective evaluation indicators.

[0186] To verify the performance of the HFCKANet designed in this invention, it was compared with several advanced deep learning segmentation networks, including UKAN, UNet, PSPNet, HRNet, CPFNet, TransUNet, TransAttUNet, UCTransNet, and H2Former.

[0187] As shown in Table 1, Table 1 illustrates the segmentation performance of different segmentation models on dataset I.

[0188] Table 1

[0189]

[0190] As shown in Table 1, the choroidal neovascularization segmentation model (HFCKANet) designed in this invention achieves the best performance across all metrics, with Dice, Jaccard, Sensitivity, and ACC reaching 90.69%, 83.35%, 90.31%, and 99.75%, respectively. The segmentation Dice of UKAN and Unet networks based on ordinary encoder-decoder structures are 88.13% and 88.24%, respectively, which are relatively good but not as good as the method of this invention. This may be because while these methods have certain feature extraction capabilities, they may be insufficient in feature fusion and detail preservation. These networks extract features during the encoding stage and then perform upsampling and feature fusion during the decoding stage. This process may result in the loss of some important details, leading to less refined segmentation results. The segmentation Dice of HRNet based on high-resolution feature representation learning and CPFNet based on a context pyramid structure are 88.27% and 89.02%, respectively, showing some improvement in CNV segmentation. Both utilize multi-scale features; contextual information at different scales helps to better understand image content, thereby improving segmentation accuracy. TransUNet, TransAttUNet, and H2Former are hybrid architecture networks proposed in recent years that combine the advantages of Transformer and CNN. Their segmentation Dice on dataset I is 89.72%, 89.19%, and 89.19%, respectively. By combining the global modeling capability of Transformer with the local feature extraction capability of CNN, these networks have improved the accuracy of CNV segmentation tasks to some extent, but their performance is still lower than that of the method in this invention.

[0191] As shown in Table 2, Table 2 illustrates the segmentation performance of different segmentation models on dataset II.

[0192] Table 2

[0193]

[0194] As shown in Table 2, the method proposed in this invention achieves 87.42%, 78.12%, 86.86%, and 99.20% respectively in the four indicators of Dice, Jaccard, Sensitivity, and ACC on dataset II, which is better than other methods, indicating that the proposed method has good generalization ability.

[0195] Figure 4 This is a visualization of the comparative experimental results of the method of this invention with other segmentation methods on dataset I. Figure 4 In the image (a), the OCT image to be segmented is from dataset I. Figure 4 In (b), the CNV segmentation gold standard for the OCT image to be segmented in dataset I is... Figure 4(c) in the image shows the segmentation result using UNet. Figure 4 (d) in the figure represents the segmentation result using CPFNet. Figure 4 (e) in the figure represents the segmentation result using TransUNet. Figure 4 In the diagram, (f) represents the segmentation result using TransAttUnet. Figure 4 In the diagram, (g) represents the segmentation result using UCTransNet. Figure 4 (h) in the figure represents the segmentation result using the method of the present invention.

[0196] Figure 5 This is a visualization of the comparative experimental results of the method of this invention with other segmentation methods on dataset II. Figure 5 In the image (a), the OCT image to be segmented is from dataset II. Figure 5 In (b), CNV segmentation is the gold standard for the OCT images to be segmented in dataset II. Figure 5 (c) in the image shows the segmentation result using UNet. Figure 5 (d) in the figure represents the segmentation result using CPFNet. Figure 5 (e) in the figure represents the segmentation result using TransUNet. Figure 5 In the diagram, (f) represents the segmentation result using TransAttUnet. Figure 5 In the diagram, (g) represents the segmentation result using UCTransNet. Figure 5 (h) in the figure represents the segmentation result using the method of the present invention.

[0197] Depend on Figure 4 , Figure 5 It is known that UKAN and UNet are both U-shaped network structures. Due to insufficient receptive field, holes often appear inside the segmentation of large targets. PSPNet enhances the extraction capability of multi-scale features through pyramid pooling modules, which alleviates the discontinuity of image segmentation to some extent. HRNet processes images in parallel through multi-resolution sub-networks, maintaining high-resolution features and performing excellently in detail preservation, but it has high computational resource requirements. CPFNet enhances feature extraction capabilities through a context pyramid fusion network and performs well in handling complex backgrounds. TransUNet combines Transformer and UNet, and performs excellently in handling long-range dependencies and complex scenes. The segmentation results of the HFCKANet method in this invention, especially in the preservation of edge details, are superior to other methods.

[0198] To verify the effectiveness of the heterogeneous feature alignment module MSHFAM, curvature KAN guided attention module CKANA, and deep and shallow dual-path decoder DSDPD designed in this invention, related ablation experiments were conducted.

[0199] As shown in Table 3, Table 3 presents the ablation experimental results of the method of the present invention on dataset I.

[0200] Table 3

[0201]

[0202] Using TransUNet as the baseline network, different combinations of the heterogeneous feature alignment module MSHFAM, the curvature KAN-guided attention module CKANA, and the deep and shallow dual-path decoder DSDPD designed in this invention were added to the baseline network, and the model improved in Dice metric. Specifically, the segmentation Dice metric of the baseline network was 89.72%. When only the heterogeneous feature alignment module MSHFAM was added, the Dice metric improved to 90.16%, indicating that the heterogeneous feature alignment module MSHFAM can better align the features extracted by the convolutional neural network and the visual extractor. When only the curvature KAN-guided attention module CKANA was added, the Dice metric improved to 90.10%, indicating that the curvature KAN-guided attention module CKANA improved the model's utilization of geometric feature constraints. When only the deep and shallow dual-path decoder DSDPD was added, the Dice metric improved to 90.27%, indicating that the deep and shallow dual-path decoder DSDPD effectively solved the problem of insufficient model capture ability for small target lesions. When the heterogeneous feature alignment module MSHFAM, the curvature KAN guided attention module CKANA, and the deep and shallow dual-path decoder DSDPD are added simultaneously, it becomes the HFCKANet designed in this invention. The segmentation Dice index reaches 90.69%, which is the best segmentation performance, verifying the effectiveness of the heterogeneous feature alignment module MSHFAM, the curvature KAN guided attention module CKANA, and the deep and shallow dual-path decoder DSDPD designed in this invention.

[0203] As shown in Table 4, Table 4 presents the ablation experimental results of the method of the present invention on dataset II.

[0204] Table 4

[0205]

[0206] By adding different combinations of the heterogeneous feature alignment module MSHFAM, the curvature KAN guided attention module CKANA, and the deep and shallow dual-path decoder DSDPD to the baseline network, the model improved the Dice index. In particular, adding all three modules improved the segmentation Dice index to 87.42%, and the experimental results also verified the effectiveness of these three modules.

[0207] Thus, an HFCKANet method suitable for automatic CNV segmentation has been implemented and validated. Based on the heterogeneous feature alignment module MSHFAM, the curvature KAN-guided attention module CKANA, and the deep / shallow dual-path decoder DSDPD designed in this invention, HFCKANet effectively solves the problems of insufficient fusion of features extracted by visual extractors and convolutional neural networks, insufficient utilization of geometric feature constraints, and insufficient ability to capture small target lesions, thereby improving segmentation accuracy and making it suitable for automatic CNV segmentation tasks.

[0208] This embodiment three provides an OCT image choroidal neovascularization segmentation system, including:

[0209] A memory for storing computer programs; a processor for executing the computer programs to implement the steps of the above-described OCT image choroidal neovascularization segmentation method.

[0210] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0211] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0212] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0213] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0214] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. An OCT image choroidal neovascularization segmentation method, characterized in that, The curvature KAN guided attention module comprises: The OCT image to be segmented is input into the choroidal neovascularization segmentation model, which outputs the segmentation result of the OCT image. The choroidal neovascularization segmentation model is obtained by setting a curvature KAN-guided attention module between each convolutional neural network stage and its corresponding decoder layer of the TransUNet model. A heterogeneous feature alignment module is set after each convolutional neural network stage to... The local feature map of the first stage of the convolutional neural network and the first stage The global feature map of the first visual converter stage is obtained through the first... The heterogeneous feature alignment module is obtained to achieve the first... The fused feature map of the heterogeneous feature alignment modules includes: The first The global feature map in the first visual converter stage is sequentially processed through deformable convolution, multi-level transposed convolution, batch normalization, and activation functions to obtain the result that is consistent with the first visual converter stage. The local feature maps of the i-th convolutional neural network stage are of uniform size, and are used as the i-th convolutional neural network stage. Alignment feature maps for each visual converter stage; where... , This represents the number of stages in the convolutional neural network. The first Alignment feature map of the first visual converter stage and the first After concatenating the local feature maps from the first stage of the convolutional neural network, a 1×1 convolution is performed to obtain the second stage. Fusion feature map of heterogeneous feature alignment modules; The first The fused feature map of the heterogeneous feature alignment module is used as the first... The curvature KAN corresponding to each stage of the convolutional neural network guides the input of the attention module; The input feature map of the curvature KAN guided attention module is extracted through the corresponding curvature KAN guided attention module, and the geometric feature map of the curvature KAN guided attention module is extracted. The input feature map of the curvature KAN guided attention module is extracted through the corresponding curvature KAN guided attention module, and the geometric feature map of the curvature KAN guided attention module is extracted. For each channel of the local curvature feature map corresponding to the input feature map of the curvature KAN guided attention module, the feature values of all spatial positions in the channel are summed to obtain a channel curvature weight vector composed of the feature value sums of the channels. After the input feature map of the curvature KAN guided attention module is normalized, the channel curvature weight vector is optimized through the KAN network. The input feature map of the curvature KAN guided attention module is multiplied element by element with the corresponding optimized channel curvature weight vector to obtain the geometric feature map of the curvature KAN guided attention module. 2.The OCT image choroidal neovascularization segmentation method of claim 1, wherein, The segmentation result of the OCT image to be segmented is output by inputting the OCT image to be segmented into the choroidal neovascularization segmentation model, comprising: The OCT image to be segmented is passed through a convolutional neural network to extract a local feature map of a convolutional neural network stage; The first After downsampling the local feature maps of each convolutional neural network stage, they are divided into multiple patch blocks. After linear mapping of each patch block, position information is embedded to obtain a patch sequence with embedded position information. passing the patch sequence embedded with the position information through a visual transformer to extract global feature maps of a visual transformer stage; The first The local feature map of the first convolutional neural network stage and the first... The global feature map of the first visual converter stage is obtained through the first... The heterogeneous feature alignment module is obtained to achieve the first... The fused feature map of three heterogeneous feature alignment modules; among them... ; The fusion feature map of each heterogeneous feature alignment module is extracted through the corresponding curvature KAN guided attention module, and the geometric feature map of each curvature KAN guided attention module is extracted. The first The global feature map of each visual converter stage and the geometric feature map of each curvature KAN-guided attention module are input into the decoder, and the segmentation result of the OCT image to be segmented is output.

3. The method of claim 2, wherein the method further comprises: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The first The global feature map of the first visual converter stage is sequentially passed through transposed convolution, batch normalization layer, and ReLU activation function to obtain the first... Global feature map after upsampling at each visual converter stage; The first The global feature map upsampled after the first visual converter stage and the first... After concatenating the geometric feature maps of the curvature KAN-guided attention module, the result is obtained by passing two cascaded 3×3 convolutions, batch normalization layers, and ReLU activation functions. Output feature maps of each deep decoder layer; The first The output feature map of the deep decoder layer is sequentially passed through transposed convolution, batch normalization layer, and ReLU activation function to obtain the upsampled feature map of the th layer. Output feature maps of each deep decoder layer; After upsampling, the first The output feature map of the deep decoder layer and the first After concatenating the geometric feature maps of the curvature KAN-guided attention module, the result is obtained by passing two cascaded 3×3 convolutions, batch normalization layers, and ReLU activation functions. The target feature map of each deep decoder layer; among which... ; The target feature map of the first deep decoder layer is input into the segmentation head to obtain the segmentation result of the OCT image to be segmented.

4. The method of claim 2, wherein the method further comprises: The decoder further comprises a shallow decoder layer, and the geometric feature map of the first curvature KAN guided attention module and the geometric feature map of the second curvature KAN guided attention module are input into the shallow decoder layer to obtain a shallow feature map, comprising: The geometric feature map of the second curvature KAN guided attention module is sequentially input into the bilinear interpolation, batch normalization layer and ReLU activation function to obtain the up-sampled geometric feature map of the second curvature KAN guided attention module; The up-sampled geometric feature map of the second curvature KAN guided attention module is concatenated with the geometric feature map of the first curvature KAN guided attention module, and then input into two cascaded 3x3 convolution, batch normalization layer and ReLU activation function to obtain the shallow feature map. The shallow feature map is concatenated with the feature map output by the first deep decoder layer along the channel dimension, and then input into the segmentation head to obtain the segmentation result of the OCT image to be segmented.

5. The method of claim 4, wherein the method further comprises: The segmentation head comprises a 3x3 convolution and a bilinear interpolation connected in sequence.

6. The method of claim 2, wherein the method further comprises: When the OCT image to be segmented is passed through the convolutional neural network, the local feature maps of each convolutional neural network stage are extracted, including: The OCT image to be segmented is input into a first convolutional neural network stage through a convolution kernel with a size of 7x7 and a step of 2, a group normalization and a ReLU activation function connected in sequence to obtain a local feature map of the first convolutional neural network stage; The local feature map of the first convolutional neural network stage is input into a first stage residual transformation feature map through a max-pooling with a kernel size of 3x3 and a step of 2, a group normalization, a ReLU activation function, a 3x3 convolution with a step of 1, a group normalization and a ReLU activation function connected in sequence. After the first-stage residual transformation feature map is added to the local feature map of the first convolutional neural network stage element by element, a ReLU activation function is used to obtain the local feature map of the second convolutional neural network stage; The local feature map of the second convolutional neural network stage is sequentially processed by a 1x1 convolution, group normalization, a ReLU activation function, a 3x3 convolution with a step of 1, group normalization, and a ReLU activation function to obtain a second-stage residual transformation feature map; After the second-stage residual transformation feature map is added to the local feature map of the second convolutional neural network stage element by element, a ReLU activation function is used to obtain the local feature map of the third convolutional neural network stage.

7. The method of claim 2, wherein the method further comprises: Each visual converter stage is stacked by three Transformer layers, and each Transformer layer includes a multi-head self-attention module and a feedforward neural network connected in sequence.

8. The method of claim 1, wherein the method further comprises: The weight parameters of the 3x3 curvature convolution kernel are as follows: ， wherein, are the weight parameters of the 3x3 curvature convolution kernel.

9. An OCT image choroidal neovascularization segmentation system, comprising: The method comprises the following steps: A memory for storing a computer program; A processor for executing the computer program to implement the steps of the OCT image choroidal neovascularization segmentation method in any one of claims 1 to 8.

Images